Test spec¶

A Chainsaw test is mostly made of steps.

That being said, there are a couple of other interesting fields too.

Syntax¶

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: example
spec:
  # test configuration
  concurrent: false
  bindings:
  - name: foo
    value: bar
  timeouts:
    apply: 1s
    assert: 2m
    delete: 30s
  ...
  steps:
  # step 1
  - try: ...
  # step 2
  - try: ...
    catch: ...
  # step 3
  - try: ...
    catch: ...
    finally: ...

Reference¶

The full structure of the TestSpec is documented here.

Lifecycle¶

Cleanup¶

At the end of the test, Chainsaw cleans up resources it created during the test, in the opposite order of creation.

By default, when a step fails, Chainsaw stops the execution and the remaining steps are not executed. The cleanup process starts at the moment the test stops executing.

Tip

Note that when a failure happens during cleanup, the test is marked as failed and Chainsaw continues executing cleanup for the remaining steps.

Without failure¶

sequenceDiagram
    autonumber
    participant T as Test
    participant S1 as Step 1
    participant S2 as Step 2
    participant S3 as Step 3

    T  ->> S1: execute
    S1 ->> S2: execute
    S2 ->> S3: execute

    S3 -->> S2: cleanup
    S2 -->> S1: cleanup
    S1 -->> T: cleanup

Test starts by executing Step 1
Step 1 terminates -> Step 2 starts executing
Step 2 terminates -> Step 3 starts executing
Step 3 terminates -> Cleanup for Step 3 starts
Cleanup for Step 3 terminates -> Cleanup for Step 2 starts
Cleanup for Step 2 terminates -> Cleanup for Step 1 is executed

With failure¶

sequenceDiagram
    autonumber
    participant T as Test
    participant S1 as Step 1
    participant S2 as Step 2
    participant S3 as Step 3

    T  ->> S1: execute
    S1 ->> S2: execute (fail)

    Note left of S3: Step 3 is NOT executed

    S2 -->> S1: cleanup
    S1 -->> T: cleanup

Test starts by executing Step 1
Step 1 terminates -> Step 2 starts executing
Step 2 fails -> Cleanup for Step 2 starts
Cleanup for Step 2 terminates -> Cleanup for Step 1 is executed

Supported elements¶

Namespace¶

The namespace the test should run into, see Namespace selection.

Namespace template¶

Eventually provide a template if you something specific, see Namespace template.

Timeouts¶

All timeouts can be specified per test, see Control your timeouts.

Clusters¶

Additional clusters can be registered at the test level, see Multi-cluster options.

Cluster¶

The cluster (by name) used to run the test, see Multi-cluster setup.

Bindings¶

Bindings can be registered at the test level and inherited in all steps, see Bindings.

Catch¶

Catch blocks can be specified at the test level to declare common catch statements.

Template¶

Chainsaw allows templating configuration on a per test basis.

Concurrency¶

Controlling concurrency per test is also possible, see Concurrency control.

Skip¶

In case you need to skip a test for whatever reason, use skip: true.

Steps¶

Steps are what tests will execute when they are run, see Test step spec dedicated section.