Skip to content

What is a test step?

A test step is made of four main components used to determine the actions Chainsaw will perform when executing the step.

  1. The try statement (required)
  2. The catch statement (optional)
  3. The finally statement (optional)
  4. The cleanup statement (optional)

Syntax

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: example
spec:
  steps:
    # `try` defines operations to execute in the step
  - try: [...]
    # `catch` defines operations to execute when the step fails
    catch: [...]
    # `finally` defines operations to execute at the end of the step
    finally: [...]
    # `cleanup` defines operations to execute at the end of the test
    cleanup: [...]

Reference

The full structure of the TestStepSpec is documented here.

Lifecycle

Try, Catch, Finally flow

Operations defined in the try block are executed first, then:

  • If an operation fails to execute, Chainsaw won't execute the remaining operations and will execute all operations defined in the catch block instead (if any).
  • If all operations succeed, Chainsaw will NOT execute operations defined in the catch block (if any).
  • Regardless of the step outcome (success or failure), Chainsaw will execute all operations defined in the finally block (if any).

Tip

Note that all operations coming from the catch or finally blocks are executed. If one operation fails, Chainsaw will mark the test as failed and continue executing with the next operations.

Without failure

sequenceDiagram
    autonumber

    participant S as Step N

    box Try block
    participant T1 as Op 1
    participant T2 as Op N
    end
    box Catch block
    end
    box Finally block
    participant F1 as Op 1
    participant F2 as Op N
    end
    participant S1 as Step N+1

    S  -->> T1 : try
    T1 ->>  T2 : success
    T2 -->> S  : done
    S  -->> F1 : finally
    F1 ->>  F2 : done
    F2 -->> S  : done
    S  -->> S1 : next step
  1. Step starts by executing operations in the try block
  2. Operations in the try block execute sequentially
  3. All operations in the try block terminate
  4. Step starts executing operations in the finally block
  5. Operations in the finally block execute sequentially
  6. All operations in the finally block terminate
  7. Next step starts executing

With failure

sequenceDiagram
    autonumber

    participant S as Step N

    box Try block
    participant T1 as Op 1
    participant T2 as Op N
    end
    box Catch block
    participant C1 as Op 1
    participant C2 as Op N
    end
    box Finally block
    participant F1 as Op 1
    participant F2 as Op N
    end

    S  -->> T1 : try
    T1 ->>  T2 : success
    T2 -->> S  : error
    S  -->> C1 : catch
    C1 ->>  C2 : done
    C2 -->> S  : done
    S  -->> F1 : finally
    F1 ->>  F2 : done
    F2 -->> S  : done
  1. Step starts by executing operations in the try block
  2. Operations in the try block execute sequentially until an error happens
  3. Operations in the try block stop when an error occurs
  4. Step starts executing operations in the catch block
  5. Operations in the catch block execute sequentially
  6. All operations in the catch block terminate
  7. Step starts executing operations in the finally block
  8. Operations in the finally block execute sequentially
  9. All operations in the finally block terminate

Cleanup

In addition to try, catch and finally blocks, the cleanup block of steps is better illustrated in the Test lifecycle diagrams.