TestSteps based syntax¶

The TestSteps based syntax is more verbose than the manifests based one but offers more flexibility to influence how a test step runs.

It allows providing additional configuration per specific step, and makes it easier to reuse files accross multiple tests.

This document focuses on understanding how TestSteps work.

Keep in mind that a TestStep, like with the manifest based syntax, relies on file naming conventions. On the other hand it doesn't suffer the unsupported deletion limitation and can be combined with manifests based syntax when defining a step.

The TestStep resource¶

A TestStep resource, like any Kubernetes resource has an apiVersion, kind and metadata section.

It also comes with a spec section used to declaratively represent the step operations and other configuration elements belonging to the step being defined.

The full structure of the TestStep resource is documented here.

TestStep loading process¶

In addition to the manifests loading process, if the manifest being loading is a TestStep it is directly aggregated in the steps that compose the test (it would not make sense to consider the TestStep a resource to be applied in the target cluster).

Manifests that are not a TestStep are processed as usual and participate to the test step being loaded in the same way they do with manifests based syntax.

Note that it's only allowed to have a single TestStep resource for a given test step.

Raw Resource Support¶

Chainsaw now allows the specification of Kubernetes resources directly within the TestStep definition. This raw resource feature enhances flexibility by allowing inline resource definitions, particularly useful for concise or reusable configurations.

Example¶

01-test-step.yaml¶

The manifest below contains a TestStep in a file called 01-test-step.yaml. Chainsaw will load the TestStep in step 01. The TestStep defines a custom timeout for step 01 and references a config map manifest from a relative path.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: TestStep
metadata:
  name: test-step-name
spec:
  skipDelete: true
  # these timeouts are applied per operation
  # it would not be possible to override the timeouts
  # with a manifests based approach
  timeouts:
    apply: 45s
  try:
  # apply a configmap to the cluster
  # the path to the configmap is relative to the folder
  # containing the test, hence allow reusing manifests
  # across multiple tests
  - apply:
      file: ../resources/configmap.yaml

02-assert.yaml¶

The manifest below contains an assertion statement in a file called 02-assert.yaml. Chainsaw will associate this manifest with an assert operation in step 02. Note that this file doesn't contain a TestStep resource.

apiVersion: v1
kind: ConfigMap
metadata:
  name: chainsaw-quick-start
data:
  foo: bar

02-error.yaml¶

The manifest below contains a TestStep in a file called 02-error.yaml. Chainsaw will load the TestStep and aggregate it in step 02. The TestStep defines a custom timeout for step 02 and references an error statement manifest from a relative path.

This illustrates how a TestStep resource can be combined with raw manifests to compose a test step in Chainsaw.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: TestStep
metadata:
  name: test-step-name
spec:
  # these timeouts are applied per operation
  # it would not be possible to override the timeouts
  # with a manifests based approach
  timeouts:
    error: 20s
  try:
  # evaluate an error statement against resources
  # present in the cluster
  - error:
      file: ../resources/configmap-error.yaml

02-assert-exec.yaml¶

This manifest contains both an assertion and an exec operation, showing how you can mix operations in a single step.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: TestStep
metadata:
  name: test-step-name-02
spec:
  try:
  - assert:
      file: ../resources/configmap-assert.yaml
  - command:
      entrypoint: "echo"
      args:
      - "Hello Chainsaw"

Example Raw Resource¶

01-test-step.yaml¶

This TestStep defines a custom timeout and applies a ConfigMap directly within the step using the raw resource feature.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: TestStep
metadata:
  name: test-step-name
spec:
  skipDelete: true
  timeouts:
    apply: 45s
  try:
  - apply:
      resource:
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: chainsaw-quick-start
        data:
          foo: bar

02-assert.yaml¶

This manifest contains an assertion statement for a ConfigMap and does not include a TestStep resource.

apiVersion: v1
kind: ConfigMap
metadata:
  name: chainsaw-quick-start
data:
  foo: bar

02-error.yaml¶

The manifest below contains a TestStep in a file called 02-error.yaml. Chainsaw will load the TestStep and aggregate it in step 02.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: TestStep
metadata:
  name: test-step-name
spec:
  timeouts:
    error: 20s
  try:
  - error:
      file: ../resources/configmap-error.yaml

Conclusion¶

This test will create a config map in the first step. The second step will both assert that the content of the config map contains the foo: bar data, and verify that the configmap does not contain the lorem: ipsum data.

The first step is made of a single TestStep resource, while the second step is the combination of a TestStep resource and a raw manifest.