Helm Chart
The available Helm Chart is the easiest way to install and configure Policy Reporter. Depending on your usage, it provides several values to configure features and disable anything else.
values.yaml
of the related chart for reference.Usage
Add the Policy Reporter Helm Repository
helm repo add policy-reporter https://kyverno.github.io/policy-reporterhelm repo update
Installation
helm install policy-reporter policy-reporter/policy-reporter -f values.yaml --create-namespace -n policy-reporter
Configuration
By default all available features are disabled. So it's up to the user to enable or configure the features needed. This approach reduces the required resources to a bare minimum.
Enable Metrics Endpoint
Metrics will be available at http://policy-reporter:8080/metrics
metrics: enabled: true
Enable REST Endpoints
REST APIs will be available at http://policy-reporter:8080/v1/
see API Reference for details.
rest: enabled: true
Enable API BasicAuth (since v2.20.0)
It is possible to enable global HTTP Basic Authentication for the API and metric endpoints
global: basicAuth: # username: username # password: password secretRef: auth-secret
The credentials can be configured directly or as reference to an already existing secret with username
and password
key/value pairs.
Policy Reporter UI will automatically do authorized API calls. If monitoring
is enabled the ServiceMonitor resources will also get the required basicAuth
configuration.
Enable Targets / Notification
Policy Reporter supports several targets to which notifications can be sent. You can configure as many targets as you like, and also configure different targets for different priorities or sources (like Kyverno, Kube Bench or Falco). Channels in combination with filters allow you to configure multiple clients of the same type of targets. This is useful, for example, to forward different priorities or results of certain namespaces to a separate Slack channel.
For example, you can configure Grafana Loki by providing an accessible host or Slack with different channels:
target: loki: # loki host address host: "http://loki.loki-stack:3100" # loki API path, defaults to "/api/prom/push" (deprecated) path: "/loki/api/v1/push" # minimum priority "" < info < warning < critical < error minimumPriority: "warning" # Skip already existing PolicyReportResults on startup skipExistingOnStartup: true # Send only results from the given sources customLabels: cluster: rancher-desktop sources: - kyverno - falco slack: minimumPriority: "warning" skipExistingOnStartup: true channels: - webhook: "https://hooks.slack.com/services/123..." filter: namespaces: include: ["team-a-*"] - webhook: "https://hooks.slack.com/services/456..." filter: namespaces: include: ["team-b-*"]
Use secretRef
Instead of defining your credentials or webhooks directly in the values.yaml
, you can also use the secretRef
configuration to define an already existing Secret by name. If the secret does not exist, the target is skipped.
The Secret should contain the related configuration as key
. Supported keys are host, webhook, username, password, token, accessKeyID and secretAccessKey - depending on the related target. Only exception is token
, which is dedicated for webhook
targets and is added as Authorization header.
The secretRef
is also supported for channels, so you can use different secrets for different channels.
# values.yamltarget: slack: minimumPriority: "warning" skipExistingOnStartup: true channels: # secretRef configuration - secretRef: "team-a-slack-webhook" filter: namespaces: include: ["team-a-*"] # inline configuration - webhook: "https://hooks.slack.com/services/456..." filter: namespaces: include: ["team-b-*"]
related Secret:
apiVersion: v1kind: Secretmetadata: name: team-a-slack-webhooktype: Opaquedata: webhook: aHR0cHM6Ly9ob29rcy5zbGFjay5jb20vc2VydmljZXMvVDAuLi4= # https://hooks.slack.com/services/T0...
Enable E-Mail Reports
Sends automatically and regularly email summary reports over a configured SMTP server to one ore more emails. It supports filter
and channels
to send only a subset of namespaces or sources to dedicated emails, this is useful in multi tenant environments.
emailReports: clusterName: "" # (optional) - displayed in the E-Mail Report if configured smtp: # (optional) name of a existing secret with all or a subset of the following SMTP configurations # its an alternative way to configure your SMTP server without exposing credentials in the values.yaml secret: "" host: "" port: 465 username: "" password: "" from: "" # Displayed From E-Mail Address encryption: "" # default is none, supports ssl/tls and starttls certificate: "" # path to your customize cert skipTLS: false # skip TLS verification # basic summary report summary: enabled: false schedule: "* 8 * * *" # CronJob schedule defines when the report will be send activeDeadlineSeconds: 300 # timeout in seconds backoffLimit: 3 # retry counter ttlSecondsAfterFinished: 0 restartPolicy: Never # pod restart policy to: [] # list of receiver email addresses filter: {} # optional filters # disableClusterReports: false # remove ClusterPolicyResults from Reports # namespaces: # include: [] # exclude: [] # sources: # include: [] # exclude: [] channels: [] # (optional) channels can be used to to send only a subset of namespaces / sources to dedicated email addresses #- to: ['team-a@company.org'] # filter: # disableClusterReports: true # namespaces: # include: ['team-a-*'] # sources: # include: ['Kyverno'] # violation summary report violations: enabled: false schedule: "* 8 * * *" # CronJob schedule defines when the report will be send activeDeadlineSeconds: 300 # timeout in seconds backoffLimit: 3 # retry counter ttlSecondsAfterFinished: 0 restartPolicy: Never # pod restart policy to: [] # list of receiver email addresses filter: {} # optional filters # disableClusterReports: false # remove ClusterPolicyResults from Reports # namespaces: # include: [] # exclude: [] # sources: # include: [] # exclude: [] channels: [] # (optional) channels can be used to to send only a subset of namespaces / sources to dedicated email addresses #- to: ['team-a@company.org'] # filter: # disableClusterReports: true # namespaces: # include: ['team-a-*'] # sources: # include: ['Kyverno']
PolicyReport CRD Filter
Filter processed PolicyReport resources by namespace - you can either define an include or exclude list of namespaces with wildcard support. See report filter for details.
# Filter PolicyReport resources to processreportFilter: namespaces: # Process only PolicyReport resources from an included namespace, wildcards are supported include: [] # Ignore all PolicyReport resources from a excluded namespace, wildcards are supported # exclude will be ignored if an include filter exists exclude: [] clusterReports: # Disable the processing of ClusterPolicyReports disabled: false
Metric customization
Metric Modes
Reduce the cardinality of the PolicyReportResult metrics by customize the provided labels.
Available metric modes are:
- detailed (default): provides all existing label information but has a high cardinality.
Creates 1 metric per resource / policy / rule - simple: provides a predefined subset of the available labels with a lower cardinality.
Provides "namespace", "policy", "status", "severity", "category", "source" - custom: provides the configured labels as list via the
metrics.customLabels
value.
It supports all labels that are also available in the detailed mode.
See the API Reference for details.
metrics: enabled: true mode: custom customLabels: ["namespace", "policy", "source", "status"]
Metric Report Label
Use (Cluster)PolicyReport labels as additional metric labels in custom
mode. Invalid metric label characters will be replaced with _
.
Example
Add the app
label of each (Cluster)PolicyReport, if available, as metric label.
apiVersion: wgpolicyk8s.io/v1alpha2kind: PolicyReportmetadata: labels: app: nginx name: cpol-disallow-host-path namespace: test
metrics: enabled: true mode: custom customLabels: ["status","label:app"]
# HELP policy_report_result List of all PolicyReport Results
# TYPE policy_report_result gauge
policy_report_result{app="nginx",status="pass"} 1
policy_report_result{app="",status="pass"} 1
Metric Property Label
Use PolicyReportResult properties
as additional labels in custom
mode. Invalid metric label characters will be replaced with _
.
Example
Add the score
property of PolicyReportResults, if available, as metric label.
apiVersion: wgpolicyk8s.io/v1alpha2kind: PolicyReport name: trivy-vuln-polr-nginx-5fbc65fff namespace: test ...results:- category: Vulnerability Scan message: 'apt: integer overflows and underflows while parsing .deb packages' policy: CVE-2020-27350 properties: artifact.repository: library/nginx artifact.tag: "1.17" score: "5.7"
metrics: enabled: true mode: custom customLabels: ["property:score", "property:artifact.tag", "status"]
# HELP policy_report_result List of all PolicyReport Results
# TYPE policy_report_result gauge
policy_report_result{artifact_tag="1.17",score="5.7",status="pass"} 1
policy_report_result{artifact_tag="",score="",status="pass"} 1
Metric Filter
Configure the processed namespaces, sources, policies, severities, kinds and/or status for metrics to get rid of unnecessary metrics. You can either define exclude or include rules, with wildcard support, for each available filter and combine them as needed.
metrics: enabled: true filter: namespaces: include: ["prod", "stage"] sources: include: ["Trivy*", "Kyverno"] status: exclude: ["skip"] kinds: exclude: ["Pod"]
Priority Mapping
Define a custom mapping for fail results based on the related policy. You can also overwrite the default priority for fail results without a severity. See priority mapping for details.
policyPriorities: # used for all fail results without severity or concrete mapping default: warning # used for all fail results of the require-ns-labels policy independent of the severity require-ns-labels: error
External Databases (AppVersion >= 2.15.0)
By default, Policy Reporter uses an internal SQLite database to convert (cluster) policies into an internal SQL schema. The REST API uses this schema to execute performant queries and filter or group information according to various criteria.
While this works well in most scenarios, SQLite can reach its limits in large environments. To overcome these limitations, it is possible to configure an external SQL database. Currently PostgreSQL, MySQL and MariaDB are supported.
In a HA setup, only one pod writes to the database, all other pods only read from it. In case of a new leader promotion, the new leader cleans up the DB and recreates the current state to reflect possible changes during the downtime of the previous leader.
database: type: "postgres" database: "policy-reporter" username: "username" password: "password" host: "localhost:5432" enableSSL: false # instead of configure the individual values you can also provide an DSN string # example postgres: postgres://postgres:password@localhost:5432/postgres?sslmode=disable # example mysql: root:password@tcp(localhost:3306)/test?tls=false dsn: "" # configure an existing secret as source for your values # supported fields: username, password, host, dsn, database secretRef: "" # use an mounted secret as source for your values, required the information in JSON format # supported fields: username, password, host, dsn, database mountedSecret: ""
External Caching Storage
To reduce the memory usage of Policy Reporter in bigger clusters, it is possible to configure Redis
as external caching service.
redis: # enables the feature enabled: false # address of the redis service address: "redis:6379" # used redis database database: 1 # prefix for each key prefix: "policy-reporter" # optional authentication username: "" password: ""
Logging
Since AppVersion 2.14.0
its possible to customize the logger output like encoding
and logLevel
. It is also possible to enable API access debug logging.
logging: encoding: console # possible encodings are console and json logLevel: 0 # default info development: false # more human readable structure, enables stacktraces and removes log samplingapi: logging: false # enable debug API access logging, sets logLevel to debug
High Available Setup
The High Available setup makes it possible to deploy more then one instance of Policy Reporter without the issue of duplicated pushes.
By default HA mode will be enabled if the replicaCount
is higher then 1
.
Policy Reporter uses LeaderElection
, to ensure that only one instance is responsible to send pushes for new PolicyReportResults
. Other features like API requests and metrics are loadbalanced between each instance.
The High Available setup also adds an PodDisruptionBudget
with a default minAvailable
of 1
. Both, LeaderElection
as well as the PodDisruptionBudget
can be configured to you personal needs.
replicaCount: 3# enabled if replicaCount > 1podDisruptionBudget: # -- Configures the minimum available pods for policy-reporter disruptions. # Cannot be used if `maxUnavailable` is set. minAvailable: 1 # -- Configures the maximum unavailable pods for policy-reporter disruptions. # Cannot be used if `minAvailable` is set. maxUnavailable:# required when policy-reporter runs in HA mode and you have targets configured# if no targets are configured, leaderElection is disabled automatically# will be enabled when replicaCount > 1leaderElection: enabled: false releaseOnCancel: true leaseDuration: 15 renewDeadline: 10 retryPeriod: 2
Ingress
Serve the API over a hostname with the integrated Ingress support. This is mainly needed for the Multi Tenance Feature of Policy Reporter UI. In this case make sure that the API is not reachable for the outside world.
ingress: enabled: true annotations: nginx.ingress.kubernetes.io/rewrite-target: /$1 hosts: - host: domain.com paths: - path: "/(.*)"
Enable NetworkPolicy
If enabled, the Helm Chart creates a NetworkPolicy resource to allow Policy Reporter egress traffic to the Kubernetes API (defaults to port 6443
) as well as ingress traffic to the Policy Reporter REST API from the Policy Reporter UI. Ingress and egress rules for additional targets or monitoring tools can be extended as needed.
networkPolicy: enabled: true egress: - to: ports: - protocol: TCP port: 6443 ingress: []
Subcharts
Extend Policy Reporter with the Policy Reporter UI and Kyverno Plugin subcharts. The Monitoring subchart helps you link Policy Reporter to your Prometheus Operator setup.
Policy Reporter UI
ui
to be clear that it has to be configured under this subchart.Enable the Policy Reporter UI subchart and the required Policy Reporter REST API.
ui: enabled: true
Display Mode
Policy Reporter UI supports a light and dark mode. The default depends on your system/browser configuration. To enforce a default mode, configure it with:
ui: displayMode: dark # available options: dark / light
Log size
Configure the maximum size of the log in the Policy Reporter UI:
ui: logSize: 200 # 200 items are stored in memory
Refresh Interval
Configure the default refresh interval for all requests in millseconds, you can set it to 0
, to disable autorefreshing.
ui: refreshInterval: 10000
Configure Views
Policy Reporter UI has different views for different kinds of information. Each view, except the dashboard as the entrypoint, can be enabled or disabled as needed. The dashboard can be customized by disabling ClusterPolicyReport or PolicyReport information.
ui: views: dashboard: policyReports: true clusterPolicyReports: true logs: true # also remove the UI as target of policy reporter policyReports: true clusterPolicyReports: true kyvernoPolicies: true kyvernoVerifyImages: true
External Clusters
Use tools like VPN, private Networks or internal Network Load Balancer to expose your APIs in a secure way to the UI
By default, the Policy Reporter UI processes only the Policy Reporter REST API running in the same cluster. If you are working in a multi-cluster environment and running Policy Reporter in each cluster, it can be annoying to switch between the different UIs of each cluster. To solve this problem, it is possible to configure additional Policy Reporter REST APIs from external clusters and switch between them as needed.
The APIs must be accessible for Policy Reporter UI, currently only HTTP Basic authentication is supported. Make sure that you make your APIs available only internally.
To activate the REST-API of your external installations of Policy Reporter, you must set rest.enabled
to true
in the respective configurations.
You can use the the ui.clusterName
configuration to set the name of the default Cluster API, default is Default
.
ui: clusterName: Dev Cluster # name used in the ClusterSelect for the default API clusters: - name: External Cluster # name used for the selection of the Cluster api: https://policy-reporter.external.cluster # reachable external Policy Reporter REST API kyvernoApi: https://kyverno-plugin.external.cluster # (optional) reachable external Policy Reporter Kyverno Plugin REST API skipTLS: false # skip SSL verification certificate: "/app/certs/root.ca" # (optional) path to a mounted root cert for custom signed domains username: username # HTTP BasicAuth Username password: password # HTTP BasicAuth Password secretRef: auth-secret # all configuration can also provided as existing secret with the related key/value pairs, except the cluster name.
External Cluster configuration
rest: enabled: true
Kyverno Plugin integration
kyvernoPlugin
subchart to be enabled.Enable the UI integration of the Kyverno Plugin to get additional views about Kyverno Policies.
ui: plugins: kyverno: true
Logging
Since AppVersion 1.18.2
its possible to customize the logger output like encoding
and logLevel
. It is also possible to enable Proxy request debug logging.
logging: encoding: console # possible encodings are console and json logLevel: 0 # default info development: false # more human readable structure, enables stacktraces and removes log samplingapi: logging: false # enable debug Proxy request logging, sets logLevel to debug
High Available Setup
Because most features are stateless, you can deploy Policy Reporter UI without additional needs in HA mode (replicaCount
> 1
). The only exception is the Log page, which receives pushes from Policy Reporter and hold them in memory by default. If you are using this feature it is recommended to configure redis
as central storage for Log entries.
The High Available setup adds an PodDisruptionBudget
with a default minAvailable
of 1
. The PodDisruptionBudget
can be configured to you personal needs.
ui: replicaCount: 3 # use redis as external log storage instead of an in memory store # recommended when using a HA setup with more then one replica # to get all logs on each instance redis: enabled: false address: "redis:6379" database: 0 prefix: "policy-reporter-ui" username: "" password: "" # enabled if replicaCount > 1 podDisruptionBudget: # -- Configures the minimum available pods for policy-reporter-ui disruptions. # Cannot be used if `maxUnavailable` is set. minAvailable: 1 # -- Configures the maximum unavailable pods for policy-reporter-ui disruptions. # Cannot be used if `minAvailable` is set. maxUnavailable:
Ingress
Serve the UI over a hostname with the integrated Ingress support.
ui: ingress: enabled: true annotations: nginx.ingress.kubernetes.io/rewrite-target: /$1 hosts: - host: domain.com paths: - path: "/(.*)"
Enable NetworkPolicy
If enabled, it creates an additional NetworkPolicy to allow ingress traffic to the Policy Reporter UI on the default Port 8080
and egress traffic to Policy Reporter and the Kyverno Plugin if enabled. Egress rules can be extended as needed.
ui: networkPolicy: enabled: true egress: []
Kyverno Plugin
kyvernoPlugin
to be clear that it has to be configured under this subchart.Enable the Policy Reporter Kyverno Plugin subchart.
kyvernoPlugin: enabled: true
Disable Metrics Endpoint
kyvernoPlugin: metrics: enabled: false
Disable REST Endpoint
kyvernoPlugin: rest: enabled: false
Enable enforce violation PolicyReports (requires Kyverno >= 1.7.0)
Because Kyverno creates PolicyReports only for audit Policies, Policy Reporters KyvernoPlugin brings the possibility to create additional PolicyReports for blocked resources as well. This makes it possible to get also metrics and notification about resources blocked by an enforce Policy.
By default PolicyReportResults are using another source (Kyverno Event
) as audit results, so that they are displayed on separate pages. This source is customizable, so you can change it for example to Kyverno
, so the results are shown in the same dashboards as the audit results.
kyvernoPlugin: blockReports: # enable the feature enabled: false # namespace where kyverno events are created # set to an empty string to watch for events in all namespaces eventNamespace: default results: # maximal results stored in a PolicyReport per namespace maxPerReport: 200 # keep only the latest result of the same violation in the report keepOnlyLatest: false # source used for the PolicyReportResults source: "Kyverno Event"
``
Logging
Since AppVersion 1.5.1
its possible to customize the logger output like encoding
and logLevel
. It is also possible to enable API access debug logging.
logging: encoding: console # possible encodings are console and json logLevel: 0 # default info development: false # more human readable structure, enables stacktraces and removes log samplingapi: logging: false # enable debug API access logging, sets logLevel to debug
High Available Setup
In HA mode KyvernoPlugin uses LeaderElection
to ensure that only one instance is responsible for managing block-PolicyReports. Other features like API requests and metrics are loadbalanced between each instance. This means, if kyvernoPlugin.blockReports.enabled
is false
, leaderElection
is not needed and will be disabled.
The High Available setup also adds an PodDisruptionBudget
with a default minAvailable
of 1
. Both, LeaderElection
as well as the PodDisruptionBudget
can be configured to you personal needs.
kyvernoPlugin: replicaCount: 3 # enabled if replicaCount > 1 podDisruptionBudget: # -- Configures the minimum available pods for policy-reporter disruptions. # Cannot be used if `maxUnavailable` is set. minAvailable: 1 # -- Configures the maximum unavailable pods for policy-reporter disruptions. # Cannot be used if `minAvailable` is set. maxUnavailable: # required if policy-reporter-kyverno-plugin should run in HA mode and the "blockReports" feature is enabled # if "blockReports" is disabled, leaderElection is also disabled automatically # will be enabled when replicaCount > 1 leaderElection: enabled: false releaseOnCancel: true leaseDuration: 15 renewDeadline: 10 retryPeriod: 2
Ingress
Serve the API over a hostname with the integrated Ingress support. This is mainly needed for the Multi Tenance Feature of Policy Reporter UI. In this case make sure that the API is not reachable for the outside world.
kyvernoPlugin: ingress: enabled: true annotations: nginx.ingress.kubernetes.io/rewrite-target: /$1 hosts: - host: domain.com paths: - path: "/(.*)"
Enable NetworkPolicy
If enabled, the Helm Chart creates a NetworkPolicy resource to allow Policy Reporter Kyverno Plugin egress traffic to the Kubernetes API (defaults to port 6443
) as well as ingress traffic to the REST API from the Policy Reporter UI. Ingress and egress rules can be extended as needed.
kyvernoPlugin: networkPolicy: enabled: true egress: - to: ports: - protocol: TCP port: 6443 ingress: []
Monitoring
monitoring
to be clear that it has to be configured under this subchart.The Monitoring Subchart integrates Policy Reporter into the Prometheus Operator.
This Subchart creates a ServiceMonitor for the available /metrics
endpoint of Policy Reporter to be fetched from Prometheus along with three preconfigured dashboards (as ConfigMaps) for the provided Grafana.
Enable the Policy Reporter Monitoring subchart.
monitoring: enabled: true
Configure the ServiceMonitor
By default, the ServiceMonitor is created in the HelmRelease namespace without additional labels. Depending on your installation you may have to add labels to match the serviceMonitorSelector
of your Prometheus CRD. If you use the Helm Chart from the Prometheus community as linked above, you need a label release
with the release name of your Prometheus operator installation. The serviceMonitor.namespace
configuration is optional and can be changed as needed.
monitoring: serviceMonitor: namespace: labels: release: prometheus-operator
Enable the Kyverno Plugin integration
kyvernoPlugin
subchart to be enabled.If enabled, a second ServiceMonitor will be created for the /metrics
endpoint of the Kyverno Plugin. This ServiceMonitor also uses the configuration provided by monitoring.serviceMonitor
.
monitoring: plugins: kyverno: true
Configure Grafana Dashboards
To ensure that the ConfigMaps are identified as dashboards for Grafana, they need a special label grafana_dashboard
and must be in the namespace of the Grafana installation.
The required label is preconfigured but can be changed if needed. The namespace must set. It is also possible to disable the ConfigMaps entirely.
monitoring: grafana: # required: namespace of your Grafana installation namespace: dashboards: # Enable the deployment of grafana dashboards enabled: true # Label to find dashboards using the k8s sidecar label: grafana_dashboard # works only if it is supported by your Grafana installation folder: # Annotation to enable folder storage using the k8s sidecar annotation: grafana_folder # Grafana folder in which to store the dashboards name: Policy Reporter
Configure the Dashboard Views
Depending on your Policy Reports, some result types like skipped
or error
are not supported. Also, depending on the size of your cluster and the number of namespaces, some parts are too small to be clearly arranged. To be flexible, the Subchart allows you to hide parts of the dashboard and change the height of the different components.
monitoring: # PolicyReport Details Dashboard policyReportDetails: # high of pass and fail Bar Charts firstStatusRow: height: 8 # high of warn and error Bar Charts secondStatusRow: enabled: true height: 2 # change the high of the Timeline or disable it statusTimeline: enabled: true height: 8 # change the high of the different status tables or disable them as you like passTable: enabled: true height: 8 failTable: enabled: true height: 8 warningTable: enabled: true height: 4 errorTable: enabled: true height: 4 # ClusterPolicyReport Details Dashboard clusterPolicyReportDetails: # high of the status counter boxes statusRow: height: 6 # change the high of the Timeline or disable it statusTimeline: enabled: true height: 8 # change the high of the different status tables or disable them as you like passTable: enabled: true height: 8 failTable: enabled: true height: 8 warningTable: enabled: true height: 4 errorTable: enabled: true height: 4 # PolicyReports Dashboard policyReportOverview: # high of the fail bar chart and counter box failingSummaryRow: height: 8 # high of the timeline failingTimeline: height: 10 # high of the PolicyReport fail table failingPolicyRuleTable: height: 10 # high of the ClusterPolicyReport fail table failingClusterPolicyRuleTable: height: 10
- Usage
- Configuration
- Enable Metrics Endpoint
- Enable REST Endpoints
- Enable API BasicAuth (since v2.20.0)
- Enable Targets / Notification
- Enable E-Mail Reports
- PolicyReport CRD Filter
- Metric customization
- Priority Mapping
- External Databases (AppVersion >= 2.15.0)
- External Caching Storage
- Logging
- High Available Setup
- Ingress
- Enable NetworkPolicy
- Subcharts
- Policy Reporter UI
- Kyverno Plugin
- Monitoring