OpenClarity Documentation

• 1: Getting Started
• 1.1: Deploy on AWS
• 1.2: Deploy on Azure
• 1.3: Deploy on Docker
• 1.4: Deploy on GCP
• 1.5: Deploy on Kubernetes
• 1.6: First Tasks on the UI
• 2: Concepts
• 2.1: Software Bill of Materials
• 2.1.1: SBOM Output Format
• 2.2: Vulnerability Scanning
• 2.3: Plugins
• 3: Using OpenClarity
• 3.1: OpenClarity Stack
• 3.2:
• 3.2.1: Example CLI Configuration
• 3.3: OpenClarity SDKs
• 4: Features
• 4.1: Kubernetes Scanning
• 4.2: Docker Scanning
• 4.3: AWS Scanning
• 4.4: GCP Scanning
• 4.5: Azure Scanning
• 4.6: Scanner Plugins
• 4.7: Cost Estimation
• 5:
• 6: API Reference
• 7:
• 8: Archive
• 8.1: VM Security
• 8.1.1: Architecture
• 8.1.2: Getting started
• 8.1.2.1: Deploy on AWS
• 8.1.2.2: Deploy on Azure
• 8.1.2.3: Deploy on Docker
• 8.1.2.4: Deploy on GCP
• 8.1.2.5: Deploy on Kubernetes
• 8.1.2.6: First tasks on the UI
• 8.1.3: Common CLI tasks
• 8.1.4: Configuration Parameters
• 8.1.5: Troubleshooting and Debugging
• 8.1.6: VMClarity development
• 8.1.6.1: End-to-End Testing Guide
• 8.1.7: API reference
• 8.1.8: Cost estimation
• 8.2: Kubernetes Security
• 8.2.1: Features
• 8.2.2: Concepts and background
• 8.2.2.1: Software bill of materials
• 8.2.2.2: Kubernetes cluster runtime scan
• 8.2.2.3: Vulnerability scanning
• 8.2.3: Getting started
• 8.2.3.1: Install the KubeClarity backend
• 8.2.3.2: Install the CLI
• 8.2.3.3: First tasks - UI
• 8.2.3.4: First tasks - CLI
• 8.2.4: Generate SBOM
• 8.2.4.1: Generate SBOM
• 8.2.4.2: Merging SBOM results
• 8.2.4.3: SBOM output format
• 8.2.5: Runtime scan
• 8.2.5.1: Run a runtime scan
• 8.2.5.2: Schedule runtime scan
• 8.2.5.3: Configure CIS benchmarks
• 8.2.6: Vulnerability scan
• 8.2.6.1: Run a vulnerability scan
• 8.2.6.2: Vulnerability scanning a local docker image
• 8.2.6.3: Remote scanner servers for CLI
• 8.2.6.3.1: Trivy
• 8.2.6.3.2: Grype
• 8.2.6.3.3: Dependency track
• 8.2.7: Private registry support
• 8.2.7.1: Private registry support for the CLI
• 8.2.7.2: Private registry support for Kubernetes
• 8.2.8: Set configuration file location for the CLI
• 8.2.9: Merge scan results
• 8.2.10: KubeClarity development
• 8.2.11: API reference
• 8.3: APIClarity
• 8.3.1: Features
• 8.3.2: Getting started
• 8.3.2.1: Install APIClarity
• 8.3.2.2: Install demo application
• 8.3.3: Enable external trace sources support
• 8.3.4: API reference
• 8.3.4.1: Core API reference
• 8.3.4.2: Common API reference
• 9: Support
• 10:
• 11: Security Policies and Procedures
• 12: Contributor Covenant Code of Conduct
• 13: Community
• 14:

1 - Getting Started

The following sections describe the installation of the OpenClarity backend and the CLI, and show you the most common tasks that you can perform with OpenClarity.

1.1 - Deploy on AWS

An AWS CloudFormation template is provided for quick deployment of the OpenClarity environment.

Note: To avoid extra costs (cross-region snapshots), you may want to deploy the OpenClarity AWS CloudFormation template in the same region where the majority of the VMs are that you want to scan with OpenClarity.

The following figure shows the basic AWS resources that the OpenClarity CloudFormation template creates:

• a VPC with a public and private subnet, and

• an AWS Internet Gateway (IGW) and NAT Gateway (NGW) into the VPC.

  

The public subnet (OpenClarityServerSubnet) hosts the OpenClarity Server (OpenClarityServer) EC2 instance. The OpenClarity server houses the scanning configuration, the UI, and other control components. The EC2 instance is assigned an external IPv4 address (EIP) for SSH and web UI access.

The private subnet (OpenClarityScannerSubnet) hosts the VM snapshot instances (EC2) that are scanned for security vulnerabilities.

Prerequisites

• Have an AWS account.
• Create an EC2 key pair.

Deployment steps

To deploy the OpenClarity AWS CloudFormation Stack, you can:

• click this quick-create link to navigate directly to the AWS CloudFormation console and jump to the wizard instructions, or
• complete the following steps.

1. Download the latest OpenClarity release.

   wget https://github.com/openclarity/openclarity/releases/download/v1.1.1/aws-cloudformation-v1.1.1.tar.gz
   

   Alternatively, copy the AWS CloudFormation template file from the project repository to deploy the latest development code and skip the next step.

2. Create a new directory and extract the files.

   mkdir aws-cloudformation-v1.1.1
   tar -xvzf aws-cloudformation-v1.1.1.tar.gz -C aws-cloudformation-v1.1.1
   

3. Log in to the AWS CloudFormation console and go to the AWS CloudFormation Stacks section, then select Create Stack > With New Resources (standard).

4. Check Template is ready and Upload a template file, then click Upload a template file/Choose file and upload the previously downloaded CFN template file.

   

5. In the OpenClarity CloudFormation Stack wizard, set the following:

   1. Enter a name for the stack.
   2. Select the InstanceType (defaults to t2.large for the OpenClarity Server, and the scanner VMs).
   3. Specify the SSH key for the EC2 instance in the KeyName field. You will need this key to connect to OpenClarity.
   4. Adjust SSHLocation according to your policies.
   5. Do not change AdvancedConfiguration, unless you are building from a custom registry.
   6. Click NEXT.
   7. (Optional) Add tags as needed for your environment. You can use the defaults unless you need to adjust for your own policies.
   8. Click NEXT, then scroll to the bottom of the screen, and check I acknowledge….
   9. Click SUBMIT.

6. Once the stack is deployed successfully, copy the OpenClarity SSH address from the Outputs tab.

   

7. Open an SSH tunnel to OpenClarity the server

   ssh -N -L 8080:localhost:80 -i  "<Path to the SSH key specified during install>" ubuntu@<OpenClarity SSH Address copied during install>
   

8. Access the OpenClarity UI.

   1. Open the OpenClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   2. (Optional) If needed, you can access the API at http://localhost:8080/api. For details on the API, see the API reference.

   

Next steps

Complete the First Tasks on the UI.

1.2 - Deploy on Azure

Prerequisites

• Have an Azure subscription.

• Create an SSH public key for Linux. Please follow these instructions for Linux and Mac users or these for Windows users. Once you have an RSA private key, convert it to an SSH2 public key with:

  ssh-keygen -e -f ~/.ssh/id_rsa.pub > ~/.ssh/id_rsa2.pub
  

Deployment steps

1. Click here to deploy OpenClarity’s custom template.

2. Fill out the required Project details and Instance details in the Basics tab.

   

   You can set the following parameters:

   Parameter	Required	Description
   Subscription	True	Azure subscription where resources will be billed.
   Region	False	Azure region where resources will be deployed.
   OpenClarity Deploy Postfix	True	Postfix for Azure resource group name (e.g. openclarity-<postfix>).
   OpenClarity Server SSH Username	True	SSH Username for the OpenClarity Server Virtual Machine.
   OpenClarity Server SSH Public Key	True	SSH Public Key for the OpenClarity Server Virtual Machine. Paste the contents of ~/.ssh/id_rsa2.pub here.
   OpenClarity Server VM Size	True	The size of the OpenClarity Server Virtual Machine.
   OpenClarity Scanner VMs Size	True	The size of the OpenClarity Scanner Virtual Machines.
   Security Type	False	Security Type of the OpenClarity Server Virtual Machine, e.g. TrustedLaunch (default) or Standard.

3. (Optional) In the Advanced tab, modify the Container Image for each service if a specific OpenClarity version is required. Then, select the delete policy and the database.

   

   Parameter	Required	Description
   Service Container Image	True	Docker Container Image to use for each service.
   Asset Scan Delete Policy	True	Delete Policy for resources created when performing an asset scan, e.g. Always, OnSuccess or Never.
   Database To Use	True	Database type to use, e.g. SQLite, PostgreSQL or External PostgreSQL.

4. Click Review + create to create the deployment.

5. Once the deployment is completed successfully, copy the OpenClarity SSH address from the Outputs tab.

   

6. Open an SSH tunnel to OpenClarity the server

   ssh -N -L 8080:localhost:80 -i  "<Path to the SSH key specified during install>" ubuntu@<OpenClarity SSH Address copied during install>
   

7. Access the OpenClarity UI.

   1. Open the OpenClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   2. (Optional) If needed, you can access the API at http://localhost:8080/api. For details on the API, see the API reference.

   

Next steps

Complete the First Tasks on the UI.

1.3 - Deploy on Docker

Prerequisites

• Install Docker.

Deployment steps

To run OpenClarity in Docker on a local machine, complete the following steps.

1. Download the latest OpenClarity release.

   wget https://github.com/openclarity/openclarity/releases/download/v1.1.1/docker-compose-v1.1.1.tar.gz
   

2. Create a new directory, extract the files and navigate to the directory.

   mkdir docker-compose-v1.1.1
   tar -xvzf docker-compose-v1.1.1.tar.gz -C docker-compose-v1.1.1
   cd docker-compose-v1.1.1
   

3. Start every control plane element with the docker compose file.

   docker compose --project-name openclarity --file docker-compose.yml up -d --wait --remove-orphans
   

   The output should be similar to:

   [+] Running 14/14
   ⠿ Network openclarity                        Created                                                       0.2s
   ⠿ Volume "openclarity_grype-server-db"       Created                                                       0.0s
   ⠿ Volume "openclarity_apiserver-db-data"     Created                                                       0.0s
   ⠿ Container openclarity-orchestrator-1       Healthy                                                      69.7s
   ⠿ Container openclarity-yara-rule-server-1   Healthy                                                      17.6s
   ⠿ Container openclarity-exploit-db-server-1  Healthy                                                      17.7s
   ⠿ Container openclarity-swagger-ui-1         Healthy                                                       7.8s
   ⠿ Container openclarity-trivy-server-1       Healthy                                                      26.7s
   ⠿ Container openclarity-uibackend-1          Healthy                                                      17.6s
   ⠿ Container openclarity-ui-1                 Healthy                                                       7.7s
   ⠿ Container openclarity-freshclam-mirror-1   Healthy                                                       7.8s
   ⠿ Container openclarity-grype-server-1       Healthy                                                      37.3s
   ⠿ Container openclarity-gateway-1            Healthy                                                       7.7s
   ⠿ Container openclarity-apiserver-1          Healthy                                                      17.7s
   

   Please note that the image_override.env file enables you to use the images you build yourself. You can override parameters in the docker-compose.yml by passing a custom env file to the docker compose up command via the --env-file flag. The /installation/docker/image_override.env file contains an example overriding all the container images.

4. Check the running containers in the Docker desktop.

   

5. Access the OpenClarity UI. Navigate to http://localhost:8080/ in your browser.

   

Next steps

Complete the First Tasks on the UI.

Clean up steps

1. After you’ve finished your tasks, stop the running containers.

   docker compose --project-name openclarity --file docker-compose.yml down --remove-orphans
   

1.4 - Deploy on GCP

Prerequisites

• You can install OpenClarity using the CLI, so you have to have gcloud on your computer available beforehand. For details on installing and configuring gcloud, see the official installation guide.
• If you have already installed OpenClarity before and want to reinstall it, you have to manually restore deleted roles that were created during the previous installation.

Deployment steps

To install OpenClarity on Google Cloud Platform (GCP), complete the following steps.

1. Download the newest GCP deployment release from GitHub and extract it to any location.

   wget https://github.com/openclarity/openclarity/releases/download/v1.1.1/gcp-deployment-v1.1.1.tar.gz
   

2. Create a new directory, extract the files and navigate to the directory.

   mkdir gcp-deployment-v1.1.1
   tar -xvzf gcp-deployment-v1.1.1.tar.gz -C gcp-deployment-v1.1.1
   cd gcp-deployment-v1.1.1
   

3. Copy the example configuration file and rename it.

   cp openclarity-config.example.yaml openclarity-config.yaml
   

4. The following table contains all the fields that can be set in the openclarity-config.yaml file. You have to set at least the required ones.

   Field	Required	Default	Description
   zone	yes		The Zone to locate the OpenClarity server.
   machineType	yes		The machine type for the OpenClarity server.
   region	yes		The region to locate OpenClarity.
   scannerMachineType		e2-standard-2	Machine type to use for the Scanner instances.
   scannerSourceImage		projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230630	Source image to use for the Scanner instances.
   databaseToUse		SQLite	The database that OpenClarity should use.
   apiserverContainerImage		ghcr.io/openclarity/openclarity-apiserver:1.1.1	The container image to use for the apiserver.
   orchestratorContainerImage		ghcr.io/openclarity/openclarity-orchestrator:1.1.1	The container image to use for the orchestrator.
   uiContainerImage		ghcr.io/openclarity/openclarity-ui:1.1.1	The container image to use for the ui.
   uibackendContainerImage		ghcr.io/openclarity/openclarity-ui-backend:1.1.1	The container image to use for the uibackend.
   scannerContainerImage		ghcr.io/openclarity/openclarity-cli:1.1.1	The container image to use for the scanner.
   exploitDBServerContainerImage		ghcr.io/openclarity/exploit-db-server:v0.2.4	The container image to use for the exploit db server.
   trivyServerContainerImage		docker.io/aquasec/trivy:0.41.0	The container image to use for the trivy server.
   grypeServerContainerImage		ghcr.io/openclarity/grype-server:v0.7.0	The container image to use for the grype server.
   freshclamMirrorContainerImage		ghcr.io/openclarity/freshclam-mirror:v0.2.0	The container image to use for the fresh clam mirror server.
   postgresqlContainerImage		docker.io/bitnami/postgresql:12.14.0-debian-11-r28	The container image to use for the postgresql server.
   assetScanDeletePolicy		Always	When asset scans should be cleaned up after scanning.
   postgresDBPassword			Postgres DB password. Only required if DatabaseToUse is Postgresql.
   externalDBName			DB to use in the external DB. Only required if DatabaseToUse is External.
   externalDBUsername			Username for the external DB. Only required if the DatabaseToUse is External.
   externalDBPassword			Password for the external DB. Only required if the DatabaseToUse is External.
   externalDBHost			Hostname or IP for the external DB. Only required if the DatabaseToUse is External.
   externalDBPort			Port for the external DB. Only required if the DatabaseToUse is External.

5. Deploy OpenClarity using gcloud deployment-manager.

   gcloud deployment-manager deployments create <openclarity deployment name> --config openclarity-config.yaml
   

6. Open an SSH tunnel to the OpenClarity server with gcloud. For further information on how to create an SSH connection with gcloud to one of your instances check the official page.

   gcloud compute ssh --project=<project id> --zone=<zone name> <name of your VM> -- -NL 8080:localhost:80
   

7. Access the OpenClarity UI.

   1. Open the OpenClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   2. (Optional) If needed, you can access the API at http://localhost:8080/api. For details on the API, see the API reference.

   

Next steps

Complete the First Tasks on the UI.

Uninstall OpenClarity

1. You can uninstall OpenClarity using the gcloud manager.

   gcloud deployment-manager deployments delete <openclarity deployment name>
   

Restore deleted roles

1. On the IAM & Admin page on GCP, open the Roles tab, then search for OpenClarity in the filter input.

2. Now manually undelete the Discoverer Snapshotter and Scanner roles to set their statuses from Deleted to Enabled.

   

1.5 - Deploy on Kubernetes

Prerequisites

• Install a tool to run local Kubernetes clusters. Here, Kind is used as the default option for creating a local cluster.
• Helm to install OpenClarity.

Deployment steps

To deploy OpenClarity to your Kubernetes cluster, complete the following steps.

1. Create a Kubernetes cluster.

   kind create cluster --name openclarity-k8s
   

2. Ensure the Kubernetes cluster is up and running. If you’re using kind, you can check the status of your clusters with the following command:

   kind get clusters
   

3. Use Helm to install OpenClarity. Run the following command:

   helm install openclarity oci://ghcr.io/openclarity/charts/openclarity --version 1.1.1 \
       --namespace openclarity --create-namespace \
       --set orchestrator.provider=kubernetes \
       --set orchestrator.serviceAccount.automountServiceAccountToken=true
   

4. Verify that all the OpenClarity pods have been successfully deployed by executing the following command:

   kubectl get pods -n openclarity
   

5. Wait until all pods are in the Running state or have completed their initialization.

6. Once the pods are ready, start port forwarding to access the OpenClarity gateway service. Use the following command to forward traffic from your local machine to the cluster:

   kubectl port-forward -n openclarity service/openclarity-gateway 8080:80
   

7. Access the OpenClarity UI by navigating to http://localhost:8080/ in your web browser.

   

Next steps

Complete the First Tasks on the UI.

Clean up steps

1. Uninstall OpenClarity with Helm. Run the following command:

   helm uninstall openclarity --namespace openclarity
   

2. Delete the Kubernetes cluster.

   kind delete clusters openclarity-k8s
   

1.6 - First Tasks on the UI

Configure your first scan

1. Open the UI.

   1. Open the OpenClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   2. (Optional) If needed, you can access the API at http://localhost:8080/api. For details on the API, see the API reference.

   

2. Click on the Scans icon. In the Scans window, you can create a new scan configuration.

   

3. Click New scan configuration.

   

4. Follow the steps of the New scan config wizard to name the scan, and optionally narrow the scope down with an OData query.

   

5. Enable the scan types you want to perform.

   

6. Select the time and/or frequency of the scans. To run the scan immediately, select Now.

   

7. Optionally, adjust the number of scanners to run in parallel and whether to use spot instances on cloud providers, or not.

   

8. Click Save. The new scan appears on the Scan Configurations tab.

   

9. Once a scan is finished, you can browse around the various OpenClarity UI features and investigate the security scan reports.

   

2 - Concepts

The following sections give you the concepts and background information about the scans provided by OpenClarity.

2.1 - Software Bill of Materials

A software bill of materials (SBOM) is a list of all the components, libraries, and other dependencies that make up a software application, along with information about the versions, licenses, and vulnerabilities associated with each component. They are formal, structured documents detailing the components of a software product and its supply chain relationships.

SBOMs are important because organizations increasingly rely on open source and third-party software components to build and maintain their applications. These components can introduce security vulnerabilities and must be adequately managed and updated. SBOMs help you understand what open source and third-party components are used in your applications, and identify and address any security vulnerabilities.

Under specific scenarios, generating and publishing SBOMs is mandatory for compliance with regulations and industry standards that require organizations to disclose the use of open source and third-party software in their products.

SBOM Standards

There are several related standards, for example, CycloneDX, SPDX, SWID.

SPDX (Software Package Data Exchange) is a standard format for communicating a software package’s components, licenses, and copyrights. It is commonly used to document the open source components included in a proprietary software product. SPDX files can be easily read and understood by humans and machines, making it easy to track and manage open source components in a software project. SPDX format is supported by Linux Foundation.

CycloneDX is an open source standard for creating software bill of materials files. It is like SPDX in that it documents the components and licenses associated with a software package, but it is specifically designed for use in software supply chain security. CycloneDX is a more lightweight format compared to SPDX, which is intended to be more detailed. CycloneDX format is supported by OWASP.

SBOM Architecture

A typical SBOM architecture can be laid out as a tree-like dependency graph with the following key elements:

• Component inventory: Information about the components, libraries, and other assets used in the software, including version numbers, licenses, and vulnerabilities.
• Dependency mapping: A map of relationships between different components and libraries, showing how they depend on each other and how changes to one may impact the other.
• License management: It should also include information about the licenses of the components and libraries used to ensure that the software complies with legal and ethical obligations.

SBOM Generators

There are two typical ways to generate SBOM: during the build process, or after the build and deployment using a Software Composition Analysis tool. Trivy and Syft are two noteworthy open source generators among many other generators, including open source and commercial. Both use CycloneDX format. It is also important to note that not all SBOMs can be generated equally. Each generator may pick up a few language libraries better than the others based on its implementation. It might take multiple runs through a few different types of generators to draw comprehensive insights.

OpenClarity content analyzer integrates with the following SBOM generators:

• Syft
• Cyclonedx-gomod
• Trivy

Multiple SBOMs for Accuracy

OpenClarity can run multiple SBOM generators in parallel, and unify their results to generate a more accurate document.

In such cases, OpenClarity compiles a merged SBOM from multiple open source analyzers, and delivers a comprehensive SBOM document report. Although OpenClarity does not generate SBOMs, it integrates with popular generators so that a combined document can provide amplified inputs that can be further analyzed using vulnerability scanners. Leveraging multiple SBOM documents can improve visibility into software dependency posture.

OpenClarity formats the merged SBOM to comply with the input requirements of vulnerability scanners before starting vulnerability scans.

Note: OpenClarity can merge vulnerability scans from various sources like Grype and Trivy to generate a robust vulnerability scan report.

Scan SBOM Documents for Vulnerabilities

You can feed the generated SBOM documents to vulnerability scanners, which analyze the SBOMs and generate a vulnerability report detailing all known and fixed CVEs of the software components listed by SBOM.

2.1.1 - SBOM Output Format

The openclarity-cli scan command can format the resulting SBOM into different formats to integrate with another system. The supported formats are:

To configure the openclarity-cli to use a format other than the default, the sbom.output_format config parameter can be used with the configuration name from above:

# Create config based on https://github.com/openclarity/openclarity/blob/main/.families.yaml
cat <<EOF > config.yml
sbom:
  enabled: true
  analyzers_list:
    - "syft"
  inputs:
    - input: "/dir-to-scan"
      input_type: "rootfs"
  output_format: "cyclonedx-json"
EOF

# Run scan
openclarity-cli scan --config config.yml

For more information the CLI configuration, see the Example CLI Configuration.

2.2 - Vulnerability Scanning

Vulnerability scanning identifies weak spots in software code and dependencies. Vulnerability scanners can identify infrastructure, networks, applications, or website vulnerabilities. These tools scan various target systems for security flaws that attackers could exploit.

The scanners use the information contained in the SBOM to identify vulnerabilities and potential security risks within software applications. Vulnerability scanners use SBOM information to:

• Identify vulnerable components: Scanners use the SBOM to identify a software application’s components, then cross-reference this information with known vulnerabilities and security issues to identify vulnerable components within the software.
• Prioritize vulnerabilities: After the vulnerability scanner has identified all vulnerable components within the software application, it uses the SBOM to prioritize the vulnerabilities so you can focus on the most critical vulnerabilities.
• Identify supply chain risks: SBOMs provide visibility into the software supply chain, enabling vulnerability scanners to identify third-party or security risks. As a result, organizations can mitigate supply chain risks and reduce their overall security exposure.
• Track changes and updates: Software vulnerability scanners use SBOM information to determine whether software changes have introduced new vulnerabilities or security risks.

The SBOM is a critical tool for vulnerability scanners, providing the information needed to identify, prioritize, and mitigate security risks within software applications. In addition, scanners also rely on other types of inputs, as listed below.

OpenClarity and Vulnerability Scanning

OpenClarity isn’t a vulnerability scanner but integrates with top opensource vulnerability scanners. It also helps with prioritization and risk management by visualization and filtering. It is often necessary to prioritize CVEs because of the sheer volume of identified CVEs. With OpenClarity’s vulnerability trending dashboard and APIs, you can locate and double-click into a specific CVE in your application or infrastructure.

OpenClarity features a range of flexible and dynamic filters that help map CVEs down to an application->package->Image level. Additionally, it normalizes reports from multiple scanners and calculates missing CVSS (Common Vulnerability Scoring System) scores.

OpenClarity vulnerability scanner integrates with the following scanners:

• Grype
• Dependency-Track
• Trivy

OpenClarity supports both automatic scans to find common vulnerabilities quickly and efficiently, and manual scans to help verify automated scans, and also to help identify more complex and less common vulnerabilities. In addition to conventional scans, OpenClarity also provides multi-scanner integration.

Multi-scanner Architecture

OpenClarity infrastructure enables multiple scanners’ configuration and simultaneous operation. Scanners in OpenClarity are designed to work in parallel.

The following figure shows the multi-scanner architecture for vulnerability scanning: OpenClarity preprocesses the SBOMs so they conform to the specific formatting requirements of the specific scanner. Each scanner may have different types and unique formatting expectations. The scanners analyze the incoming data and generate vulnerability outputs in their native formats.

OpenClarity can merge the vulnerability reports of different scanners, to include severity levels, sources, and available fixes. These reports serve as valuable outputs, allowing you to filter and focus on specific areas of vulnerabilities for further investigation and resolution.

Run Vulnerability Scans

For details on running vulnerability scans with OpenClarity, see the Getting Started.

2.3 - Plugins

Plugins provide additional scanning capabilities to OpenClarity ecosystem. Project structure:

• runner - Provides necessary logic to manage scanner plugins in OpenClarity.
• sdk - Language-specific libraries, templates, and examples to aid with the implementation of scanner plugins.
• store - Collection of available plugins that can be directly used in OpenClarity.

Requirements

Scanner plugins are distributed as containers and require Docker Engine on the host that runs the actual scanning via OpenClarity CLI to work.

Support

List of supported environments:

1. AWS
2. GCP
3. Azure
4. Docker

List of unsupported environments:

• Kubernetes - We plan on adding plugin support to Kubernetes once we have dealt with all the security considerations.

Note: Plugin support has been tested against OpenClarity installation artifacts for the given environments.

Usage

You can start using plugins via Plugins Store. For example, you can pass the .families.yaml scan config file defined below to the OpenClarity CLI scan command. This configuration uses KICS scanner to scan /tmp dir for IaC security misconfigurations.

# --- .families.yaml
plugins:
  enabled: true
  scanners_list:
    - "kics"
  inputs: 
    - input: "/tmp"
      input_type: "rootfs"
  scanners_config:
    kics:
      image_name: "ghcr.io/openclarity/openclarity-plugin-kics:latest"
      config: "{}"

SDKs

You can use one of available SDKs in your language of choice to quickly develop scanner plugins for OpenClarity.

List of supported languages:

• Golang
• Python

3 - Using OpenClarity

3.1 - OpenClarity Stack

Today, OpenClarity has two halves, the OpenClarity control plane, and the OpenClarity CLI.

The OpenClarity control plane includes several microservices:

• API Server: The OpenClarity API for managing all objects in the OpenClarity system. This is the only component in the system which talks to the DB.

• Orchestrator: Orchestrates and manages the life cycle of OpenClarity scan configs, scans and asset scans. Within the Orchestrator there is a pluggable “provider” which connects the orchestrator to the environment to be scanned and abstracts asset discovery, VM snapshotting as well as creation of the scanner VMs. (Note The only supported provider today is AWS, other hyperscalers are on the roadmap)

• UI Backend: A separate backend API which offloads some processing from the browser to the infrastructure to process and filter data closer to the source.

• UI Webserver: A server serving the UI static files.

• DB: Stores the OpenClarity objects from the API. Supported options are SQLite and Postgres.

• Scanner Helper services: These services provide support to the OpenClarity CLI to offload work that would need to be done in every scanner, for example downloading the latest vulnerability or malware signatures from the various DB sources. The components included today are:

  • grype-server: A rest API wrapper around the grype vulnerability scanner
  • trivy-server: Trivy vulnerability scanner server
  • exploitDB server: A test API which wraps the Exploit DB CVE to exploit mapping logic
  • freshclam-mirror: A mirror of the ClamAV malware signatures

The OpenClarity CLI contains all the logic for performing a scan, from mounting attached volumes and all the pluggable infrastructure for all the families, to exporting the results to OpenClarity API.

These components are containerized and can be deployed in a number of different ways. For example our cloudformation installer deploys OpenClarity on a VM using docker in an dedicated AWS Virtual Private Cloud (VPC).

Once the OpenClarity server instance has been deployed, and the scan configurations have been created, OpenClarity will discover VM resources within the scan range defined by the scan configuration (e.g., by region, instance tag, and security group). Once the asset list has been created, snapshots of the assets are taken, and a new scanner VM are launched using the snapshots as attached volumes. The OpenClarity CLI running within the scanner VM will perform the configured analysis on the mounted snapshot, and report the results to the OpenClarity API. These results are then processed by the OpenClarity backend into findings.

3.2 -

Initiate scan using the cli

Reporting results into file

./cli/bin/openclarity-cli scan --config ~/testConf.yaml -o outputfile

If we want to report results to the OpenClarity backend, we need to create asset and asset scan object before scan because it requires asset-scan-id

Reporting results to OpenClarity backend

ASSET_ID=$(./cli/bin/openclarity-cli asset-create --file assets/dir-asset.json --server http://localhost:8080/api) --jsonpath {.id}
ASSET_SCAN_ID=$(./cli/bin/openclarity-cli asset-scan-create --asset-id $ASSET_ID --server http://localhost:8080/api) --jsonpath {.id}
./cli/bin/openclarity-cli scan --config ~/testConf.yaml --server http://localhost:8080/api --asset-scan-id $ASSET_SCAN_ID

Using one-liner:

./cli/bin/openclarity-cli asset-create --file docs/assets/dir-asset.json --server http://localhost:8080/api --update-if-exists --jsonpath {.id} | xargs -I{} ./cli/bin/openclarity-cli asset-scan-create --asset-id {} --server http://localhost:8080/api --jsonpath {.id} | xargs -I{} ./cli/bin/openclarity-cli scan --config ~/testConf.yaml --server http://localhost:8080/api --asset-scan-id {}

3.2.1 - Example CLI Configuration

This section provides a sample configuration for scanner families supported by the OpenClarity CLI tool. Each family can be enabled or disabled, and configured with specific options. The configuration is used to define which scanners to run, what inputs to scan, and the configuration for each scanner.

  
# This file contains a sample configuration for scanner families supported by the OpenClarity CLI tool.
# Each family can be enabled or disabled, and configured with specific options.
# The configuration is used to define the scanners to run, the inputs to scan, and the configuration for each scanner.

# SBOM (Software Bill of Materials) scanner family
sbom:
  enabled: false # Enable or disable SBOM scanner family
  analyzers_list: # List of analyzers to run
    - "syft"
    - "trivy"
    - "windows"
    - "gomod"
  inputs: # List of inputs to scan
    - input: "node:slim"
      input_type: "image" # Type of input (image, rootfs, etc.)
    # - input: "/mnt"
    #   input_type: "rootfs"
    # - input: "nginx:1.10"
    #   input_type: "image"
  # merge_with: # Merge multiple SBOMs into one
  #  - sbom_path: "nginx.11.cdx.json" # Path to SBOM file to merge with
  local_image_scan: true # Scan images from local docker daemon (true) or from remote registry (false)
  registry:
    skip-verify-tls: false # Skip TLS verification
    use-http: false # Use HTTP instead of HTTPS
    auths: # Registry authentication
      authority: "authority"
      username: "username"
      password: "password"
      token: "token"
  output_format: "cyclonedx-json" # Output format for SBOMs (cyclonedx-json, cyclonedx-xml, spdx-json, spdx-tv, syft-json)
  analyzers_config: # Configuration for each analyzer
    syft:
      scope: "Squashed" # Scope of the scan (squashed, all-layers)
      exclude_paths: # Paths to exclude from the scan
        - "./dev"
        - "./proc"
      ## Overrides parent sbom configs
      # local_image_scan: ...
      # registry: ...
    trivy:
      timeout: 300 # Timeout in seconds
      cache_dir: /tmp/.trivy/cache # Cache directory
      temp_dir: /tmp/.trivy/ # Temp directory
      ## Overrides parent sbom configs
      # local_image_scan: ...
      # registry: ...

# Vulnerabilities scanner family
vulnerabilities: 
  enabled: false # Enable or disable vulnerabilities scanner family
  scanners_list: # List of scanners to run
    - "grype"
    - "trivy"
  inputs: # List of inputs to scan
    - input: "nginx:1.12"
      input_type: "image" # Type of input (image, rootfs, etc.)
    # - input: "nginx:1.13"
    #   input_type: "image"
    - input: "/mnt/"
      input_type: "sbom"
  local_image_scan: true # Scan images from local docker daemon (true) or from remote registry (false)
  registry: # Registry configuration
    skip-verify-tls: false
    use-http: false
    auths:
      authority: "authority"
      username: "username"
      password: "password"
      token: "token"
  scanners_config: # Configuration for each scanner
    grype:
      mode: "LOCAL" # Mode of operation (LOCAL, REMOTE). LOCAL uses local database, REMOTE uses Grype server.
      local_grype_config:
        update_db: true # Update the database
        db_root_dir: "/tmp/" # Database root directory
        listing_url: "https://toolbox-data.anchore.io/grype/databases/listing.json" # Listing URL
        max_allowed_built_age: "120h" # Max allowed built age
        listing_file_timeout: "60s" # Listing file timeout
        update_timeout: "60s" # Update timeout
        scope: "squashed" # Scope of the scan (squashed, all-layers)
        ## Overrides parent sbom configs
        # local_image_scan: ...
        # registry: ...
      remote_grype_config:
        grype_server_address: "" # Grype server address
        grype_server_timeout: "2m" # Grype server timeout
        grype_server_schemes: [] # Grype server schemes
    trivy:
      timeout: 300 # Timeout in seconds
      cache_dir: /tmp/.trivy/cache # Cache directory
      temp_dir: /tmp/.trivy/ # Temp directory
      server_addr: "trivy.example.com" # Trivy server address
      server_token: "token" # Trivy server token
      ## Overrides parent sbom configs
      # registry: ...

# Secrets scanner family
secrets:
  enabled: false # Enable or disable secrets scanner family
  scanners_list: # List of scanners to run
    - "gitleaks"
  strip_input_paths: false # Strip input paths from the output
  inputs: # List of inputs to scan
    - input: "/"
      input_type: "rootfs"
  scanners_config: # Configuration for each scanner
    gitleaks:
      binary_path: "/usr/local/bin/gitleaks" # Path to gitleaks binary

# Exploits scanner family
exploits:
  enabled: false # Enable or disable exploits scanner family
  scanners_list: # List of scanners to run
    - "exploitdb"
  inputs: # List of inputs to scan
    - input: "CVE-2024-5535,CVE-2023-3446"
      input_type: "csv"
  scanners_config: # Configuration for each scanner
    exploit_db:
      base_url: "http://localhost:1326" # Base URL for the ExploitDB server

# Misconfigurations scanner family
misconfiguration:
  enabled: false # Enable or disable misconfigurations scanner family
  scanners_list: # List of scanners to run
    - "cisdocker"
    - "lynis"
    - "fake"
  strip_input_paths: false # Strip input paths from the output
  inputs: # List of inputs to scan
    - input: "/"
      input_type: "rootfs"
  scanners_config: # Configuration for each scanner
    cisdocker:
      timeout: "60s" # Timeout
      registry: # Registry configuration
        skip-verify-tls: false
        use-http: false
        auths:
          authority: "authority"
          username: "username"
          password: "password"
          token: "token"
    lynis:
      binary_path: "/usr/local/bin/lynis" # Path to Lynis binary

# InfoFinder scanner family
infofinder:
  enabled: false # Enable or disable infofinder scanner family
  scanners_list: # List of scanners to run
    - "sshTopology"
  strip_input_paths: false # Strip input paths from the output
  inputs: # List of inputs to scan
    - input: "/"
      input_type: "rootfs"
  scanners_config: {}

# Malware scanner family
malware:
  enabled: false # Enable or disable malware scanner family
  scanners_list: # List of scanners to run
    - "clam"
    - "yara"
  strip_input_paths: false # Strip input paths from the output
  inputs: # List of inputs to scan
    - input: "/"
      input_type: "rootfs"
  scanners_config: # Configuration for each scanner
    clam:
      freshclam_binary_path: "/usr/local/bin/freshclam" # Path to freshclam binary
      freshclam_config_path: "/etc/clamav/freshclam.conf" # Path to freshclam configuration file
      alternative_freshclam_mirror_url: "" # Alternative freshclam mirror URL. Config option cannot include servers under *.clamav.net.
      use_native_clamscan: false # Scan using native clamscan command (true) instead of daemon clamdscan (false)
      clamscan_binary_path: "/usr/local/bin/clamscan" # Path to clamscan binary
      clamscan_exclude_files: # Files to exclude from the scan
        - "^.*\\.log$"
      clamscan_exclude_dirs: # Directories to exclude from the scan
        - "^/sys"
      clam_daemon_binary_path: "/usr/local/bin/clamd" # Path to clamd binary
      clam_daemon_config_path: "/etc/clamav/clamd.conf" # Path to clamd configuration file
      clam_daemon_client_binary_path: "/usr/local/bin/clamdscan" # Path to clamdscan binary
    yara:
      yara_binary_path: "/usr/local/bin/yara" # Path to yara binary
      compiled_rule_url: "" # URL to download compiled rules
      rule_sources: # List of rule sources
        - name: ""
          url: ""
      yarac_binary_path: "/usr/local/bin/yarac" # Path to yarac binary
      cache_dir: "/tmp/.yara" # Cache directory
      directories_to_scan: [] # Directories to scan

# Rootkits scanner family
rootkits:
  enabled: false # Enable or disable rootkits scanner family
  scanners_list: # List of scanners to run
    - "chkrootkit"
  strip_input_paths: false # Strip input paths from the output
  inputs: # List of inputs to scan
    - input: "/"
      input_type: "rootfs"
  scanners_config: # Configuration for each scanner
    chkrootkit:
      binary_path: "/usr/local/bin/chkrootkit" # Path to chkrootkit binary

# Plugins scanner family
plugins:
  enabled: false # Enable or disable plugins scanner family
  binary_mode: false # Use binary mode for plugins
  binary_artifacts_path: "" # Path to binary artifacts
  binary_artifacts_clean: true # Clean binary artifacts after execution
  scanners_list: # List of scanners to run
    - "kics"
  inputs: # List of inputs to scan
    - input: "/"
      input_type: "rootfs"
  scanners_config: # Configuration for each scanner
    kics:
      image_name: "ghcr.io/openclarity/openclarity-plugin-kics:latest" # Image name for KICS plugin
      config: "{\"preview-lines\": 3, \"report-formats\": [\"json\" ], \"platform\": [], \"max-file-size-flag\": 100, \"disable-secrets\": true, \"query-exec-timeout\": 60, \"silent\": true, \"minimal\": true}" # Configuration example for KICS

  

3.3 - OpenClarity SDKs

See the scanner module for the SDK.

4 - Features

OpenClarity provides a wide range of features for asset scanning and discovery:

• Dashboard
  • Fixable vulnerabilities per severity
  • Top 5 vulnerable elements (applications, resources, packages)
  • New vulnerabilities trends
  • Package count per license type
  • Package count per programming language
  • General counters
• Applications
  • Automatic application detection in K8s runtime
  • Create/edit/delete applications
  • Per application, navigation to related:
    • Resources (images/directories)
    • Packages
    • Vulnerabilities
    • Licenses in use by the resources
• Application Resources (images/directories)
  • Per resource, navigation to related:
    • Applications
    • Packages
    • Vulnerabilities
• Packages
  • Per package, navigation to related:
    • Applications
    • Linkable list of resources and the detecting SBOM analyzers
    • Vulnerabilities
• Vulnerabilities
  • Per vulnerability, navigation to related:
    • Applications
    • Resources
    • List of detecting scanners
• K8s Runtime scan
  • On-demand or scheduled scanning
  • Automatic detection of target namespaces
  • Scan progress and result navigation per affected element (applications, resources, packages, vulnerabilities)
  • CIS Docker benchmark
• CLI (CI/CD)
  • SBOM generation using multiple integrated content analyzers (Syft, cyclonedx-gomod)
  • SBOM/image/directory vulnerability scanning using multiple integrated scanners (Grype, Dependency-track)
  • Merging of SBOM and vulnerabilities across different CI/CD stages
  • Export results to OpenClarity backend
• API
  • See the API Reference.

Runtime environment

The following table lists all supported environments and asset types that can be discovered and scanned by OpenClarity.

Scanning

The following table lists all supported scanners that can be used when performing a scan on an asset, such as a container image or a directory.

Integrated SBOM Generators and Vulnerability Scanners

OpenClarity content analyzer integrates with the following SBOM generators:

• Syft
• Cyclonedx-gomod
• Trivy

OpenClarity vulnerability scanner integrates with the following scanners:

• Grype
• Dependency-Track
• Trivy

4.1 - Kubernetes Scanning

Perform Scan

For details on performing runtime scans with OpenClarity, see the First Tasks on the UI.

Asset Discovery

The OpenClarity stack supports the automatic discovery of assets in Kubernetes:

4.2 - Docker Scanning

Perform Scan

For details on performing runtime scans with OpenClarity, see the First Tasks on the UI.

Asset Discovery

The OpenClarity stack supports the automatic discovery of assets in Docker:

4.3 - AWS Scanning

Perform Scan

For details on performing runtime scans with OpenClarity, see the First Tasks on the UI.

Asset Discovery

The OpenClarity stack supports the automatic discovery of assets in AWS:

4.4 - GCP Scanning

Perform Scan

For details on performing runtime scans with OpenClarity, see the First Tasks on the UI.

Asset Discovery

The OpenClarity stack supports the automatic discovery of assets in GCP:

4.5 - Azure Scanning

Perform Scan

For details on performing runtime scans with OpenClarity, see the First Tasks on the UI.

Asset Discovery

The OpenClarity stack supports the automatic discovery of assets in Azure:

4.6 - Scanner Plugins

Plugins provide additional scanning capabilities to OpenClarity ecosystem. Project structure:

• runner - Provides necessary logic to manage scanner plugins in OpenClarity.
• sdk - Language-specific libraries, templates, and examples to aid with the implementation of scanner plugins.
• store - Collection of available plugins that can be directly used in OpenClarity.

Requirements

Scanner plugins are distributed as containers and require Docker Engine on the host that runs the actual scanning via OpenClarity CLI to work.

Support

List of supported environments:

1. AWS
2. GCP
3. Azure
4. Docker

List of unsupported environments:

• Kubernetes - We plan on adding plugin support to Kubernetes once we have dealt with all the security considerations.

Note: Plugin support has been tested against OpenClarity installation artifacts for the given environments.

Usage

You can start using plugins via Plugins Store. For example, you can pass the .families.yaml scan config file defined below to the OpenClarity CLI scan command. This configuration uses KICS scanner to scan /tmp dir for IaC security misconfigurations. See the KICS documentation for further information.

# --- .families.yaml
plugins:
  enabled: true
  scanners_list:
    - "kics"
  inputs: 
    - input: "/tmp"
      input_type: "rootfs"
  scanners_config:
    kics:
      image_name: "ghcr.io/openclarity/openclarity-plugin-kics:latest"
      config: "{}"

SDKs

You can use one of available SDKs in your language of choice to quickly develop scanner plugins for OpenClarity.

List of supported languages:

• Golang
• Python

4.7 - Cost Estimation

Available in version 0.6.0 and later. Currently, this feature is exclusively available on AWS.

You can get a preliminary cost estimation before initiating a security scan with OpenClarity. This helps you plan and budget your security assessments more effectively, ensuring that you have a clear understanding of the financial implications before taking action.

To start a new estimation, complete the following steps.

1. Create a new resource called ScanEstimation in the API server. For example, if your POST’s body is the following JSON, it will estimate an SBOM scan on your workload with id i-123456789.

   Use the same same scanTemplate in the ScanEstimation than in the ScanConfiguration.

   {
     "assetIDs": ["i-123456789"],
     "state": {
       "state": "Pending"
     },
     "scanTemplate": {
       "scope": "contains(assetInfo.tags, '{\"key\":\"scanestimation\",\"value\":\"test\"}')",
       "assetScanTemplate": {
         "scanFamiliesConfig": {
           "sbom": {
             "enabled": true
           }
         }
       }
     }
   }
   

2. Retrieve the object from the <apiserver IP address>:8888/scanEstimations endpoint, and wait for the state to be Done. The totalScanCost of the summary property shows your scan’s cost in USD:

   {
      "assetIDs":[
         "d337bd07-b67f-4cf0-ac43-f147fce7d1b2"
      ],
      "assetScanEstimations":[
         {
            "id":"23082244-0fb6-4aca-8a9b-02417dfc95f8"
         }
      ],
      "deleteAfter":"2023-10-08T17:33:52.512829081Z",
      "endTime":"2023-10-08T15:33:52.512829081Z",
      "id":"962e3a10-05fb-4c5d-a773-1198231f3103",
      "revision":5,
      "scanTemplate":{
         "assetScanTemplate":{
            "scanFamiliesConfig":{
               "sbom":{
                  "enabled":true
               }
            }
         },
         "scope":"contains(assetInfo.tags, '{\"key\":\"scanestimation\",\"value\":\"test\"}')"
      },
      "startTime":"2023-10-08T15:33:37.513073573Z",
      "state":{
         "state":"Done",
         "stateMessage":"1 succeeded, 0 failed out of 1 total asset scan estimations",
         "stateReason":"Success"
      },
      "summary":{
         "jobsCompleted":1,
         "jobsLeftToRun":0,
         "totalScanCost":0.0006148403,
         "totalScanSize":3,
         "totalScanTime":12
      },
      "ttlSecondsAfterFinished":7200
   }
   

5 -

Configuration

API server

Orchestrator

Provider

AWS

UI backend

6 - API Reference

7 -

Troubleshooting and debugging OpenClarity

Table of Contents

• How to debug the Scanner VMs
  • Docker and Kubernetes
  • Cloud providers

How to debug the Scanner VMs

Docker and Kubernetes provider

For Docker provider, scanners are created as containers, while as pods in case of Kubernetes. In both cases, you can access them directly and check the logs.

Cloud providers

On cloud providers (AWS, Azure, GCP) OpenClarity is configured to create the Scanner VMs with the same key-pair that the OpenClarity server has. The Scanner VMs run in a private network, however the OpenClarity Server can be used as a bastion/jump host to reach them via SSH.

ssh -i <key-pair private key> -J ubuntu@<openclarity server public IP> ubuntu@<scanner VM private IP address>

Once SSH access has been established, the status of the VM’s start up configuration can be debugged by checking the cloud-init logs:

sudo journalctl -u cloud-final

And the openclarity-scanner service logs:

sudo journalctl -u openclarity-scanner

8 - Archive

This section contains the documentation for VMClarity, KubeClarity, and APIClarity.

Announcement: OpenClarity Unification

We have reached the final step in the OpenClarity unification roadmap. Previously, we successfully enriched VMClarity with Kubernetes scanning and runtime features, achieving feature parity with KubeClarity.

We are currently in the process of unifying the two projects - KubeClarity and VMClarity - into one consolidated project under OpenClarity.

The VMClarity and KubeClarity repositories will not be maintained in the future. Thank you for your support and contributions as we continue to enhance and streamline the OpenClarity ecosystem.

8.1 - VM Security

VMClarity is an open source tool for agentless detection and management of Virtual Machine Software Bill Of Materials (SBOM) and security threats such as vulnerabilities, exploits, malware, rootkits, misconfigurations and leaked secrets.

VMClarity is the tool responsible for VM Security in the OpenClarity platform.

Join VMClarity’s Slack channel to hear about the latest announcements and upcoming activities. We would love to get your feedback!

Why VMClarity?

Virtual machines (VMs) are the most used service across all hyperscalers. AWS, Azure, GCP, and others have virtual computing services that are used not only as standalone VM services but also as the most popular method for hosting containers (e.g., Docker, Kubernetes).

VMs are vulnerable to multiple threats:

• Software vulnerabilities
• Leaked Secrets/Passwords
• Malware
• System Misconfiguration
• Rootkits

There are many very good open source and commercial-based solutions for providing threat detection for VMs, manifesting the different threat categories above.

However, there are challenges with assembling and managing these tools yourself:

• Complex installation, configuration, and reporting
• Integration with deployment automation
• Siloed reporting and visualization

The VMClarity project is focused on unifying detection and management of VM security threats in an agentless manner.

Overview

VMClarity uses a pluggable scanning infrastructure to provide:

• SBOM analysis
• Package and OS vulnerability detection
• Exploit detection
• Leaked secret detection
• Malware detection
• Misconfiguration detection
• Rootkit detection

The pluggable scanning infrastructure uses several tools that can be enabled/disabled on an individual basis. VMClarity normalizes, merges and provides a robust visualization of the results from these various tools.

These tools include:

• SBOM Generation and Analysis
  • Syft
  • Trivy
  • Cyclonedx-gomod
• Vulnerability detection
  • Grype
  • Trivy
  • Dependency-Track
• Exploits
  • Go exploit db
• Secrets
  • gitleaks
• Malware
  • ClamAV
  • YARA (Available in version 0.6.0 and later)
• Misconfiguration
  • Lynis
• Rootkits
  • Chkrootkit

A high-level architecture overview is available in Architecture.

Roadmap

VMClarity project roadmap is available here.

8.1.1 - Architecture

Today, VMClarity has two halves, the VMClarity control plane, and the VMClarity CLI.

The VMClarity control plane includes several microservices:

• API Server: The VMClarity API for managing all objects in the VMClarity system. This is the only component in the system which talks to the DB.

• Orchestrator: Orchestrates and manages the life cycle of VMClarity scan configs, scans and asset scans. Within the Orchestrator there is a pluggable “provider” which connects the orchestrator to the environment to be scanned and abstracts asset discovery, VM snapshotting as well as creation of the scanner VMs. (Note The only supported provider today is AWS, other hyperscalers are on the roadmap)

• UI Backend: A separate backend API which offloads some processing from the browser to the infrastructure to process and filter data closer to the source.

• UI Webserver: A server serving the UI static files.

• DB: Stores the VMClarity objects from the API. Supported options are SQLite and Postgres.

• Scanner Helper services: These services provide support to the VMClarity CLI to offload work that would need to be done in every scanner, for example downloading the latest vulnerability or malware signatures from the various DB sources. The components included today are:

  • grype-server: A rest API wrapper around the grype vulnerability scanner
  • trivy-server: Trivy vulnerability scanner server
  • exploitDB server: A test API which wraps the Exploit DB CVE to exploit mapping logic
  • freshclam-mirror: A mirror of the ClamAV malware signatures

The VMClarity CLI contains all the logic for performing a scan, from mounting attached volumes and all the pluggable infrastructure for all the families, to exporting the results to VMClarity API.

These components are containerized and can be deployed in a number of different ways. For example our cloudformation installer deploys VMClarity on a VM using docker in an dedicated AWS Virtual Private Cloud (VPC).

Once the VMClarity server instance has been deployed, and the scan configurations have been created, VMClarity will discover VM resources within the scan range defined by the scan configuration (e.g., by region, instance tag, and security group). Once the asset list has been created, snapshots of the assets are taken, and a new scanner VM are launched using the snapshots as attached volumes. The VMClarity CLI running within the scanner VM will perform the configured analysis on the mounted snapshot, and report the results to the VMClarity API. These results are then processed by the VMClarity backend into findings.

8.1.2 - Getting started

This chapter guides you through the installation of the VMClarity backend and the CLI, and shows you the most common tasks that you can perform with VMClarity.

8.1.2.1 - Deploy on AWS

An AWS CloudFormation template is provided for quick deployment of the VMClarity environment.

Note: To avoid extra costs (cross-region snapshots), you may want to deploy the VMClarity AWS CloudFormation template in the same region where the majority of the VMs are that you want to scan with VMClarity.

The following figure shows the basic AWS resources that the VMClarity CloudFormation template creates:

• a VPC with a public and private subnet, and

• an AWS Internet Gateway (IGW) and NAT Gateway (NGW) into the VPC.

  

The public subnet (VmClarityServerSubnet) hosts the VMClarity Server (VmClarityServer) EC2 instance. The VMClarity server houses the scanning configuration, the UI, and other control components. The EC2 instance is assigned an external IPv4 address (EIP) for SSH and web UI access.

The private subnet (VmClarityScannerSubnet) hosts the VM snapshot instances (EC2) that are scanned for security vulnerabilities.

Prerequisites

• Have an AWS account.
• Create an EC2 key pair.

Deployment steps

To deploy the VMClarity AWS CloudFormation Stack, you can:

• click this quick-create link to navigate directly to the AWS CloudFormation console and jump to the wizard instructions, or
• complete the following steps.

1. Download the latest VMClarity release.

   wget https://github.com/openclarity/vmclarity/releases/download/v1.1.1/aws-cloudformation-v1.1.1.tar.gz
   

   Alternatively, copy the AWS CloudFormation template file from the project repository to deploy the latest development code and skip the next step.

2. Create a new directory and extract the files.

   mkdir aws-cloudformation-v1.1.1
   tar -xvzf aws-cloudformation-v1.1.1.tar.gz -C aws-cloudformation-v1.1.1
   

3. Log in to the AWS CloudFormation console and go to the AWS CloudFormation Stacks section, then select Create Stack > With New Resources (standard).

4. Check Template is ready and Upload a template file, then click Upload a template file/Choose file and upload the previously downloaded CFN template file.

   

5. In the VMClarity CloudFormation Stack wizard, set the following:

   1. Enter a name for the stack.
   2. Select the InstanceType (defaults to t2.large for the VMClarity Server, and the scanner VMs).
   3. Specify the SSH key for the EC2 instance in the KeyName field. You will need this key to connect to VMClarity.
   4. Adjust SSHLocation according to your policies.
   5. Do not change AdvancedConfiguration, unless you are building from a custom registry.
   6. Click NEXT.
   7. (Optional) Add tags as needed for your environment. You can use the defaults unless you need to adjust for your own policies.
   8. Click NEXT, then scroll to the bottom of the screen, and check I acknowledge….
   9. Click SUBMIT.

6. Once the stack is deployed successfully, copy the VMClarity SSH address from the Outputs tab.

   

7. Open an SSH tunnel to VMClarity the server

ssh -N -L 8080:localhost:80 -i  "<Path to the SSH key specified during install>" ubuntu@<VmClarity SSH Address copied during install>

1. Open the VMClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   

2. (Optional) If needed, you can access the API athttp://localhost:8080/api. For details on the API, see API Reference.

Next steps

Complete the First tasks on the UI.

8.1.2.2 - Deploy on Azure

Prerequisites

• Have an Azure subscription.

• Create an SSH public key for Linux. Please follow these instructions for Linux and Mac users or these for Windows users. Once you have an RSA private key, convert it to an SSH2 public key with:

  ssh-keygen -e -f ~/.ssh/id_rsa.pub > ~/.ssh/id_rsa2.pub
  

Deployment steps

1. Click here to deploy VMClarity’s custom template.

2. Fill out the required Project details and Instance details in the Basics tab.

   

   You can set the following parameters:

   Parameter	Required	Description
   Subscription	True	Azure subscription where resources will be billed.
   Region	False	Azure region where resources will be deployed.
   VMClarity Deploy Postfix	True	Postfix for Azure resource group name (e.g. vmclarity-<postfix>).
   VMClarity Server SSH Username	True	SSH Username for the VMClarity Server Virtual Machine.
   VMClarity Server SSH Public Key	True	SSH Public Key for the VMClarity Server Virtual Machine. Paste the contents of ~/.ssh/id_rsa2.pub here.
   VMClarity Server VM Size	True	The size of the VMClarity Server Virtual Machine.
   VMClarity Scanner VMs Size	True	The size of the VMClarity Scanner Virtual Machines.
   Security Type	False	Security Type of the VMClarity Server Virtual Machine, e.g. TrustedLaunch (default) or Standard.

3. (Optional) In the Advanced tab, modify the Container Image for each service if a specific VMClarity version is required. Then, select the delete policy and the database.

   

   Parameter	Required	Description
   Service Container Image	True	Docker Container Image to use for each service.
   Asset Scan Delete Policy	True	Delete Policy for resources created when performing an asset scan, e.g. Always, OnSuccess or Never.
   Database To Use	True	Database type to use, e.g. SQLite, PostgreSQL or External PostgreSQL.

4. Click Review + create to create the deployment.

5. Once the deployment is completed successfully, copy the VMClarity SSH address from the Outputs tab.

   

6. Open an SSH tunnel to VMClarity the server

ssh -N -L 8080:localhost:80 -i  "<Path to the SSH key specified during install>" ubuntu@<VmClarity SSH Address copied during install>

1. Open the VMClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   

2. (Optional) If needed, you can access the API athttp://localhost:8080/api. For details on the API, see API Reference.

Next steps

Complete the First tasks on the UI.

8.1.2.3 - Deploy on Docker

Prerequisites

• Install Docker.

Deployment steps

To run VMClarity in Docker on a local machine, complete the following steps.

1. Download the latest VMClarity release.

   wget https://github.com/openclarity/vmclarity/releases/download/v1.1.1/docker-compose-v1.1.1.tar.gz
   

2. Create a new directory, extract the files and navigate to the directory.

   mkdir docker-compose-v1.1.1
   tar -xvzf docker-compose-v1.1.1.tar.gz -C docker-compose-v1.1.1
   cd docker-compose-v1.1.1
   

3. Start every control plane element with the docker compose file.

   docker compose --project-name vmclarity --file docker-compose.yml up -d --wait --remove-orphans
   

   The output should be similar to:

   [+] Running 14/14
   ⠿ Network vmclarity                        Created                                                       0.2s
   ⠿ Volume "vmclarity_grype-server-db"       Created                                                       0.0s
   ⠿ Volume "vmclarity_apiserver-db-data"     Created                                                       0.0s
   ⠿ Container vmclarity-orchestrator-1       Healthy                                                      69.7s
   ⠿ Container vmclarity-yara-rule-server-1   Healthy                                                      17.6s
   ⠿ Container vmclarity-exploit-db-server-1  Healthy                                                      17.7s
   ⠿ Container vmclarity-swagger-ui-1         Healthy                                                       7.8s
   ⠿ Container vmclarity-trivy-server-1       Healthy                                                      26.7s
   ⠿ Container vmclarity-uibackend-1          Healthy                                                      17.6s
   ⠿ Container vmclarity-ui-1                 Healthy                                                       7.7s
   ⠿ Container vmclarity-freshclam-mirror-1   Healthy                                                       7.8s
   ⠿ Container vmclarity-grype-server-1       Healthy                                                      37.3s
   ⠿ Container vmclarity-gateway-1            Healthy                                                       7.7s
   ⠿ Container vmclarity-apiserver-1          Healthy                                                      17.7s
   

   Please note that the image_override.env file enables you to use the images you build yourself. You can override parameters in the docker-compose.yml by passing a custom env file to the docker compose up command via the --env-file flag. The /installation/docker/image_override.env file contains an example overriding all the container images.

4. Check the running containers in the Docker desktop.

   

5. Access the VMClarity UI. Navigate to http://localhost:8080/ in your browser.

   

Next steps

Complete the First tasks on the UI.

Clean up steps

1. After you’ve finished your tasks, stop the running containers.

   docker compose --project-name vmclarity --file docker-compose.yml down --remove-orphans
   

8.1.2.4 - Deploy on GCP

Prerequisites

• You can install VMClarity using the CLI, so you have to have gcloud on your computer available beforehand. For details on installing and configuring gcloud, see the official installation guide.
• If you have already installed VMClarity before and want to reinstall it, you have to manually restore deleted roles that were created during the previous installation.

Deployment steps

To install VMClarity on Google Cloud Platform (GCP), complete the following steps.

1. Download the newest GCP deployment release from GitHub and extract it to any location.

   wget https://github.com/openclarity/vmclarity/releases/download/v1.1.1/gcp-deployment-v1.1.1.tar.gz
   

2. Create a new directory, extract the files and navigate to the directory.

   mkdir gcp-deployment-v1.1.1
   tar -xvzf gcp-deployment-v1.1.1.tar.gz -C gcp-deployment-v1.1.1
   cd gcp-deployment-v1.1.1
   

3. Copy the example configuration file and rename it.

   cp vmclarity-config.example.yaml vmclarity-config.yaml
   

4. The following table contains all the fields that can be set in the vmclarity-config.yaml file. You have to set at least the required ones.

   Field	Required	Default	Description
   zone	yes		The Zone to locate the VMClarity server.
   machineType	yes		The machine type for the VMClarity server.
   region	yes		The region to locate VMClarity.
   scannerMachineType		e2-standard-2	Machine type to use for the Scanner instances.
   scannerSourceImage		projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20230630	Source image to use for the Scanner instances.
   databaseToUse		SQLite	The database that VMClarity should use.
   apiserverContainerImage		ghcr.io/openclarity/vmclarity-apiserver:1.1.1	The container image to use for the apiserver.
   orchestratorContainerImage		ghcr.io/openclarity/vmclarity-orchestrator:1.1.1	The container image to use for the orchestrator.
   uiContainerImage		ghcr.io/openclarity/vmclarity-ui:1.1.1	The container image to use for the ui.
   uibackendContainerImage		ghcr.io/openclarity/vmclarity-ui-backend:1.1.1	The container image to use for the uibackend.
   scannerContainerImage		ghcr.io/openclarity/vmclarity-cli:1.1.1	The container image to use for the scanner.
   exploitDBServerContainerImage		ghcr.io/openclarity/exploit-db-server:v0.2.4	The container image to use for the exploit db server.
   trivyServerContainerImage		docker.io/aquasec/trivy:0.41.0	The container image to use for the trivy server.
   grypeServerContainerImage		ghcr.io/openclarity/grype-server:v0.7.0	The container image to use for the grype server.
   freshclamMirrorContainerImage		ghcr.io/openclarity/freshclam-mirror:v0.2.0	The container image to use for the fresh clam mirror server.
   postgresqlContainerImage		docker.io/bitnami/postgresql:12.14.0-debian-11-r28	The container image to use for the postgresql server.
   assetScanDeletePolicy		Always	When asset scans should be cleaned up after scanning.
   postgresDBPassword			Postgres DB password. Only required if DatabaseToUse is Postgresql.
   externalDBName			DB to use in the external DB. Only required if DatabaseToUse is External.
   externalDBUsername			Username for the external DB. Only required if the DatabaseToUse is External.
   externalDBPassword			Password for the external DB. Only required if the DatabaseToUse is External.
   externalDBHost			Hostname or IP for the external DB. Only required if the DatabaseToUse is External.
   externalDBPort			Port for the external DB. Only required if the DatabaseToUse is External.

5. Deploy VMClarity using gcloud deployment-manager.

   gcloud deployment-manager deployments create <vmclarity deployment name> --config vmclarity-config.yaml
   

6. Open an SSH tunnel to the VMClarity server with gcloud. For further information on how to create an SSH connection with gcloud to one of your instances check the official page.

   gcloud compute ssh --project=<project id> --zone=<zone name> <name of your VM> -- -NL 8080:localhost:80
   

7. Open the VMClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   

8. (Optional) If needed, you can access the API athttp://localhost:8080/api. For details on the API, see API Reference.

Next steps

Complete the First tasks on the UI.

Uninstall VMClarity

1. You can uninstall VMClarity using the gcloud manager.

   gcloud deployment-manager deployments delete <vmclarity deployment name>
   

Restore deleted roles

1. On the IAM & Admin page on GCP, open the Roles tab, then search for VMClarity in the filter input.

2. Now manually undelete the Discoverer Snapshotter and Scanner roles to set their statuses from Deleted to Enabled.

   

8.1.2.5 - Deploy on Kubernetes

Prerequisites

• Install a tool to run local Kubernetes clusters. Here, Kind is used as the default option for creating a local cluster.
• Helm to install VMClarity.

Deployment steps

To deploy VMClarity to your Kubernetes cluster, complete the following steps.

1. Create a Kubernetes cluster.

   kind create cluster --name vmclarity-k8s
   

2. Ensure the Kubernetes cluster is up and running. If you’re using kind, you can check the status of your clusters with the following command:

   kind get clusters
   

3. Use Helm to install VMClarity. Run the following command:

   helm install vmclarity oci://ghcr.io/openclarity/charts/vmclarity --version 1.1.1 \
       --namespace vmclarity --create-namespace \
       --set orchestrator.provider=kubernetes \
       --set orchestrator.serviceAccount.automountServiceAccountToken=true
   

4. Verify that all the VMClarity pods have been successfully deployed by executing the following command:

   kubectl get pods -n vmclarity
   

5. Wait until all pods are in the Running state or have completed their initialization.

6. Once the pods are ready, start port forwarding to access the VMClarity gateway service. Use the following command to forward traffic from your local machine to the cluster:

   kubectl port-forward -n vmclarity service/vmclarity-gateway 8080:80
   

7. Access the VMClarity UI by navigating to http://localhost:8080/ in your web browser.

   

Next steps

Complete the First tasks on the UI.

Clean up steps

1. Uninstall VMClarity with Helm. Run the following command:

   helm uninstall vmclarity --namespace vmclarity
   

2. Delete the Kubernetes cluster.

   kind delete clusters vmclarity-k8s
   

8.1.2.6 - First tasks on the UI

Configure your first scan

1. Open the VMClarity UI in your browser at http://localhost:8080/. The dashboard opens.

   

2. (Optional) If needed, you can access the API athttp://localhost:8080/api. For details on the API, see API Reference.

3. Click on the Scans icon. In the Scans window, you can create a new scan configuration.

   

4. Click New scan configuration.

   

5. Follow the steps of the New scan config wizard to name the scan, and identify the AWS scope (region, VPC, security groups, etc). The following example shows the AWS us-east-2 region, a specific VPC, and the vmclarity-demo-vm EC2

   

6. Enable the scan types you want to perform.

   

7. Select the time and/or frequency of the scans. To run the scan immediately, select Now.

   

8. Click Save. The new scan appears on the Scan Configurations tab.

   

9. Once a scan is finished, you can browse around the various VMClarity UI features and investigate the security scan reports.

   

8.1.3 - Common CLI tasks

Initiate a scan using the CLI

Reporting results into file:

./cli/bin/vmclarity-cli scan --config ~/testConf.yaml -o outputfile

If we want to report results to the VMClarity backend, we need to create asset and asset scan object before scan because it requires asset-scan-id

Reporting results to VMClarity backend:

ASSET_ID=$(./cli/bin/vmclarity-cli asset-create --file assets/dir-asset.json --server http://localhost:8080/api) --jsonpath {.id}
ASSET_SCAN_ID=$(./cli/bin/vmclarity-cli asset-scan-create --asset-id $ASSET_ID --server http://localhost:8080/api) --jsonpath {.id}
./cli/bin/vmclarity-cli scan --config ~/testConf.yaml --server http://localhost:8080/api --asset-scan-id $ASSET_SCAN_ID

Using one-liner:

./cli/bin/vmclarity-cli asset-create --file docs/assets/dir-asset.json --server http://localhost:8080/api --update-if-exists --jsonpath {.id} | xargs -I{} ./cli/bin/vmclarity-cli asset-scan-create --asset-id {} --server http://localhost:8080/api --jsonpath {.id} | xargs -I{} ./cli/bin/vmclarity-cli scan --config ~/testConf.yaml --server http://localhost:8080/api --asset-scan-id {}

8.1.4 - Configuration Parameters

Orchestrator

Provider

AWS

8.1.5 - Troubleshooting and Debugging

How to debug the Scanner VMs

How to debug the Scanner VMs can differ per provider these are documented below.

Debug Scanner VM on AWS

On AWS VMClarity is configured to create the Scanner VMs with the same key-pair that the VMClarity server has. The Scanner VMs run in a private network, however the VMClarity Server can be used as a bastion/jump host to reach them via SSH.

ssh -i <key-pair private key> -J ubuntu@<vmclarity server public IP> ubuntu@<scanner VM private IP address>

Once SSH access has been established, the status of the VM’s start up configuration can be debugged by checking the cloud-init logs:

sudo journalctl -u cloud-final

And the vmclarity-scanner service logs:

sudo journalctl -u vmclarity-scanner

8.1.6 - VMClarity development

Building VMClarity Binaries

Makefile targets are provided to compile and build the VMClarity binaries. make build can be used to build all of the components, but also specific targets are provided, for example make build-cli and make build-backend to build the specific components in isolation.

Building VMClarity Containers

make docker can be used to build the VMClarity containers for all of the components. Specific targets for example make docker-cli and make docker-backend are also provided.

make push-docker is also provided as a shortcut for building and then publishing the VMClarity containers to a registry. You can override the destination registry like:

DOCKER_REGISTRY=docker.io/tehsmash make push-docker

You must be logged into the docker registry locally before using this target.

Linting

make lint can be used to run the required linting rules over the code. golangci-lint rules and config can be viewed in the .golangcilint file in the root of the repo.

make fix is also provided which will resolve lint issues which are automatically fixable for example format issues.

make license can be used to validate that all the files in the repo have the correctly formatted license header.

To lint the cloudformation template, cfn-lint can be used, see https://github.com/aws-cloudformation/cfn-lint#install for instructions on how to install it for your system.

Unit tests

make test can be used run all the unit tests in the repo. Alternatively you can use the standard go test CLI to run a specific package or test like:

go test ./cli/cmd/... -run Test_isSupportedFS

Generating API code

After making changes to the API schema in api/openapi.yaml, you can run make api to regenerate the model, client and server code.

Testing End to End

For details on how to test VMClarity end to end please see End-to-End Testing Guide.

8.1.6.1 - End-to-End Testing Guide

Installing a specific VMClarity build on AWS

1. Build the containers and publish them to your docker registry

   DOCKER_REGISTRY=<your docker registry> make push-docker
   

2. Install VMClarity cloudformation

   1. Ensure you have an SSH key pair uploaded to AWS Ec2
   2. Go to CloudFormation -> Create Stack -> Upload template.
   3. Upload the VMClarity.cfn file.
   4. Follow the wizard through to the end
      1. Set the VMClarity Backend Container Image and VMClarity Scanner Container Image parameters in the wizard to use custom images (from step 1.) for deployment.
      2. Change the Asset Scan Delete Policy to OnSuccess or Never if debugging scanner VMs is required.
   5. Wait for install to complete

3. Ensure that VMClarity backend is working correctly

   1. Get the IP address from the CloudFormation stack’s Output Tab

   2. ssh ubuntu@<ip address>

   3. Check the VMClarity Logs

      sudo journalctl -u vmclarity
      

Performing an end to end test

1. Copy the example scanConfig.json into the ubuntu user’s home directory

   scp scanConfig.json ubuntu@<ip address>:~/scanConfig.json
   

2. Edit the scanConfig.json

   1. Give the scan config a unique name

   2. Enable the different scan families you want:

      "scanFamiliesConfig": {
        "sbom": {
          "enabled": true
        },
        "vulnerabilities": {
          "enabled": true
        },
        "exploits": {
          "enabled": true
        }
      },
      

   3. Configure the scope of the test

      • By Region, VPC or Security group:

        "scope": "contains(assetInfo.location, '<name of region>/<name of vpc>') and contains(assetInfo.securityGroups, '{\"id\":\"<name of sec group>\"}')"
        

      • By tag:

        "scope": "contains(assetInfo.tags, '{\"key\":\"<key>\",\"value\":\"<value>\"}')"
        

   • All:

     ```yaml
     "scope": ""
     ```
     

   1. Set operationTime to the time you want the scan to run. As long as the time is in the future it can be within seconds.

3. While ssh’d into the VMClarity server run

   curl -X POST http://localhost:8080/api/scanConfigs -H 'Content-Type: application/json' -d @scanConfig.json
   

4. Check VMClarity logs to ensure that everything is performing as expected

   sudo journalctl -u vmclarity
   

5. Monitor the asset scans

   • Get scans:

     curl -X GET http://localhost:8080/api/scans
     

     After the operationTime in the scan config created above there should be a new scan object created in Pending.

     Once discovery has been performed, the scan’s assetIDs list should be populated will all the assets to be scanned by this scan.

     The scan will then create all the “assetScans” for tracking the scan process for each asset. When that is completed the scan will move to “InProgress”.

   • Get asset scans:

     curl -X GET http://localhost:8080/api/assetScans
     

8.1.7 - API reference

8.1.8 - Cost estimation

Available in version 0.6.0 and later. Currently, this feature is exclusively available on AWS.

You can get a preliminary cost estimation before initiating a security scan with VMClarity. This helps you plan and budget your security assessments more effectively, ensuring that you have a clear understanding of the financial implications before taking action.

To start a new estimation, complete the following steps.

1. Create a new resource called ScanEstimation in the API server. For example, if your POST’s body is the following JSON, it will estimate an SBOM scan on your workload with id i-123456789.

   Use the same same scanTemplate in the ScanEstimation than in the ScanConfiguration.

   {
     "assetIDs": ["i-123456789"],
     "state": {
       "state": "Pending"
     },
     "scanTemplate": {
       "scope": "contains(assetInfo.tags, '{\"key\":\"scanestimation\",\"value\":\"test\"}')",
       "assetScanTemplate": {
         "scanFamiliesConfig": {
           "sbom": {
             "enabled": true
           }
         }
       }
     }
   }
   

2. Retrieve the object from the <apiserver IP address>:8888/scanEstimations endpoint, and wait for the state to be Done. The totalScanCost of the summary property shows your scan’s cost in USD:

   {
      "assetIDs":[
         "d337bd07-b67f-4cf0-ac43-f147fce7d1b2"
      ],
      "assetScanEstimations":[
         {
            "id":"23082244-0fb6-4aca-8a9b-02417dfc95f8"
         }
      ],
      "deleteAfter":"2023-10-08T17:33:52.512829081Z",
      "endTime":"2023-10-08T15:33:52.512829081Z",
      "id":"962e3a10-05fb-4c5d-a773-1198231f3103",
      "revision":5,
      "scanTemplate":{
         "assetScanTemplate":{
            "scanFamiliesConfig":{
               "sbom":{
                  "enabled":true
               }
            }
         },
         "scope":"contains(assetInfo.tags, '{\"key\":\"scanestimation\",\"value\":\"test\"}')"
      },
      "startTime":"2023-10-08T15:33:37.513073573Z",
      "state":{
         "state":"Done",
         "stateMessage":"1 succeeded, 0 failed out of 1 total asset scan estimations",
         "stateReason":"Success"
      },
      "summary":{
         "jobsCompleted":1,
         "jobsLeftToRun":0,
         "totalScanCost":0.0006148403,
         "totalScanSize":3,
         "totalScanTime":12
      },
      "ttlSecondsAfterFinished":7200
   }
   

8.2 - Kubernetes Security

KubeClarity is a tool for detection and management of Software Bill Of Materials (SBOM) and vulnerabilities of container images and filesystems. It scans both runtime K8s clusters and CI/CD pipelines for enhanced software supply chain security.

KubeClarity is the tool responsible for Kubernetes Security in the OpenClarity platform.

Why?

SBOM & Vulnerability Detection Challenges

• Effective vulnerability scanning requires an accurate Software Bill Of Materials (SBOM) detection:
  • Various programming languages and package managers
  • Various OS distributions
  • Package dependency information is usually stripped upon build
• Which one is the best scanner/SBOM analyzer?
• What should we scan: Git repos, builds, container images or runtime?
• Each scanner/analyzer has its own format - how to compare the results?
• How to manage the discovered SBOM and vulnerabilities?
• How are my applications affected by a newly discovered vulnerability?

Solution

• Separate vulnerability scanning into 2 phases:
  • Content analysis to generate SBOM
  • Scan the SBOM for vulnerabilities
• Create a pluggable infrastructure to:
  • Run several content analyzers in parallel
  • Run several vulnerability scanners in parallel
• Scan and merge results between different CI stages using KubeClarity CLI
• Runtime K8s scan to detect vulnerabilities discovered post-deployment
• Group scanned resources (images/directories) under defined applications to navigate the object tree dependencies (applications, resources, packages, vulnerabilities)

Architecture

Limitations

1. Supports Docker Image Manifest V2, Schema 2 (https://docs.docker.com/registry/spec/manifest-v2-2/). It will fail to scan earlier versions.

Roadmap

• Integration with additional content analyzers (SBOM generators)
• Integration with additional vulnerability scanners
• CIS Docker benchmark in UI
• Image signing using Cosign
• CI/CD metadata signing and attestation using Cosign and in-toto (supply chain security)
• System settings and user management

8.2.1 - Features

• Dashboard
  • Fixable vulnerabilities per severity
  • Top 5 vulnerable elements (applications, resources, packages)
  • New vulnerabilities trends
  • Package count per license type
  • Package count per programming language
  • General counters
• Applications
  • Automatic application detection in K8s runtime
  • Create/edit/delete applications
  • Per application, navigation to related:
    • Resources (images/directories)
    • Packages
    • Vulnerabilities
    • Licenses in use by the resources
• Application Resources (images/directories)
  • Per resource, navigation to related:
    • Applications
    • Packages
    • Vulnerabilities
• Packages
  • Per package, navigation to related:
    • Applications
    • Linkable list of resources and the detecting SBOM analyzers
    • Vulnerabilities
• Vulnerabilities
  • Per vulnerability, navigation to related:
    • Applications
    • Resources
    • List of detecting scanners
• K8s Runtime scan
  • On-demand or scheduled scanning
  • Automatic detection of target namespaces
  • Scan progress and result navigation per affected element (applications, resources, packages, vulnerabilities)
  • CIS Docker benchmark
• CLI (CI/CD)
  • SBOM generation using multiple integrated content analyzers (Syft, cyclonedx-gomod)
  • SBOM/image/directory vulnerability scanning using multiple integrated scanners (Grype, Dependency-track)
  • Merging of SBOM and vulnerabilities across different CI/CD stages
  • Export results to KubeClarity backend
• API
  • See API reference.

Integrated SBOM generators and vulnerability scanners

KubeClarity content analyzer integrates with the following SBOM generators:

• Syft
• Cyclonedx-gomod
• Trivy

KubeClarity vulnerability scanner integrates with the following scanners:

• Grype
• Dependency-Track
• Trivy

8.2.2 - Concepts and background

The following sections give you the concepts and background information about the scans provided by KubeClarity.

8.2.2.1 - Software bill of materials

A software bill of materials (SBOM) is a list of all the components, libraries, and other dependencies that make up a software application, along with information about the versions, licenses, and vulnerabilities associated with each component. They are formal, structured documents detailing the components of a software product and its supply chain relationships.

SBOMs are important because organizations increasingly rely on open source and third-party software components to build and maintain their applications. These components can introduce security vulnerabilities and must be adequately managed and updated. SBOMs help you understand what open source and third-party components are used in your applications, and identify and address any security vulnerabilities.

Under specific scenarios, generating and publishing SBOMs is mandatory for compliance with regulations and industry standards that require organizations to disclose the use of open source and third-party software in their products.

SBOM standards

There are several related standards, for example, CycloneDX, SPDX, SWID.

SPDX (Software Package Data Exchange) is a standard format for communicating a software package’s components, licenses, and copyrights. It is commonly used to document the open source components included in a proprietary software product. SPDX files can be easily read and understood by humans and machines, making it easy to track and manage open source components in a software project. SPDX format is supported by Linux Foundation.

CycloneDX is an open source standard for creating software bill of materials files. It is like SPDX in that it documents the components and licenses associated with a software package, but it is specifically designed for use in software supply chain security. CycloneDX is a more lightweight format compared to SPDX, which is intended to be more detailed. CycloneDX format is supported by OWASP.

SBOM architecture

A typical SBOM architecture can be laid out as a tree-like dependency graph with the following key elements:

• Component inventory: Information about the components, libraries, and other assets used in the software, including version numbers, licenses, and vulnerabilities.
• Dependency mapping: A map of relationships between different components and libraries, showing how they depend on each other and how changes to one may impact the other.
• License management: It should also include information about the licenses of the components and libraries used to ensure that the software complies with legal and ethical obligations.

SBOM generators

There are two typical ways to generate SBOM: during the build process, or after the build and deployment using a Software Composition Analysis tool. Trivy and Syft are two noteworthy open source generators among many other generators, including open source and commercial. Both use CycloneDX format. It is also important to note that not all SBOMs can be generated equally. Each generator may pick up a few language libraries better than the others based on its implementation. It might take multiple runs through a few different types of generators to draw comprehensive insights.

KubeClarity content analyzer integrates with the following SBOM generators:

• Syft
• Cyclonedx-gomod
• Trivy

Multiple SBOMs for accuracy

KubeClarity can run multiple SBOM generators in parallel, and unify their results to generate a more accurate document.

In such cases, KubeClarity compiles a merged SBOM from multiple open source analyzers, and delivers a comprehensive SBOM document report. Although KubeClarity does not generate SBOMs, it integrates with popular generators so that a combined document can provide amplified inputs that can be further analyzed using vulnerability scanners. Leveraging multiple SBOM documents can improve visibility into software dependency posture.

KubeClarity formats the merged SBOM to comply with the input requirements of vulnerability scanners before starting vulnerability scans.

Note: KubeClarity can merge vulnerability scans from various sources like Grype and Trivy to generate a robust vulnerability scan report.

Scan SBOM documents for vulnerabilities

You can feed the generated SBOM documents to vulnerability scanners, which analyze the SBOMs and generate a vulnerability report detailing all known and fixed CVEs of the software components listed by SBOM.

Generate SBOM

For details on generating SBOMs with KubeClarity, see the Getting started and Generate SBOM.

8.2.2.2 - Kubernetes cluster runtime scan

Scanning your runtime Kubernetes clusters is essential to proactively detect and address vulnerabilities in real-time, ensuring the security and integrity of your applications and infrastructure. By continuously monitoring and scanning your clusters, you can mitigate risks, prevent potential attacks, and maintain a strong security posture in the dynamic Kubernetes environment.

Runtime scan features

KubeClarity enhance the runtime scanning experience:

Faster runtime scan

KubeClarity optimizes the scanning process, reducing the time required to detect vulnerabilities. This allows for quicker identification and remediation of potential security risks.

Reduce image TAR pulling

KubeClarity uses an efficient approach that avoids the unnecessary overhead of fetching the complete image tar.

Cache SBOMs

If an image has already been scanned, KubeClarity uses the cached SBOM data, avoiding time-consuming image retrieval and recomputing, improving overall efficiency.

Runtime scan architecture

The following figure illustrates the structure of a runtime scanning architecture. This layout visually represents the components and their interconnections within the runtime scanning system.

Perform runtime scan

For details on performing runtime scans with KubeClarity, see the Getting started and Runtime scan.

8.2.2.3 - Vulnerability scanning

Vulnerability scanning identifies weak spots in software code and dependencies. Vulnerability scanners can identify infrastructure, networks, applications, or website vulnerabilities. These tools scan various target systems for security flaws that attackers could exploit.

The scanners use the information contained in the SBOM to identify vulnerabilities and potential security risks within software applications. Vulnerability scanners use SBOM information to:

• Identify vulnerable components: Scanners use the SBOM to identify a software application’s components, then cross-reference this information with known vulnerabilities and security issues to identify vulnerable components within the software.
• Prioritize vulnerabilities: After the vulnerability scanner has identified all vulnerable components within the software application, it uses the SBOM to prioritize the vulnerabilities so you can focus on the most critical vulnerabilities.
• Identify supply chain risks: SBOMs provide visibility into the software supply chain, enabling vulnerability scanners to identify third-party or security risks. As a result, organizations can mitigate supply chain risks and reduce their overall security exposure.
• Track changes and updates: Software vulnerability scanners use SBOM information to determine whether software changes have introduced new vulnerabilities or security risks.

The SBOM is a critical tool for vulnerability scanners, providing the information needed to identify, prioritize, and mitigate security risks within software applications. In addition, scanners also rely on other types of inputs, as listed below.

KubeClarity and vulnerability scanning

KubeClarity isn’t a vulnerability scanner but integrates with top opensource vulnerability scanners. It also helps with prioritization and risk management by visualization and filtering. It is often necessary to prioritize CVEs because of the sheer volume of identified CVEs. With KubeClarity’s vulnerability trending dashboard and APIs, you can locate and double-click into a specific CVE in your application or infrastructure.

KubeClarity features a range of flexible and dynamic filters that help map CVEs down to an application->package->Image level. Additionally, it normalizes reports from multiple scanners and calculates missing CVSS (Common Vulnerability Scoring System) scores.

KubeClarity vulnerability scanner integrates with the following scanners:

• Grype
• Dependency-Track
• Trivy

KubeClarity supports both automatic scans to find common vulnerabilities quickly and efficiently, and manual scans to help verify automated scans, and also to help identify more complex and less common vulnerabilities. In addition to conventional scans, KubeClarity also provides multi-scanner integration.

Multi-scanner architecture

KubeClarity infrastructure enables multiple scanners’ configuration and simultaneous operation. Scanners in KubeClarity are designed to work in parallel.

The following figure shows the multi-scanner architecture for vulnerability scanning: KubeClarity preprocesses the SBOMs so they conform to the specific formatting requirements of the specific scanner. Each scanner may have different types and unique formatting expectations. The scanners analyze the incoming data and generate vulnerability outputs in their native formats.

KubeClarity can merge the vulnerability reports of different scanners, to include severity levels, sources, and available fixes. These reports serve as valuable outputs, allowing you to filter and focus on specific areas of vulnerabilities for further investigation and resolution.

Run vulnerability scans

For details on running vulnerability scans with KubeClarity, see the Getting started and Vulnerability scan.

8.2.3 - Getting started

This chapter guides you through the installation of the KubeClarity backend and the CLI, and shows you the most common tasks that you can perform with KubeClarity.

8.2.3.1 - Install the KubeClarity backend

You can install the KubeClarity backend using Helm, or you can build and run it locally.

Prerequisites

KubeClarity requires these Kubernetes permissions:

Prerequisites for AWS

If you are installing KubeClarity on AWS, complete the following steps. These are needed because KubeClarity uses a persistent PostgreSQL database, and that requires a volume.

1. Make sure that your EKS cluster is 1.23 or higher.
2. Install the EBS CSI Driver EKS add-on. For details, see Amazon EKS add-ons.
3. Configure the EBS CSI Driver with IAMServiceRole and policies. For details, see Creating the Amazon EBS CSI driver IAM role.

Install using Helm

1. Add the Helm repository.

   helm repo add kubeclarity https://openclarity.github.io/kubeclarity
   

2. Save the default KubeClarity chart values.

   helm show values kubeclarity/kubeclarity > values.yaml
   

3. (Optional) Check the configuration in the values.yaml file and update the required values if needed. You can skip this step to use the default configuration.

   • To enable and configure the supported SBOM generators and vulnerability scanners, check the analyzer and scanner configurations under the vulnerability-scanner section. You can skip this step to use the default configuration settings.

4. Deploy KubeClarity with Helm.

   • If you have customized the values.yaml file, run:

     helm install --values values.yaml --create-namespace kubeclarity kubeclarity/kubeclarity --namespace kubeclarity
     

   • To use the default configuration, run:

     helm install --create-namespace kubeclarity kubeclarity/kubeclarity --namespace kubeclarity
     

   • For an OpenShift Restricted SCC compatible installation, run:

     helm install --values values.yaml --create-namespace kubeclarity kubeclarity/kubeclarity --namespace kubeclarity --set global.openShiftRestricted=true \
     --set kubeclarity-postgresql.securityContext.enabled=false --set kubeclarity-postgresql.containerSecurityContext.enabled=false \
     --set kubeclarity-postgresql.volumePermissions.enabled=true --set kubeclarity-postgresql.volumePermissions.securityContext.runAsUser="auto" \
     --set kubeclarity-postgresql.shmVolume.chmod.enabled=false
     

5. Wait until all the pods are in ‘Running’ state. Check the output of the following command:

   kubectl get pods --namespace kubeclarity
   

   The output should be similar to:

   NAME                                                    READY   STATUS    RESTARTS   AGE
   kubeclarity-kubeclarity-7689c7fbb7-nlhh5                1/1     Running   0          82s
   kubeclarity-kubeclarity-grype-server-79b6fb4b88-5xtbh   1/1     Running   0          82s
   kubeclarity-kubeclarity-postgresql-0                    1/1     Running   0          82s
   kubeclarity-kubeclarity-sbom-db-6895d97d5d-55jnj        1/1     Running   0          82s
   

6. Port-forward to the KubeClarity UI.

   kubectl port-forward --namespace kubeclarity svc/kubeclarity-kubeclarity 9999:8080
   

7. (Optional) Install a sample application (sock shop) to run your scans on.

   1. Create a namespace for the application.

      kubectl create namespace sock-shop
      

   2. Install the application.

      kubectl apply -f https://raw.githubusercontent.com/microservices-demo/microservices-demo/master/deploy/kubernetes/complete-demo.yaml
      

   3. Check that the installation was successful.

      kubectl get pods --namespace sock-shop
      

      Expected output:

      NAME                            READY   STATUS    RESTARTS   AGE
      carts-5dc994cf5b-4rhfj          2/2     Running   0          44h
      carts-db-556cbbd5fb-64qls       2/2     Running   0          44h
      catalogue-b7b968c97-b9k8p       2/2     Running   0          44h
      catalogue-db-f7547dd6-smzk2     2/2     Running   0          44h
      front-end-848c97475d-b7sl8      2/2     Running   0          44h
      orders-7d47794476-9fjsx         2/2     Running   0          44h
      orders-db-bbfb8f8-7ndr6         2/2     Running   0          44h
      payment-77bd4bbdf6-hkzh7        2/2     Running   0          44h
      queue-master-6d4cf8c4ff-pzk68   2/2     Running   0          44h
      rabbitmq-9dd69888f-6lzfh        3/3     Running   0          44h
      session-db-7d9d77c495-zngsn     2/2     Running   0          44h
      shipping-67fff9d476-t87jw       2/2     Running   0          44h
      user-7b667cd8d-q8bg8            2/2     Running   0          44h
      user-db-5599d45948-vxpq6        2/2     Running   0          44h
      

8. Open the KubeClarity UI in your browser at http://localhost:9999/. The KubeClarity dashboard should appear. KubeClarity UI has no data to report vulnerabilities after a fresh install, so there is no data on the dashboard.

   

9. If you also want to try KubeClarity using its command-line tool, Install the CLI. Otherwise, you can run runtime scans using the dashboard.

Uninstall using Helm

Later if you have finished experimenting with KubeClarity, you can delete the backend by completing the following steps.

1. Helm uninstall

   helm uninstall kubeclarity --namespace kubeclarity
   

2. Clean the resources. By default, Helm doesn’t remove the PVCs and PVs for the StatefulSets. Run the following command to delete them all:

   kubectl delete pvc -l app.kubernetes.io/instance=kubeclarity --namespace kubeclarity
   

Build and run locally with demo data

1. Build the UI and the backend and start the backend locally, either using Docker, or without it:

   • Using docker:

     1. Build UI and backend (the image tag is set using VERSION):

        VERSION=test make docker-backend
        

     2. Run the backend using demo data:

        docker run -p 9999:8080 -e FAKE_RUNTIME_SCANNER=true -e FAKE_DATA=true -e ENABLE_DB_INFO_LOGS=true -e DATABASE_DRIVER=LOCAL ghcr.io/openclarity/kubeclarity:test run
        

   • Local build:

     1. Build UI and backend

        make ui && make backend
        

     2. Copy the built site:

        cp -r ./ui/build ./site
        

     3. Run the backend locally using demo data:

        FAKE_RUNTIME_SCANNER=true DATABASE_DRIVER=LOCAL FAKE_DATA=true ENABLE_DB_INFO_LOGS=true ./backend/bin/backend run
        

2. Open the KubeClarity UI in your browser: http://localhost:9999/

3. Install the CLI.

8.2.3.2 - Install the CLI

KubeClarity includes a CLI that can be run locally and is especially useful for CI/CD pipelines. It allows you to analyze images and directories to generate SBOM, and scan it for vulnerabilities. The results can be exported to the KubeClarity backend.

You can install the KubeClarity CLI using the following methods:

Next step

Check the common tasks you can do using the web UI.

8.2.3.3 - First tasks - UI

After you have installed the KubeClarity backend and the KubeClarity CLI, complete the following tasks to see the basic functionality of KubeClarity web UI.

Runtime scan

To start a runtime scan, complete the following steps.

1. Open the UI in your browser at http://localhost:9999/.

2. From the navigation bar on the left, select Runtime Scan.

   

3. Select the namespace you want to scan, for example, the sock-shop namespace if you have installed the demo application, then click START SCAN. You can select multiple namespaces.

   

4. Wait until the scan is completed, then check the results. The scan results report the affected components such as Applications, Application Resources, Packages, and Vulnerabilities.

   

5. Click on these elements for details. For example, Applications shows the applications in the namespace that have vulnerabilities detected.

   

6. Now that you have run a scan, a summary of the results also appears on the dashboard page of the UI.

   

Vulnerability scan

1. To see the results of a vulnerability scan, select the Vulnerabilities page in KubeClarity UI. It shows a report including the vulnerability names, severity, the package of origin, available fixes, and attribution to the scanner that reported the vulnerability.

   

2. You can click on any of these fields to access more in-depth information. For example, click on the name of a vulnerability in the VULNERABILITY NAME column.

   

3. Select CVSS to show the CVSS scores and other details reported from the scanning process.

   

4. Navigate back to the Vulnerabilities view to explore the filtering options. Filtering helps you reduce noise and improve efficiency in identifying and potentially fixing crucial vulnerabilities.

   

5. The KubeClarity Dashboard gives you insights into vulnerability trends and fixable vulnerabilities.

   

Next step

Check the common tasks you can do using the CLI tool.

8.2.3.4 - First tasks - CLI

After you have installed the KubeClarity backend and the KubeClarity CLI, and completed the first tasks on the UI, complete the following tasks to see the basic functionality of the KubeClarity CLI.

Generate SBOM

To generate the Software Bill of Materials (SBOM), complete the following steps.

1. Run the following command.

   kubeclarity-cli analyze <image/directory name> --input-type <dir|file|image(default)> -o <output file or stdout>
   

   For example:

   kubeclarity-cli analyze --input-type image nginx:latest -o nginx.sbom
   

   Example output:

   INFO[0000] Called syft analyzer on source registry:nginx:latest  analyzer=syft app=kubeclarity
   INFO[0004] Skipping analyze unsupported source type: image  analyzer=gomod app=kubeclarity
   INFO[0004] Sending successful results                    analyzer=syft app=kubeclarity
   INFO[0004] Got result for job "syft"                     app=kubeclarity
   INFO[0004] Got result for job "gomod"                    app=kubeclarity
   INFO[0004] Skip generating hash in the case of image    
   

2. Verify that the ngnix.sbom file is generated and explore its contents as in below:

   head ngnix.sbom
   

   Example output:

   {
     "bomFormat": "CycloneDX",
     "specVersion": "1.4",
     "serialNumber": "urn:uuid:8cca2aa3-1aaa-4e8c-9d44-08e88b1df50d",
     "version": 1,
     "metadata": {
       "timestamp": "2023-05-19T16:27:27-07:00",
       "tools": [
         {
           "vendor": "kubeclarity",
   

3. To run also the trivy scanner and merge the output into a single SBOM, run:

   ANALYZER_LIST="syft gomod trivy" kubeclarity-cli analyze --input-type image nginx:latest -o nginx.sbom
   

   Example output:

   INFO[0000] Called syft analyzer on source registry:nginx:latest  analyzer=syft app=kubeclarity
   INFO[0004] Called trivy analyzer on source image nginx:latest  analyzer=trivy app=kubeclarity
   INFO[0004] Skipping analyze unsupported source type: image  analyzer=gomod app=kubeclarity
   INFO[0005] Sending successful results                    analyzer=syft app=kubeclarity
   INFO[0005] Sending successful results                    analyzer=trivy app=kubeclarity
   INFO[0005] Got result for job "trivy"                    app=kubeclarity
   INFO[0005] Got result for job "syft"                     app=kubeclarity
   INFO[0005] Got result for job "gomod"                    app=kubeclarity
   INFO[0005] Skip generating hash in the case of image   
   

Vulnerability scan

You can scan vulnerabilities by running the appropriate commands. The CLI provides flexibility and automation capabilities for integrating vulnerability scanning into your existing workflows or CI/CD pipelines. The tool allows scanning an image, directory, file, or a previously generated SBOM.

Usage:

kubeclarity-cli scan <image/sbom/directory/file name> --input-type <sbom|dir|file|image(default)> -f <output file>

Example:

kubeclarity-cli scan nginx.sbom --input-type sbom

You can list the vulnerability scanners to use using the SCANNERS_LIST environment variable separated by a space (SCANNERS_LIST="<Scanner1 name> <Scanner2 name>"). For example:

SCANNERS_LIST="grype trivy" kubeclarity-cli scan nginx.sbom --input-type sbom

Example output:

INFO[0000] Called trivy scanner on source sbom nginx.sbom  app=kubeclarity scanner=trivy
INFO[0000] Loading DB. update=true                       app=kubeclarity mode=local scanner=grype
INFO[0000] Need to update DB                             app=kubeclarity scanner=trivy
INFO[0000] DB Repository: ghcr.io/aquasecurity/trivy-db  app=kubeclarity scanner=trivy
INFO[0000] Downloading DB...                             app=kubeclarity scanner=trivy
INFO[0010] Gathering packages for source sbom:nginx.sbom  app=kubeclarity mode=local scanner=grype
INFO[0010] Found 136 vulnerabilities                     app=kubeclarity mode=local scanner=grype
INFO[0011] Sending successful results                    app=kubeclarity mode=local scanner=grype
INFO[0011] Got result for job "grype"                    app=kubeclarity
INFO[0012] Vulnerability scanning is enabled             app=kubeclarity scanner=trivy
INFO[0012] Detected SBOM format: cyclonedx-json          app=kubeclarity scanner=trivy
INFO[0012] Detected OS: debian                           app=kubeclarity scanner=trivy
INFO[0012] Detecting Debian vulnerabilities...           app=kubeclarity scanner=trivy
INFO[0012] Number of language-specific files: 1          app=kubeclarity scanner=trivy
INFO[0012] Detecting jar vulnerabilities...              app=kubeclarity scanner=trivy
INFO[0012] Sending successful results                    app=kubeclarity scanner=trivy
INFO[0012] Found 136 vulnerabilities                     app=kubeclarity scanner=trivy
INFO[0012] Got result for job "trivy"                    app=kubeclarity
INFO[0012] Merging result from "grype"                   app=kubeclarity
INFO[0012] Merging result from "trivy"                   app=kubeclarity
NAME              INSTALLED                FIXED-IN  VULNERABILITY     SEVERITY    SCANNERS
curl              7.74.0-1.3+deb11u7                 CVE-2023-23914    CRITICAL    grype(*), trivy(*)
curl              7.74.0-1.3+deb11u7                 CVE-2023-27536    CRITICAL    grype(*), trivy(*)
libcurl4          7.74.0-1.3+deb11u7                 CVE-2023-27536    CRITICAL    grype(*), trivy(*)
libdb5.3          5.3.28+dfsg1-0.8                   CVE-2019-8457     CRITICAL    grype(*), trivy(*)
libcurl4          7.74.0-1.3+deb11u7                 CVE-2023-23914    CRITICAL    grype(*), trivy(*)
perl-base         5.32.1-4+deb11u2                   CVE-2023-31484    HIGH        grype(*), trivy(*)
libss2            1.46.2-2                           CVE-2022-1304     HIGH        grype(*), trivy(*)
bash              5.1-2+deb11u1                      CVE-2022-3715     HIGH        grype(*), trivy(*)

Export results to KubeClarity backend

To export the CLI results to the KubeClarity backend, complete the following steps.

1. To export CLI-generated results to the backend, from the left menu bar select Applications, then copy the ID from the KubeClarity UI. If your application is not listed yet, select + New Application, and create a new pod.

   

2. To export the generated SBOMs to a running KubeClarity backend pod, use the -e flag and the ID as the <application ID> value in the following command.

   BACKEND_HOST=<KubeClarity backend address> BACKEND_DISABLE_TLS=true kubeclarity-cli analyze <image> --application-id <application ID> -e -o <SBOM output file>
   

   For example:

   BACKEND_HOST=localhost:9999 BACKEND_DISABLE_TLS=true kubeclarity-cli analyze nginx:latest --application-id 23452f9c-6e31-5845-bf53-6566b81a2906 -e -o nginx.sbom
   

   Example output:

   INFO[0000] Called syft analyzer on source registry:nginx:latest  analyzer=syft app=kubeclarity
   INFO[0004] Called trivy analyzer on source image nginx:latest  analyzer=trivy app=kubeclarity
   INFO[0004] Skipping analyze unsupported source type: image  analyzer=gomod app=kubeclarity
   INFO[0004] Sending successful results                    analyzer=syft app=kubeclarity
   INFO[0004] Got result for job "syft"                     app=kubeclarity
   INFO[0004] Got result for job "gomod"                    app=kubeclarity
   INFO[0004] Sending successful results                    analyzer=trivy app=kubeclarity
   INFO[0004] Got result for job "trivy"                    app=kubeclarity
   INFO[0004] Skip generating hash in the case of image    
   INFO[0004] Exporting analysis results to the backend: localhost:8080  app=kubeclarity
   

3. To export the vulnerability scan results to the KubeClarity backend, set the BACKEND_HOST environment variable and the -e flag.

   Note: Until TLS is supported, set BACKEND_DISABLE_TLS=true.

   BACKEND_HOST=<KubeClarity backend address> BACKEND_DISABLE_TLS=true kubeclarity-cli scan <image> --application-id <application ID> -e
   

   For example:

   SCANNERS_LIST="grype" BACKEND_HOST=localhost:9999 BACKEND_DISABLE_TLS=true kubeclarity-cli scan nginx.sbom --input-type sbom  --application-id 23452f9c-6e31-5845-bf53-6566b81a2906 -e
   

4. Now you can see the exported results on the UI, for example, on the Dashboard page.

Next step

Now that you have finished the getting started guide, explore the UI, or check the documentation for other use cases.

8.2.4 - Generate SBOM

A software bill of materials (SBOM) is a list of all the components, libraries, and other dependencies that make up a software application, along with information about the versions, licenses, and vulnerabilities associated with each component. They are formal, structured documents detailing the components of a software product and its supply chain relationships.

KubeClarity exposes SBOM generator integration settings via the values.yaml file.

OpenClarity content analyzer integrates with the following SBOM generators:

• Syft
• Cyclonedx-gomod
• Trivy

Trivy has an extensive vulnerability database, which includes CVEs from various sources such as NVD, Red Hat, and Debian. It can detect vulnerabilities in multiple programming languages, including Java, Python, and Ruby.

Syft’s vulnerability database is smaller and primarily focuses on detecting vulnerabilities in Python libraries.

KubeClarity, by default, enables Syft and CycloneDX gomod analyzers. To enable the Trivy scanner, edit the values. yaml file like this:

    analyzer:
      ## Space separated list of analyzers. (syft gomod)
      analyzerList: "syft gomod trivy"
      analyzerScope: "squashed"

      trivy:
        ## Enable trivy scanner, if true make sure to add it to list above  
        enabled: true
        timeout: "300"

SBOM database

KubeClarity automatically deploys an SBOM database pod and caches the generated SBOMs in the SBOM DB. The database is a lightweight SQLite DB that avoids persistent volume storage overheads. It stores and retrieves SBOM documents in a string format and serves as a caching function for rendering SBOM data. The DB does not store or query JSON objects to parse or query the SBOMs. However, it supports a gzip compression and base64 encoded storage to reduce memory footprint.

Here is the corresponding configuration snippet from the values.yaml file:

## KubeClarity SBOM DB Values

kubeclarity-sbom-db:
  ## Docker Image values.
  docker:
    ## Use to overwrite the global docker params
    ##
    imageName: ""

  ## Logging level (debug, info, warning, error, fatal, panic).
  logLevel: warning

  servicePort: 8080

  resources:
    requests:
      memory: "20Mi"
      cpu: "10m"
    limits:
      memory: "100Mi"
      cpu: "100m"

## End of KubeClarity SBOM DB Values

8.2.4.1 - Generate SBOM

To generate the Software Bill of Materials (SBOM), complete the following steps.

1. Run the following command.

   kubeclarity-cli analyze <image/directory name> --input-type <dir|file|image(default)> -o <output file or stdout>
   

   For example:

   kubeclarity-cli analyze --input-type image nginx:latest -o nginx.sbom
   

   Example output:

   INFO[0000] Called syft analyzer on source registry:nginx:latest  analyzer=syft app=kubeclarity
   INFO[0004] Skipping analyze unsupported source type: image  analyzer=gomod app=kubeclarity
   INFO[0004] Sending successful results                    analyzer=syft app=kubeclarity
   INFO[0004] Got result for job "syft"                     app=kubeclarity
   INFO[0004] Got result for job "gomod"                    app=kubeclarity
   INFO[0004] Skip generating hash in the case of image    
   

2. Verify that the ngnix.sbom file is generated and explore its contents as in below:

   head ngnix.sbom
   

   Example output:

   {
     "bomFormat": "CycloneDX",
     "specVersion": "1.4",
     "serialNumber": "urn:uuid:8cca2aa3-1aaa-4e8c-9d44-08e88b1df50d",
     "version": 1,
     "metadata": {
       "timestamp": "2023-05-19T16:27:27-07:00",
       "tools": [
         {
           "vendor": "kubeclarity",
   

3. To run also the trivy scanner and merge the output into a single SBOM, run:

   ANALYZER_LIST="syft gomod trivy" kubeclarity-cli analyze --input-type image nginx:latest -o nginx.sbom
   

   Example output:

   INFO[0000] Called syft analyzer on source registry:nginx:latest  analyzer=syft app=kubeclarity
   INFO[0004] Called trivy analyzer on source image nginx:latest  analyzer=trivy app=kubeclarity
   INFO[0004] Skipping analyze unsupported source type: image  analyzer=gomod app=kubeclarity
   INFO[0005] Sending successful results                    analyzer=syft app=kubeclarity
   INFO[0005] Sending successful results                    analyzer=trivy app=kubeclarity
   INFO[0005] Got result for job "trivy"                    app=kubeclarity
   INFO[0005] Got result for job "syft"                     app=kubeclarity
   INFO[0005] Got result for job "gomod"                    app=kubeclarity
   INFO[0005] Skip generating hash in the case of image