OpenShift and Kubernetes Compliance Operator

Introduction

Discover the power of the Compliance Operator, a security tool that ensures adherence to security compliance rules in OpenShift and Kubernetes clusters. Learn how to deploy this operator from Red Hat Advanced Cluster Management, automate compliance scanning, and generate insightful reports for effective monitoring and assessment.

So let’s start!

How to deploy the Compliance Operator from Red Hat Advanced Cluster Management to a set of Managed clusters

The Compliance Operator is an operator that runs OpenSCAP and allows you to keep your Red Hat OpenShift Container Platform cluster compliant with the security benchmark that you need.

You can install the Compliance Operator on your managed cluster by using the Compliance Operator policy.

On the Governance section of RHACM Create policy

After that create the following policy

# This policy verifies the installation of the official & supported version of
# the Compliance Operator on the managed clusters.
#
# If set to "enforce" it'll install the operator.
#
# Note that OpenShift 4.6 is required.
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  name: policy-comp-operator
  annotations:
    policy.open-cluster-management.io/standards: NIST SP 800-53
    policy.open-cluster-management.io/categories: CA Security Assessment and Authorization
    policy.open-cluster-management.io/controls: CA-2 Security Assessments, CA-7 Continuous Monitoring
spec:
  remediationAction: inform
  disabled: false
  policy-templates:
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: comp-operator-ns
        spec:
          remediationAction: inform # will be overridden by remediationAction in parent policy
          severity: high
          object-templates:
            - complianceType: musthave
              objectDefinition:
                apiVersion: v1
                kind: Namespace
                metadata:
                  name: openshift-compliance
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: comp-operator-operator-group
        spec:
          remediationAction: inform # will be overridden by remediationAction in parent policy
          severity: high
          object-templates:
            - complianceType: musthave
              objectDefinition:
                apiVersion: operators.coreos.com/v1
                kind: OperatorGroup
                metadata:
                  name: compliance-operator
                  namespace: openshift-compliance
                spec:
                  targetNamespaces:
                    - openshift-compliance
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: comp-operator-subscription
        spec:
          remediationAction: inform  # will be overridden by remediationAction in parent policy
          severity: high
          object-templates:
            - complianceType: musthave
              objectDefinition:
                apiVersion: operators.coreos.com/v1alpha1
                kind: Subscription
                metadata:
                  name: compliance-operator
                  namespace: openshift-compliance
                spec:
                  installPlanApproval: Automatic
                  name: compliance-operator
                  source: redhat-operators
                  sourceNamespace: openshift-marketplace
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-policy-comp-operator
placementRef:
  name: placement-policy-comp-operator
  kind: PlacementRule
  apiGroup: apps.open-cluster-management.io
subjects:
  - name: policy-comp-operator
    kind: Policy
    apiGroup: policy.open-cluster-management.io
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-policy-comp-operator
spec:
  clusterConditions:
    - status: "True"
      type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
      - {key: name, operator: In, values: ["f90a8d71a1"]}

Such policy will create the following resources on the managed cluster(s)
- A compliance operator namespace (openshift-compliance) for the operator installation
- An operator group (compliance-operator) to specify the target namespace
- A subscription (comp-operator-subscription) to reference the name and channel. The subscription pulls the profile, as a container, that it supports
Use the following settings
Once the policy is created it will inform whether the deployment has been done successfully on the selected Managed clusters

And it will enforce the Operator deployment in the selected cluster

How to automate compliance scanning over the cluster?

The ScanSetting and ScanSettingBinding APIs are recommended to run compliance scans with the Compliance Operator and use the Compliance Operator deployment for increment security in the complete cluster.

ScanSetting

So in order to create such a scanning configuration we start checking the ScanSetting “TAB”.

In there you can find 2 default configurations:

default: only provides scanning
default-auto-apply: will apply all automated remediations

In addition the ScanSetting points to the roles that will be scanned (workers and master by default)

How to automate compliance scanning over the cluster

For the current implementation, I will use the default configuration for demonstration purposes.

ScanSettingBindings

To initiate a scan, we need to create a ScanSettingBinding. In the ScanSettingBinding tab, simply click on “Create ScanSettingBinding,” and the operator creates a YAML file that defaults to using the rhcos-moderate profile, which will run against the masters and workers. However, we need to change that profile.

Privileged Containers

In order to fulfill the requirement for privileged container detection we need to use the following profile:

CIS Red Hat OpenShift Container Platform 4 Benchmark

YAML Representation

Finally, the configuration should look like this:

As soon as you create and save this binding, the system will initiate a scan. If you look on the ComplianceScan tab. You’ll see the status of each scan normally as either running, aggregating, or done.

How to generate reports out of the tool in order to monitor and assess the monitoring?

There are multiple ways to check the results initially we will use the OpenShift CLI.

OpenShift CLI report

List all compliance results

oc get compliancecheckresult -n openshift-compliance

As a result, you will have a list of all rules and outcome

NAME                                                               STATUS           SEVERITY
ocp4-cis-accounts-restrict-service-account-tokens                  MANUAL           medium
ocp4-cis-accounts-unique-service-account                           MANUAL           medium
ocp4-cis-api-server-admission-control-plugin-alwaysadmit           PASS             medium
ocp4-cis-api-server-admission-control-plugin-alwayspullimages      PASS             high
ocp4-cis-api-server-admission-control-plugin-namespacelifecycle    PASS             medium
ocp4-cis-api-server-admission-control-plugin-noderestriction       PASS             medium
ocp4-cis-api-server-admission-control-plugin-scc                   PASS             medium
ocp4-cis-api-server-admission-control-plugin-securitycontextdeny   PASS             medium
ocp4-cis-api-server-admission-control-plugin-serviceaccount        PASS             medium
...
ocp4-cis-scc-drop-container-capabilities                           MANUAL           medium
ocp4-cis-scc-limit-container-allowed-capabilities                  PASS             medium
ocp4-cis-scc-limit-ipc-namespace                                   MANUAL           medium
ocp4-cis-scc-limit-net-raw-capability                              MANUAL           medium
ocp4-cis-scc-limit-network-namespace                               MANUAL           medium
ocp4-cis-scc-limit-privilege-escalation                            MANUAL           medium
ocp4-cis-scc-limit-privileged-containers                           MANUAL           medium
ocp4-cis-scc-limit-process-id-namespace                            MANUAL           medium
ocp4-cis-scc-limit-root-containers                                 MANUAL           medium

You can also observe the possible automatic remediation of possible compliance-detected issues.

Asset Reporting Format (ARF)

The results from those scans are stored in a persistent volume claim (PVC), one for each scan type. If you go to the openshift-compliance project in the console, you’ll see that it has one PVC assigned to it:

In order to pull the files the scan generated, fetch the raw results by spawning a pod that mounts the volume and copying the results

apiVersion: "v1"
kind: Pod
metadata:
  name: pv-extract
spec:
  containers:
    - name: pv-extract-pod
      image: registry.access.redhat.com/ubi8/ubi
      command: ["sleep", "3000"]
      volumeMounts:
      - mountPath: "/workers-scan-results"
        name: workers-scan-vol
  volumes:
    - name: workers-scan-vol
      persistentVolumeClaim:
        claimName: ocp4-csi

Create the POD

oc apply -f pv-extract-compliance.yaml

Extract the results

oc cp pv-extract:/workers-scan-results .

Don’t forget to delete the POD

oc delete pod pv-extract

OpenSCAP Evaluation Report

How to transform that output into an OpenSCAP Evaluation Report

sudo yum install -y openscap openscap-utils scap-security-guide

Generate report

oscap xccdf generate report ocp4-cis-api-checks-pod.xml.bzip2 > csi_check.html

Final Thoughts

Imagine a world where compliance and security go hand in hand. In this blog post, we delve into the deployment of the powerful Compliance Operator from Red Hat Advanced Cluster Management to managed clusters. Discover how to automate compliance scanning and generate insightful reports for monitoring and assessment. Uncover the secrets of deploying the Compliance Operator using policies and harness the potential of ScanSetting and ScanSettingBinding APIs. With OpenShift CLI and Asset Reporting Format (ARF), you gain the ability to effortlessly navigate the compliance landscape. Embrace default configurations and privileged containers for a secure and efficient cluster environment. Let us embark on this journey of compliance and discover a realm of possibilities. And remember how to create a strong application on top of it.