Analysing Kubernetes policy enforcement using OPA Gatekeeper and Kyverno

Policy and Governance in Kubernetes:

In brief, policy and governance provide a set of rules which define a guideline that either can be enforced or audited. In the kubernetes ecosystem the governance model plays a crucial role in managing and organising the infrastructure, security and consistency of the resources and deployments.Kubernetes provides Role-based Access Control (RBAC) which is an authorization mechanism for creating and managing permission around Kubernetes resources. But RBAC doesn’t allow admins to control the specification of the kubernetes resources. By controlling these specifications could help the administrators to be able to define the policy boundaries and limits.

These limitations could be overcome by using Kubernetes native Policy management and Governance tools. Advantages of using these policy and governance tools:

• Restrict running pods as the root user
• Required labels to group resources
• Provide resource requests and limits
• Whitelisting of trusted container registries and images
• Permitting the publicly exposed Load-Balancer services
• Restrict Updates and Deletes
• Restrict deprecated resources and a lot more..

OPA Gatekeeper:

• OPA Gatekeeper is a customizable kubernetes admission webhook that helps to enforce custom policies on any kubernetes object at its creation time. It is also a CNCF graduated open source project.

Architecture:

OPA gatekeeper is undoubtedly the most widely adopted tool for implementing Kubernetes policies. OPA uses a “policy-as-code” approach by decoupling the policy from the code and combining the policy enforcement. OPA uses a Domain Specific Language (DSL) called Rego to describe its policies. Gatekeeper policies run with the help of kubernetes admission controller webhooks which are triggered whenever a resource is created, updated or deleted inside a kubernetes cluster. In general Gatekeeper is a validating (mutating TBA as per official doc) webhook that enforces CRD-based policies executed by Open Policy Agent.

Kyverno:

Kyverno (meaning “govern” in Greek) is a policy engine that is specifically designed only for Kubernetes. Kyverno works by using a dynamic admission controller that verifies each and every request you send via Kubectl to the Kube API server. Then it will match the request with the policy and take appropriate action (mutate, validate, etc..) accordingly. Also, Kyverno is an open-source and a part of CNCF Sandbox Project.

Architecture:

In Kyverno, policies are managed as Kubernetes resources and no new language (yaml) is required to write policies. This allows using familiar tools such as kubectl, git, and kustomize to manage policies. Kyverno policies can validate, mutate, and generate Kubernetes resources. Also, Kyverno can be used to scan the existing workloads in search of best practices or any regulation/compliance standards. The Kyverno CLI can be used to test policies and validate resources as part of a CI/CD pipeline.

Side by Side Analysis:

Analysing Real Time Examples:

Here we use a scenario to compare both OPA Gatekeeper and Kyverno. We are using a policy that will not allow you to create a namespace that doesn’t have a specific label.

OPA Gatekeeper:

OPA Gatekeeper has been already deployed on the AKS Cluster. The OPA gatekeeper will have an audit and controller manager deployed, audit functionality will enable periodic evaluations of replicated resources against the Constraints enforced in the cluster to detect any existing misconfigurations.

• Deploying an OPA gatekeeper policy, the OPA gatekeeper uses OPA Constraint framework to describe and enforce the policy. OPA policies have a constraint and a constraint template, you must define the constraint template first. The constraint template describes both the Rego that enforces the constraint and the schema of the constraint.
• Constraint Template that restricts the creation of namespaces that don’t have a label “gatekeeper‘.

$ kubectl apply -f k8srequiredlabels_template.yaml

constrainttemplate.templates.gatekeeper.sh/k8srequiredlabels created

k8srequiredlabels_template.yaml

apiVersion: templates.gatekeeper.sh/v1

kind: ConstraintTemplate

metadata:

  name: k8srequiredlabels

spec:

  crd:

    spec:

      names:

        kind: K8sRequiredLabels

      validation:

        # Schema for the `parameters` field

        openAPIV3Schema:

          type: object

          properties:

            labels:

              type: array

              items:

                type: string

  targets:

    – target: admission.k8s.gatekeeper.sh

      rego: |

        package k8srequiredlabels

        violation[{“msg”: msg, “details”: {“missing_labels”: missing}}] {

          provided := {label | input.review.object.metadata.labels[label]}

          required := {label | label := input.parameters.labels[_]}

          missing := required – provided

          count(missing) > 0

          msg := sprintf(“you must provide labels: %v”, [missing])

        }

Constraint CRD:

$ kubectl apply -f all_ns_must_have_gatekeeper.yaml  

 k8srequiredlabels.constraints.gatekeeper.sh/ns-must-have-gk created

apiVersion: constraints.gatekeeper.sh/v1beta1

kind: K8sUniqueLabel 

metadata:

  name: ns-gk-label-unique

spec:

  match:

    kinds:

      – apiGroups: [“”]

        kinds: [“Namespace”]

  parameters:

    label: gatekeeper

• Listing the constraints that are currently active.
• Testing the OPA Gatekeeper by creating a namespace without the label.
• Deploying a namespace that have a label “gatekeeper”
  

  Kyverno:

  Kyverno has already been installed on the cluster and it will run as a Dynamic Admission Controller. Kyverno verifies the api requests and applies matching policies that are intended to enforce admission policies or reject requests with the help of validating and mutating webhook controllers.

  • Listing all the Kubernetes resources that are being deployed with the Kyverno.
  • Getting the validating and mutating webhook configurations.
    • Kyverno policies are a collection of rules which consist of a match and exclude(optional) declaration, and one of a validate, mutate, generate, or verify Imagesdeclaration. Here, applying a Kyverno policy that is intended to reject the namespaces that don’t have a label “Kyverno”.

    $ kubectl apply -f all_ns_must_have_kyverno.yaml                        

    clusterpolicy.kyverno.io/require-labels created

    apiVersion: kyverno.io/v1

    kind: ClusterPolicy

    metadata:

    name: require-labels

      annotations:

        policies.kyverno.io/title: Require Labels

        policies.kyverno.io/category: Best Practices

        policies.kyverno.io/severity: medium

        policies.kyverno.io/subject: Namespace, Label

        policies.kyverno.io/description: >-

          Define and use labels that identify semantic attributes of your application or Deployment.

          A common set of labels allows tools to work collaboratively, describing objects in a common manner that

          all tools can understand. The recommended labels describe applications in a way that can be

          queried. This policy validates that the label `kyverno` is specified with some value.      

    spec:

      validationFailureAction: enforce

      background: true

      rules:

      – name: check-for-labels

        match:

          resources:

            kinds:

            – Namespace

        validate:s

          message: “The label `kyverno` is required. example: `(kyverno: value)`”

          pattern:

            metadata:

              labels:

                kyverno: “?*”

    • Listing all the resource types of Kyverno
    • Listing the cluster policies that are active, here you can find the already deployed policy.
    • Testing the policy by creating a namespace without any labels.
    • Now checking with a matching scenario, deploying a namespace with the label “kyverno”.
      

      Conclusion:

      Both OPA Gatekeeper and Kyverno are great policy-as-code tools that are potent in implementing best security and governance at an organisation. Each of these tools have its own advantages, OPA Gatekeeper can handle very complex policies because of the usage of its own DSL (Rego). As it has predefined libraries, it is easy to write the policies from the scratch. Likewise, Kyverno is flexible and there is no need for learning extra DSLs. So the one who knows Kubernetes YAML can write and understand policies easily. Kyverno has good mutation ability and HA support.

      Looking for the help to implement a good Devops strategy? Please reach out to us.