Blog Image
Analysing Kubernetes Policy Enforcement Using OPA Gatekeeper & Kyverno
10 October 2022

Analysing Kubernetes policy enforcement using OPA Gatekeeper and Kyverno

Policy and Governance in Kubernetes:

In brief, policy and governance provide a set of rules which define a guideline that either can be enforced or audited. In the kubernetes ecosystem the governance model plays a crucial role in managing and organising the infrastructure, security and consistency of the resources and deployments.Kubernetes provides Role-based Access Control (RBAC) which is an authorization mechanism for creating and managing permission around Kubernetes resources. But RBAC doesn’t allow admins to control the specification of the kubernetes resources. By controlling these specifications could help the administrators to be able to define the policy boundaries and limits.

These limitations could be overcome by using Kubernetes native Policy management and Governance tools. Advantages of using these policy and governance tools:

  • Restrict running pods as the root user
  • Required labels to group resources
  • Provide resource requests and limits
  • Whitelisting of trusted container registries and images
  • Permitting the publicly exposed Load-Balancer services
  • Restrict Updates and Deletes
  • Restrict deprecated resources and a lot more..
OPA Gatekeeper:

OPA Gatekeeper is a customizable kubernetes admission webhook that helps to enforce custom policies on any kubernetes object at its creation time. It is also a CNCF graduated open source project.

Architecture:

OPA gatekeeper is undoubtedly the most widely adopted tool for implementing Kubernetes policies. OPA uses a “policy-as-code” approach by decoupling the policy from the code and combining the policy enforcement. OPA uses a Domain Specific Language (DSL) called Rego to describe its policies. Gatekeeper policies run with the help of kubernetes admission controller webhooks which are triggered whenever a resource is created, updated or deleted inside a kubernetes cluster. In general Gatekeeper is a validating (mutating TBA as per official doc) webhook that enforces CRD-based policies executed by Open Policy Agent.

Kyverno:

Kyverno (meaning “govern” in Greek) is a policy engine that is specifically designed only for Kubernetes. Kyverno works by using a dynamic admission controller that verifies each and every request you send via Kubectl to the Kube API server. Then it will match the request with the policy and take appropriate action (mutate, validate, etc..) accordingly. Also, Kyverno is an open-source and a part of CNCF Sandbox Project.

Architecture:

In Kyverno, policies are managed as Kubernetes resources and no new language (yaml) is required to write policies. This allows using familiar tools such as kubectl, git, and kustomize to manage policies. Kyverno policies can validate, mutate, and generate Kubernetes resources. Also, Kyverno can be used to scan the existing workloads in search of best practices or any regulation/compliance standards. The Kyverno CLI can be used to test policies and validate resources as part of a CI/CD pipeline.

Side by Side Analysis:

OPA Gatekeeper Kyverno
Advantages
  • Capable of handling very complex policy
  • CNCF graduated and more established in the community
  • Supports multiple replicas for availability and scalability
  • Easiness in writing policies
  • Excellent mutation abilities
  • Supports multiple replicas for availability and scalability
  • Generation feature and syncing ability is unique and allows new use cases
  • Quick validation and high degree of flexibility
Disadvantages
  • Requires to learn a new programming language
  • Mutation ability is in nascent stage
  • Limited to validation use cases
  • Policy is complex and often lengthy
  • Highly complex policy may not be possible since no programming language is integrated
  • Young, still largely unproven due to lower establishments

Analysing Real Time Examples:

Here we use a scenario to compare both OPA Gatekeeper and Kyverno. We are using a policy that will not allow you to create a namespace that doesn’t have a specific label.

OPA Gatekeeper:
  • OPA Gatekeeper has been already deployed on the AKS Cluster. The OPA gatekeeper will have an audit and controller manager deployed, audit functionality will enable periodic evaluations of replicated resources against the Constraints enforced in the cluster to detect any existing misconfigurations.

  • Deploying an OPA gatekeeper policy, the OPA gatekeeper uses OPA Constraint framework to describe and enforce the policy. OPA policies have a constraint and a constraint template, you must define the constraint template first. The constraint template describes both the Rego that enforces the constraint and the schema of the constraint.
  • Constraint Template that restricts the creation of namespaces that don’t have a label “gatekeeper‘.
$ kubectl apply -f k8srequiredlabels_template.yaml
constrainttemplate.templates.gatekeeper.sh/k8srequiredlabels created

k8srequiredlabels_template.yaml

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels
        violation[{"msg": msg, "details": {"missing_labels": missing}}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("you must provide labels: %v", [missing])
        }

Constraint CRD:

$ kubectl apply -f all_ns_must_have_gatekeeper.yaml  
 k8srequiredlabels.constraints.gatekeeper.sh/ns-must-have-gk created
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sUniqueLabel 
metadata:
  name: ns-gk-label-unique
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    label: gatekeeper
  • Listing the constraints that are currently active.

  • Testing the OPA Gatekeeper by creating a namespace without the label.

  • Deploying a namespace that have a label “gatekeeper”

Kyverno:

Kyverno has already been installed on the cluster and it will run as a Dynamic Admission Controller. Kyverno verifies the api requests and applies matching policies that are intended to enforce admission policies or reject requests with the help of validating and mutating webhook controllers.

  • Listing all the Kubernetes resources that are being deployed with the Kyverno.

  • Getting the validating and mutating webhook configurations.

  • Kyverno policies are a collection of rules which consist of a match and exclude(optional) declaration, and one of a validate, mutate, generate, or verify Imagesdeclaration. Here, applying a Kyverno policy that is intended to reject the namespaces that don’t have a label “Kyverno”.
$ kubectl apply -f all_ns_must_have_kyverno.yaml                        
clusterpolicy.kyverno.io/require-labels created
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-labels
  annotations:
    policies.kyverno.io/title: Require Labels
    policies.kyverno.io/category: Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Namespace, Label
    policies.kyverno.io/description: >-
      Define and use labels that identify semantic attributes of your application or Deployment.
      A common set of labels allows tools to work collaboratively, describing objects in a common manner that
      all tools can understand. The recommended labels describe applications in a way that can be
      queried. This policy validates that the label `kyverno` is specified with some value.      
spec:
  validationFailureAction: enforce
  background: true
  rules:
  - name: check-for-labels
    match:
      resources:
        kinds:
        - Namespace
    validate:s
      message: "The label `kyverno` is required. example: `(kyverno: value)`"
      pattern:
        metadata:
          labels:
            kyverno: "?*"
  • Listing all the resource types of Kyverno

  • Listing the cluster policies that are active, here you can find the already deployed policy.

  • Testing the policy by creating a namespace without any labels.

  • Now checking with a matching scenario, deploying a namespace with the label “kyverno”.

Conclusion:

Both OPA Gatekeeper and Kyverno are great policy-as-code tools that are potent in implementing best security and governance at an organisation. Each of these tools have its own advantages, OPA Gatekeeper can handle very complex policies because of the usage of its own DSL (Rego). As it has predefined libraries, it is easy to write the policies from the scratch. Likewise, Kyverno is flexible and there is no need for learning extra DSLs. So the one who knows Kubernetes YAML can write and understand policies easily. Kyverno has good mutation ability and HA support.

Looking for the help to implement a good Devops strategy? Please reach out to us.