Open Policy Agent with Kubernetes - Tutorial (Pt. 2)
Published: 2023-02-22 16:26:15

In my previous articles, we discussed what Policy-as-Code is, why we need it, and how to use the Open Policy Agent (OPA) tool. If you haven't read the introduction yet, please take some time to read it first:

[What is Policy-as-Code? An Introduction to Open Policy Agent](https://blog.gitguardian.com/what-is-policy-as-code-an-introduction-to-open-policy-agent/)

Following the OPA introduc how to use OPA to enforce policies inside a Kubernetes cluster. Here's the link to the first part of the tutorial:

[Open Policy Agent with Kubernetes - Tutorial (Pt. 1)](https://blog.gitguardian.com/open-policy-agent-with-kubernetes-tutorial-pt-1/)

**TL;DR**: we used OPA as the admission controller with the `kube-mgmt` sidecar enforcing ConfigMap-based policies.  
Today, we will look at another way of using OPA in a Kubernetes cluster: Gatekeeper - policy controller for Kubernetes.

---

## Gatekeeper Introduction

The OPA [Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is a project under the OPA umbrella. Although I did not mention Gatekeeper in the previous tutorial, the technique I described there (using OPA with its sidecar `kube-mgmt`) is also referred to as **Gatekeeper v1.0**.

Today, we will get familiar with **Gatekeeper v3** (hereafter: just "Gatekeeper", omitting the "v3" part), which builds a Kubernetes admission controller around the policy engine to integrate OPA and the Kubernetes API service.

### CRD-Based Policies

Gatekeeper's most significant value is the ability to configure OPA policies dynamically using Gatekeeper's **Custom Resource Definitions (CRDs)**. [Custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) are extensions of the Kubernetes API that allows for the customization of a Kubernetes installation.

CRD-based policies allow for a deeper integration of OPA within the Kubernetes ecosystem: it enables the creation of policy templates for Rego policies, the creation of policies as CRDs, and the storage of audit results on policy CRDs.

### How is Gatekeeper different from OPA?

Compared to using OPA with its sidecar `kube-mgmt` (aka Gatekeeper v1.0), Gatekeeper introduces the following functionality:

-   An extensible, parameterized policy library
-   Native Kubernetes CRDs for instantiating the policy library (aka "constraints")
-   Native Kubernetes CRDs for extending the policy library (aka "constraint templates")
-   Native Kubernetes CRDs for mutation support
-   Audit functionality
-   External data support

This may sound a bit too abstract, so let's get down to the nitty-gritty of how Gatekeeper works.

---

## Install Gatekeeper

First, let's start minikube:

```shell
minikube start
```

For this tutorial, we will deploy a released version of Gatekeeper in our minikube cluster with a prebuilt image:

```shell
kubectl apply -f https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper.yaml
```

> Note: it's also possible to deploy it with Helm:

```shell
helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts 
helm install gatekeeper/gatekeeper --name-template=gatekeeper --namespace gatekeeper-system --create-namespace
```

After the deployment, a new namespace `gatekeeper-system` will be created, and the following resources will be created:

```shell
tiexin@mbp ~ $ kubectl get deployments
NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
gatekeeper-audit                1/1     1            1           2m20s
gatekeeper-controller-manager   3/3     3            3           2m20s

tiexin@mbp ~ $ kubectl get services
NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
gatekeeper-webhook-service   ClusterIP   10.103.86.204   <none>        443/TCP   2m22s

tiexin@mbp ~ $ kubectl get crd
NAME                                                 CREATED AT
assign.mutations. Gatekeeper.sh                       2023-01-01T08:59:54Z
assignmetadata.mutations.gatekeeper.sh               2023-01-01T08:59:54Z
configs.config.gatekeeper.sh                         2023-01-01T08:59:55Z
constraintpodstatuses.status.gatekeeper.sh           2023-01-01T08:59:55Z
constrainttemplatepodstatuses.status.gatekeeper.sh   2023-01-01T08:59:55Z
constrainttemplates.templates.gatekeeper.sh          2023-01-01T08:59:55Z
expansiontemplate.expansion. Gatekeeper.sh            2023-01-01T08:59:55Z
modifyset.mutations. Gatekeeper.sh                    2023-01-01T08:59:55Z
mutatorpodstatuses.status. Gatekeeper.sh              2023-01-01T08:59:55Z
providers.externaldata. Gatekeeper.sh                 2023-01-01T08:59:55Z
```

---

## Gatekeeper Concepts: Constraints and Constraint Templates

Before moving on to the actual tutorial, let's have a look at two essential concepts of Gatekeeper with concrete examples: **constraints** and **constraint templates**.

In short, constraints use constraint templates to inform Gatekeeper what policies to be enforced and how.

I know this sounds a bit confusing, so let's have a look at an example:

The `ConstraintTemplate` example below contains Rego code which checks if a resource object has a label named "team":

File `constraint_template.yaml`:

```yaml
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: teamlabel
spec:
  crd:
    spec:
      names:
        kind: TeamLabel
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package teamlabel
 
        labels := input.review.object.metadata.labels
 
        has_team {
          labels.team
        }
 
        violation[{"msg": msg}] {
          not has_team
          msg := "You should have the team label"
        }
```

The `ConstraintTemplate` object above doesn't trigger policy enforcement on its own. However, it creates a new custom resource in our cluster of the type `TeamLabel`. If we want to enforce our `TeamLabel` policy, we create a constraint by using that new resource type:

File `constraint.yaml`:

```yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: TeamLabel
metadata:
  name: teampods
spec:
  match:
    kinds:
    - apiGroups: [""]
      kinds: ["Pod"]
    excludedNamespaces:
    - kube-system
  parameters: {}
```

This constraint uses the TeamLabel constraint template above to let Gatekeeper enforce our TeamLabel policy for all pods not in the `kube-system` namespace.

The Gatekeeper service also continually and constantly monitors and audits existing cluster objects to detect policy violations.

---

## How to Use Gatekeeper in Kubernetes

Create files `constraint_template.yaml` and `constraint.yaml` with the content in the previous section, and apply them:

```bash
kubectl apply -f constraint_template.yaml
kubectl apply -f constraint.yaml
```

Now let's deploy a simple pod into the default namespace as a test:

```bash
kubectl apply -n default -f https://k8s.io/examples/pods/simple-pod.yaml
```

We will get the following error:

```shell
Error from server (Forbidden): error when creating "https://k8s.io/examples/pods/simple-pod.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [teampods] You should have the team label
```

Let's get in compliance and add a "team" label to the simple pod:

File `simple-pod.yaml`:

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    team: test
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
```



Create this file with the above content and apply it:

```bash
kubectl apply -n default -f simple-pod.yaml
```

We should encounter no error, and the pod will be created successfully:

```bash
tiexin@mbp ~ $ kubectl get pods -n default
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          34s
```

---

## How to Scale OPA?

So far, we have demonstrated a simple use case of OPA/Kubernetes integration with Gatekeeper.

However, the policies can be much more complicated in real-world scenarios, and you will have multiple policies. How does OPA work at a much larger scale?

Let's talk about three crucial topics on this subject matter:

-   repository structure
-   testing policies
-   other tools

### Repository Structure

A well-structured directory helps to make your code more manageable:

```bash
.
└── team-label-policy
    ├── README.md
    ├── constraint.yaml
    ├── constraint_template.yaml
    ├── simple-pod.yaml
    ├── src.rego
    └── src_test.rego
```

In the example above, we put everything that belongs to the team label policy under the folder "team-label-policy", which contains the core files:

-   `src.rego`: the OPA Rego code
-   `src_test.rego`: the corresponding test cases for our policy Rego
-   `constraint_template.yaml`: ConstraintTemplate for our policy. Note that this file also contains the code from `src.rego` inline, but the OPA tool cannot parse the manifest YAML, so we need to  the Rego code to a separate file for testing. If you use this layout for your policies, **you must remember to synchronize code changes between the two files**.
-   `constraint.yaml`: the manifest for a test of the constraint template
-   `simple-pod.yaml`: a minimalist pod definition to demonstrate the constraint in practice
-   `README.md`

### How to Test a Rego Policy

For essential use cases and complicated policies, we should also write tests for those policies.

When writing test coverage for your Gatekeeper policy, you want to consider the following points carefully:

-   What Kubernetes API resource fields do my policy query? Are any of them optional? Can they appear more than once in a spec?
-   How many positive test cases do I need to write to ensure my policy will do what I expect?
-   How many negative test cases do I need to write to ensure my policy will not produce results I do not want?

Policy tests are also written in Rego.

By convention, they live in the same directory as the source file. In our case, we can  the policy from the `constraint_template.yaml` file into `src.rego` and write tests in the file `src_test.rego`.

Note the matching package name at the top of each file:

File `src.rego`:

```yaml
package teamlabel

# copied from file constraint_template.yaml

labels := input.review.object.metadata.labels

has_team {
  labels.team
}

violation[{"msg": msg}] {
  not has_team
  msg := "You should have the team label"
}
```

File `src_test.rego`:

```yaml
package teamlabel
import future.keywords

test_pod_allowed if {
  results := violation with input as {"review": {"object": {"metadata": {"labels": { "team": "test" }}}}}
  count(results) == 0
}

test_pod_denied if {
  results := violation with input as {"review": {"object": {"metadata": {"labels": {}}}}}
  count(results) > 0
}
```

Test method names should always begin with the prefix `test_`.

We can use the OPA command-line tool to evaluate our tests:

```bash
tiexin@mbp ~ $ opa test --explain fails src.rego src_test.rego
src_test.rego:
data.teamlabel.test_pod_allowed: PASS (7.852791ms)
data.teamlabel.test_pod_denied: PASS (301.75µs)
--------------------------------------------------------------------------------
PASS: 2/2
```

If you haven't installed the OPA CLI tool, follow the instructions [here](https://blog.gitguardian.com/what-is-policy-as-code-an-introduction-to-open-policy-agent/).

### Helpful Tools

There are some exciting tools (for example, [this one](https://www.styra.com/styra-das/)) available that can help integrate OPA with your systems and provide ways of writing policies and deploying them across your infrastructure, as well as tools for unit testing, monitoring policy usage, and more. But, even without them, OPA can be managed at a larger scale.

---

## Going from Here

In this 2-part OPA tutorial mini-series, we did two demos on integrating OPA with Kubernetes. If you are interested in OPA and OPA/Kubernetes integration, here is some more reading material for you:

-   [A more complicated real-world example](https://cloud.redhat.com/blog/better-kubernetes-security-with-open-policy-agent-opa-part-2) with [code](https://github.com/stackrox/blog-examples/tree/master/code/opa-gatekeeper-taint-tolerations)
-   [Open Policy Agent Docs](https://www.openpolicyagent.org/docs/latest/)
-   [Gatekeeper Docs](https://open-policy-agent.github.io/gatekeeper/website/docs/howto/)

If you like the OPA introduction and the tutorials, please like, comment, and subscribe. See you in the next one!