Policy enforced deployments for your Kubernetes resources
Tags: conftest, kubernetes, github
Published: 2022-02-14 09:10:21

---
title: Policy enforced deployments for your Kubernetes resources
published: true
date: 2022-02-13 00:00:00 UTC
tags: conftest, kubernetes, github
canonical_url: https://www.visualstudiogeeks.com/azure/kubernetes/devops/policy-enforcement-for-k8s
---

As your team starts to deploy resources to Kubernetes regularly, it becomes necessary for you as a cluster administrator to maintain good standards and consistency of the Kubernetes resources. Be it, ensuring all the resources have set of labels, or ensuring you only pull images from your enterprise container registry. [Gatekeeper](https://github.com/open-policy-agent/gatekeeper) is a well known policy enforcement tool using [Open Policy Agent (OPA)](https://www.openpolicyagent.org) - which is a opensource, Cloud Native Computing Foundation (CNCF) project.

<!-- more -->

However, Gatekeeper is installed on the cluster and thus ensures no policy is broken at deployment time. This means that any validation of policies happen only when you are trying to deploy resources to cluster. While this ensures that no resource violates the policy, you would like to know about these policies much earlier in your CI/CD pipeline. Doing policy validations much to the left of your deployment pipeline ensures your deployment going smooth when necessary.

This is where [Conftest](https://www.conftest.dev) helps. Conftest relies on OPA and policies are written using [Rego](https://www.openpolicyagent.org/docs/latest/#rego) - thus the policies you write for Gatekeeper will be compatible with Conftest. But more importantly **with Conftest, you can validate your local manifests against OPA policies locally and ensure your resources are compliant before you deploy them**.

## Installation

Installation is really easy if you are on Mac - For other platforms refer to the [documentation](https://www.conftest.dev/install/)

```
brew install conftest
```

## Folder structure

By default, Conftest expects you to maintain your policies under `policy` folder at the same location as your Kubernetes resources. If you prefer a different path, you will want to pass it using CLI or set environment variable `CONFTEST_POLICY`.

```
📂 src
    📂 k8s
        📄 deployment.yml
        📄 service.yml
        📁 policy
            📄 replica.rego
            📄 labels.rego
    📂 app
        📄 main.ts
        📄 package.json
```

## Writing Policies

As mentioned previously, policies are written in [Rego](https://www.openpolicyagent.org/docs/latest/#rego). I struggled to write policies initially and constantly went back to documentation. However, once you write couple of policies, you will get a hang of it. Take a look at the simple policy to _check every deployment has at least 2 replicas_.

```
package main

deny_replicas[msg] {
    input.kind == "Deployment" # check if it is a Deployment
    input.spec.replicas < 2 # And the replicas are < 2
    msg := "Deployments must have 2 or more replicas" # show the error message and fail the test
}
```

`input` is the complete yaml document from our deployment yaml (see below) and we we are checking if `kind` is equal to `Deployment`. If its deployment, we move to next line in the constraint and check if `spec.replicas` is less that 2.

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mynodeapi-dep
  labels:
    app: dep-k8s-nodejs-api
spec:
  replicas: 1
  selector:
    ...
```

Lets see how we can test our `Deployment` resource using Conftest.

## Testing

The command to test resources using Conftest is `conftest test <PATH>`. Since I would like to test all resources under `k8s` folder, I pass folder path as below.

```
conftest test ./k8s
```

Running this you will see the output as below (ignore other errors as I have other policies). The test failed because we have set `deny` rule if `spect.replicas < 2` and in our case our deployment yaml has `replicas: 1` (see `spec` section in the deployment yaml above).

![conftest error](https://www.visualstudiogeeks.com/images/screenshots/utkarsh/2022/feb/policy-enabled-deployments-for-k8s/conftest-error.png)
_Conftest failing due to policy violation_

## Using Conftest in GitHub Actions

Making Conftest work in your Continuous Integration (CI) process is simple. For demo purposes, I am using GitHub Actions in my repo [here](https://github.com/onlyutkarsh/k8s-nodejs-api). If you run the tests, you will see the action fails with errors - [see the output](https://github.com/onlyutkarsh/k8s-nodejs-api/runs/5173696729?check_suite_focus=true#step:4:6)

## Conclusion

As you can see, Conftest lets you validate and govern your Kubernetes resources efficiently and can easily be integrated with your CI workflows. This lets your team standardise the common practices, go through PR review process before eventually deploying to the cluster. Once deployed to cluster, you can use Gatekeeper to validate as well to full proof your workloads.