Kubernetes-101: Security concepts
Tags: kubernetes
Published: 2023-06-23 10:00:00

Security is in general a large and complex topic, and security in Kubernetes is no different. This is especially true if you host your own Kubernetes cluster. In this article I will go through some of the security related concepts that Kubernetes has to offer. Specifically I will will discuss:

- How we can restrict network traffic between Pods in our Kubernetes cluster using **NetworkPolicies**.
- How we can provide identities to our Pods using **ServiceAccounts**, as well as how to provide permissions for these identities using **Roles**, **RoleBindings**, **ClusterRoles**, and **ClusterRoleBindings**.
- How we can define privileges and access control settings for Pods and containers in our Pods using **security contexts**.

It is important to note that the security concepts I am discussing in this article concern security _inside_ of our cluster and our containers. We must also take care of securing access to our cluster using authentication, authorization, and network-related restrictions. That is a whole different topic and is outside the scope of this article.

## Restrict network communication using NetworkPolicies

A **NetworkPolicy** is a Kubernetes resource that is used to define allowed incoming and outgoing traffic to and from a Pod.

Like an Ingress, a NetworkPolicy resource has no effect unless there is a corresponding controller installed in your cluster. And just like Ingress controllers there are a number of them available, for instance [Calico](https://www.tigera.io/project-calico/). In this article I will only go through NetworkPolicies from a theoretical point of view, I will not install anything in my cluster.

Let's look at an example of a NetworkPolicy resource that displays many of the features that it supports and then go through it in more detail:

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: my-network-policy
  namespace: my-application-namespace
spec:
  podSelector:
    matchLabels:
      app: nginx
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - ipBlock:
            cidr: 10.0.0.0/24
        - namespaceSelector:
            matchLabels:
              project: source-namespace
        - podSelector:
            matchLabels:
              role: frontend
      ports:
        - protocol: TCP
          port: 6379
  egress:
    - to:
        - ipBlock:
            cidr: 10.0.1.0/24
      ports:
        - protocol: TCP
          port: 8080
```

In this manifest for a NetworkPolicy we see the following properties:

- The `.apiVersion` is set to `networking.k8s.io/v1`, i.e. this resource belongs to the `networking.k8s.io` API group and it is using the `v1` version.
- The `.kind` is set to `NetworkPolicy`.
- In `.metadata` I have defined both a `name` and a `namespace`, i.e. a NetworkPolicy resource is a namespaced resource.
- In `.spec` we find the details of what Network traffic we allow:
  - We select the Pods that this NetworkPolicy applies to using `.spec.podSelector` with a `matchLabels` statement. In this example I select Pods that have a label named `app` with a value of `nginx`.
  - In `.spec.policyTypes` I specify that this NetworkPolicy includes rules for both `Ingress` (incoming traffic) and `Egress` (outgoing traffic).
  - In `.spec.ingress` I configure what incoming traffic is allowed, and in `.spec.egress` I configure what outgoing traffic is allowed. See additional details for these two properties in their own sections below.

Once we have our manifest describing our NetworkPolicy we can create it using `kubectl apply`, list it using `kubectl get networkpolicies` (or `kubectl get netpol` using the short-hand version), as well as viewing additional details of it using `kubectl describe networkpolicy`. However, for brevity I will not run these commands in this article.

### Ingress rules

In the sample above I defined the following rule for ingress traffic:

```yaml
ingress:
  - from:
      - ipBlock:
          cidr: 10.0.0.0/24
      - namespaceSelector:
          matchLabels:
            project: source-namespace
      - podSelector:
          matchLabels:
            role: frontend
    ports:
      - protocol: TCP
        port: 6379
```

This rule says that any traffic destined for port `6379` originating from one of the following sources:

- an IP in the CIDR-block `10.0.0.0/24`
- a Pod in the Namespace named `source-namespace`
- a Pod with the label `role: frontend`

will be allowed. Note that it is enough that one of the three sources is a match for the traffic to be allowed, as long as it is destined to port `6379`.

I could add additional rules if I wish.

### Egress rules

In the sample above I defined the following rule for egress traffic:

```yaml
egress:
  - to:
      - ipBlock:
          cidr: 10.0.1.0/24
    ports:
      - protocol: TCP
        port: 8080
```

This rule says that my Pods (identified by the label `app: nginx`) are allowed to send traffic on port `8080` to IP-addresses in the CIDR-block `10.0.1.0/24`. Any outgoing traffic outside of this rule will be blocked.

I could add additional rules if I wish.

## Restrict what Pods can do using ServiceAccounts

Each Pod we run in our Kubernetes cluster has an associated **ServiceAccount**. A ServiceAccount is a non-human principal that we can assign roles and permissions to.

In this Kubernetes-101 series of articles we have created many Pods, but we have not assigned any ServiceAccounts to our Pods. If we do not explicitly assign a ServiceAccount to a Pod it will be assigned a default ServiceAccount instead.

Each Namespace in your cluster has a default ServiceAccount. You can create additional ServiceAccounts scoped to a given Namespace.

Why would you want to use a ServiceAccount? If your application must perform any action inside the cluster then you would need a ServiceAccount with the correct permissions to do so. Another situation is when you have a CI/CD pipeline, e.g. GitHub Actions, where you need an identity that can create resources inside of your cluster. In that situation you can create a ServiceAccount and obtain a token for this account that you store as a secret in your CI/CD pipeline. You can then use the token to perform actions as the ServiceAccount. In this article I will only go through how to create a ServiceAccount and assign it to a Pod in a cluster.

A manifest for a basic ServiceAccount resource looks like this:

```yaml
# service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  namespace: my-namespace
```

The ServiceAccount manifest has an `.apiVersion`, a `.kind`, and `.metadata`. This is all we need to create a ServiceAccount, although there are additional options you might want to specify - but I try to keep things simple here.

I can create my ServiceAccount from my manifest using `kubectl apply`, note that the Namespace named `my-namespace` must already exist:

```console
$ kubectl apply -f service-account.yaml

serviceaccount/my-service-account created
```

I can list all my ServiceAccounts for a given Namespace using `kubectl get serviceaccounts`:

```console
$ kubectl get serviceaccounts --namespace my-namespace

NAME                 SECRETS   AGE
default              0         37s
my-service-account   0         20s
```

We see that `my-service-account` is listed, but we can also see the `default` ServiceAccount that was created when my Namespace was created. To shorten the previous command a bit we can use the short form of `serviceaccounts` which is `sa`, so the previous command becomes `kubectl get sa --namespace my-namespace`.

As usual I can see additional details of a given ServiceAccount using `kubectl describe`:

```console
$ kubectl describe serviceaccount my-service-account --namespace my-namespace

Name:                my-service-account
Namespace:           my-namespace
Labels:              <none>
Annotations:         <none>
Image pull secrets:  <none>
Mountable secrets:   <none>
Tokens:              <none>
Events:              <none>
```

My ServiceAccount is not very interesting since I kept the configuration to a minimum.

### Assign a ServiceAccount to a Pod

To assign a ServiceAccount to a Pod we simply specify the name of the ServiceAccount in `.spec.serviceAccountName` property of the Pod manifest:

```yaml
# pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  namespace: my-namespace
spec:
  serviceAccountName: my-service-account
  containers:
    - name: nginx
      image: nginx:latest
```

I create my Pod using `kubectl apply`:

```console
$ kubectl apply -f pod.yaml

pod/my-pod created
```

Then to verify the ServiceAccount is set I can use `kubectl describe`:

```console
$ kubectl describe pod my-pod --namespace my-namespace

Name:             my-pod
Namespace:        my-namespace
Priority:         0
Service Account:  my-service-account
Node:             minikube/192.168.49.2
Start Time:       Wed, 14 Feb 2023 20:33:17 +0100
Labels:           <none>
Annotations:      <none>
Status:           Running
IP:               172.17.0.2
... (output truncated) ...
```

The output indicates that `my-service-account` is correctly assigned to the Pod.

### Role-Based Access Control (RBAC)

A ServiceAccount is of little use unless we can assign it permissions to perform operations in our Kubernetes cluster. To do this we have the option of using RBAC. RBAC in Kubernetes involves four new resource types:

- Role
- ClusterRole
- RoleBinding
- ClusterRoleBinding

A **Role** is a set of permissions, scoped to a Namespace. A **ClusterRole** is likewise a set of permissions, but it is not scoped to a single Namespace. A **RoleBinding** assigns a given Role, or ClusterRole, to a subject (e.g. a ServiceAccount). A **ClusterRoleBinding** assigns a ClusterRole to a subject.

To keep things light in this article I will concentrate on Roles and RoleBindings. Keep in mind that if you need to assign permissions across Namespaces or for cluster-scoped resources (i.e. a Node) then you would need to use ClusterRoles and ClusterRoleBindings instead.

A basic Role that specifies permissions to read Pods might look like this:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: my-role
  namespace: my-namespace
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]
```

The Role manifest contains `.apiVersion`, `.kind`, and `.metadata`. However, unlike most other manifests we have seen there is no `.spec`. Instead there is `.rules`. A rule identifies `resources` (like Pods, Deployments, Jobs, Services, etc) from a specified `apiGroup` (an empty value `""` refers to the _core_ API group where most objects we are familiar with lives, e.g. Pods), and finally a rule also contains `verbs` which specifies what actions can be done on the resources. So our single rule says that this role gives permissions to `get`, `watch`, and `list` resources of type `pods` in the core API group.

Assuming we have a ServiceAccount named `my-service-account` we can now assign this Role to our ServiceAccount using a RoleBinding:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: my-role-binding
  namespace: my-namespace
subjects:
  - kind: ServiceAccount
    name: my-service-account # my service-account from above
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: my-role # the name of my role from above
  apiGroup: rbac.authorization.k8s.io
```

We see that the RoleBinding manifest ties together my `subject`, which is the ServiceAccount (the identity), with my Role. From the manifest it is also clear that we could assign the same Role to many subjects in the same RoleBinding if we wish.

## Define privileges and access control settings for Pods and containers

A security context allows us to specify what user ID a container should run as, what group it should belong to, if we want to run as a privileged user or not, use Security Enhanced Linux (SELinux) features, and much more. This is an advanced topic and in this article I will just show you how you specify a security context.

### Set the security context for a Pod

If you specify a security context for a Pod it will be set for each container in the Pod. A basic example of a Pod with a security context looks like this:

```yaml
# pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  securityContext:
    runAsUser: 1010
  containers:
    - name: my-container
      image: busybox:1.28
      command: ["sh", "-c", "sleep 1h"]
```

The security context is specified in `.spec.securityContext`. In this example I specify that each container in my Pod should run all processes as the user with ID `1010`. The container named `my-container` starts a `sleep` process at startup. We can verify that this process is indeed started as user `1010`. I begin by creating my Pod using `kubectl apply`:

```console
$ kubectl apply -f pod.yaml

pod/my-pod created
```

Then I `exec` into my Pod and look at the current processes:

```console
$ kubectl exec -it my-pod -- sh
$ ps

PID   USER     TIME  COMMAND
    1 1010      0:00 sleep 1h
    7 1010      0:00 sh
   13 1010      0:00 ps
```

From the output we can see that the `sleep` process is indeed started as user `1010`.

### Set the security context for a container

We can also specify a specific security context for a given container, this will override any clashing security context that we specify for the whole Pod. Let us modify the previous example and add a security context for the container:

```yaml
# pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  securityContext:
    runAsUser: 1010
  containers:
    - name: my-container
      image: busybox:1.28
      command: ["sh", "-c", "sleep 1h"]
      securityContext:
        runAsUser: 2020
```

In this manifest I have added `.spec.containers[0].securityContext`. In the security context I specify that I want the container to run any process it starts as user with ID `2020`. This overrides the security context specified for the whole Pod. I repeat the process of creating my Pod and then `exec`ing into it:

```console
$ kubectl apply -f pod.yaml

pod/my-pod created

$ kubectl exec -it my-pod -- sh
$ ps

PID   USER     TIME  COMMAND
    1 2020      0:00 sleep 1h
    7 2020      0:00 sh
   13 2020      0:00 ps
```

The output shows that the `sleep` process has been started as user `2020`.

## Summary

This was mostly a theoretical lesson in three important concepts related to security in your Kubernetes cluster. In summary, we looked at:

- NetworkPolicies
- ServiceAccounts (together with Role, RoleBindings, ClusterRole, and ClusterRoleBindings)
- Security contexts

Next article will be the last in this series of Kubernetes-101 articles. There I will do a high-level overview of the Kubernetes landscape to see what else there is to learn. Kubernetes is a platform, and although there are many concepts to learn you can still master plain Kubernetes in a relatively short time. The Kubernetes landscape includes a lot more than plain Kubernetes, and this is where things really get out of hand. There are at least 100 tools to do any task that you want to do in your cluster (that was an exaggeration, but probably close to the truth). Do you want to deploy applications using GitOps? Should you use Flux or Argo CD? Or something else? I will scratch this Kubernetes landscape surface in the next article, see you there!