Kubernetes: ServiceAccounts, JWT-tokens, authentication, and RBAC authorization
Tags: security, kubernetes, todayilerned
Published: 2021-02-02 08:12:57

---
title: Kubernetes: ServiceAccounts, JWT-tokens, authentication, and RBAC authorization
published: true
date: 2020-11-22 07:32:59 UTC
tags: security,kubernetes,todayilerned
canonical_url: https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/
---

![](https://cdn-images-1.medium.com/max/1024/1*LwQ5TrmSSdHxgqBn6oahvw.png)

For the authentification and authorization, Kubernetes has such notions as _User Accounts_ and _Service Accounts_.

User Accounts  -  common user profiles used to access a cluster from the outside, while Service Accounts are used to grant access from inside of the cluster.

ServiceAccounts are intended to provide an identity for a Kubernetes Pod to be used by its container to authenticate and authorize them when performing API-requests to the Kubernetes API-server.

### Content

- [Default ServiceAccount](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#Default_ServiceAccount)
- [default token](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#default_token)
- [JWT token](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#JWT_token)
- [JWT token and authentification](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#JWT_token_and_authentification)
- [ServiceAccounts, and RBAC](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#ServiceAccounts,_and_RBAC)
- [RoleBindig for ServiceAccount](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#RoleBindig_for_ServiceAccount)
- [ServiceAccounts and security](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#ServiceAccounts_and_security)
- [Useful links](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/#Useful_links)

### Default ServiceAccount

Every Kubernetes Namespace has its own default ServiceAccount (SA) which is created when creating a namespace.

Let’s check the _default_ namespace:

```
$ kubectl --namespace default get serviceaccount
NAME SECRETS AGE
default 1 176d
```

For each ServiceAccount a token is generated and stored as a Kubernetes Secret.

Check the default SA:

```
$ kubectl --namespace default get serviceaccount default -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: “2020–05–25T12:04:49Z”
name: default
namespace: default
resourceVersion: “296”
selfLink: /api/v1/namespaces/default/serviceaccounts/default
uid: 19cc2b5f-fbc3–403e-a7c7-d62361a4038a
secrets:
- name: default-token-292g9
```

Here is the token for this SA — the _default-token-292g9_ Secret:

```
…
secrets:
- name: default-token-292g9
```

#### default token

Now, check the Secret’s content:

```
$ kubectl get secret default-token-292g9 -o yaml
apiVersion: v1
data:
ca.crt: LS0…sdA==
token: ZXl…TWc=
kind: Secret
metadata:
annotations:
kubernetes.io/service-account.name: default
kubernetes.io/service-account.uid: 19cc2b5f-fbc3–403e-a7c7-d62361a4038a
creationTimestamp: “2020–05–25T12:04:49Z”
name: default-token-292g9
namespace: default
resourceVersion: “294”
selfLink: /api/v1/namespaces/default/secrets/default-token-292g9
uid: 07a46645–0083–45a0-a640–6e6a78ebd9b1
type: kubernetes.io/service-account-token
```

At first, its type is the `kubernetes.io/service-account-token`.

Another interesting part here is the data that keeps two records - `ca.cert` и `token`.

If a token is not from the default namespace — there will be a third field specifying a namespace to which this token belongs.

Theca.cert is signed by the cluster's master key so the cluster is playing the Certificate Authority role, and allows a pod or an application to verify the API-server.

And now, let’s go to investigate the tokenpart.

#### JWT token

To make it easier to work from the terminal — save the `data.token` value to a variable:

```
$ token=”ZXl…TWc=”
```

Use the base64 get its content:

```
$ echo $token | base64 -d
eyJ[…]iJ9.eyJ[…]ifQ.g5I[…]3Mg
```

Here I’ve removed some data with the […], but we can see that the value is divided into three parts with dots:

- the _header_ — describes how the token was signed
- the _payload_ — actual data of the token, such as expiration date, who issued it, etc see the [RFC-7519](https://tools.ietf.org/html/rfc7519#section-4.1)
- the _signature_ — is used to verify that the token wasn’t modified and can be used to validate the sender

See the [documentation>>>](https://jwt.io/introduction/).

To check the token’s content we can use the `jwt` utility or on the [jwt.io](https://jwt.io/) website.

In our case, the payload section has the following lines:

```
{
  "iss": "kubernetes/serviceaccount",
  "kubernetes.io/serviceaccount/namespace": "default",
  "kubernetes.io/serviceaccount/secret.name": "default-token-s8m4t",
  "kubernetes.io/serviceaccount/service-account.name": "default",
  "kubernetes.io/serviceaccount/service-account.uid": "b4514006-4c9a-4c30-92c8-1cc1c058b31c",
  "sub": "system:serviceaccount:default:default"
}
```

here in the [`sub`](https://tools.ietf.org/html/rfc7519#section-4.1.2) filed we can see the ServiceAccount name, i.e. - who is presenting this token to the Kubernetes API-server so the server will know from who this token came.

Okay, but what about a password? In the sub there is a "login" - but where is his "password"?

And here is the third part is playing — the _signature_.

#### JWT token and authentification

I wasn’t able to see these details in any from the googled materials, see the [Useful links](https://rtfm.co.ua/en/?p=25299#Useful_links) section of this post, although as for me — this is the most interesting part of the scheme.

Let’s go back to the first section of the token — the _header_, which in our case has the RS256 algorithm type defined i.e. RSA (_Rivest-Shamir-Adleman_) — the asymmetric algorithm with private and public keys and uses SHA-256 algorithm for the signature.

Let’s check our token on the [jwt.io](jwt.io):

![](https://cdn-images-1.medium.com/max/1024/0*Kx7AcDiWQfBHD6Ih.png)

**_Invalid Signature_**  — as we not provided the private and public keys to verify the token.

Because the masters’ private key on AWS Elastic Kubernetes Service is stored on the ConrolPlane nodes and we can’t access them — let’s use [minikube](https://rtfm.co.ua/kubernetes-zapusk-minikube-na-arch-linux/) for the testing.

Run a local cluster:

```
$ minikube start
```

In its default namespace we can see already existing token:

```
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-s8m4t kubernetes.io/service-account-token 3 2m44s
```

Grab the `token` field and decode it with `base64`:

```
$ kubectl get secrets -o jsonpath=’{.items[0].data.token}’ | base64 -d
eyJhbGciO[…]61O_LxbM_-tiLjyjeCZw
```

Go back to the jwt.io, paste the string received above:

![](https://cdn-images-1.medium.com/max/1024/0*4NhKpUZFB7v1BHEt.png)

Still **_Invalid Signature_**  — but go to your minikube and take its public certificate - the `~/.minikube/ca.crt` file:

```
$ cat ~/.minikube/ca.crt
 — — -BEGIN CERTIFICATE — — -
MIIDBjCCAe6gAwIBAgIBATANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwptaW5p
…
0g+FhVM92T+yV38vYLO/HaKeiOzIcgHHkAoLJZd/K/Mu7crwIuGlcCVhrjcHoa3p
Md34ZTeqxA4J3w==
 — — -END CERTIFICATE — — -
```

Paster it to the _Public Key or Certificate_ field.

Find the private key of the minikube cluster - actually, it is also used to sing the ca.crt and tokens, the ~/.minikube/ca.key file:

```
$ cat ~/.minikube/ca.key
 — — -BEGIN RSA PRIVATE KEY — — -
MIIEowIBAAKCAQEAtDRDag2D7UBaBmWQwTKVLjuKTuat4eD/oThRgfi5bcCnwooG
…
xnL96EHthflb3NaS4GKuJYzNAPhfOdMw96Ce8KtNYpMYjRhNF9TN
 — — -END RSA PRIVATE KEY — — -
```

Paste it to the _Private Key_ field:

![](https://cdn-images-1.medium.com/max/1024/0*ioH0eKFxi7UPCrUi.png)

**_Signature Verified_**  — yup, it works! The authenticity of the bearer of the token is verified.

So, going back to the ServiceAccounts:

- for a ServiceAccount a token is created which keep the SA name
- the token is signed by the master key of the Kubernetes cluster
- a pod make a request to the API server using this token to authenticate him
- the API server validates the token by using its public key and verify that the token wasn’t modified and is relly issued by this Kubernetes clutserм

Now, let’s go to see in practice how this is working and how Kubernetes RBAC is used here.

### ServiceAccounts, and RBAC

For each Pod that has no ServiceAccount specified the _default_ ServiceAccount is attached and its _default_ token is mounted.

Go back to our EKS cluster and run a Pod:

```
$ kubectl run -i --tty --rm ca-test-pod --image=radial/busyboxplus:curl
kubectl run — generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run — generator=run-pod/v1 or kubectl create instead.
If you don’t see a command prompt, try pressing enter.
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$
```

Check it `volumeMounts`, `serviceAccount`, and `volumes`:

```
$ kubectl get pod ca-test-pod-5c96c78d7f-wqlsq -o yaml
apiVersion: v1
kind: Pod
…
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-292g9
readOnly: true
…
serviceAccount: default
serviceAccountName: default
…
volumes:
- name: default-token-292g9
secret:
defaultMode: 420
secretName: default-token-292g
```

Inside of the pod check the `/var/run/secrets/kubernetes.io/serviceaccount` directory content:

```
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$ ls -1 /var/run/secrets/kubernetes.io/serviceaccount
ca.crt
namespace
token
```

And recall the content of the data section of the _default-token-292g9_ Secret:

```
$ kubectl get secret default-token-292g9 -o yaml
apiVersion: v1
data:
ca.crt: LS0t[…]
namespace: ZGVmYXVsdA==
token: ZXlKaGJ
…
```

Try to perform a request to the API-server without authentification — use the special Service _kubernetes_, add the `-k` or `--insecure` to the `curl` to skip server's certificate validation

```
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$ curl -k [https://kubernetes](https://kubernetes)
{
“kind”: “Status”,
“apiVersion”: “v1”,
“metadata”: {
},
“status”: “Failure”,
“message”: “forbidden: User \”system:anonymous\” cannot get path \”/\””,
“reason”: “Forbidden”,
“details”: {
},
“code”: 403
}
```

Cool — we got the 403, Forbidden.

Now, add two variables — one with the `ca.crt` and with the `token`:

```
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$ CERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$ TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
```

And run `curl` again - let's try to get a list of the pods in our namespace, this time without `--insecure` and with authorization by using the `Authorization` header:

```
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$ curl — cacert $CERT -H “Authorization: Bearer $TOKEN” “https://kubernetes/api/v1/namespaces/default/pods/"
{
“kind”: “Status”,
“apiVersion”: “v1”,
“metadata”: {
},
“status”: “Failure”,
“message”: “pods is forbidden: User \”system:serviceaccount:default:default\” cannot list resource \”pods\” in API group \”\” in the namespace \”default\””,
“reason”: “Forbidden”,
“details”: {
“kind”: “pods”
},
“code”: 403
}
```

At this time, we are able to see our user — the User "`system:serviceaccount:default:default`", but it has no permissions to perform requests as by default all users and ServiceAccounts have no privileges (the principle of the least privileges, POLP).

#### RoleBindig for ServiceAccount

To give our SericeAccount permissions we need to create a [RoleBinding](https://rtfm.co.ua/kubernetes-znakomstvo-chast-5-rbac-avtorizaciya-i-primery-role-i-rolebinding/) or ClusterRoleBinding as for normal users.

Create a RoleBinding mapping to the default ClusterRole _view_, see [User-facing roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles):

```
$ kubectl create rolebinding ca-test-view --clusterrole=view --serviceaccount=default:default
rolebinding.rbac.authorization.k8s.io/ca-test-view created
```

And run `curl` again:

```
[root@ca-test-pod-5c96c78d7f-wqlsq:/]$ curl — cacert $CERT -H “Authorization: Bearer $TOKEN” “https://kubernetes/api/v1/namespaces/default/pods/"
{
“kind”: “PodList”,
“apiVersion”: “v1”,
“metadata”: {
“selfLink”: “/api/v1/namespaces/default/pods/”,
“resourceVersion”: “66892356”
},
“items”: [
{
“metadata”: {
“name”: “ca-test-pod-5c96c78d7f-wqlsq”,
“generateName”: “ca-test-pod-5c96c78d7f-”,
“namespace”: “default”,
“selfLink”: “/api/v1/namespaces/default/pods/ca-test-pod-5c96c78d7f-wqlsq”,
“uid”: “f0d77cfe-38ab-48e9-aaf3-f344f1d343f3”,
“resourceVersion”: “66888089”,
“creationTimestamp”: “2020–11–17T16:08:09Z”,
“labels”: {
“pod-template-hash”: “5c96c78d7f”,
“run”: “ca-test-pod”
},
…
“qosClass”: “BestEffort”
}
}
]
```

### ServiceAccounts and security

Remember, that having access to Secrets and ServiceAccounts any pod can have any token attached and thus can be able to perform actions allowed by such a token.

For example, by using the ServiceAccount of the [ExternalDNS](https://rtfm.co.ua/en/kubernetes-update-aws-route53-dns-from-an-ingress/) — such a pod can make a mess in our AWS Route53.

That’s why it is important to divide access to resources by using RBAC rules and roles for users, for example by allowing access to resources from only one namespace.

### Useful links

- [Using RBAC with Service Accounts in Kubernetes](https://dzone.com/articles/using-rbac-with-service-accounts-in-kubernetes)
- [Kubernetes Access Control: Exploring Service Accounts](https://thenewstack.io/kubernetes-access-control-exploring-service-accounts/)
- [Kubernetes Tips: Using a ServiceAccount](https://medium.com/better-programming/k8s-tips-using-a-serviceaccount-801c433d0023)
- [What is JSON Web Token?](https://jwt.io/introduction/)
- [kubernetes.io: Service Accounts](https://jamesdefabia.github.io/docs/user-guide/service-accounts/)
- [kubernetes.io: Authenticating](https://kubernetes.io/docs/reference/access-authn-authz/authentication/)
- [kubernetes.io: Using RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
- [The Dark Arts of IAM & RBAC — Read-only Kubernetes Access](https://telegraphhillsoftware.com/the-dark-arts-of-iam-rbac-read-only-kubernetes-access/)
- [Kubernetes Authentication](https://www.magalix.com/blog/kubernetes-authentication)
- [Istio End-User Authentication for Kubernetes using JSON Web Tokens (JWT) and Auth0](https://itnext.io/securing-kubernetes-withistio-end-user-authentication-using-json-web-tokens-jwt-97f4bb8e71c6)
- [Kubernetes RBAC 101: Authentication](https://www.cncf.io/blog/2020/07/31/kubernetes-rbac-101-authentication/)
- [How to Authorize Non-Kubernetes Clients With Istio on Your K8s Cluster](https://medium.com/better-programming/how-to-authorize-non-kubernetes-clients-with-istio-on-your-k8s-cluster-8a90fe95b137)
- [Kubernetes Client Authentication on Amazon EKS](https://itnext.io/how-does-client-authentication-work-on-amazon-eks-c4f2b90d943b)
- [Verifying EKS digital certificates](https://www.nickaws.net/aws/kubernetes/2018/08/16/Verifying-digital-certificates.html)
- [3 Realistic Approaches to Kubernetes RBAC](https://thenewstack.io/three-realistic-approaches-to-kubernetes-rbac/)

_Originally published at_ [_RTFM: Linux, DevOps и системное администрирование_](https://rtfm.co.ua/en/kubernetes-serviceaccounts-jwt-tokens-authentication-and-rbac-authorization/)_._

* * *