Kubernetes: Automatic Let's Encrypt Certificates for Services
Tags: kubernetes, tutorial
Published: 2020-08-03 16:10:39

Service exposed from a Kubernetes cluster should by encrypted with TLS. Learn how to get fully automatic, self-renewed Let's Encrypt certificates.

Kubernetes is an industry standard based on security best practices. By default, all deployments are cluster-internal only, you need to explicitly expose them with a Kubernetes resource of type `Service`. But when you open your application to public traffic, you should provide strong TLS encryption.

In this article, I show you the essential, easy to apply steps to expose services and automatically get self-renewed [Let’s Encrypt](https://letsencrypt.org/) certificates.

_This article originally appeared at [my blog](https://admantium.com/blog/kube02_tls_encryption/)_.

## Preparation: Install Nginx Ingress

K3S, the Kubernetes distribution that I'm using, uses the Traefik Ingress per default. We need to install the Nginx-Ingress manually. To do this, we will use the great helper tool [arkade](https://github.com/alexellis/arkade).

```sh
curl -sLS https://dl.get-arkade.dev | sudo sh
```

Then, we can install Nginx with a simple one liner.

```sh
> arkade install nginx-ingress

Release "nginx-ingress" has been upgraded. Happy Helming!
NAME: nginx-ingress
LAST DEPLOYED: Fri May  8 14:11:09 2020
NAMESPACE: default
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The nginx-ingress controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
...
```

## Deploy an Application

For demonstration purposes, let's start a webserver that returns a simple message. We will use the following Kubernetes `Deployment` resource.

```yaml
apiVersion: apps/v1
kined: Deployment
namespace: demo
metadata:
  name: echo
  namespace: demo
spec:
  selector:
    matchLabels:
      app: echo
  replicas: 2
  template:
    metadata:
      labels:
        app: echo
    spec:
      containers:
        - name: echo
          image: hashicorp/http-echo
          args: ['--listen', ':5678', '--text', 'echo']
          ports:
            - containerPort: 5678
```

Let's deploy this and check that the pods are created.

```sh
> kb apply -f deployment.yaml

> kb get all

NAME                        READY   STATUS    RESTARTS   AGE
pod/echo-7b86d65bc8-6crzv   1/1     Running   0          9s

NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/echo   1/1     2            2           9s
```

## Configure Ingress

In order to access the webserver, we will define a Service resource of type `ClusterIP`. This means that all `echo` pods will be accessible from within the cluster.

```yml
apiVersion: v1
kind: Service
metadata:
  name: echo
spec:
  ports:
    - port: 80
      targetPort: 5678
  selector:
    app: echo
```

To make this service available from the outside, we need to route from the internet through the Nginx ingress to the services. For this, we need to define the following `Ingress` resource.

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo
  namespace: demo
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: echo.admantium.com
      http:
        paths:
          - backend:
              serviceName: echo
              servicePort: 5678
```

Inside the `spec` part, you see that this rule applies when the request hostname is `echo.admantium.com`. All requests will be forwarded to the service `echo` on port `5678`. For this to work, you need of course configure the DNS entry for this domain to point to your Kubernetes cluster.

Apply this rule, and then check its status with the `describe` command.

```sh
> kubectl apply -f echo-service.yml

> kubectl describe ingress echo

Name:             echo
Namespace:        default
Address:          49.12.45.26
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  letsencrypt-staging terminates echo.admantium.com
Rules:
  Host                Path  Backends
  ----                ----  --------
  echo.admantium.com
                         echo:5678 (10.42.1.155:5678,10.42.2.165:5678)
Annotations:          cert-manager.io/cluster-issuer: letsencrypt-staging
                      kubernetes.io/ingress.class: nginx
Events:
  Type    Reason  Age                   From                      Message
  ----    ------  ----                  ----                      -------
  Normal  UPDATE  2m25s (x4 over 159m)  nginx-ingress-controller  Ingress default/echo
```

Now you can already access the echo server. But it’s not TLS encrypted.

## Install cert-manager

In Kubernetes, certificate management is a central responsibility that can be realized with the `cert-manager`. Internally, this tool consists of Kubernetes resources like Pods, Services and Deployments. In your application, you configure to use the cert-manager as a provide for TLS certificates. Then, it will automatically issue certificates that are stored as secrets inside your cluster. It will also check and renew certificates automatically before they expire.

First of all, we will install the `cert-manager`.

```sh
arkade install cert-manager
Using helm3
Client: x86_64, Darwin

...

NAME: cert-manager
LAST DEPLOYED: Mon Apr 27 19:58:05 2020
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
cert-manager has been deployed successfully!
```

## Configuring Cert-Manager

In order to issue TLS certificates, you need to decide which certificate issuer you want to use. In our case, we will use [Let’s Encrypt](https://letsencrypt.org/). This issuer needs to be configured as a Kubernetes resource of type `ClusterIssuer`. For Let’s Encrypt, there are two issuers: staging and production. It is essential that you use the staging issuer until the configuration completely works.

```yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
  namespace: cert-manager
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: staging@admantium.com
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
      - http01:
          ingress:
            class: nginx
```

Apply this file, and then check the cert-managers log file to see that the cluster issuer is created correctly. This can take one minute or two.

```sh
kubectl logs -n cert-manager deploy/cert-manager -f
```

When the `ClusterIssuer` is successfully created, we can execute the last step.

## Configuring Ingress Resource to use TLS

Inside our `Ingress` resource, we need to add two new configuration options. Inside `metadata.annotations`, add `cert-manager.io/cluster-issuer`. And inside `spec`, add a `tls` block, with the domain name and a `secretName` which is the same as the `ClusterIssuer`-

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-staging
spec:
  rules:
    - host: echo.admantium.com
      http:
        paths:
          - backend:
              serviceName: echo
              servicePort: 80
  tls:
    - hosts:
        - echo.admantium.com
      secretName: letsencrypt-staging
```

When we apply this ingress, we can follow the cert-managers logfiles to see the progress.

```sh
I0508 12:19:24.176712       1 pod.go:58] cert-manager/controller/challenges/http01/selfCheck/http01/ensurePod "level"=0 "msg"="found one existing HTTP01 solver pod" "dnsName"="echo.admantium.com" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-pmn4v" "related_resource_namespace"="default" "resource_kind"="Challenge" "resource_name"="letsencrypt-staging-1985468592-3302894823-3409218764" "resource_namespace"="default" "type"="http-01"
----
I0508 12:29:41.771005       1 acme.go:166] cert-manager/controller/certificaterequests-issuer-acme/sign "level"=0 "msg"="certificate issued" "related_resource_kind"="Order" "related_resource_name"="letsencrypt-staging-1985468592-3302894823" "related_resource_namespace"="default" "resource_kind"="CertificateRequest" "resource_name"="letsencrypt-staging-1985468592" "resource_namespace"="default"
```

The certificate is issued. Now you can access the service in a browser - and check its certificate.

![echo_server_in_browser1](https://admantium.com/images/blog/kube02_echo_server_in_browser1.png)

![echo_server_in_browser2](https://admantium.com/images/blog/kube02_echo_server_in_browser2.png)

## Conclusion

With Kubernetes, you can automate the creation of TLS certificates. Once properly setup, the cert-manager takes care of creating certificates, checking their expiration date and re-creating new certificates. To apply certificates, you add an annotation and a TLS block to your deployment specification. That is all you need.