Exposing public applications on AWS EKS with Traefik
Tags: aws, kubernetes, eks, traefik
Published: 2024-01-12 07:19:13

In this short guide, I'll walk you through the process of exposing applications on AWS EKS using Traefik. Despite Traefik often being perceived as having complex configurations and unclear documentation, I aim to equip you with a straightforward setup that you can easily apply to your solution. Let's streamline the process and make exposing applications on the cloud a hassle-free experience.

## Prerequisites

- preconfigure EKS Cluster with [AWS Load Balancer Controller](https://github.com/kubernetes-sigs/aws-load-balancer-controller) and [External DNS](https://github.com/kubernetes-sigs/external-dns) installed - you can leverage my repo that will help you set up EKS cluster with `eksctl` - [GitHub repo](https://github.com/luafanti/eksctl-labs-cluster)
- Route53 domain with ACM Certificate

As always I encourage you to install [kubectx + kubens](https://github.com/ahmetb/kubectx) and configure shorter aliases to navigate Kubernetes easily.

## Install & Configure Traefik

All resources you can find in my [GitHub repo](https://github.com/luafanti/eks-traefik)

First, check if the Ingress Class has been added with installation of AWS Load Balancer Controller from my [EKS setup repo](https://github.com/luafanti/eksctl-labs-cluster). 

```bash
kubectl get ingressclas

-------------------------
NAME      CONTROLLER         
alb       ingress.k8s.aws/alb
```

We use this Ingress Controller to provision the main NLB pointing to our Traefik proxy.

Before we install the official [Traefik helm chart](https://github.com/traefik/traefik-helm-chart) let's take a few minutes to look at the helm values file `helm/traefik.yaml`.

```yaml
#1
ports:
  web:
    expose: false
  websecure:
    port: 8443
    expose: true
    exposedPort: 443
    protocol: TCP
    tls:
      enabled: false
  traefik:
    port: 9000
    expose: true
    exposedPort: 9000
    protocol: TCP
    tls:
      enabled: false
  postgres:
    port: 5432
    expose: true
    exposedPort: 5432
    protocol: TCP
  rabbitmq:
    port: 5672
    expose: true
    exposedPort: 5672
    protocol: TCP

#2
ingressRoute:
  dashboard:
    enabled: false

service:
  #3
  annotations:
    external-dns.alpha.kubernetes.io/hostname: traefik.<YOUR_DOMAIN>, rabbitmq.<YOUR_DOMAIN>, postgres.<YOUR_DOMAIN> #4
    service.beta.kubernetes.io/aws-load-balancer-type: external
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443, 9000"
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <YOUR_CERTIFICATE_ARN> #5
    service.beta.kubernetes.io/load-balancer-source-ranges: 0.0.0.0/0
globalArguments:
  - "--api.insecure=false"

#6
logs:
  general:
    level: INFO
  access:
    enabled: false
```

At the top we define the Trafik port **(#1)**. There are the network entry points into Traefik. We see dedicated ports defined for PostgreSQL and AMQP for RabbitMQ. The `web` port responsible for HTTP has been disabled and we have only left the HTTPS `websecure` open. Next, we disable default Ingress Route for Traefik dashboard **(#2)**. Later we will create custom route for it. At the Traefik `service` section **(#3)** we define specific annotations for External DNS and AWS Load Balancer Controller. Here you need to know that by using these annotations you configure the AWS NLB and DNS records for Route53. Adjust `external-dns.alpha.kubernetes.io/hostname` with your root domain **(#4)** and  `service.beta.kubernetes.io/aws-load-balancer-ssl-cert` with ACM certificate ARN **(#5).** 

For more information please refer to the documentation:

- [AWS Load Balancer Controller docs](https://kubernetes-sigs.github.io/external-dns/v0.14.0/annotations/annotations/)
- [External DNS docs](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/guide/service/annotations/)

The logging section **(#6)** is obvious. I advise setting log level to `debug` during the first setup as the info level is not too verbose.

So now, after adjustment in `helm/traefik.yaml` you are ready to install Traefik: 

```bash
helm repo add traefik https://helm.traefik.io/traefik
helm install traefik traefik/traefik --create-namespace --namespace=network --values=helm/traefik.yaml
```

Right after, you should see NLB being created with the following list of listeners.


![AWS NLB Listeners](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nkuffffu4jtvx9hsla6o.png)

DNS records should also appear in the Route53 console.


![Route53 DNS records](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ws303y3pao912u5fbvcs.png)

If something doesn't happen, look for the answer in logs.

```bash
kubectl logs -f aws-load-balancer-controller- -n kube-system
kubectl logs -f external- -n kube-system
kubectl logs -f traefik- -n network
```

The next step will be adding Traefik dashboard route.

With the installation of Traefik, its [CRD's](https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/) are installed. Check our first router definitions in `network/traefik-dashboard-route.yaml` . First, we add a k8s secret **(#1)** that will hold the credentials for the Basic Auth, then we define the BA as Traefik `Middleware` **(#2)** and finally we create an `IngressRoute` which will be our routing path **(#3)**. Remember to change your domain in the `Host` rule.

```yaml
#1
apiVersion: v1
kind: Secret
metadata:
  name: dashboard-basic-auth-creds
  namespace: network
type: kubernetes.io/basic-auth
stringData:
  username: admin
  password: admin

---
#2
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: dashboard-basic-auth
  namespace: network
spec:
  basicAuth:
    secret: dashboard-basic-auth-creds

---
#3
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: dashboard
  namespace: network
spec:
  entryPoints:
    - traefik
  routes:
    - kind: Rule
      match: Host(`traefik.<YOUR_DOMAIN>`) #4
      middlewares:
        - name: dashboard-basic-auth
          namespace: network
      services:
        - kind: TraefikService
          name: api@internal
```

Apply the dashboard route. 

```bash
kubectl apply -f network/traefik-dashboard-route.yaml
```

After a few seconds, when Traefik react to adding a new definition, the dashboard should be available in the browser under the below URL (**The trailing slash is mandatory**):

```bash
https://traefik.<YOUR_DOMAIN>:9000/dashboard/
```

## Install sample apps (RabbitMQ & PostgreSQL)

Now let's add some more apps on different ports to better check Traefik's capabilities.

```bash
helm repo add bitnami https://charts.bitnami.com/bitnami

helm install rabbitmq bitnami/rabbitmq --create-namespace --namespace=messaging -f helm/rabbitmq.yaml
helm install postgres bitnami/postgresql --create-namespace --namespace=database -f helm/postgres.yaml
```

Add Traefik routes for:

- Postgres port 5432
- AMQP RabbitMQ port 5672
- HTTPS RabbitMQ management panel

```bash
kubectl apply -f network/postgres-route.yaml
kubectl apply -f network/rabbitmq-route.yaml
```

List all Traefik routes 

```bash
kubectl get ingressroute,ingressroutetcps --all-namespaces

-------------------------
NAMESPACE   NAME                                          
messaging   ingressroute.traefik.containo.us/rabbitmq-http
network     ingressroute.traefik.containo.us/dashboard     

NAMESPACE   NAME                                            
database    ingressroutetcp.traefik.containo.us/postgres     
messaging   ingressroutetcp.traefik.containo.us/rabbitmq-amqp
```

You should see two HTTP routes (Traefik dashboard, RabbitMQ panel) and two TCP routes (PostgreSQL and AQMP RabbitMQ). This state should be reflected in the Traefik panel. I encourage you to explore it because it is really rich in information and intuitive.


![Traefik dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rzmbbzrh26qm2l7wkune.png)

The final test will be to check if routing to test applications works correctly.

To check PostgreSQL try to connect with:

```yaml
psql -h postgres.<YOUR_DOMAIN> -p 5432 -U root
# pass 'root' password
```

To check RabbitMQ AMQP connectivity execute my python script. Update URL in `utils/rabbitmq_ampq_test.py` before:

```yaml
python3 utils/rabbitmq_ampq_test.py
```

RabbitMQ admin panel should be accessible on:

```yaml
https://rabbitmq.<YOUR_DOMAIN>
```

## Final Thoughts

As you can see, working with Traefik can be simple. In our lab we needed a single AWS NLB to expose several applications from different Kubernetes namespaces and on different ports. It’s a good benchmark of simplicity. This sort of approach has also limitations, but it’s a different topic. Be aware that other are other solutions in this Ingress segment such as [Nginx Ingress Controller](https://github.com/kubernetes/ingress-nginx) and [HAProxy Ingress Controller](https://github.com/haproxytech/kubernetes-ingress). As always, I encourage you to experiment and form your own opinion. However, I hope I have helped you get through the basic setup more conveniently.