Autoscaling Ingress controllers in Kubernetes
Tags: kubernetes, devops
Published: 2023-04-17 12:29:34

How do you deal with peaks of traffic in Kubernetes?

To autoscale the Ingress controller based on incoming requests, you need the following:

1. **Metrics** (e.g. the requests per second).
2. A **metrics collector** (to store the metrics).
3. An **autoscaler** (to act on the data).

![What you need to scale the ingress controller](https://res.cloudinary.com/learnk8s/image/upload/v1681721495/threads/nginx-scaling-3_hrljaw.png)

_Let's start with metrics._

[The nginx-ingress can be configured to expose Prometheus metrics.](https://kubernetes.github.io/ingress-nginx/user-guide/monitoring/)

You can use `nginx_connections_active` to count the number of active requests.

![nginx_connections_active to count the number of active requests](https://res.cloudinary.com/learnk8s/image/upload/v1681721496/threads/nginx-scaling-4_nrkcoj.png)

Next, you need a way to scrape the metrics.

As you've already guessed, you can install Prometheus to do so.

Since Nginx-ingress uses annotations for Prometheus, I [installed the server without the Kubernetes operator.](https://github.com/prometheus-community/helm-charts)

```bash
$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
"prometheus-community" has been added to your repositories
$ helm install prometheus prometheus-community/prometheus
NAME: prometheus
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
```

I used Locust to generate some traffic to the Ingress to check that everything was running smoothly.

With the Prometheus dashboard open, I checked that the metrics increased as more traffic hit the controller.

![Testing active connections](https://res.cloudinary.com/learnk8s/image/upload/v1681721496/threads/active-connections-preview_thfb6c.gif)

The last piece of the puzzle was the autoscaler.

I decided to go with [KEDA](https://keda.sh) because:

1. It's an autoscaler with a [metrics server](https://github.com/kubernetes-sigs/metrics-server) (so I don't need to install 2 different tools).
2. It's easier to configure than the Prometheus adapter.
3. I can use the [Horizontal Pod Autoscaler with PromQL.](https://keda.sh/docs/2.10/scalers/prometheus/)

![How KEDA works](https://res.cloudinary.com/learnk8s/image/upload/v1681721495/threads/nginx-scaling-6_pksbtk.png)

Once I installed KEDA, I only had to [create a ScaledObject](https://keda.sh/docs/2.10/concepts/scaling-deployments/#scaledobject-spec), configure the source of the metrics (Prometheus), and scale the Pods (with a PromQL query).

![Example of a ScaledObject for Prometheus scaler in KEDA](https://res.cloudinary.com/learnk8s/image/upload/v1681721496/threads/nginx-scaling-7_va2pwa.png)

**KEDA automatically creates the HPA for me.**

I repeated the tests with Locust and watched the replicas increase as more traffic hit the Nginx Ingress controller!

![Scaling the Ingress Controllers with KEDA](https://res.cloudinary.com/learnk8s/image/upload/v1681721497/threads/end-to-end-preview_zofo1y.gif)

_Can this pattern be extended to any other app?_

_Can you autoscale all microservices on the number of requests received?_

Unless they expose the metrics, the answer is no.

However, there's a workaround.

[KEDA ships with an HTTP add-on to enable HTTP scaling.](https://github.com/kedacore/http-add-on)

_How does it work!?_

**KEDA injects a sidecar proxy in your pod so that all the HTTP traffic is routed first.**

Then it measures the number of requests and exposes the metrics.

With that data at hand, you can trigger the autoscaler finally.

![Keda HTTP add-on architecture](https://res.cloudinary.com/learnk8s/image/upload/v1681721495/threads/nginx-scaling-8_ceoiqp.png)

_KEDA is not the only option, though._

**You could install the [Prometheus Adapter.](https://github.com/kubernetes-sigs/prometheus-adapter)**

The metrics will flow from Nginx to Prometheus, and then the Adapter will make them available to Kubernetes.

From there, they are consumed by the Horizontal Pod Autoscaler.

![Prometheus adapter architecture](https://res.cloudinary.com/learnk8s/image/upload/v1681730145/threads/nginx-scaling-9_dzaikl.png)

_Is this better than KEDA?_

They are similar, as both have to query and buffer metrics from Prometheus.

However, KEDA is pluggable, and the Adapter works exclusively with Prometheus.

![Similarity between KEDA & the Prometheus Adapter](https://res.cloudinary.com/learnk8s/image/upload/v1681730145/threads/nginx-scaling-10_morsxq.png)

_Is there a competitor to KEDA?_

A promising project called the [Custom Pod Autoscaler aims to make the pod autoscaler pluggable.](https://custom-pod-autoscaler.readthedocs.io/en/latest/)

However, the project focuses more on how those pods should be scaled (i.e. algorithm) than the metrics collection.

During my research, I found these links helpful:

- https://keda.sh/docs/2.10/scalers/prometheus/
- https://sysdig.com/blog/kubernetes-hpa-prometheus/
- https://github.com/nginxinc/nginx-prometheus-exporter#exported-metrics
- https://learnk8s.io/scaling-celery-rabbitmq-kubernetes

And finally, if you've enjoyed this thread, you might also like:

- [The Kubernetes workshops that we run at Learnk8s.](https://learnk8s.io/training)
- [This collection of past threads.](https://twitter.com/danielepolencic/status/1298543151901155330)
- [The Kubernetes newsletter I publish every week.](https://learnk8s.io/learn-kubernetes-weekly)