Stress testing Karpenter with EKS and Qovery
Published: 2024-04-02 06:46:59

Let’s unroll my tweet (yes, I’m not over saying Tweet even with X), where I stressed Karpenter with EKS and Qovery.

> ⚠️ This is a stress test where I spent 1:30 hour; it’s not a benchmark. I’ll probably run some benchmarking to compare with Kubernetes node-autoscaler at some point, but not today :)

**Setup**
- 30 environments
- Each environment: 3 containerized apps + 2 container databases (Redis and Postgres) with EBS storage
- 1 EKS cluster with Karpenter deployed with Qovery

If you’re not familiar with [Karpenter](https://karpenter.sh) — watch my quick intro. But in a nutshell, Karpenter is a better node autoscaler for Kubernetes (say goodbye to wasted compute resources). It is open-source and built by the AWS team. Qovery is an Internal Developer Platform I’m a co-founder) that we’ll use to spin up our EKS cluster with Karpenter.

Let’s see how things behaved.

## 30 Environments Deployed

{% embed https://www.loom.com/share/f96a164d66ec453e95cee4941c1d5f6d %}

I created and deployed my 30 environments — and we can see that my apps are scheduled on my cluster. The EC2 nodes are dynamically allocated to the cluster (we can see t3.smallinstances). I don’t even have to choose the instance type. Karpenter is smart to choose the right instance depending on the following criteria:

- The workload of my apps
- The instance types available
- The best choice in terms of performance vs. costs

It’s truly a game-changer!

![Note also that I’ve requested to allocate spot instances. (configurable via Qovery cluster interface)](https://miro.medium.com/v2/resize:fit:4800/format:webp/1*rlY__3BWCmjffhKJ7eyVjg.png)

_Note also that I’ve requested to allocate spot instances. (configurable via Qovery cluster interface)_

## Dynamic Node Allocation

{% embed https://www.loom.com/share/cd272c5780ca4704a5767d9545b109eb %}

I changed the CPU and RAM of one app from my env-1 environment to see how Karpenter will reallocate resources.

- From: 250m CPU and 256 MB RAM
- To: 2000m CPU and 8GB RAM

It just works nicely, and we can see that Karpenter scheduled a t3a.xlarge instance to handle my app requirement.

## Cluster Scaled Down

If you are familiar with the default Kubernetes node autoscaler — you know how hard it is to downscale your cluster. For many reasons that I’ll not enumerate here… But in the case of Karpenter. It just works once again!

{% embed https://www.loom.com/share/a30432e5556547658c8c3a32bd3024d6 %}

I stopped all my environments, and as expected, my cluster is down-scaled in less than 10 minutes as well. So I don’t have wasted resources!

## Cluster Scaled Up.. Again

I wanted to see the cluster scaling up from 3 nodes to x nodes and see how fast it was... In less than 15 minutes, I got all my nodes provisioned and my 30 environments up and running.
Use Cases

{% embed https://www.loom.com/share/8566807bee484ba695d991acc6fe4480 %}

At Qovery, we have hundreds of customers running testing environments with our Preview Environment product. Karpenter will solve many issues that they were facing previously. Just to mention a few...

- No need to allocate a specific node pool size now
- Cluster Scale-down is well handled, and no more resources are wasted (reducing AWS costs)

Interested in giving it a shot? [Sign up to Qovery](https://start.qovery.com) and create your EKS cluster with Karpenter :)