CPU requests and limits in Kubernetes
Tags: kubernetes, devops
Published: 2023-03-06 12:38:19

_In Kubernetes, what should I use as CPU requests and limits?_

Popular answers include:

- Always use limits!
- NEVER use limits, only requests!
- I don't use either; is it OK?

Let's dive into it.

In Kubernetes, you have two ways to specify how much CPU a pod can use:

1. **Requests** are usually used to determine the average consumption.
1. **Limits** set the max number of resources allowed.

The Kubernetes scheduler uses requests to determine where the pod should be allocated in the cluster.

Since the scheduler doesn't know the consumption (the pod hasn't started yet), it needs a hint.

But it doesn't end there.

![The Kubernetes scheduler uses requests to decide how to assign a pod to a node](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-3_lwfc76.png)

CPU requests are also used to repart the CPU to your containers.

Let's have a look at an example:

- A node has a single CPU.
- Container A has requests equal to 0.1 vCPU.
- Container B has requests equal to 0.2 vCPU.

_What happens when both containers try to use 100% of the available CPU?_

![CPU usage in two containers](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-4_jjafrb.png)

Since the CPU request doesn't limit consumption, both containers will use all available CPUs.

However, since container B's request is doubled compared to the other, the final CPU distribution is: **Container 1 uses 0.3vCPU and the other 0.6vCPU (double the amount).**

![Both container use all available CPU but they keep proportional quotas](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-5_dq1jic.png)

Requests are suitable for:

- Setting a baseline (give me at least X amount of CPU).
- Setting relationships between pods (this pod A uses twice as much CPU as the other).

But do not help set hard limits.

For that, you need CPU limits.

**When you set a CPU limit, you define a period and quota.**

Example:

- period: 100000 microseconds (0.1s).
- quota: 10000 microseconds (0.01s).

I can only use the CPU for 0.01 seconds every 0.1 seconds.

That's also abbreviated as "100m".

![Quota and period in CPU limits](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-6_vd40de.png)

**If your container has a hard limit and wants more CPU, it has to wait for the next period.**

Your process is throttled.

![A process being CPU throttled](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-7_cnv3jf.png)

_So what should you use as CPU requests and limits in your Pods?_

A simple (but not accurate) way is to calculate the smallest CPU unit as:

```
REQUEST = NODE_CORES * 1000 / MAX_NUM_PODS_PER_NODE
```

For a 1 vCPU node and a limit of 10 Pods, that's a `1 * 1000 / 10 = 100Mi` request.

**Assign the smallest unit or a multiplier of it to your containers.**

![Assigning CPU requests to Pods and containers](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-8_c0nqm2.png)

For example, if you don't know how much CPU you need for Pod A, but you identified it is twice as Pod B, you could set:

- Request A: 1 unit
- Request B: 2 units

If the containers use 100% CPU, they repart the CPU according to their weights (1:2).

![Two pods competing for CPU resources](https://res.cloudinary.com/learnk8s/image/upload/v1678078845/threads/cpu-request-limit-9_qzqjao.png)

**A better approach is to monitor the app and derive the average CPU utilization.**

You can do this with your existing monitoring infrastructure or use the [Vertical Pod Autoscaler](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler) to monitor and report the average request value.

![Vertical Pod Autoscaler with Goldilocks](https://res.cloudinary.com/learnk8s/image/upload/v1678078846/threads/demo_agsihk.gif)

_How should you set the limits?_

1. Your app might already have "hard" limits. (Node.js is single-threaded and uses up to 1 core even if you assign 2).
1. You could have: limit = 99th percentile + 30-50%.

You should profile the app (or use the VPA) for a more detailed answer.

![99th percentile for CPU](https://res.cloudinary.com/learnk8s/image/upload/v1678078846/threads/cpu-request-limit-10_egg6vb.png)

_Should you always set the CPU request?_

**Absolutely, yes.**

This is a standard good practice in Kubernetes and helps the scheduler allocate pods more efficiently.

_Should you always set the CPU limit?_

This is a bit more controversial, but, in general, I think so.

You can find a deeper dive here: https://dnastacio.medium.com/why-you-should-keep-using-cpu-limits-on-kubernetes-60c4e50dfc61

Also, if you want to dig in more a few relevant links:

- https://learnk8s.io/setting-cpu-memory-limits-requests
- https://medium.com/@betz.mark/understanding-resource-limits-in-kubernetes-cpu-time-9eff74d3161b
- https://nodramadevops.com/2019/10/docker-cpu-resource-limits/

And finally, if you've enjoyed this thread, you might also like:

- The Kubernetes workshops that we run at Learnk8s https://learnk8s.io/training
- This collection of past threads https://twitter.com/danielepolencic/status/1298543151901155330
- The Kubernetes newsletter I publish every week https://learnk8s.io/learn-kubernetes-weekly