What is Kubernetes HPA and How Can It Help You Save on the Cloud?
Tags: cloud, kubernetes, devops, finops
Published: 2022-09-13 15:46:39

<!-- wp:paragraph -->
<p>Autoscaling is a core capability of Kubernetes. The tighter you configure the scaling mechanisms - <strong>HPA, VPA, and Cluster Autoscaler</strong> - the lower the waste and costs of running your application.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Kubernetes comes with three types of autoscaling mechanisms: Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA), and Cluster Autoscaler. Each of these adds a unique ingredient to your overarching goal of <a href="https://cast.ai/blog/guide-to-kubernetes-autoscaling-for-cloud-cost-optimization/" target="_blank" rel="noreferrer noopener">autoscaling for cloud cost optimization</a>.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>In this article, we focus on horizontal pod autoscaling. You can control how many pods run based on various metrics by configuring the Horizontal Pod Autoscaler (HPA) settings on your cluster. This gives you the ability to scale up or down according to demand.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Keep reading to learn what Kubernetes HPA is and how it works in a hands-on example.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-let-s-start-with-a-quick-recap-of-kubernetes-autoscaling">Let’s start with a quick recap of Kubernetes autoscaling</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Before diving into Horizontal Pod Autoscaler (HPA), let’s look at Kubernetes autoscaling mechanisms.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Kubernetes supports three types of autoscaling:&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><strong>Horizontal Pod Autoscaler (HPA)</strong>, which scales the number of replicas of an application.&nbsp;</li><li><strong>Vertical Pod Autoscaler (VPA)</strong>, which scales the resource requests and limits of a container.&nbsp;</li><li><strong>Cluster Autoscaler</strong>, which adjusts the number of nodes of a cluster.&nbsp;</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>These autoscalers work on one of two Kubernetes levels: pod and cluster. While Kubernetes HPA and VPA methods adjust resources at the pod level, the Cluster Autoscaler scales up or down the number of nodes in a cluster.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-what-is-kubernetes-horizontal-pod-autoscaler-hpa">What is Kubernetes Horizontal Pod Autoscaler (HPA)?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In many applications, usage changes over time - for example, more people visit an e-commerce store in the evening than around noon. When the demands of your application change, you can use the Horizontal Pod Autoscaler (HPA) to add or remove pods automatically based on CPU utilization.</p>
<!-- /wp:paragraph -->

<!-- wp:quote {"className":"is-style-plain"} -->
<blockquote class="wp-block-quote is-style-plain"><!-- wp:paragraph -->
<p>HPA makes autoscaling decisions based on metrics that you provide externally or custom metrics. </p>
<!-- /wp:paragraph --></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>To get started, you need to define how many replicas should run at any given time using the MIN and MAX values. Once configured, the Horizontal Pod Autoscaler controller takes care of checking metrics and making adjustments as necessary. It checks metrics every 15 seconds by default.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-how-does-horizontal-pod-autoscaler-work">How does Horizontal Pod Autoscaler work?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Configuring the HPA controller will monitor your deployment's pods and understand whether the number of pod replicas needs to change. To determine this, HPA takes a weighted mean of a per-pod metric value and calculates whether removing or adding replicas would bring that value closer to its target value.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Example scenario</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Imagine that your deployment has a target CPU utilization of 50%. You currently have five pods running there, and the mean CPU utilization is 75%. In this scenario, the HPA controller will add three replicas to bring the pod average closer to the target of 50%.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":13149,"sizeSlug":"large","linkDestination":"none"} -->
<figure class="wp-block-image size-large"><img src="https://cast.ai/wp-content/uploads/2022/09/DES-273-HPA-Visual-1200x628.png" alt="Kubernetes HPA" class="wp-image-13149"/></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2 id="h-when-to-use-kubernetes-hpa">When to use Kubernetes HPA?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Horizontal Pod Autoscaler is an autoscaling mechanism that comes in handy for scaling stateless applications. But you can also use it to support scaling stateful sets.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To achieve cost savings for workloads that experience regular changes in demand, <strong>use HPA in combination with cluster autoscaling</strong>. This will help you reduce the number of active nodes when the number of pods decreases.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-limitations-of-horizontal-pod-autoscaler">Limitations of Horizontal Pod Autoscaler&nbsp;</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Note that HPA comes with some limitations:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>It might require architecting your application with a scale-out in mind so that distributing workloads across multiple servers is possible.&nbsp;</li><li>HPA might not always keep up with unexpected demand peaks since new virtual machines may take a few minutes to load.</li><li>If you fail to set CPU and memory limits on your pods, they may frequently terminate or waste resources if you choose to do the opposite.&nbsp;</li><li>If the cluster is out of capacity, HPA cannot scale up until new nodes are added to the cluster. Cluster Autoscaler (CA) can automate this process.&nbsp;</li></ul>
<!-- /wp:list -->

<!-- wp:heading -->
<h2 id="h-what-do-you-need-to-run-horizontal-pod-autoscaler">What do you need to run Horizontal Pod Autoscaler?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Horizontal Pod Autoscaler (HPA) is a feature of the Kubernetes cluster manager that watches the CPU usage of pod containers, automatically resizing them as necessary to maintain a target level of utilization.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To do that, HPA requires a source of metrics. For example, when scaling based on CPU usage, it uses <a href="https://github.com/kubernetes-sigs/metrics-server" target="_blank" rel="noreferrer noopener">metrics-server</a>. If you want to use custom or external metrics for HPA scaling, you need to deploy a service implementing the custom.metrics.k8s.io API or external.metrics.k8s.io API; this provides an interface with a monitoring service or metrics source.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Custom metrics include network traffic, memory, or any value that relates to the pod's application. And if your workloads use the standard CPU metric, make sure to configure the CPU resource limits for containers in the pod spec.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-expert-tips-for-running-kubernetes-hpa">Expert tips for running Kubernetes HPA</h2>
<!-- /wp:heading -->

<!-- wp:heading {"level":3} -->
<h3 id="h-1-install-metrics-server">1. Install metrics-server&nbsp;</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Kubernetes HPA needs to access per-pod resource metrics to make scaling decisions. These values are retrieved from the metrics.k8s.io API provided by the metrics-server.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3 id="h-2-configure-resource-requests-for-all-pods">2. Configure resource requests for all pods</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Another key source of information for HPA’s scaling decisions is observed CPU utilization values of pods. But how are these values calculated? They are a percentage of the resource requests from individual pods.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you miss resource request values for some containers, these calculations might become entirely inaccurate and lead to suboptimal operation and poor scaling decisions. That’s why it’s worth configuring resource request values for all containers of every pod that’s part of the Kubernetes controller scaled by the HPA.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3 id="h-3-configure-custom-and-external-metrics">3. Configure custom and external metrics</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p><strong>Custom metrics</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can configure Horizontal Pod Autoscaler (HPA) to scale based on custom metrics, which are internal metrics that you collect from your application. HPA supports two types of custom metrics:&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Pod metrics - averaged across all the pods in an application, which support only the target type of AverageValue.&nbsp;</li><li>Object metrics - metrics describing any other object in the same namespace as your application and supporting target types of Value and AverageValue.&nbsp;</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Remember to use the correct target type for pod and object metrics when configuring custom metrics.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>External metrics</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>These metrics allow HPA to autoscale applications based on metrics that are provided by third-party monitoring systems. External metrics support target types of Value and AverageValue.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>When deciding between custom and external metrics, go for custom metrics because securing an external metrics API is more difficult than getting an internal one.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3 id="h-4-verify-that-your-hpa-and-vpa-policies-don-t-clash">4. Verify that your HPA and VPA policies don’t clash</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Vertical Pod Autoscaler automates requests and limits configuration, reducing overhead and achieving cost savings. Horizontal Pod Autoscaler, on the other hand, aims to scale out rather than up or down.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Double-check that your binning and packing density settings <strong>aren't in conflict with each other</strong> when designing clusters for business or purpose-class tier of service.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3 id="h-5-use-instance-weighting-scores">5. Use instance weighting scores</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Suppose one of your workloads ends up consuming more than it requested. Is this happening because the resources are needed? Or did the workload consume them because they were available but not critically required?</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Use instance weighting</strong> when choosing instance sizes and types for autoscaling. Instance weighting is useful, especially when you adopt a diversified allocation strategy and use spot instances.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-example-hpa-demo">Example: HPA demo</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>As this is one of core Kubernetes features, the cloud service provider we use shouldn’t matter. But for this example we will be using GKE.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can create a cluster via the UI, or via the gcloud utils like so:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>gcloud container \

&nbsp;&nbsp;&nbsp;&nbsp;--project "&#91;your-project]" clusters create "&#91;cluster-name]" \

&nbsp;&nbsp;&nbsp;&nbsp;--release-channel None \

&nbsp;&nbsp;&nbsp;&nbsp;--zone "europe-west3-c" \

&nbsp;&nbsp;&nbsp;&nbsp;--node-locations "europe-west3-c" \

&nbsp;&nbsp;&nbsp;&nbsp;--machine-type "e2-standard-2" \

&nbsp;&nbsp;&nbsp;&nbsp;--image-type "COS_CONTAINERD" \

&nbsp;&nbsp;&nbsp;&nbsp;--disk-size "50" \

&nbsp;&nbsp;&nbsp;&nbsp;--enable-autorepair \

&nbsp;&nbsp;&nbsp;&nbsp;--num-nodes "3"</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>We can then connect to the cluster using:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>gcloud container clusters get-credentials &#91;cluster-name] --zone europe-west3-c --project &#91;your-project]</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>This should also switch your context to the cluster, so whenever you use kubectl, you’ll be within this cluster’s context.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After we’ve done that, we can verify that we can see the nodes with&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl get nodes

NAME&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; STATUS &nbsp; ROLES&nbsp; &nbsp; AGE &nbsp; &nbsp; VERSION

gke-valdas-1-default-pool-cf1cd6be-cvc4 &nbsp; Ready&nbsp; &nbsp; &lt;none&gt; &nbsp; 4m50s &nbsp; v1.22.11-gke.400

gke-valdas-1-default-pool-cf1cd6be-q62h &nbsp; Ready&nbsp; &nbsp; &lt;none&gt; &nbsp; 4m49s &nbsp; v1.22.11-gke.400

gke-valdas-1-default-pool-cf1cd6be-xrf0 &nbsp; Ready&nbsp; &nbsp; &lt;none&gt; &nbsp; 4m50s &nbsp; v1.22.11-gke.400</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p><br>GKE comes with the metrics server preinstalled; we can verify that using</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl get pods -n kube-system | grep metrics

gke-metrics-agent-n92rl&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1/1 &nbsp; &nbsp; Running &nbsp; 0&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 7m34s

gke-metrics-agent-p5d49&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1/1 &nbsp; &nbsp; Running &nbsp; 0&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 7m33s

gke-metrics-agent-tf96r&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1/1 &nbsp; &nbsp; Running &nbsp; 0&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 7m34s

metrics-server-v0.4.5-fb4c49dd6-knw6v&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2/2 &nbsp; &nbsp; Running &nbsp; 0&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 7m20s</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Note, if your cluster doesn’t have a metrics server, you can easily install it using one command:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>You can find more info here: <a href="https://github.com/kubernetes-sigs/metrics-server" target="_blank" rel="noreferrer noopener">https://github.com/kubernetes-sigs/metrics-server</a>&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We can also use the `top` command to verify that the metrics are collected:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl top pods -A

NAMESPACE &nbsp; &nbsp; NAME &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; CPU(cores) &nbsp; MEMORY(bytes)

kube-system &nbsp; event-exporter-gke-5479fd58c8-5rt4b&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 12Mi

kube-system &nbsp; fluentbit-gke-5r6v5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 5m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 22Mi

kube-system &nbsp; fluentbit-gke-nwcx9&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 4m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 25Mi

kube-system &nbsp; fluentbit-gke-zz6gl&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 4m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 25Mi

kube-system &nbsp; gke-metrics-agent-n92rl&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 26Mi

kube-system &nbsp; gke-metrics-agent-p5d49&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 3m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 27Mi

kube-system &nbsp; gke-metrics-agent-tf96r&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 3m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 26Mi

kube-system &nbsp; konnectivity-agent-6fbc8c774c-4vvff&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 6Mi

kube-system &nbsp; konnectivity-agent-6fbc8c774c-lsk26&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 7Mi

kube-system &nbsp; konnectivity-agent-6fbc8c774c-rnvp2&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 6Mi

kube-system &nbsp; konnectivity-agent-autoscaler-555f599d94-f2w5n &nbsp; &nbsp; &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 4Mi

kube-system &nbsp; kube-dns-85df8994db-lvf55&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 30Mi

kube-system &nbsp; kube-dns-85df8994db-mvxv5&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 30Mi

kube-system &nbsp; kube-dns-autoscaler-f4d55555-pctcj &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 11Mi

kube-system &nbsp; kube-proxy-gke-valdas-1-default-pool-cf1cd6be-cvc4 &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 22Mi

kube-system &nbsp; kube-proxy-gke-valdas-1-default-pool-cf1cd6be-q62h &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 23Mi

kube-system &nbsp; kube-proxy-gke-valdas-1-default-pool-cf1cd6be-xrf0 &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 26Mi

kube-system &nbsp; l7-default-backend-69fb9fd9f9-fctch&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1Mi

kube-system &nbsp; metrics-server-v0.4.5-fb4c49dd6-knw6v&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 27m&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 24Mi

kube-system &nbsp; pdcsi-node-mf6pt &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 9Mi

kube-system &nbsp; pdcsi-node-sxdrg &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 3m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 9Mi

kube-system &nbsp; pdcsi-node-wcnw7 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 3m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 9Mi</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Now, let’s create a single replica deployment with resource requests and limits:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: apps/v1

kind: Deployment

metadata:

&nbsp;name: hpa-demo

&nbsp;labels:

&nbsp;&nbsp;&nbsp;app: nginx

spec:

&nbsp;replicas: 1

&nbsp;selector:

&nbsp;&nbsp;&nbsp;matchLabels:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;app: nginx

&nbsp;template:

&nbsp;&nbsp;&nbsp;metadata:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;labels:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;app: nginx

&nbsp;&nbsp;&nbsp;spec:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;containers:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- name: nginx

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;image: k8s.gcr.io/nginx-slim:0.8
&nbsp; &nbsp; &nbsp; ports:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- containerPort: 80

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;resources:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;requests:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cpu: 200m

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;limits:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;cpu: 1000m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Let’s save this to a demo.yaml and run</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl apply -f demo.yaml</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>We can check that it’s deployed via</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl get deploy -n default

NAME &nbsp; &nbsp; &nbsp; READY &nbsp; UP-TO-DATE &nbsp; AVAILABLE &nbsp; AGE

hpa-demo &nbsp; 1/1 &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 17m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Lastly, before configuring HPA, we need to expose a service that we can call to increase the load, which HPA will act upon.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Let’s create a service.yaml like so:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: v1

kind: Service

metadata:

name: hpa-demo

labels:

&nbsp;&nbsp;app: nginx

spec:

ports:

- port: 80

selector:

&nbsp;&nbsp;app: nginx</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>And apply it using</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl apply -f service.yaml</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>We can verify that the service is working like so:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl get services
NAME &nbsp; &nbsp; &nbsp; &nbsp; TYPE&nbsp; &nbsp; &nbsp; &nbsp; CLUSTER-IP &nbsp; &nbsp; EXTERNAL-IP &nbsp; PORT(S) &nbsp; AGE

hpa-demo &nbsp; &nbsp; ClusterIP &nbsp; 10.92.62.191 &nbsp; &lt;none&gt;&nbsp; &nbsp; &nbsp; &nbsp; 80/TCP&nbsp; &nbsp; 7m35s

kubernetes &nbsp; ClusterIP &nbsp; 10.92.48.1 &nbsp; &nbsp; &lt;none&gt;&nbsp; &nbsp; &nbsp; &nbsp; 443/TCP &nbsp; 48m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>As you can see, the hpa-demo service exists.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Finally, we need to configure HPA. To do that, we can create a file called hpa.yaml and fill it in with:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: autoscaling/v1

kind: HorizontalPodAutoscaler

metadata:

name: hpa-demo

spec:

scaleTargetRef:

&nbsp;&nbsp;apiVersion: apps/v1

&nbsp;&nbsp;kind: Deployment

&nbsp;&nbsp;name: hpa-demo

minReplicas: 1

maxReplicas: 5

targetCPUUtilizationPercentage: 60</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Once again, we apply it using</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl apply -f hpa.yaml

Now, let’s watch HPA in action using

&gt;kubectl get hpa -w

NAME &nbsp; &nbsp; &nbsp; REFERENCE &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; TARGETS &nbsp; MINPODS &nbsp; MAXPODS &nbsp; REPLICAS &nbsp; AGE

hpa-demo &nbsp; Deployment/hpa-demo &nbsp; 0%/60%&nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2m31s</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p><br>As you can see, nothing is happening. But remember how we created a service earlier? Let’s start generating some load using busybox:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl run -i --tty load-generator --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://hpa-demo; done"</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>We should immediately see a load increase under our HPA watch command:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>hpa-demo &nbsp; Deployment/hpa-demo &nbsp; 16%/60% &nbsp; &nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 8m47s</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>The load is still not enough to reach our target, but for the sake of this demo, let’s set the targetCPUUtilizationPercentage: 10 and re-apply the hpa.yaml.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Under the watch command, we should see that our target is exceeded and a replica gets added.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>hpa-demo &nbsp; Deployment/hpa-demo &nbsp; 16%/10% &nbsp; &nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 15m

hpa-demo &nbsp; Deployment/hpa-demo &nbsp; 16%/10% &nbsp; &nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; &nbsp; 2&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 15m

hpa-demo &nbsp; Deployment/hpa-demo &nbsp; 9%/10%&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; &nbsp; 2&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 16m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>We can verify this via</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl get deploy
NAME &nbsp; &nbsp; &nbsp; READY &nbsp; UP-TO-DATE &nbsp; AVAILABLE &nbsp; AGE

hpa-demo &nbsp; 2/2 &nbsp; &nbsp; 2&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 2 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 40m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>As you can see, we have two pods while the deployment specified only one - so our HPA policy worked.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We can proceed to test downscaling by deleting the load-gen pod:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl delete pod load-generator</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>After a while, we should see:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>NAME &nbsp; &nbsp; &nbsp; REFERENCE &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; TARGETS &nbsp; MINPODS &nbsp; MAXPODS &nbsp; REPLICAS &nbsp; AGE

hpa-demo &nbsp; Deployment/hpa-demo &nbsp; 0%/10%&nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; 5 &nbsp; &nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 26m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>This means that downscaling worked.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We can verify that using:</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>&gt; kubectl get deploy

NAME &nbsp; &nbsp; &nbsp; READY &nbsp; UP-TO-DATE &nbsp; AVAILABLE &nbsp; AGE

hpa-demo &nbsp; 1/1 &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 1 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 48m</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>That’s basically it. We’ve seen how we can easily upscale and downscale based on pod resource usage.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-gain-real-time-cost-visibility-when-using-hpa">Gain real-time cost visibility when using HPA</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Increased scalability poses a challenge to cost monitoring and control in Kubernetes because autoscalers constantly adjust capacity.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>CAST AI provides a free cost monitoring product you can use to get an hourly, daily, weekly, and monthly overview of your cloud cost.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Connect your cluster in one minute or less to instantly see your current costs in real time and access months of past cost data for comprehensive reporting.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:group {"style":{"spacing":{"padding":{"top":"2rem","right":"2rem","bottom":"2rem","left":"2rem"},"margin":{"top":"2rem","bottom":"2rem"}}},"backgroundColor":"cast-black","className":"pattern-cta-text-btn","layout":{"type":"flex","flexWrap":"wrap","justifyContent":"space-between","verticalAlignment":"center"}} -->
<div class="wp-block-group pattern-cta-text-btn has-cast-black-background-color has-background" style="margin-top:2rem;margin-bottom:2rem;padding-top:2rem;padding-right:2rem;padding-bottom:2rem;padding-left:2rem"><!-- wp:group {"style":{"spacing":{"padding":{"top":"0rem","right":"0rem","bottom":"0rem","left":"0rem"},"blockGap":"0.5rem"}},"layout":{"type":"default","contentSize":"520px"}} -->
<div class="wp-block-group" style="padding-top:0rem;padding-right:0rem;padding-bottom:0rem;padding-left:0rem"><!-- wp:heading {"level":5} -->
<h5 id="h-cast-ai-clients-save-an-average-of-63-on-their-kubernetes-bills">CAST AI clients <mark style="background-color:rgba(0, 0, 0, 0)" class="has-inline-color has-cast-green-color">save an average of 63%</mark> on their Kubernetes bills</h5>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Connect your cluster and see your costs in 5 min, no credit card required.</p>
<!-- /wp:paragraph --></div>
<!-- /wp:group -->