Cloud cost reporting alone isn’t going to cut your AWS bill
Tags: aws, kubernetes, devops, startup
Published: 2021-07-22 16:12:27

<!-- wp:paragraph -->
<p>As containerization rises in popularity, traditional cost optimization methods like monitoring and reporting are put to the test. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Getting a detailed report of your cloud spend is just <strong>one side of the coin</strong>. The other - and arguably more important one - is <strong>doing something about these findings</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Keep on reading to find out why adding automated optimization techniques to the mix makes all the difference (and generates some serious savings).</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Cloud cost reporting is a must-have, there’s no doubt about it</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Teams usually use Kubernetes in dynamic and multi-tenant environments. Since increasing resources is so easy, losing control over cloud spend is even easier. If left unchecked, a missed bug or architecture oversight might easily snowball into a massive expense.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Consider this story:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>One of <strong>Adobe’s</strong> development teams once <strong>burned $80,000 per day</strong> because of a computing job left running on Azure. Before someone discovered it and pulled the plug, <strong>the bill snowballed to more than <a href="https://www.teampay.co/insights/manage-cloud-costs/">half a million dollars</a></strong>. One simple alert would be enough to prevent this.</p>
<!-- /wp:paragraph -->

<!-- wp:quote {"className":"is-style-large"} -->
<blockquote class="wp-block-quote is-style-large"><p>That’s why users need a solid cost reporting tool that brings them cost visibility in detailed reports and sends alerts when certain parts of the infrastructure go beyond the set thresholds.&nbsp;</p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>Cost reporting is incredibly important, but like I said - it’s just one side of the cost optimization coin.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>But cost reporting alone doesn’t cut the chase</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>You need more than cost reports to make a real difference in your setup. After all, what cost monitoring and reporting tools give you are static recommendations that require a human to implement them.</p>
<!-- /wp:paragraph -->

<!-- wp:quote {"className":"is-style-large"} -->
<blockquote class="wp-block-quote is-style-large"><p>If you have a small environment, a human engineer can make it work. But think about the scale of a mid-sized company or enterprise. </p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>Applying all the recommendations manually and on a regular basis translates into time, which translates into cost (not to mention all the lost optimization opportunities for sudden traffic spikes and such).</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>What you need is to add <strong>automated cost optimization</strong> to the mix.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you’re wondering how the combination of Kubernetes cost reporting and optimization works in practice, here’s a case study that shows it step by step.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>How automated cost optimization helped reduce Amazon EKS costs in 15 minutes</h2>
<!-- /wp:heading -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh5.googleusercontent.com/WL4gQe96MxZzUmjME-U_DtSnAO5hvq9KpTU2utUhZvXWy0qqL8nTQcffz5Ff6GfQ3z_8uT-qKtlx4UDBbvivWW6k2bjnx3w09C74QYeCILFvyQHovesDw-Musp3T2BGbnuIltvh1" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p><strong>TL;DR</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>I provisioned an e-commerce app (<a href="https://github.com/castai/examples">see it here</a>) on an EKS cluster with 6 m5 nodes (2 vCPU, 8 GiB) on the Amazon EKS platform. Then, I deployed <a href="https://cast.ai/eks-optimizer/">an AI engine</a> to go through my application and suggest some optimizations. I got some pretty interesting recommendations in the Savings Report, so I activated the automated optimization feature.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Here are the results:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><strong>Initial cluster cost: $414 / month.</strong></li><li>Within <strong>15 minutes</strong> after turning the automation on, the cluster cost got down to <strong>$207 (a 50% reduction) </strong>by eliminating 3 nodes.</li><li>And <strong>5 minutes</strong> later, the solution added Spot Instances which made the cluster costs go down to <strong>$138 per month (a 66% reduction!)</strong>.</li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":3} -->
<h3>Here’s a more detailed take on this case study</h3>
<!-- /wp:heading -->

<!-- wp:heading {"level":3} -->
<h3>Step 1: Generating the Savings Report</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>After creating and deploying my EKS cluster, I connected it to CAST AI by creating a free account and selecting <em>Connect your cluster</em>.</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/fc3rkSJxugy-Q1AyBnB3a27HN4PAVYJOJNCtNTLYF14CuWFjF2_-cMeapNRR494-dW6_O1vyjOtn8-NVJTinEUc6Z3MmVJT2iC7fEMuewQIyI5UmvTGW31j9rR8jesrPrvjRBvKK" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh5.googleusercontent.com/oSRjPaBQifDoRwW_v1t3-9IyuI6KiFEqKv92dQWBaeG22G7JfcahK2dgmh1w6cZKLB-Aj6AT8-5F2_KBVM_bnGUWxnCj3_4z9fvh5b594uumkvPaakQvfbQkHs8OrRfNhKutVMOL" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>I then copied and ran the script successfully in my terminal.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The CAST AI agent analyzed my EKS cluster in read-only mode and generated this Savings Report:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh5.googleusercontent.com/ibksdy3608iGxRkN9wEJ-KvTr4ClJWi3xuV44AJK3b28yCV8FQw9oQuKYOK70LbCOq_ro5tSNSGjWPZa89nKT5_xLDbbIx8rRK6LbZkPPXp_Rn_LqmRoVU1LSCpfaW7Oi6sm2NCB" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>As it turns out, if I switched my 6 m5.large to 3 c5a.large, I could slash my bill by almost 60%.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>And I could get even higher savings (66.5%!) if I decided to use Spot Instances:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh6.googleusercontent.com/Dur8N8u7MUgcwIzcpinxLiihk2BCv36sShMGSVolYw68ZRJLlnwTZMxFo4a5TI2ThD6e0Lxg5teSTgTll0gERUwujFt4jfHmFDntaLk55AUibDIwRO46dISvaFMH8roM2oApMqeR" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":3} -->
<h3>Step 2: Activating the cost optimization</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Turning automated optimization on made a lot of sense, so I had to grab my AWS access key ID and Secret access key, and add both to the platform.&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To get the access keys, I had to run this script:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh4.googleusercontent.com/gyvjtYMafi6cMNAb-G9wJZH1MztwY-pQxrJgCmCuLTYkNpiLJRD_znqs_6BDGEWMw0QpbWnouBKcWLvumJwUBqYQcAgat4F-R4evh97YdywQvaFmCWmECQH_brmmVdYz_NXiYCLm" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":3} -->
<h3>Step 3: Enabling cost policies</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To achieve maximum cost savings, I turned on all the policies available in CAST AI:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>CPU Policy: I used this policy to set the maximum budget of 200 CPUs.</li></ul>
<!-- /wp:list -->

<!-- wp:list -->
<ul><li>Node autoscaler: Whenever I get unscheduled pods, CAST AI will start looking for a place to run them - starting with Spot Instances (if they’re Spot Instance-friendly) or On-Demand instances.</li></ul>
<!-- /wp:list -->

<!-- wp:list -->
<ul><li>Node Deletion + Evictor: Evictor is a background process that continuously reduces the cluster to the minimum number of nodes by bin-packing pods. Once a node becomes empty, it’s instantly deleted.&nbsp;</li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":3} -->
<h3>Step 4: Running Evictor</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>This is what the last policy and Evictor tool look like in action:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>One node (marked in red) is identified as a good candidate for eviction.</li><li>Evictor automatically moves pods to other nodes for “bin-packing.”</li><li>Once the node becomes empty, it’s deleted from the cluster.</li><li>Evictor returns to step 1 and keeps on looking for other nodes that could be deleted to cut costs.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>After about 10 minutes, Evictor deleted 3 nodes and left just 3 nodes running. CPUs were at a much healthier 80% rate.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The cost of my cluster is now $207.36 per month - so, 50% of the initial cost ($414 per month_.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Step 5: Moving workloads to new optimized nodes</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>This is a more advanced and optional step where CAST AI actively replaced my current nodes with more optimized ones - for example, Spot Instances. To do that, CAST AI cordons the cluster, drains nodes, and then replaces them with more optimized nodes.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>So, my nodes were cordoned:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh3.googleusercontent.com/rkJiSSY6WiznSOo2EHu7XY5C-TMNpP53qZyQOX2UFayBkySWO3TF1ndw4Qfh8d1O0FRUlepUsrrt00xNmHuYfoBrR-IqNeYNlBZA3-F1ioCoCcDg9aFGvuHlecxzdtBQjUz3dP15" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>The first two nodes were drained, and the AI engine selected the best instance type for these nodes. Here’s what I saw in my CAST AI dashboard:</p>
<!-- /wp:paragraph -->

<!-- wp:image -->
<figure class="wp-block-image"><img src="https://lh3.googleusercontent.com/HIUzBn-QnMHndSrzb2qATRve_a6t0YF2gN2wHVRsvq1fVfwxWEXuapwEMjn4cZdlLjko4pQ1FWaPhC_E_HxH9m8-y6FZAB1zdd_i6jY4MObMN7yi96sByIp_DGotY2pw2lhdpT99" alt=""/></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>As you can see, my cluster is now running on 2 nodes that cost me $138 per month. It’s hard to believe that I initially got a monthly EKS bill of $414.72!</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Running clusters on Amazon EKS? Give automated optimization a try</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Run the free <a href="https://cast.ai/eks-optimizer/">CAST AI Savings Report</a> to check how much you could potentially save. It also gives you actionable steps to reduce the bill that other tools ask you to pay for. And when you’re ready, turn automated cost optimization on to see how your cloud bill shrinks with every minute.&nbsp;</p>
<!-- /wp:paragraph -->