Optimising your Cloud Infrastructure Spending
By: John Easton
Tags: Cloud Infrastructure, Iaas, Cloud Migration, Cloud, Cloud Technology

The Cloud Engagement Hub blog, here on Medium, talks a lot about different technologies and techniques used by organisations on their respective cloud journeys. Containerisation is clearly the strategic choice that many clients are making, though the reality is that a lot of workloads that you want to move to cloud are complex and many will be best suited to lift as virtual machines (VMs) for the time being. If this applies to you, be very aware: if you don’t change how you manage these VMs, it will cost you much more than it does now. The following article will talk about the patterns you should adopt, and the automation required to change the way you work. Without these “new ways of working”, cloud is never cheaper.

Let us start by revisiting our journey map in Figure 1. The modernisation approaches we are talking about in this article are the top two of those in blue; namely “Virtual > Virtual” and “Physical > Virtual”.

Figure 1 Cloud Application Modernisation Journey Map

Before we dive into the details, let’s dispel some myths. Most client journeys start with the consumption of IaaS services despite the fact that more value is realised in the application space than down in the infrastructure. One of the simple reasons for this is that enterprises spend more on application development and maintenance (ADM) than infrastructure — on average approximately $1.50 spend on ADM for every $1 on infrastructure.

Secondly, although there is a significant move towards containers and containerisation — see here and here for some related content — many organisations are still consuming virtual machines as their primary cloud infrastructure platform. They expect to see cost savings with this “lift-n-shift” approach; rehosting existing applications to cloud-based VMs with minimal architectural changes; but all too frequently, savings fail to materialise. Even though containerisation has a higher implementation cost, the ROI it provides is higher (310%) than the lift-n-shift approach (250%). But that theoretical lift-n-shift ROI depends upon you changing the way you work. It’s all about doing things differently.

That said, even though there is more value to be realised from the application space and through containerisation, there are still many things that can be done to optimise infrastructure spending. So, what are those patterns that can be used to better optimise your infrastructure spend?

Note: throughout this article, example prices are shown (in US dollars) to illustrate the points being made and to quantify potential savings. Though correct at the time of writing, prices change frequently and so these are not able to be guaranteed as correct in the future.

Pattern 1: “Turn it off!”

If your cloud is costing you more than your on-premises infrastructure to do the same thing, then chances are, you’re running it in the same way. Let’s face it, in many cases, cloud is only cheaper when you turn it off. If I create VMs in a cloud and leave them running for years as I might have done on-premises, then it’s almost always going to cost more. Which brings us neatly to the ability to suspend or hibernate a virtual machine. This allows you to temporarily stop using a VM while keeping it intact and without the need to re-provision or reconfigure it. It can be booted up again and you don’t pay for the time it’s not being used. Well, not quite true. It’s always important to understand what you stop paying for and what you will continue to be charged while it’s suspended (typically storage and maybe network addresses). There may be minimum usage charges or limitations on whether a VM can indeed be suspended at all and for how long. Remember also that suspended images are not patched or updated while suspended.

For systems supporting office hours working, use hourly billing and suspend these overnight. If the system has minimal configuration above the standard image or if the configuration activity is automated, it may prove more cost-effective to delete these resources and recreate them the following day.

The relative cost effects can be seen below. We use for this example, an hourly priced VM with SAN storage which costs $0.14/hr

Table 1. Cost effects of suspension and deletion of VMs when not used

You can see from this that suspending the VM saves 53% compared to just leaving it running. Deleting it saves 76%. Because you are always paying for the storage, a small VM with a lot of storage may not save you as much when suspended as in this example.

Pattern 2: “Right-size” your VMs

There is an oft-quoted statistic that the majority of x86 systems running within an enterprise are less than 5% utilised. Note that this is not the same as physical server utilisation. The use of virtualisation has tended to drive up average server utilisation as systems are consolidated onto the same physical hardware, but even here, average server utilisation is unlikely to be above 50%. The move to cloud provides an opportunity to “right-size” both your physical and virtual server fleet and reduce the amount of wasted resource and hence, cost.

If we consider a number of different VM sizes, we might see a range that looks like this:

Table 2. Cost effects of different VM sizes

If I have the equivalent of an X-Large server today and this is less than 50% utilised, as would be likely the case based on industry metrics, I should be able to right-size the physical server when moving to the cloud. I could similarly right-size an on-premises VM. This would allow me to reduce costs by about 50%. If I have servers which are less than 25% utilised, then these savings are of the order of 75%.

Doing this well requires you to have a good understanding of not only how the existing VMs or physical servers are being used, but also the constraining factor(s) for the workload. Most cloud providers offer a range of profiles that are optimised for particular workload needs (computation or memory) in addition to a more typical “balanced” option. The relative costs of different VM profiles for the same compute performance can be seen here:

Table 3. Cost effects of different VM profiles

While we can see the potential to save here, many organisations seemingly struggle with both the ‘t-shirt’ sizing approach and standardisation in general because again it’s different to what they did before. There are experts in many organisations whose role seems to be to exactly size a server to meet a specific application need. Selecting processors, memory, storage, even specific network adapters to precisely fit. In those IT shops where it took weeks or months to procure a physical server, this might have had some merit. Also, companies where there was no sharing of resources or budgets between teams wanted to get this choice just right. But cloud just isn’t like this. If I get my sizing wrong, I simply delete the ‘incorrect’ server and provision a ‘better’ one. If that isn’t perfect, I can repeat the process until I get it optimally sized for my workload’s needs. And it doesn’t take weeks or months to do this, rather minutes! The ‘t-shirt’ sizing approach frequently means that it’s never going to be perfect, but it’s good enough. Automation and ruthless standardisation are key to making this work well — but that is bugbear for another article.

There are other levers that we can pull to further optimise our VM costs. The first of these is to use the right technology generation for your workload needs. Most organisations have some technical debt and see cloud as a way to reduce this by selecting the latest technologies available in the service catalog. The question you need to be asking though is, at what cost? Consider the following example of a bare metal server. In this example, a dual processor / 16 cores device with memory, storage etc remaining the same:

Table 4. Cost effects of different processor generations

Obviously, the latest generation processor on this list has a clock speed almost twice as fast as the earlier generation ones, but if you don’t need, or your applications cannot take advantage of this, then there is potential for a 40% cost saving by not taking this latest device.

Datacentre location can also have an impact too due to the different taxation regimes operating in different countries. Below are the monthly costs for similarly configured VMs in a number of different countries:

Table 5. Cost effects of cloud datacentre location

You may not have the ability to choose the datacentre in which your workloads can run for regulatory or policy reasons, but if you can, it is worth checking the price differential between different locations. As we can see from this example, up to a 20% price differential across different datacentre locations might not be uncommon.

Some IT departments have a good understanding of the utilisation of their existing physical and virtual servers, but many don’t. If you’re in the latter category, figuring out the correct size of a new cloud resource may be problematic. Remember the point above that this doesn’t have to be perfect and you can always remove and re-deploy. The key here is to monitor the new system for a while and then decide whether action is needed. It is all too easy to over-estimate the resource needs or err on the safe side and go larger than you might need. If you then forget about this and hence leave it oversized, you’ll be paying more in the long run. There are many ways to measure resource consumption. The key point here is to actually do it and then decide, based on real data, whether the VM should be resized or not. In most clouds, compute resources double between t-shirt sizes: 2 vCPUs — 4 vCPUs — 8 vCPUs etc. The same is true for memory: 4GB — 8GB — 16GB. If you find your VMs are less than 50% utilised, then downsizing is likely to be possible. And if you do get it wrong, you can always go back again with minimal effort.

Pattern 3: Make use of transient (spot priced) VMs

Transient or spot priced VMs make use of unused cloud capacity and are offered at a lower (spot) price compared to a normal VM. The available capacity will vary based on where you want to provision the resources, the time of day, day of the week etc. As their name suggests however, the downside is that access to these resources is not guaranteed, they typically have no SLAs associated with them and they can be reclaimed by the cloud at very short notice. There may also be limits on how many spot instances you have access to at any one time.

Because they can be interrupted at any time, or indeed may not be able to be provisioned due to a lack of capacity, spot instances are most often used for workloads that can be interrupted or where there are less stringent “time to result” requirements. These are typically, but not uniquely, non-production workloads such as development and testing, but batch processing and also high-performance computation are also good candidates.

If you have workloads that can take advantage of them, the savings can be considerable as shown below:

Table 6. Cost effects of spot pricing on different VM sizes

Some clouds offer options to control when spot priced VMs are reclaimed based on price thresholds. This gives you better budgetary control

As with “right-sizing”, if you don’t have a particular VM size or locality requirement you may be able to find even cheaper options by shopping around. The key here is to understand which workloads you run may be suitable to run on spot priced VMs.

Consider a compute intensive workload that an organisation runs at the end of each week. It is highly variable in nature, based upon the business completed, and can require 30% additional capacity to complete for a busy week. Let us assume for the basis of this example that this may take 8 hours to run on 100 nodes (VMs). If we assume that this is managed by a workload scheduler which can resubmit units of work on ‘failed’ compute nodes, we can see the implications here:

Table 8. Cost effects of spot pricing on different weekly usage profiles.

If there is no deadline by which time a result needs to be calculated, you could solely use spot instances and realise savings in the range of 70% however it has to be stressed that not every workload can take advantage of these spot-prices so choose carefully.

Pattern 4: Reserve capacity for things that run 24x7

Spot price instance availability isn’t guaranteed. If you need to guarantee access to VM resources at some later date, enter the “reserved instance”. This guarantees you capacity and by making a longer-term commitment to the cloud platform, you will see lower unit prices. The longer the commitment, the greater the discount.

Reserved instances usually have restrictions or limitations, so you need to understand what you are getting into here. These restrictions might include not being available for every option in the service catalog and limits on the number of reservations you can make. Reservations typically don’t renew automatically so you need to keep on top of when the term ends so that you don’t suddenly find yourself without capacity when you need it or faced with a monthly bill larger than you were expecting.

Remember that you are reserving capacity and that if you don’t use that reservation, you are still paying for it as opposed to the Pay-As-You-Go model where you wouldn’t be. Often your cancellation options are limited, and you may have to pay some penalty (a cancellation fee) to get out of the commitment you previously made.

While the discounted price arising from the longer commitment, may look attractive you need to be sure that you can utilise them effectively. For predictable workloads running all the time, it makes sense to use reserved instances. Where there is variable usage or a lack of predictability, the case is much less clear and the savings you might make by using normal PAYG instances and turning them off when they are unused likely outweighs the reserved instance discounted price. You need to have a good view of your past and future utilisation to make the best choice.

To give some comparison of the “commitment effect” here let us consider what this might mean in practice:

Table 9. Cost effects of increased commitment to the cloud platform

Again, the savings realised through this approach can be considerable, but only where you are able to fully utilise the capacity you have reserved. Even if you are getting a lower unit price, you still run the risk of wasting money if you can’t.

A further ‘flavour’ of increased commitment available in some cloud platforms is the “Dedicated Host”. These are virtualised, single-tenant, dedicated servers into which your VMs can be placed. Mostly, these tend to be used to get isolation from other cloud users, whether that is for security reasons or to avoid ‘noisy neighbour’ problems. They do however also offer financial benefit because the monthly cost of the entire physical server is usually lower than purchasing the same capacity in standard VMs.

Table 10. Cost effects of dedicated servers vs standard VMs

As you can see, the percentage saving in this example is quite low. Some dedicated host offerings also allow you to control resource overcommitment allowing greater savings to be realised by putting more VMs onto the server. As with any “commitment” offering, you need to be sure that you can actually take full advantage of it or you will be wasting money if not.

The key role of automation

Automation and standardisation are fundamental to making these four patterns work well. When dealing with clients, concerns are often expressed about what happens if you forget to turn off VMs at the end of a day or over a weekend. Automation that takes this off your hands and operates according to policies is thus key.

Creation, configuration and deletion of VMs needs to be fully automated. Whether you are using Terraform, Ansible, Spinnaker or many of the other offerings in this space, doesn’t matter. The key thing is that human intervention is not required for any of these tasks. These automation capabilities then need the ability to be activated either at specific times (e.g. the end of the working day) or when other conditions are true (e.g. there are no users using a particular service).

This then brings us neatly to talk about the use of autoscaling. Rather than pre-configuring a pool of servers to support a workload, consider using autoscaling functionality provided by the cloud to dynamically allocate and deallocate resources as your workload demands change over time. While adding instances as more load is placed on your applications and services may be helpful, more interesting here is the removal of instances as load subsides. This removes unnecessary resources and hence reduces your costs. When working well, this ensures that you don’t have more resources provisioned than you need to, to support business workload requirements. Plus, you don’t need to forget about turning things off as the cloud platform will do it for you. Typically, you set minimum and maximum numbers of VMs to control your costs within autoscaling environments fall within known (pre-determined) limits. For autoscaling to work, however, your application needs to be able to support it — typically stateless or with state held outside of the application itself.

For the most part, we have been looking at individual VMs or systems, however many systems have complex interactions with others. Your automation capabilities need to handle these cases too. Treating these as an ‘environment’ rather than a ‘server’ is one way to do this. Again, there are many tools out there that can be used to create complete DevTest environments from scratch with all of the tooling that is needed.

The combination of these patterns, automated and deployed into the running environment allows us to drive potentially significant cost benefits, as the following example demonstrates.

Bringing it all together

Consider a global organisation running approximately 9300 applications across 50,000 servers in their own datacentres.

About 400 of these applications are out of scope for this exercise. These are either “desktop” applications running on a standalone device or applications supporting telecommunications. The remainder are what we might term “exotic” in that they run on about 7000 servers (Mainframe, various Unix flavours, Tandem etc). While there is surely potential for a Unix to Linux migration as part of any journey to cloud, we will ignore these applications for the time being though, as we can see from Figure 1 above, they all have a role to play here.

So, let us apply our patterns to this remaining population of about 8900 applications. Of these, 6250 are “production” applications. The other 2650 (approx. 30%) are used for development, test and other non-production use. These are potential targets for suspension or turning off when not in use, but this is not all. Of the 6250 production applications, 1550 (approx. 25%) are only used “9-to-5” so are also potential suspension / turn off targets. Thus, 4200 applications (47%) of the total population are hence potentially able to be turned off or suspended for the majority of any 24-hour period. If we assume that 50% can be suspended and the remainder turned off when not used, we can see from applying Pattern 1 that there is potential to save ~64% of the hosting costs for these applications.

Pattern 2 (right-sizing) is another area where significant benefits should be possible. Traditional x86 servers are frequently under-utilised so migrating workloads to smaller virtual servers gives the potential for reducing costs. If we don’t have actual utilisation data, we might choose to err on the side of caution and estimate 40% of the server population to be under-utilised. Half of this 40% could be reduced in size by 50%. Of the remaining 20%, half can be reduced by 25% and half by 75%.

For the Development & Test population (2650 applications), it may be possible to use spot-priced instances (Pattern 3) for some of these. Even if we assume that only 30% of this can be provided this way, there is another potential to drive further savings here.

Pattern 4 is the use of longer-term commitments to drive down the cost of those machines which are required to be on and available all the time. To keep the sums easy, let’s assume that all of our “7x24 applications” should have a longer term (in this case 3 year) commitment which reduces costs by another 60% for this population.

The results of applying these different optimisations can be seen in the chart below:

Figure 2. Potential impacts of infrastructure optimisation approaches

In this article, we’ve looked at potential ways to optimise infrastructure spending within the cloud at the pure IaaS level. There are many other business benefits realisable through a journey to cloud such as technical debt reduction and the reduction of maintenance costs. We’ve also not considered the potential upside of decommissioning old applications and the infrastructure that they run on in this. Though, as stated at the beginning, there is more financial benefit to be realised by containerising applications rather than undertaking a lift-n-shift to the cloud, this is still the journey that most organisations are on. The next step for these companies is to start to realise the potential cost benefits that cloud promised, but maybe they have yet to realise. Part of this comes down to changing the way that you work, and by adopting the patterns described above, I would hope to have shown a few pointers to show you how to do this.