Redefine Your Cloud Journey With Terraform and Packer
By: Gaurav Agarwal
Tags: Programming, Soft, DevOps, Technology, Terraform

Redefine Your Cloud Journey With Terraform and Packer

Photo by Łukasz Łada on Unsplash.

For most DevOps professionals, creating a VM usually consists of spinning it up on a cloud using Terraform and then using a config management tool (e.g. Ansible or Puppet) or a bootstrap script (e.g. cloud-init) to convert the raw Virtual Machine to a purposeful server.

We all have been doing it for a long time and it works for most cases, but it comes with some drawbacks.

I will give you an example from personal experience. We have a horizontally scalable web server running on GCP using managed instance groups (MIG). Whenever the MIG wants to add new nodes to the group, it takes some time for the node to be ready.

After the server is up, Ansible takes a few minutes to install packages, do configurations, and make the node ready to process requests. For a production server processing huge volumes, the time lag means dropped requests that hamper the customer experience. To work around this, we had to drop the CPU and memory threshold so that the MIG can spin up a new server before it is too late. That also means that there are too many false positives and we’re wasting valuable infrastructure.

Also, consider the amount it would take to patch every server and ensure that the patches would be right on every node. The complexity and risk increase with the number of nodes.

Well, we figured out that we were doing it wrong and there is a better way: by using Immutable Infrastructure. Now, what do we mean by this term? Well, it means that instead of spinning up infrastructure and configuring it after it is ready, we can pre-bake all configuration within the OS image so that when the machine is up, it is ready. It also means that you are not supposed to configure it any further, as the next time a copy of the machine is launched, it will not have your settings — hence the immutability. It’s the concept of containers within the VM space.

While this concept is not new, there were challenges with the approach because there weren’t tools available then. Luckily, with the modern DevOps and CI/CD practices, we can easily bake a new image and release it when we need to. You can also run tests on the new image before you decide to release it. It helps in reducing a lot of pain that you might incur in production. Rolling back a change is easy as well. Simply deploy the old image. You can do canary deployments with this approach if you need to and do A/B testing with ease.

HashiCorp has an impressive DevOps stack that relates to both managing infrastructure and server configuration. While HashiCorp’s Terraform is the leading multi-cloud Infrastructure as Code solution, HashiCorp Packer provides you with the power to create Immutable Infrastructure.

Packer helps you take OS images from a base, run it in a temporary VM, customise your image by running shell scripts or even Ansible playbooks, test the configuration, build the image, and push it to your image repository. So next time, when you want to release a new configuration, you need to update your instance template.

In this article, we will have a look at some of these features in the hands-on demo. As we work primarily on GCP, we will use the GCP example, but the principle is the same and the tool supports any public cloud platform.