Infrastructure as Code — Pros vs Pros
By: Pedram Ataee
Tags: Technology, Software Development, Startup, Cloud Computing, Towards Data Science

These days most software companies must release their products frequently to remove bugs or to introduce new features. And, if they have cloud-based products they need to provision and manage cloud services on a regular basis. These tasks may be executed manually but that is very inefficient.

You can provision and manage a cloud service through writing codes. This methodology is called infrastructure as code or IaC. In this article, I first describe the benefits of using IaC solutions. Then, I share my experience using this methodology.

IaC methodology enables us to:

rebuild the cloud service countlessly,

the cloud service countlessly, monitor the cloud configuration easily,

the cloud configuration easily, automate cloud provisioning within the development pipeline, and

cloud provisioning within the development pipeline, and select the cloud service provider with more flexibility.

Let me explain more.

I. Why infrastructure as code?

— Repetition

You must configure the cloud infrastructure (e.g., network, security, and storage) many times during the development process due to many reasons. For example, if an instance shuts down in the cluster, your service stops working. In this case, you must redo the cloud provisioning and you want to be completely consistent with the previous configuration.

— Monitoring

In IaC solutions, everything is written in human and machine-readable languages such as YAML or JSON. So, the configuration file can be archived and tagged in the version control system for any future purpose. So, a team lead or a CTO can easily monitor or review this configuration. Note that you just need to monitor the configuration file without extra files created by IaC solutions.

Example. Let’s assume the engineering team selected AWS infrastructure and allocated several EC2 instances with 16 CPUs to deliver the product. Using the IaC solution, the lead can easily observe the type of EC2 instances and prevent the company from spending unnecessary amounts. This is a transparency that is highly needed in complex products.

— Automation

You may want to delicately embed the cloud provisioning step in the continuous integration and deployment or CI/CD pipeline. It is not recommended to execute the cloud provisioning step every time you want to deploy the solution, especially in the development environment. However, you may want to have the cloud provisioning step in the product release pipeline especially during the early days when the cloud configuration is changing.

Example. You want to build stage and release environments with identical configuration. But, the configuration gets changed constantly since you are not still sure about many details. You can use an IaC solution to ensure having identical environments. Plus, you did some modifications to the YAML file, and the service stop working. You have a deadline and do not have time to debug. You just need to roll it back to a functional point in the past. Thanks to IaC solutions, you have access to the previous cloud configurations.

— Flexibility

It would be costly if you lock yourself to a cloud service provider by using all the native technology stacks. The native technology stack of a cloud service provider may not be compatible with its counterparts. So, you may not be able to migrate easily to other services that may lead to unaffordable monthly bills.

II. What was my experience with IaC?

In my recent experience, I was in charge of building and managing the CI/CD pipeline. I worked closely with the product manager for releases. I could not deliver tasks without an IaC solution. Here, I want to share my experiences.

It is costly to lock the development to a specific vendor.

During the early days, we received several thousand dollars worth of credit to use AWS. We could start using all the AWS technology stack to develop our solution. However, contrary to many peers, I believe to be vendor-agnostic even though that vendor is Amazon.

If we locked in ourselves to a vendor, we will not have any control over the monthly bill and we could not minimize it easily. For example, if we receive credits to use other cloud services it would not be easy for us to migrate services to the new host. So, I selected Terraform instead of CloudFormation to configure the cloud infrastructure with more flexibility.

Note that I have no intention to meticulously describe the pros and cons of CloudFormation vs Terraform in this article.

Even AWS EC2 instances may shut down permanently.

Several weeks after we set up our cloud services on AWS, we received an email from Amazon that one of the EC2 instances in the cluster will be shut down permanently due to a hardware issue. We noted that issue in a JIRA card, and we put that in our backlog but it slipped our attention easily.

On a beautiful Monday, we suddenly encountered an unusual error in our cloud services. Knowing we had a deadline to deliver, it was not that beautiful anymore! After an investigation, we found that the error was raised due to the EC2 instance shutdown. Thanks to Terraform, we were able to rebuild our EC2 clusters reliably and quickly.

The stage and release environments must be completely identical.

When several clients started using our solution regularly, we added the stage environment to our development workflow. The stage environment lowers the risk of failure or encountering unexpected behaviors in releases.

We had to be confident that stage and release environments are always identical. Even a small difference could result in a failure. Using IaC methodology is the best practice to build identical development environments. So, we used it as well.

We had to configure cloud infrastructure many times.

During the early days, we only had a single development environment. So, we did not think of the necessity of IaC solutions. However, we reached a point where we had three development environments: dev, stage, and release. So, we had to configure cloud infrastructures three times more than before.

Plus, we did not know what cloud configurations we need for each environment. Therefore, we had to run experiments to find the best configuration. Obviously, we had no chance of running those experiments without an IaC solution.

We needed to copy a file into EC2 instances every time.

At some point in our development process, we decided to make our DockerHub account private. To ensure the CI/CD pipeline working smoothly, we had to store the account credentials and retrieve them whenever needed.

To follow the best practices, we should have stored credentials in a password vault such as AWS secret manager. Since we were on a firm deadline and we wanted to close our DockerHub account to the public, we chose an easier method. We managed to store the credential files during the stage of cloud provisioning in EC2 instances. This was not the best method but at least relatively improve the security.

No company has industry standards for everything.

We had to develop as fast as possible. We had competitors and we wanted to be far ahead of them. So, we always used solutions that provided us better user experiences. For example, We found Terraform, in comparing to its alternatives, provided us:

a human-friendly development language,

very well-written and updated documents, and

an easy to understand development pipeline.

Companies like Amazon have their own solutions for all the cloud services; however, they are not necessarily the best-in-class or the most popular ones. For example, AWS has CloudFormation for its IaC solution, but Terraform is becoming the lead solution in this context.

The Last Words

All of my experiences just make me more excited about infrastructure as code or IaC solutions; especially Terraform. You may not need to use the IaC solution in an advanced way. However, it is essential to start using this methodology to, step by step, address your challenges and needs. If you start using this methodology, you will definitely enjoy it.