Continuous Integration for Terraform Using GitLab CI
By: Emmanuel Sys
Tags: Programming, Gitlab, Cloud Computing, DevOps, Terraform

Implementing the Pipeline

In this pipeline, we will consider 3 stages:

Check will run some quality checks against the code.

will run some quality checks against the code. Plan generates and saves the Terraform execution plan to be reviewed by a team member.

generates and saves the Terraform execution plan to be reviewed by a team member. Apply where the team member will manually trigger a job to apply the changes on infrastructure if the plan review is OK. In addition, this stage will only occur on the master branch.

We can also consider an additional stage: reverting the last applied set of changes. But as these operations may be sensible, I personally keep them out of CI.

Setup

To implement this pipeline, you need to configure some secrets. Most people are using a remote bucket for storing the Terraform state. Access to this storage requires specific credentials depending on your cloud provider.

Let’s demonstrate using Google Cloud Platform and a storage bucket.

First create a new service account for GitLab and save the credential file. Give this service account the appropriate rights on the Terraform bucket (Storage Owner) Configure the service account credentials as a file variable in your GitLab project CI/CD settings.

Configure the google credentials file in GitLab CI

The process is similar if you are using AWS: you have to configure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables with the credentials of an appropriately scoped user.

Initialize the GitLab CI Pipeline

Create a .gitlab-ci.yml file at the project root with the following content:

This base job will be inherited by all the subsequent job and inject the credentials. We take car of triggering our pipeline only if Terraform files are changed in the context of a merge request or a push on a branch.

Static Check for Terraform Code

First thing we want is to ensure the pushed code:

Is formatted correctly as per Terraform’s format command

Has no syntax errors courtesy of the validate command

These checks will be the first jobs in our pipeline. They are run in parallel to spare some time. Note that the validate command requires Terraform to be initialized.

Generate Plan

Next we want to generate the Terraform plan. We need to initialize Terraform first and then, if you are using the workspace feature, select the adequate workspace. Finally we call the plan command with optional tfvars files if required.

There are some interesting things going on here:

First we save the plan as a file to feed it into the apply command in the next stage. This is the only way to be absolutely sure what we plan will be what will be applied to our infrastructure.

For the same reason, we save the .terraform folder to keep all the provider modules in the exact same version as the one used to generate the plan. In Terraform 0.14, the dependency lock file will solve this problem more elegantly.

folder to keep all the provider modules in the exact same version as the one used to generate the plan. In Terraform 0.14, the dependency lock file will solve this problem more elegantly. We save the files using the GitLab CI artifacts keyword to make them available to jobs in later stage.

Applying the changes

Last thing to do is to apply the changes.

As we have explained before, we want this step to be manual after reviewing the plan. The reason is that infrastructure changes may be critical or destructive and automatic validation with no human eye may be impossible to implement.