How to use Terraform without server ?
By: Guillaume Blaquiere
Tags: Cloud Computing, Google Cloud Platform, Continuous Delivery, Terraform, Serverless

Terraform is trendy. Many ops use it: Multi cloud support, up-to-date, large community and templates,… Its advantages are multiple. However, I’m a big fan of serverless and I preferred Deployment manager. My ops did not, because it’s sticky to GCP. And they are right.

In fact, I never liked Terraform because of this endless loop :

For clean deployment of a server, use Terraform to deploy this server. Terraform has to be installed on a server; Use Terraform to deploy this server...

Thereby, I never used Terraform… until this unrelated post of Steren

What? Use Cloud Build for not building containers? It’s so smart!

Thanks to this tweet, an amazing idea comes to me: Use Terraform in serverless mode through Cloud Build!

First (bad) idea

Where to start? I need to execute Terraform and, as much as I (not) know Terraform, I need to keep tfstate files.

First step: does a docker image exist ?

Yes, Terraform installation is packaged in a container and available on Docker Hub. Thereby, it will be easy to use it with Cloud Build.

Second step: How to manage tfstate files?

Steren tweet shows how to push a file from Cloud Build environment to Google Cloud Storage at the end of the process. And for getting them from Storage at the beginning, use Cloud Builder images. That’s done!

I’m going to test and deep dive into Terraform mechanisms and documentation and… Bad first approach!

Terraform is mature

In fact, Terraform is far more mature than I expected. I discovered that a backend definition exists. It allows to retrieve and to store the deployment states remotely.

A GCS implementation exists and it’s very easy to use. Simply define the backend bucket and path in your main.tf file.

terraform {

backend "gcs" {

bucket = "my-state-bucket"

prefix = "terraform/state"

}

}

My second step is settled by 4 lines of configuration! It’s perfect!

Now, let’s focus on Terraform commands. There are only 2 steps to define in the cloudbuild.yaml file:

Run init command to set up the Cloud Build ephemeral environment

- name: 'hashicorp/terraform'

args: ['init']

Run apply command to execute the deployment/update in auto-approve mode

- name: 'hashicorp/terraform'

args: ['apply','-var=PROJECT_ID=$PROJECT_ID','-auto-approve']

The apply command requires to know the PROJECT_ID . The trick is to valorize a Terraform variable with the Cloud Build environment variables PROJECT_ID . Terraform’s main.tf file handles the PROJECT_ID variable like this :

variable "PROJECT_ID" {

type = string

} provider "google" {

//credentials = NOT NEEDED, use Cloud Build service account permissions

project = var.PROJECT_ID

region = "us-central1"

}

Leverage Cloud Build

The combination of Cloud Build and Terraform produces more than I expected initially: run Terraform on a server that I don’t have to manage.

Authentication

It’s another pain point not mentioned before, and that annoyed my ops: security. Indeed, on a managed server, dedicated to deployment of several different projects, you have to manage different service-accounts, with different permissions on these different projects.

For that, you have to store different JSON security key files. You have to rotate these JSON key files, to ensure their confidentiality/access right on the VM,… Lots of manual work, time consuming, and not always state of art security best practices.

Cloud Build solves this point. You simply use the permissions of a Cloud Build service account for the project. The security is now ensured by the platform.

Automation

With a server dedicated to deployment, you have to connect to it, to pull the code updates, and to launch the correct commands to apply changes. Without manual mistakes.

Cloud Build comes with automation and triggers. Thereby, a simple commit on the infrastructure definition files and the update is done automatically some minutes after.

History

With local execution on a server, you have only 1 file of history ( terraform.tfstate.backup ). Moreover, what is the resilience of this file on 1 server ? How is it protected against deletion/update by mistake ?

Here again, another advantage of this solution thanks to Terraform backend feature. The tfstate files are stored in Google Cloud Storage with high resilience and durability. And for keeping the history of changes, Terraform documentation recommends to activate versioning on the bucket dedicated of state storage