Using Terratest to test your infrastructure
Tags: testing, infrastructure, docker, terraform
Published: 2021-04-19 07:47:43

This post is going to get you started with using [Terratest](https://terratest.gruntwork.io) to help test your infrastructure changes. Terratest describes itself as:

> A Go library that provides patterns and helper functions for testing infrastructure, with 1st-class support for Terraform, Packer, Docker, Kubernetes, AWS, GCP, and more.

Now that infrastructure is becoming code and configuration, we want to write tests that check that our infrastructure code works as intended. This is where Terratest can help. What I personally like about Terratest is the fact it can test many different systems. This means my team can utilise one tool for many testing scenarios.

Terratest has many [packages](https://terratest.gruntwork.io/docs/getting-started/packages-overview/) that you can import into your project. For this post, I'm going to cover:

- Terraform
- AWS
- Docker
- Test Structure

This post requires the following pre-requisites:

- [Docker](https://www.docker.com/products/docker-desktop)
- [Terraform](https://www.terraform.io/downloads.html)
- [Go 1.16+](https://golang.org/dl/)
- An AWS account that you can create resources in.
- [The AWS CLI (v2)](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)

To focus on what the code does, rather than copying and pasting, you can get the code from [here](https://github.com/benmatselby/terratest-examples). I will still paste relevant code snippets into this post to help understanding.

## AWS

For this section we are going to define some Terraform that creates a new S3 bucket, with versioning enabled. We are then going to use the `terraform` and `aws` Terratest packages to test that the code works. Terratest allows us to actually deploy, test, and then destroy the infrastructure from the tests.

If you don't know about Terraform, then checkout my "[Terraform knowledge to get you through the day](https://dev.to/benmatselby/terraform-knowledge-to-get-you-through-the-day-17kk)" post first.

Let's take a look at the [aws/s3.tf](https://github.com/benmatselby/terratest-examples/blob/main/aws/s3.tf) Terraform file.

```terraform
terraform {
  required_version = "0.15.0"
}

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.36.0"
    }
  }
}

provider "aws" {
  region = "eu-west-2"
}

variable "bucket_name" {
  description = "The name of the bucket"
  default     = "-example"
}

resource "aws_s3_bucket" "terratest_bucket" {
  bucket = "terratest${var.bucket_name}"
  versioning {
    enabled = true
  }
}

output "bucket_id" {
  value = aws_s3_bucket.terratest_bucket.id
}
```

There are three main items to focus on:

- The `bucket_name` variable.
  - This means we can provide a unique suffix for the tests, but defaults to `-example` if not overridden.
- The `aws_s3_bucket` resource block.
  - Here we are defining the `versioning.enabled` attribute to be `true`.
  - This will form the test we want to make.
- The `bucket_id` output.
  - We can use this to then query AWS for the bucket information.

Let's take a look at the [tests/bucket_test.go](https://github.com/benmatselby/terratest-examples/blob/main/tests/bucket_test.go) Go test.

```go
package test

import (
  "fmt"
  "strings"
  "testing"

  "github.com/gruntwork-io/terratest/modules/aws"
  "github.com/gruntwork-io/terratest/modules/random"
  "github.com/gruntwork-io/terratest/modules/terraform"
  "github.com/stretchr/testify/assert"
)

// Standard Go test, with the "Test" prefix and accepting the *testing.T struct.
func TestS3Bucket(t *testing.T) {
  // I work in eu-west-2, you may differ
  awsRegion := "eu-west-2"

  // This is using the terraform package that has a sensible retry function.
  terraformOpts := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
    // Our Terraform code is in the /aws folder.
    TerraformDir: "../aws/",

    // This allows us to define Terraform variables. We have a variable named
    // "bucket_name" which essentially is a suffix. Here we are are using the
    // random package to get a unique id we can use for testing, as bucket names
    // have to be unique.
    Vars: map[string]interface{}{
      "bucket_name": fmt.Sprintf("-%v", strings.ToLower(random.UniqueId())),
    },

    // Setting the environment variables, specifically the AWS region.
    EnvVars: map[string]string{
      "AWS_DEFAULT_REGION": awsRegion,
    },
  })

  // We want to destroy the infrastructure after testing.
  defer terraform.Destroy(t, terraformOpts)

  // Deploy the infrastructure with the options defined above
  terraform.InitAndApply(t, terraformOpts)

  // Get the bucket ID so we can query AWS
  bucketID := terraform.Output(t, terraformOpts, "bucket_id")

  // Get the versioning status to test that versioning is enabled
  actualStatus := aws.GetS3BucketVersioning(t, awsRegion, bucketID)

  // Test that the status we get back from AWS is "Enabled" for versioning
  assert.Equal(t, "Enabled", actualStatus)
}
```

I've curated the code above so you get an explanation of what each line is doing. Now we want to see the tests in action, so let's get the dependencies.

```shell
go get ./...
```

Now we can run the tests:

```shell
❯ go test -count=1 -v ./...
ok  github.com/benmatselby/terratest-examples/tests 16.702s
```

If everything went to plan, the tests will pass. We have used `-count=1` so there is no Go test caching.

Let's check what happens if the tests fail. If we open `aws/s3.tf` and update the following code block:

```terraform
resource "aws_s3_bucket" "terratest_bucket" {
  bucket = "terratest${var.bucket_name}"
  versioning {
    enabled = true
  }
}
```

and set it to:

```terraform
resource "aws_s3_bucket" "terratest_bucket" {
  bucket = "terratest${var.bucket_name}"
  versioning {
    enabled = false
  }
}
```

If we run the tests now, they will fail as versioning is "Suspended":

```shell
❯ go test -count=1 -v ./...
[truncated]
    bucket_test.go:41:
          Error Trace:  bucket_test.go:41
          Error:        Not equal:
                        expected: "Enabled"
                        actual  : "Suspended"

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1 +1 @@
                        -Enabled
                        +Suspended
          Test:         TestS3Bucket
[truncated]
```

You can, if required, also pull in the [aws-sdk-go](https://aws.amazon.com/sdk-for-go/) package for more detailed API calls to test your infrastructure. I've used the ECR and ECS packages to help understand what is running in an ECS cluster etc.

## Docker

This is my personal favourite of the Terratest suite of packages. This package allows us to build our Docker images, and then run a suite of tests over them. This is essentially testing your software runtime environments.

We are going to use a very basic example. See the [docker/Dockerfile](https://github.com/benmatselby/terratest-examples/blob/main/docker/Dockerfile) file:

```dockerfile
# A standard image
FROM node:14

# We want to show something inside the image,
# so creating a file
RUN touch /tmp/testing.txt
```

As you can see, nothing special at all. We are using a base version of `node`, and adding a `testing.txt` file. This gives us enough to showcase the `docker` package within Terratest.

Let's now look at the test for it in the [tests/docker_test.go](https://github.com/benmatselby/terratest-examples/blob/main/tests/docker_test.go) file.

```go
package test

import (
  "testing"

  "github.com/gruntwork-io/terratest/modules/docker"
  test_structure "github.com/gruntwork-io/terratest/modules/test-structure"
  "github.com/stretchr/testify/assert"
)

// Standard Go test, with the "Test" prefix and accepting the *testing.T struct.
func TestDockerImage(t *testing.T) {
  // Define the docker tag
  tag := "terratest-examples:docker"

  // The build options we would pass to `docker build`
  // We may want to only test images already built, so let's use the
  // `test_structure.RunTestStage` function.
  // If you want to skip this stage, then define an environment variable:
  // SKIP_docker_build=true
  test_structure.RunTestStage(t, "docker_build", func() {
    buildOptions := &docker.BuildOptions{
      Tags: []string{tag},
      OtherOptions: []string{
        "--pull",
        "--no-cache",
        "-f",
        "../docker/Dockerfile",
      },
    }

    // The wrapped docker build command, with the `../docker` folder as the
    // build context
    docker.Build(t, "../docker", buildOptions)
  })

  // A testing table to test different aspects of the image.
  tt := []struct {
    name       string
    entrypoint string
    command    string
    expected   string
  }{
    // We want to test that node 14 is installed.
    {name: "test that node is installed", entrypoint: "node", command: "--version", expected: "14"},
    // We want to test that the testing.txt file is in the image.
    {name: "test that the testing.txt is present", entrypoint: "ls", command: "/tmp/testing.txt", expected: "testing.txt"},
  }

  // Iterate over the testing table to create test cases
  for _, tc := range tt {
    tc := tc
    t.Run(tc.name, func(t *testing.T) {
      // Allow the tests to run in parallel
      t.Parallel()

      // The docker run options
      opts := &docker.RunOptions{
        // Remove the container once finished
        Remove: true,
        // Entrypoint is variable from the test table
        Entrypoint: tc.entrypoint,
        // The command we will run for the test
        Command: []string{tc.command},
      }

      // Run the container, and get the output
      output := docker.Run(t, tag, opts)

      // The test check to assert we get what we expected.
      assert.Contains(t, output, tc.expected)
    })
  }
}
```

The code is curated above, so please review each line. However, to boil it down:

- We are using the `docker` Terratest package.
- We define a load of `docker build` options, which we use to build the image.
  - This is using the `test-structure` package, which means we can optionally run the build process. See below.
- We then use the "testing table" paradigm from Go to essentially create a data provider.
- We then run the docker container with entry points to allow us to assert facts about our docker image.

Let's now run the tests:

```shell
❯ go test -count=1 -run Docker ./...
ok  github.com/benmatselby/terratest-examples/tests 2.090s
```

Let's say you have built the images, and now only care about running assertions, then you can run the following:

```shell
❯ SKIP_docker_build=true go test -count=1 -run Docker -v ./...
```

Here we have turned on verbosity using the `-v` flag. You can see now that there is no `docker build` functionality happening, make our tests substantially quicker. Clearly, you need to be aware when to use these `SKIP_*` environment variables. For example, you may have defined your Docker images, and are now adding a lot of assertions. You don't need to keep building your Docker images in this case.

## Summary

Automated testing gives us the ability to facilitate change, at pace. What I personally like about Terratest is the fact we can use the same package for many kinds of tests, be it AWS infrastructure, Docker images, wrapping up the Terraform commands, and much much more.

---

Photo by <a href="https://unsplash.com/@twinsfisch?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Isabella and Zsa Fischer</a> on <a href="https://unsplash.com/s/photos/test?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>
  
---

### See also

- [Terratest Best practices](https://terratest.gruntwork.io/docs/#testing-best-practices)
- [Terraform knowledge to get you through the day](https://dev.to/benmatselby/terraform-knowledge-to-get-you-through-the-day-17kk)