Enhancing Terraform with Terragrunt
Tags: terragrunt, terraform, tutorial
Published: 2021-04-23 08:04:59

Over the last few months, I've become very familiar with using [Terraform](https://www.terraform.io/) to deploy infrastructure across multiple cloud vendors. I'm a big fan of it, but some of its idiosyncrasies can be frustrating, particularly as the size of your project begins to increase.

Fortunately, there's a tool that helps to mitigate many of these issues; [Terragrunt](https://terragrunt.gruntwork.io/).

Below is a short post on some of the features of Terragrunt that I've found useful in my projects. I've also created an [example](https://github.com/paddymorgan84/terragrunt-tutorial) that can hopefully better illustrate the benefits I'm writing about. Provided you have the following, you should be able to spin up the example resources yourself:

- [An Azure account](https://azure.microsoft.com/en-gb/free/)
- [The Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)
- [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)
- [Terragrunt](https://terragrunt.gruntwork.io/docs/getting-started/install/)

---

## What is Terragrunt?

Terragrunt is a thin wrapper that overlays Terraform that allows you to avoid having large amounts of repetition in your infrastructure as code, as well as helping with Terraform modules and remote state.

---

## DRY Providers

You can use the [generate block](https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes#generate) in your root `terragrunt.hcl file` to inject common configuration files. One of the most common uses for it is to use it to generate your provider configuration. By adding the generate block in your root, you can delete the `provider.tf` files in your other directories. Whenever any Terragrunt command is called, the providers will be copied to the appropriate directories.

- The change in my tutorial repo can be found [here](https://github.com/paddymorgan84/terragrunt-tutorial/commit/fec1da99dee5d0ff985de452175a586dc2348efa)

---

## Executing on multiple modules

When you use Terraform, you can't run commands against all of your modules at the same time. With Terragrunt, you can. If you add a `terragrunt.hcl` file to each module, you can run commands using `run-all` from root to run every module. Terragrunt recursively looks through each of your directories for the presence of that file, and if it finds it, will run the command you've included with `run-all`.

For example, if you wanted to apply all of the modules in this solution, you can run the following commands from the root of your repo:

```bash
terragrunt run-all init
terragrunt run-all apply
```

If you subsequently wanted to tear down everything you had, you can run:

```bash
terragrunt run-all destroy
```

---

## Parallelism

Another nice feature of Terragrunt is when you run commands such as `run-all`, it will run as many of the modules that have `terragrunt.hcl` files as possible in parallel. Terragrunt doesn't put a constraint on this, which is a blessing and a curse. Your infrastructure deployment times will be faster, but if you're not careful you can get rate limited by your provider, so use it with caution.

If you do start to experience rate limits, you can specify the `--terragrunt-parallelism` flag:

```bash
terragrunt run-all apply --terragrunt-parallelism 1
```
- The change in my tutorial repo can be found [here](https://github.com/paddymorgan84/terragrunt-tutorial/commit/fec1da99dee5d0ff985de452175a586dc2348efa)

---

## DRY CLI Flags

It's quite likely that when you run Terraform you're running commands with extra arguments. For example, you may have some common variables that you pass for each module, or you may want to restrict parallelism to avoid rate limiting.

With Terragrunt, you can specify an `extra_arguments` block that makes sure that commands that you repeat each time are always included.

```hcl
terraform {
  # Ensures paralellism never exceed two modules at any time
  extra_arguments "reduced_parallelism" {
    commands  = get_terraform_commands_that_need_parallelism()
    arguments = ["-parallelism=2"]
  }

  extra_arguments "common_tfvars" {
    commands = get_terraform_commands_that_need_vars()

    required_var_files = [
      "${get_parent_terragrunt_dir()}/tfvars/common.tfvars"
    ]
  }
}
```

- The change in my tutorial repo can be found [here](https://github.com/paddymorgan84/terragrunt-tutorial/commit/ae3c882673b9faa979ddeac0474a00e85c202a68)

---

## Before and After Hooks

Terragrunt gives you the ability to run commands before and after the execution of a Terraform command. For example, you may run a script to bootstrap your environment or clean up after an `apply` has been run. You may want to copy files to certain locations, or even just add extra information that can be outputted as part of the terragrunt commands.

```hcl
  before_hook "before_hook" {
    commands     = ["apply"]
    execute      = ["echo", "Applying my terraform"]
  }

  after_hook "after_hook" {
    commands     = ["apply"]
    execute      = ["echo", "Finished applying Terraform successfully!"]
    run_on_error = false
  }
```

- Commit [here](https://github.com/paddymorgan84/terragrunt-tutorial/commit/c30c21ed81bf9c1e4df46ba14a9d8445e34996df)

---

## Auto-init

Something that people will have had to do countless times when running Terraform is running `terraform init` before running any other Terraform command. Running this command initialises your working directory by installing provider plugins and initialising your backend configuration. A feature of Terragrunt is that you don't need to explicitly call `init`, it will do this automatically before any other commands being run.

Generally auto-init works fine, but I have found occasions where it hasn't worked as expected. If you want to be totally sure that your modules are initialised before you apply/destroy/plan, you can add in your own before_hook to cover off the scenario.

```hcl
terraform {
  before_hook "auto_init" {
    commands = ["validate", "plan", "apply", "destroy", "workspace", "output", "import"]
    execute  = ["terraform", "init"]
  }
}
```

- The change in my tutorial repo can be found [here](https://github.com/paddymorgan84/terragrunt-tutorial/commit/150d0b00227d08d012e93a73479dc5e5b54c4481)

---

## CLI options

Outside of the commands we've been using with Terragrunt, much of which mirrors the commands you would see with Terraform, there are a few particularly useful CLI options to use with Terragrunt as well. This list isn't exhaustive (you can find that [here](https://terragrunt.gruntwork.io/docs/reference/cli-options/#cli-options)), it's just a list of commands I have found useful in the past:

```bash
terragrunt run-all apply --terragrunt-non-interactive
```

This will stop interactive prompts from being displayed, and will prompt all answers with a "yes". Particularly useful when running in CI/CD environments.

```bash
terragrunt run-all apply --terragrunt-working-dir registry
```

You can pass directories to Terragrunt to indicate where the Terraform command should run. Note that for any `*-all` commands, it will start from that directory but run any subsequent sub-folders with a `terragrunt.hcl` file.

```bash
terragrunt run-all apply --terragrunt-exclude-dir registry
terragrunt run-all apply --terragrunt-include-dir registry
```

The commands allow you to explicitly exclude and include certain modules (and their dependencies).

```bash
terragrunt run-all apply --terragrunt-parallelism 1
```

When passed in, limit the number of modules that are run concurrently to this number during `*-all` commands. Note that this parallelism is markedly different from the one you specify using Terraform directly. With Terraform, parallelism indicates running _resource_ creation concurrently. With Terragrunt, it indicates running entire modules concurrently.

```bash
terragrunt run-all apply --terragrunt-log-level trace
```

If you want some more detail in your output, you can modify the log levels to do so.

---

## Summary

Terragrunt is a fantastic foil for Terraform, taking you the extra yards to make your experience of using Terraform that much better. I've worked on a few projects using Terraform, and my advice would be to leave Terragrunt to the side, at least initially. It's important to understand the problems you're trying to solve, and when they start to manifest for you, that's the time to pull the trigger.

I've found that the value of Terragrunt increases exponentially with the size of your project. Smaller ones can generally be managed easily with "raw" Terraform, whereas larger, multi-module projects start to really benefit from Terragrunts features.

---

## Resources

- [Getting started with Terraform](https://learn.hashicorp.com/collections/terraform/azure-get-started)
- [Getting started with Terragrunt](https://terragrunt.gruntwork.io/docs/#getting-started)
- [All of Terragrunts features](https://terragrunt.gruntwork.io/docs/features)

---

## Credit

[Goran Ivos](https://unsplash.com/@goran_ivos) for the photo :camera: