Automating the injection of CI/CD runtime information into Terraform provider
Tags: aws, devops, terraform, bash
Published: 2023-03-31 18:35:29

As a DevOps engineer or software developer, you may have encountered scenarios where you must inject CI/CD **runtime information into your Terraform provider code**.

This information could be anything from **environment-specific variables** to runtime configuration values **only available during the CI/CD process**. However, provider usage is evaluated very early on in the Terraform run, before we have enough context to do variable interpolations, so you can't use variables there (like you can normally do with standard resources and [environment variables](https://developer.hashicorp.com/terraform/language/values/variables#environment-variables)).

I stumbled upon a real-world use case dealing with some Terraform code to create **AWS** resources, where I need to add information about the role ARN to be assumed, and this information **cannot be statically inserted** in the code, because this code needs to be executed with different roles depending on some condition and/or constraints.

Another use case is to add default tags to all providers, such as the **build number**, to ensure consistency across all created AWS resources (and maybe you can't be sure that a `default_tags` entry is present in every provider).

To automate the process of injecting CI/CD runtime information into our Terraform provider, we'll introduce the tool [hcledit](https://github.com/minamijoyo/hcledit). With `hcledit` and some other manipulation, we can insert data into the Terraform code using `regex`/`grep` to find the correct place to add it.

How did I do it? Let's see the code! Here's my Bash function:

```bash
initialize_provider() {
  # allows the last command in a pipeline to be executed in the current shell environment, rather than a subshell
  shopt -s lastpipe
  # this is a simple regex to match every provider
  regex='provider[[:blank:]]\+"\([[:alnum:]_-]\+\)"[[:blank:]]*{'

  # Search for files in the current directory that contain the regular expression
  for file in $(grep -rl "$regex" .); do
    echo "Processing file: $file"
      # Create temporary file with the same name as the original file
      temp_file="${file}.tmp"
      cp "$file" "$temp_file"
      # declare an array variable
      declare -a arr

      # Modify file contents and write to temporary file
      grep "$regex" "$file" | while IFS= read -r line; do
          # Extract the provider name, which is the second word without the quotes and braces
          pn=$(echo "$line" | sed "s/$regex/ \1/g" | tr -d '"{}' | cut -d' ' -f2)
          echo "Found provider: $pn"
          #add the provider name to the array
          arr+=("$pn")
      done

      # Iterate over the array
      for provider_name in "${arr[@]}"; do
          # Add the assume_role block to the provider
          hcledit block append provider.$provider_name assume_role -f $temp_file -u --newline
          hcledit attribute append provider.$provider_name.assume_role.role_arn \"$ROLE_ARN\" -f $temp_file -u
          # Add the default_tags block to the provider
          hcledit block append provider.$provider_name default_tags -f $temp_file -u --newline
          hcledit attribute append provider.$provider_name.default_tags.tags \"$BUILD_ID\" -f $temp_file -u
      done
      cat "$temp_file"
  done

  # Move temporary files to original file names
  for temp_file in *.tmp; do
      mv "$temp_file" "${temp_file%.tmp}"
  done
}
```

Let's clarify what it does:

1. The first line enables the `lastpipe` shell option, which allows the last command in a pipeline to be executed in the current shell environment, rather than a subshell. It is most useful if you call this function from another process (i.e. your CI/CD pipeline).
    
2. `regex` is a regular expression that matches **every** provider in the Terraform code. I need to match multiple providers because you can have more than one, for example when you deploy in different regions. **Beware** that, if you use non-AWS providers, you may change this regex to exclude them or, in general, better match your needs.
    
3. The script searches for files in the current directory that contain the `regex` regular expression.
    
4. For each file found, the script creates a temporary file with the same name as the original file and copies the contents of the original file to the temporary file.
    
5. The script declares an empty array called `arr`.
    
6. The script reads through the contents of the original file and extracts the **names of all the providers** matching the `regex` regular expression. Each provider name is added to the `arr` array.
    
7. The script iterates over the `arr` array and appends an `assume_role` block to each provider in the Terraform code, using `hcledit`, which is much more convenient than Bash directly.
    
8. For each `assume_role` block, the script appends a `role_arn` attribute with the value of `$ROLE_ARN`, using `hcledit`.
    
9. Then the script uses `hcledit` again to add a `default_tags` block to each provider in the Terraform code.
    
10. For each `default_tags` block, the script appends a `tags` attribute with a list containing the value of `$BUILD_ID`.
    
11. Finally, the script moves the temporary files to their original file names by removing the `.tmp` extension.
    

In conclusion, by automating the injection of CI/CD runtime info into your Terraform code with tools like `hcledit` and a little bit of scripting know-how, you can easily add environment-specific variables and runtime configuration values to your Terraform code, making it more efficient and less error-prone.

Happy automating!