Write Terraform Files in Go with hclwrite
Tags: go, terraform, devops
Published: 2021-08-18 07:49:37

I am one of the lead maintainers of the [PagerDuty Terraform Provider](https://registry.terraform.io/providers/PagerDuty/pagerduty/latest/docs) which means I find myself writing a lot of [Terraform](https://www.terraform.io/) definitions in [HashiCorp Configuration Language (HCL)](https://www.terraform.io/docs/language/syntax/configuration.html). The provider itself contains a healthy collection of acceptance tests, but I often still write some of my own HCL to verify use cases and make sure bug fixes address specific issues raised by users.

To create these Terraform definitions I commonly write the HCL by hand. However, this can get tedious if I need to create, say, 150 of the same resource to test [pagination](https://developer.pagerduty.com/docs/rest-api-v2/pagination/) coming from the [PagerDuty API](https://developer.pagerduty.com/api-reference/). Historically, I used a Python script that verbosely wrote out the HCL syntax like so

```python
def createService(label, id, output):
	tf_code = "resource \"pagerduty_service\" \""+id+"\" {\n\
	name = \""+label+"\"\n\
	escalation_policy = pagerduty_escalation_policy."+str(id)+".id \n\
	alert_creation = \"create_alerts_and_incidents\" \n\
}\n\n"

	output.write(tf_code)
```

Functions like this `createService` above would be written for each resource and put inside of a loop to generate the HCL resource blocks that were needed for the definition. Honestly, this method worked fine for me. At least it did until I wondered if Go provided a better way. That's when I discovered the [hclwrite](https://pkg.go.dev/github.com/hashicorp/hcl/v2/hclwrite) package from HashiCorp.   

The project documentation describes hclwrite as a package that, "deals with the problem of generating HCL configuration and of making specific surgical changes to existing HCL configurations." This turned out to be exactly the package I needed. Rather than writing out HCL syntax by hand the hclwrite package would do that for me while I just called functions to create the objects. 

This article will help you get started with the hclwrite package by walking you through how I used it to generate a Terraform configuration for creating 150 Business Services in PagerDuty. I'll step through some of the concepts I struggled with and wished there were more examples of.

The hclwrite package is imported from `github.com/hashicorp/hcl/hclwrite`. This tripped me up a bit, partly because I'm still relatively new to Go and partly because the path to the package listed in the documentation is slightly different. For the rest of you also new to Go, your imports should look something like this:

```go
import (
	"fmt"
	"os"

	"github.com/hashicorp/hcl/v2/hclwrite"
	"github.com/zclconf/go-cty/cty"
)
```
You'll notice another import for the [go-cty](https://pkg.go.dev/github.com/zclconf/go-cty/cty) package. This package (pronounced see-tie) provides some infrastructure for a type system that might be useful for applications that need to represent configuration values. You'll see it used throughout the examples when setting types to attribute values.

To get things started you'll need to create two different types of file objects. One for hclwrite and another for the filesystem. This can be done with the following code.

```go
// create new empty hcl file object
hclFile := hclwrite.NewEmptyFile()

// create new file on system
tfFile, err := os.Create("bservelist.tf")
if err != nil {
    fmt.Println(err)
    return
}
// initialize the body of the new file object
rootBody := hclFile.Body()
```
The content of every object in hclwrite is stored in a body object. To add or append anything to an object you'll need to reference its body. In the code above you see that we named the body of the HCL document object `rootBody`. The first thing we need to do with `rootBody` is set up the `provider` block for the [PagerDuty provider](https://registry.terraform.io/providers/PagerDuty/pagerduty/latest/docs). The HCL for this block looks like this

```hcl
provider "pagerduty" {
  token = "yeahRightN0tgo1ng2t3llyOuTh@t"
}
```
Constructing this block using the `hclwrite` package requires the [AppendNewBlock](https://pkg.go.dev/github.com/hashicorp/hcl/v2/hclwrite#Body.AppendNewBlock) function which is expecting two arguments. First is a string which will set the type of block and the second argument is an array of strings that act as labels for the block. In the case of this `provider` block we want to set the label simply as `pagerduty`. That would look something like this:

```go
provider := rootBody.AppendNewBlock("provider", 
    []string{"pagerduty"})
```

Inside this block we need to set a `token` attribute to the value of a PagerDuty API Key. Remember, the attribute needs to be added to the body of the block. So, we'll first set that body value to the `providerBody` variable and then call [SetAttributeValue](https://pkg.go.dev/github.com/hashicorp/hcl/v2/hclwrite#Body.SetAttributeValue), passing the label and value of the attribute as arguments.

```go
providerBody := provider.Body()
providerBody.SetAttributeValue("token", 
    cty.StringVal(os.Getenv("PAGERDUTY_TOKEN")))
```

The next block we are going to create is the `terraform` block where we define which providers we're going to use and the versions of those providers. What's interesting about these definitions is they require two nested blocks that don't have any labels. The `AppendNewBlock` function handles his by accepting a `nil` argument for the labels. Creating the two blocks looks like this:

```go
tfBlock := rootBody.AppendNewBlock("terraform", 
    nil)
tfBlockBody := tfBlock.Body()
reqProvsBlock := tfBlockBody.AppendNewBlock("required_providers", 
    nil)
reqProvsBlockBody := reqProvsBlock.Body()
```

Inside the `required_providers` block we need to define an attribute called `pagerduty` that contains a value of an object with two key-value pairs as fields. This is done by setting the value to a `cty.ObjectVal` as a map.

```go
reqProvsBlockBody.SetAttributeValue("pagerduty", 
    cty.ObjectVal(map[string]cty.Value{
        "source":  cty.StringVal("PagerDuty/pagerduty"),
        "version": cty.StringVal("1.10.1"),
}))
```

The generated HCL for this code will look like this:

```hcl
terraform {
  required_providers {
    pagerduty = {
      source  = "PagerDuty/pagerduty"
      version = "2.3.0"
    }
  }
}
```

Now it's time to start creating actual resources. Remember, our task was to create 150 Business Services. In HCL, a resource block is just like the other block types we've created. The main difference is resource blocks contain multiple labels. In this case, each resource block contains the `pagerduty_business_service` label for the resource type  along with the identifier label for the resource. Because we only care about creating a whole lot of Business Services it doesn't matter what they're named. So, we're just going to use the index `i` from our loop to put a number variable into each Business Service name. For example, the name of the first Business Service will be "Business Service 1". The code for creating those resource blocks looks like this:

```go
bs := rootBody.AppendNewBlock("resource", 
    []string{"pagerduty_business_service", 
        fmt.Sprintf("bs%v", i)})
bsBody := bs.Body()
bsBody.SetAttributeValue("name", 
    cty.StringVal(fmt.Sprintf("Business Service %v", i)))
rootBody.AppendNewline()
```

When the looping is done, and the HCL for all 150 Business Service definitions has been generated, the last thing to do is to write all the definitions out to a `.tf` file. There are a few ways to do this. I went for the way I was most familiar with, where I wrote the bytes from the `hclwrite` file object to the `tfFile`, like this:

```go
tfFile.Write(f.Bytes())
```

And that's it. You should be able to run the configurations in the `.tf` file you created and populated. To see all the code used in these examples checkout the [Create Terraform Files in Go](https://github.com/stmcallister/create-terraform-files-go) project over on GitHub. This was also only an introduction to the `hclwrite` package. Go see the [hclwrite documentation](https://pkg.go.dev/github.com/hashicorp/hcl/v2@v2.10.1/hclwrite) to see all the available functions.