Creating A Terraform Provider - Part 1
Tags: terraform, hashicorp, devops, go
Published: 2018-11-20 13:48:33

---
title: Creating A Terraform Provider - Part 1
published: true
tags: terraform,hashicorp,devops,golang
canonical_url: https://medium.com/spaceapetech/creating-a-terraform-provider-part-1-ed12884e06d7
---
Cross posting from [Medium](https://medium.com/spaceapetech/creating-a-terraform-provider-part-1-ed12884e06d7), this is a post I wrote for my employers' blog - [Space Ape Tech](https://medium.com/spaceapetech/)

A fairly common task in Ops is to figure out how to manage a service programmatically; it’s often easy to get started with a service by using the UI but this won’t scale and eventually, we’ll want to have more control over the configuration of a service; a Terraform provider can be the tool that gives you that control.

To create a Terraform provider you just need to write the logic for managing the Creation, Reading, Updating and Deletion (CRUD) of a resource and Terraform will take care of the rest; state, locking, templating language and managing the lifecycle of the resources.

Just taking a look at the list of [existing providers](https://www.terraform.io/docs/providers/index.html) shows you how versatile Terraform can be. The list includes cloud providers, Kubernetes, DNS services, GitHub, monitoring tools and TLS certificate providers among others.

At [Space Ape](https://spaceapegames.com) we decided that we wanted to start managing the configuration of our metrics service, [Wavefront](http://wavefront.com), programmatically. We were all set to write a new tool that would allow us to template Alerts and Dashboards in YAML and then create them in Wavefront via their API. We had already started writing a [Go client](https://github.com/spaceapegames/go-wavefront) for the Wavefront API when Hashicorp announced the [release of Terraform v0.10.0](https://github.com/hashicorp/terraform/blob/v0.10.0/CHANGELOG.md), which split Providers out of Terraform core allowing each provider to be developed and released independently of Terraform and for custom providers to be created.

So we set about building the [Wavefront Terraform Provider](https://github.com/spaceapegames/terraform-provider-wavefront).

### How to Build Your Own Provider

To avoid having to work against a real API for this blog post, so that you can follow along if you wish, I’ve created a small API that stores (imaginatively) “items”. The API allows you to create, read, update and delete items. An item has a name, description and a list of tags. An items’ name must be unique. I won’t go into much more detail about the working of the API, you can find a more detailed description [here](https://github.com/spaceapegames/terraform-provider-example/blob/master/Readme.md) and the code in [terraform-provider-items/api](https://github.com/spaceapegames/terraform-provider-example/tree/master/api).

The client that the Terraform provider will use to interact with the API shouldn’t be implemented within the provider itself, as that would overcomplicate the provider and mean that the client can’t be used for other purposes. So I’ve also written a client in the [API package](https://github.com/spaceapegames/terraform-provider-example/tree/master/api/client).

The full source code of the example provider and API is available on [GitHub](https://github.com/spaceapegames/terraform-provider-example).

#### Getting Started

The provider we’re going to build during this blog post will allow us to create an Item on our server using the following Terraform code:

![](https://cdn-images-1.medium.com/max/1024/1*yFSDdnXt1mBwq1qC1D5RdA.png)

Terraform Plugins are binaries that Terraform communicates with via RPC. It’s possible to write a provider in any language, but in reality, you’ll want to write it in Go; Terraform provide helper libraries in Go to aid in writing and testing providers.

The name of the repository (and therefore directory) that your provider lives in is important; all providers start with `terraform-provider-` anything after that represents the name of the provider. In this case, I’m going for the very imaginative `terraform-provider-example`.

Next, we’ll create `./main.go` which will serve as the entry point to our provider. `main.go` is just used to invoke our provider, which we will implement in a separate package, in this case, called `provider`.

{% gist https://gist.github.com/nathmclean/5a547b19cbb319facb15df7c7b676a80 %}

Now, we’ll create the provider package, within which will sit the implementation of your provider. Within the `provider` package, we’ll create `provider.go` and define the `Provider` function that our `main.go` called.

{% gist https://gist.github.com/nathmclean/bdf7e22b1d4f975699e9c91005270c79 %}

The Provider requires:

- A `Schema` which represents the various attributes we can provide to our provider via the provider block of a Terraform file. Note that if no value is provided we will check if environment variables are set. This is useful for making sure we don’t need to store secrets in the provider block of terraform files
- A `ResourceMap` defines the names of the resources the provider has and where to find the definition of those resources. In this case, you can see we have `example_item` resource, the definition of which is a `*schema.Resource` returned by the `resourceItem()` function, which we’ll define later
- A `ConfigureFunc` which can do any setup for us. In this case, we have `providerConfigure` which takes the `address`, `port` and `token` and returns a client that we’ll use to communicate with the API. Note the `providerConfigure` returns an `interface{}` so we can store anything we like here

#### Defining Our Item Resource

Before we start defining our resource there are some more naming conventions to cover:

- Resource files are named `resource_[resourceName].go`, eg `resource_item.go`
- Test files follow the usual Go naming standard of resource `_[resourceName]_test`, eg `resource_item_test.go`
- We can also write import tests which, by convention, are in files named `import_[resourceName]_test.go`, eg `import_item_test.go`. Although these tests could sit within the `resource_[resourceName]_test.go` file
- Lastly, there are also data source definitions, which we won’t be implementing here, but you may see in other providers. These are named `data_source_[resourceName].go`. Data Sources allow you to pull in information from resources that already exist, but that you don’t want to manage.

We start by creating `resource_item.go` and defining the `resourceItem()` function that we call from `provider.go`. This returns a `*schema.Resource`— the definition of our Item resource.

{% gist https://gist.github.com/nathmclean/363574c0d50e8e30d0421dbdab260225 %}

A `schema.Resource` needs us to set up a few things:

- A `Schema`— the attributes the Resources has
- A number of functions. Earlier I said you just need to set up the Create, Read, Update and Delete functions for the provider, there are also two more, Exists and Imports, I’ll cover these later on.

Let’s look at the `Schema` in more detail. The Schema element is a `_map[string]*schema.Schema`, where the string is the attribute name and the `*schema.Schema` defines what the attribute is. In this case, the attribute `name` is of a Type `TypeString`, it is required and has a short description. It also has `ForceNew` set to true, this is because the API doesn’t allow you to change the name of an item after it is created, therefore Terraform would have to destroy the resource and create it again for this to happen.

Lastly, there is a `ValidateFunc` which is a function with the signature `(v interface{}, k string) (ws []string, es []error)`

Where `v` is the value of the attribute, which you need to use type assertion on to retrieve the actual type, `k` is the name of the attribute (“name” in this case) and it returns a slice of warnings (strings) and a slice of errors.

The `ValidationFunc` used for the `name` attribute is the `validateName` function:

{% gist https://gist.github.com/nathmclean/6fef23ba02b48e3ab1b3ceb085af0e67 %}

It returns an error if the name has any whitespace (The API doesn’t allow whitespace in names). This prevents us from making API calls we know will fail.

The `description` attribute is similar to name but doesn’t force a new resource and doesn’t have a validation func.

The `tags` attribute has a few differences. It is of type `TypeSet`, which is similar to a list, but where order doesn’t matter. You can use `TypeList` when you can be sure that the API will always return a list in the same order, otherwise, you’ll always have changes to apply. You’ll also notice that tags has a `Elem` field, which lets us define the type that is stored in the Set (or List), in this case, a string.

In our case, the API purposely returns tags in a random order. You can try changing the TypeSet to TypeList to see that there will (nearly) always be changes to apply due to the reordering of the tags.

It’s also worth noting that when you have a `TypeSet` or `TypeList` the `Elem` field can be of type `&schema.Resource`, which allows for a deeper resource structure. For instance, this is how the listener blocks of an [aws\_elb resource are defined](https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_elb.go#L159).

Also notice that tags are not a required attribute, but rather than just setting `Required` to false (it’s default value), we set `Optional` to true. Either `Required` or `Optional` must be true

#### Functions

The Create, Read, Update and Delete functions are functions with the signature `(d *schema.ResourceData, m interface{})` error. Where `d` is essentially the Schema we defined above, but with values added, e.g. the “name” attribute has a value we can send to our API.

`m` is the `interface{}` returned from the `ConfigureFunc` in `provider.go`, which in our case is our Client for talking to the server.

Each of the CRUD functions essentially just call their corresponding methods in the client, but there are a few things to note:

- When you create a resource you need to set the ID of the Terraform resource, this is done using the `d.SetId method`. The ID is generally the what the API uses to uniquely identify an item, in our case name is the ID, so we use that — `d.SetId(item.Name)`
- Setting the ID to an empty string indicates to Terraform the item no longer exists. So in the `resourceDeleteItem()` function we call `d.SetId("")` after deleting the Item. We also do this in the _resourceReadItem_ function if we get an error and that error contains “not found”
- For each of these methods we obtain the Client from `m`, as it is an interface we have to assert the type as the client type — `apiClient := m.(*client.Client)`

As the underlying data structure for the `d *schema.ResourceData` ends up being an interface, rather than a solid type we end up doing a lot of type assertion. On a small resource like our Item it is easy enough to do it within the function but, for larger resources, it may be worth splitting these out into separate functions.

In the `resourceCreateItem` function, we can see that we use `d.Get` to retrieve the values from the resource that we wish to pass to the API via the client and that we must perform type assertion on the result `d.Get("name").(string)`.

{% gist https://gist.github.com/nathmclean/578548e2c771530bf5703809cae562a2 %}

The Exists function (`resourceExistsItem()`) is slightly different in that it doesn't modify Terraform state, nor does it update any resource on the server; it’s just used to check if a resource exists. It has a slightly different function signature than the CRUD functions in that it also returns a bool to indicate if the resource exists — `resourceExistsItem(d *schema.ResourceData, m interface{}) (bool, error)`

Finally, there is an import function which is used for importing a resource into Terraform, an import function is an implementation of `&schema.ResourceImporter`. Terraform provides an implementation of this called `schema.ImportStatePassthrough`, which seems to work for the majority of use cases and you can always write your own implementation if you need to.

#### End of Part 1

At this point we have a functional Terraform provider, you could compile it and start creating _Items_ with Terraform. However, there is a important piece missing —  **tests!** In part 2 we’ll add tests to our Provider and run through how we get Terraform to use the provider.

* * *