Terraform Certification (Part 4): Providers
Tags: hashicorp, terraform
Published: 2023-03-22 11:00:00

One could say that the core of Terraform is just glue between the various **providers** that we use to create different resources.

In this lesson we go through what providers are, how we can find what providers are available, and how we use providers in our Terraform configuration.

At a high-level this lesson covers the following parts of the exam curriculum:

| Part  | Content                                                |
| :---: | ------------------------------------------------------ |
| **3** | **Understand Terraform basics**                        |
|  (a)  | Install and version Terraform providers                |
|  (b)  | Describe plugin-based architecture                     |
|  (c)  | Write Terraform configuration using multiple providers |
|  (d)  | Describe how Terraform finds and fetches providers     |

## What is a provider?

A provider is an abstraction on top of an API. This API could be the API of a public cloud provider, such as AWS or Azure. The abstraction I am talking about here is that through the provider we can _declaratively_ configure a resource, such as a virtual machine, that we want to create. We do not need to know what API-calls to make to create the resource, or in what order we need to call each API-endpoint. We just declare the resource in Terraform, and the provider does the job of calling the correct APIs behind the scenes for us.

## How do we find available providers?

To find available providers we use the Terraform registry at [registry.terraform.io](https://registry.terraform.io/).

Through the Terraform registry we can discover available providers, and read the documentation about a specific provider we would like to use.

Apart from providers we can also use the Terraform registry to find:

- **Modules**: a module is a Terraform abstraction consisting of a collection of resources packaged in a reusable way. We will take a closer look at modules in a future lesson.
- **Policies**: a policy is a rule that must be fulfilled in order for a Terraform configuration to be valid. An example of this is "_you can't use an AWS virtual machine instance-size larger than m5_". Policies are written in a tool called Hashicorp Sentinel and it also uses the Hashicorp Configuration Language (HCL).
- **Run Tasks**: a run task is an integration with a third-party tool that can be configured to run at certain points in the Terraform lifecycle, for instance right before a `terraform apply`. This could be to configure tasks that run various security scans.

Policies and run tasks are **not** part of the Terraform Associate Certification, so we won't discuss them further. However, it is definitely a good thing to know that they exist!

## Using a provider

Time to get hands-on! We will use the Terraform registry to find a provider and add it a to a new Terraform project.

### Find a provider

I begin by navigating to [registry.terraform.io](https://registry.terraform.io/) and I click on _Browse Providers_:

![browse providers](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9aeoi7gh05ym94r9g8kk.png)

I see the most common providers highlighted for me:

![providers overview](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nwk1y6xrisfudj4wjnuk.png)

I select the Azure provider and arrive at the `azurerm` provider landing page:

![azure provider](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k5t32w44gduu2n7c0yi8.png)

### Add a provider to our Terraform configuration

When you have found a provider you want to use you can click on the **USE PROVIDER** button to see the required HCL to add this provider to your Terraform configuration:

![use provider](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mp9puat9xcihbwc56aed.png)

I create a new file called `main.tf` and I copy the code from the documentation:

```hcl
// main.tf
terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.37.0"
    }
  }
}

provider "azurerm" {}
```

Here we see our first example of HCL. In HCL there are two fundamental concepts called _arguments_ and _blocks_. A block consists of a _type_, zero or more _labels_, and the block content. It has the following general form:

```hcl
type "label 1" "label 2" ... "label n" {
  # block content
}
```

In the example above we saw three blocks: `terraform`, `required_providers`, and `provider`. Blocks can be nested, like the `required_providers` is nested inside of the `terraform` block. Only the `provider` block has a label `"azurerm"`.

Note that the `provider` block in the HCL above is currently empty. We will not discuss this block further in this lesson but we will come back to it in future lessons.

The other fundamental concept is an _argument_. Arguments assign a value to a name and it has the general form:

```hcl
argument_name = "argument value"
```

The value can be any type. In the example above we saw three arguments:

- `azurerm = { ... }`,
- `source = "hashicorp/azurerm"`, and
- `version = "3.37.0"`

Arguments can be nested inside of other arguments. The `azurerm = { ... }` argument has a complex type that includes the other two arguments.

The `terraform` block is where we specify all the providers that we want to use in our Terraform configuration. It is a required block as long as you want to use one or more providers.

Apart from `required_providers` another common thing to specify inside of the `terraform` block is `required_version`. This is an argument that you can use to enforce that a certain version of Terraform is used for this configuration. Let us include it in our `terraform` block, so that we now have:

```hcl
// main.tf
terraform {
  required_version = "> 1.3"

  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.37.0"
    }
  }
}

provider "azurerm" {}
```

Here I have added `required_version = "> 1.3"` inside of the `terraform` block. This argument states that this Terraform configuration requires a Terraform version that is greater than `1.3`. You would only include this argument if you are using a specific feature of Terraform that became available in a certain version. The `"> 1.3"` part is called a version constraint. You can use version constraints for both the Terraform version and for provider versions. There are a few different constraints you can specify:

- `=` allow exact one specific version, e.g. `=1.3.1`
- `!=` exclude a specific version, but allow the rest, e.g. `!=1.3.2`
- `>` greater than (e.g. `> 1.3.1`), `>=` greater than or equal (e.g. `>= 1.3.1`), `<` less than (e.g. `< 1.3.2`), `<=` less than or equal (e.g. `<= 1.3.3`)
- `~>` allow the right-most part of the semantic version number to increase, e.g. `~> 1.3.1` means that any version `1.3.X` where `X` is greater than or equal to `1` will be accepted

In the HCL we have seen so far you might have noticed that I use two types of _comments_. Comments are just like comments in any other programming language, a text block used to clarify something. In HCL there are three types of comments, illustrated in the following code block:

```hcl
// this is a comment

# this is a comment

/*
  This is a multi-line
  comment
*/
```

### Initializing Terraform

Once you have declared the required providers that your Terraform configuration needs, you can initialize your Terraform project. This is done with the `terraform init` command:

```console
$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "3.37.0"...
- Installing hashicorp/azurerm v3.37.0...
- Installed hashicorp/azurerm v3.37.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!
```

The output shows a few things going on. First of all it says `Initializing the backend...`. We have not covered the concept of _backend_ yet in this course so we will skip that part for now. The next thing that happens is `Initializing provider plugins...`, where Terraform reads the `required_providers` block and downloads the specified versions for each provider. In this case it installs version 3.37.0 of the Azure provider.

After running `terraform init` we might observe that we have additional files in our current directory:

```console
$ tree -a .

.
├── .terraform
│   └── providers
│       └── registry.terraform.io
│           └── hashicorp
│               └── azurerm
│                   └── 3.37.0
│                       └── darwin_arm64
│                           └── terraform-provider-azurerm_v3.37.0_x5
├── .terraform.lock.hcl
└── main.tf
```

In the `.terraform` directory Terraform has stored the binary for each provider it downloaded. In this case it is a single provider binary located in `.terraform/providers/registry.terraform.io/hashicorp/azurerm/3.37.0/darwin_arm64/`.

We can also see that we have a `.terraform.lock.hcl` file:

```console
$ cat .terraform.lock.hcl

# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/hashicorp/azurerm" {
  version     = "3.37.0"
  constraints = "3.37.0"
  hashes = [
    "h1:tD9TmGFgYV/oxZQu0pXuA46H+ML9nALCDwFqoaETjGg=",
    "zh:2a7bda0b7679d1c791c762103a22f333b544b6e6776c4177f33bafc9cc28c919",
    "zh:49ff49670c349f918017315838a43ece09bf6f1bf7721b992f1cadbceb273c62",
    "zh:55c9346d03380585e17616b79c4233b726d6fb9efa1921848834fc881e5d7d54",
    "zh:5ab117b56a4236ea29926e9d95c27d7bf8ae6706d0fffb76c0b1bfe67bf3a78e",
    "zh:5cfc086d5d56308edb3e68aac5f8a448ddc6e56541be7b152ae886399e9b2c69",
    "zh:7a8929ed38152aac6652711f32193c8582bc996f8fa73879a3ac7a9bf88d2460",
    "zh:895294e90a37f719975fcd2269b95e973147e48ec0ebb9c2fe472bc93531b49c",
    "zh:8baa5e2b6e5b02df5b45d253a3aea93f22619920cf9577290d682b59a6d5664b",
    "zh:b146a732c7909238c10d216b92a35092be4f72a0509a4c6742cc3245bf3b3bf3",
    "zh:cedef898ccd512a6519eae3dff7eb0d581d2c3dad8e0001992da16ad1d7fded8",
    "zh:f016d9ba94ea88476883b4d63cff88a0225974e0a8b8c3e8555f73c5de6f7119",
    "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c",
  ]
}
```

This file lists what provider versions have been installed. The `.terraform.lock.hcl` is similar to a `package-lock.json` file in a Node.js project. It should be included in your source repository.

One thing we have glossed over so far is the `source` argument we used when we specified a provider in the `required_providers` block:

```hcl
azurerm = {
  source = "hashicorp/azurerm"
  version = "3.37.0"
}
```

We say that the `azurerm` provider comes from `source = "hashicorp/azurerm"`. This is technically a short-hand for saying `source = "registry.terraform.io/hashicorp/azurerm"`. The general format of the source is `<HOSTNAME>/<NAMESPACE>/<TYPE>`, but if the `<HOSTNAME>` part is left out it will default to the Terraform registry address `registry.terraform.io`. With that said, it is perhaps clear that we could use other sources for our providers, it does not have to be the official Terraform registry.

### Upgrading a provider

When you run `terraform init` for a given Terraform configuration, Terraform will look at your `terraform` block as well as your `.terraform.lock.hcl` file if it exists to determine what version of a provider to install. If the `.terraform.lock.hcl` file exists it will take the version that is specified there, if not it will download the version that fulfills the version constraint in the `terraform` block.

If you at some point need to upgrade the provider you can edit the version constraint in the `terraform` block, and then run `terraform init -upgrade`. If I had version 3.36.0 of the Azure provider installed and I edit the version constraint to `=3.37.0` and run `terraform init -upgrade` this is what I would see:

```console
$ terraform init -upgrade

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "3.37.0"...
- Installing hashicorp/azurerm v3.37.0...
- Installed hashicorp/azurerm v3.37.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

$ tree -a .

.
├── .terraform
│   └── providers
│       └── registry.terraform.io
│           └── hashicorp
│               └── azurerm
│                   ├── 3.36.0
│                   │   └── darwin_arm64
│                   │       └── terraform-provider-azurerm_v3.36.0_x5
│                   └── 3.37.0
│                       └── darwin_arm64
│                           └── terraform-provider-azurerm_v3.37.0_x5
├── .terraform.lock.hcl
└── main.tf

9 directories, 4 files
```

I can see that now I have two versions of the Azure provider in my `.terraform` directory.

## A few common providers

There are a few common providers that could come up in the certification.

### Local provider

The `local` provider is used to read and write local files. It exposes two resources: `local_file` and `local_sensitive_file`, as well as two data sources with the same names. We have not covered data sources yet, but what they allow us to do with this provider is to read existing files.

An example of how to include the `local` provider in your `terraform` block is shown below:

```hcl
terraform {
  required_providers {
    local = {
      source = "hashicorp/local"
      version = "2.2.3"
    }
  }
}
```

Why would you use the `local` provider? If you need to read a local file containing content you wish to provide to another resource. This could be configuration data, an image, an HTML-file, or something else. You could also use it to generate a file containing data from another resource you created with Terraform, and then pass that file along somewhere else.

### Random provider

The `random` provider is used to introduce randomness into your Terraform configuration. This could be to generate a random name or a random number. It would not be very useful if it generated pure random values every time you ran Terraform, so it only generates random values once for a given input, and then holds on to those values until the inputs change.

The `random` provider is an example of a logical provider. A logical provider is a provider that does not interact with an external API, instead it does all of its work inside of Terraform itself.

An example of how to include the `random` provider in your `terraform` block is shown below:

```hcl
terraform {
  required_providers {
    random = {
      source = "hashicorp/random"
      version = "3.4.3"
    }
  }
}
```

The `random` provider only includes the following resources: `random_id`, `random_integer`, `random_password`, `random_pet`, `random_shuffle`, `random_string`, and `random_uuid`. Most resource names are self-explanatory, but you can read more about this provider and all the resources it exposes at [registry.terraform.io/providers/hashicorp/random/](https://registry.terraform.io/providers/hashicorp/random/).

Why do you want to use the `random` provider? Usually it is used to generate random prefixes or suffixes that are part of other resource names.

### Null provider

The `null` provider exposes a single resource called `null_resource` and a single data source called `null_data_source`.

An example of how to include the `null` provider in your `terraform` block is shown below:

```hcl
terraform {
  required_providers {
    null = {
      source = "hashicorp/null"
      version = "3.2.1"
    }
  }
}
```

The resource and data source in the `null` provider intentionally do nothing. To cite the documentation for what they are used for:

> ... they can be useful in various situations to help orchestrate tricky behavior or work around limitations.

I will not provide any example of when to use the `null` provider here, but we might see examples of it in a future lesson.

## Using multiple providers

How do you go about to use multiple providers in the same Terraform configuration? Do we need several `terraform` blocks, each with its own `required_providers` block? No, you only use a single `terraform` block. This is what it would look like if we wanted to use the `local`, `random`, and `null` providers in the same Terraform configuration:

```hcl
terraform {
  required_providers {
    local = {
      source = "hashicorp/local"
      version = "2.2.3"
    }

    random = {
      source = "hashicorp/random"
      version = "3.4.3"
    }

    null = {
      source = "hashicorp/null"
      version = "3.2.1"
    }
  }
}
```

## Do we need to provide a version for our providers?

What happens if we don't provide a specific version of a provider? The latest available version will be used. Let's say we have the following `main.tf`:

```hcl
// main.tf
terraform {
  required_providers {
    local = {
      source = "hashicorp/local"
    }
  }
}
```

If I run `terraform init` with this configuration I get the following output:

```console
$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/local...
- Installing hashicorp/local v2.2.3...
- Installed hashicorp/local v2.2.3 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!
```

Version `v2.2.3` was selected for me, which at the time of writing is the latest available version.

## Summary

A summary of the concepts we learned in this lesson:

- Providers:
  - A provider is an abstraction over an API.
  - A provider allows Terraform to use an API through HCL syntax.
  - You can browse available providers and read documentation on the Terraform registry.
  - We saw how to install a provider with `terraform init`
  - We saw how to upgrade a provider with `terraform init -upgrade`
  - We learned about version constraints for providers and for the Terraform version itself
  - We saw examples of three common but unusual providers: `local`, `random`, and `null`. These are used in special cases, and could come up in the certification exam.
  - We learned how to use multiple providers in the same Terraform configuration
- Hashicorp Configuration Language
  - We saw that there are two fundamental HCL concepts: _arguments_ and _blocks_.
  - We saw three examples of blocks: `terraform`, `required_providers`, and `provider`.
  - We saw how to specify a minimum required Terraform version with the `required_version` argument inside of the `terraform` block.
  - We saw how to create comments with `// comment`, `# comment`, and `/* comment */`.

What we have seen here includes 95% of everything I have ever used when it comes to providers. So I am fairly certain you won't need to know more than this about providers to pass the certification. What is included in the last 5% you ask? It is _provider aliases_ and how to pass providers to modules. We will see these concepts in a future lesson.