Terraform Enhanced Backend on Terraform Cloud with AWS access keys
Tags: aws, devops, cloud, terraform
Published: 2022-07-01 21:39:16

Enhanced backend in Terraform means a backend that can store state AND perform operations for configuration management.

Terraform has 2 enhanced backends that can store state and execute operations:
- `local` backend (default, stored locally)
- `remote` backend (used in [Terraform Cloud](https://app.terraform.io/))

We worked with `local` backend quite extensively throughout these series. This article will show how to use Terraform Cloud and `remote` backend that will allow us to store backend remotely and execute commands like `plan` and `apply` with the output showing in the local terminal.

In this article we will setup our Terraform configuration to work with Terraform Cloud and a `remote` backend.

&nbsp;

Here is the plan for this tutorial:
 1. Create a workspace
 2. Setup `remote` backend in our configuration
 3. Initialize our Terraform configuration
 4. Add AWS access tokens to the Terraform Cloud to allow remote execution of commands

&nbsp;
The code for this tutorial can be found [HERE](https://github.com/periwinkleFTW/terraform-path-to-cert/tree/master/practice-pt-5)

&nbsp;

In a [previous article](https://dev.to/af/terraform-state-4jp5) I covered how to create a Terraform Cloud account and an organization.

In order to proceed you will need to login into your [Terraform Cloud](https://app.terraform.io) account and get the name of your organization:
![Getting organization name from Terraform Cloud](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zy0dqle09bpp9a8j1ul3.png)

&nbsp;

We will also need a workspace for our. Click on "New workspace". Make sure that you have selected the correct organization.
![Creating Terraform workspace pt1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k3u1j4ckw49lsz0ldy2u.png)

&nbsp;

Select the `CLI-driven workflow`:
![Creating Terraform workspace pt2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/28qzorzhy5s2ovh1twet.png)

&nbsp;

Enter the name (optionally add description) and press `Create workspace`:
![Creating Terraform workspace pt3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/svyuxxjmvm89b0q8z4a4.png)

&nbsp;

And our organization has a new workspace. Good job!
![Creating Terraform workspace pt4](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zeamnob3v1sqbj952pey.png)
 
&nbsp;

Now we will see how to use it in our configuration. First, let's create a remote backend and point Terraform to it. Make sure that you put the correct organization and workspace names:
```terraform
# terraform.tf

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "terraform-blog"

    workspaces {
      name = "demo-app-on-aws"
    }
  }
}
```

&nbsp;

It is assumed that you done `terraform login` and added Terraform API token.

Since we changed the settings we need to reinitialize Terraform:
```bash
$ terraform init
# Output
Initializing modules...

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.
```

If you get this error:
```bash
│ The "remote" backend encountered an unexpected error while reading the
│ organization settings: unauthorized
```
It means that your Terraform Cloud API token is not valid anymore. This problem can be fixed by running `terraform login` and going the steps. You can also find a step-by-step guide in [this article](https://dev.to/af/terraform-state-4jp5).

&nbsp;

Let's see what happens when we try to execute a dry-run:
```bash
$ terraform plan
# Output
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/terraform-blog/demo-app-on-aws/runs/run-fACfrx3XtjhNh1zW

Waiting for the plan to start...

Terraform v1.2.4
on linux_amd64
Initializing plugins and modules...
╷
│ Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.
│ 
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│ 
│ Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, request send failed, Get "http://169.254.169.254/latest/meta-data/iam/security-credentials/": dial tcp 169.254.169.254:80: i/o timeout
│ 
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on main.tf line 3, in provider "aws":
│    3: provider "aws" {
│ 
╵
Operation failed: failed running terraform plan (exit 1)
```


&nbsp;

We are interested in the first part of the output indicating that plan is running remotely:
```bash
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/terraform-blog/demo-app-on-aws/

Waiting for the plan to start...
```

&nbsp;

We also got an error stating that we did not provide valid credentials for the AWS provider.

Terraform Cloud allows us to add credentials as environment variables inside the workspace. Please note that you need your `AWS_ACCESS_KEY` and `AWS_SECRET_ACCESS_KEY` to complete the next step.

Select your workspace. You will see a `Variable` tab at the top. Click that and you will arrive at the screen where you can add environment variables. That's where we will put our AWS credentials that will allow Terraform to deploy resources remotely.
![Adding environment variables to Terraform Cloud enhanced remote backend](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nahwuav6ezm4iydlzp0f.png)
 

&nbsp;

Now it's time to add our AWS access tokens to the environment. Select `Environment variable` radio button. `AWS_SECRET_ACCESS_TOKEN` will need to have the "sensitive" checkbox selected. This is important. Value field is where the actual key will go. See the picture for details:

![Adding AWS access token to Terraform Cloud](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5d65aj1c8qs0lgb5wfwq.png)
 
 

&nbsp;

This how it is supposed to look like after you are done. Pay attention how `AWS_ACCESS_KEY` is public, yet `AWS_SECRET_ACCESS_KEY` is private.
![Adding AWS access token to Terraform Cloud pt2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qmtnjal9zz2k1noarmx4.png)
 


Let's do another dry to verify that Terraform can use access to manage the infrastructure:
```bash
$ terraform plan
# output
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/terraform-blog/demo-app-on-aws/

Waiting for the plan to start...
# lots of output here with resources
Plan: 25 to add, 0 to change, 0 to destroy.
```

&nbsp;

And let's apply the changes:
```bash
$ terraform apply
# lots of output but you should see this at the end
Apply complete! Resources: 25 added, 0 changed, 0 destroyed.
```

&nbsp;

Verify that resources with the list command:
```bash
$ terraform state list
data.aws_ami.ubuntu
data.aws_availability_zones.available
data.aws_region.current
aws_eip.nat_gateway_eip
aws_internet_gateway.internet_gateway
aws_nat_gateway.nat_gateway
aws_route_table.private_route_table
aws_route_table.public_route_table
aws_route_table_association.private
aws_route_table_association.public
aws_security_group.private_sg
aws_security_group.public_sg
aws_security_group_rule.private_in
aws_security_group_rule.private_out
aws_security_group_rule.public_http_in
aws_security_group_rule.public_https_in
aws_security_group_rule.public_out
aws_security_group_rule.public_ssh_in
aws_subnet.private_subnet
aws_subnet.public_subnet
aws_vpc.vpc
module._server_from_local_module.data.aws_ami.ubuntu
module._server_from_local_module.aws_instance.web_server
module.another_server_from_a_module.data.aws_ami.ubuntu
module.another_server_from_a_module.aws_instance.web_server
module.autoscaling_from_github.data.aws_default_tags.current
module.autoscaling_from_github.data.aws_partition.current
module.autoscaling_from_github.aws_autoscaling_group.this[0]
module.autoscaling_from_github.aws_launch_template.this[0]
module.autoscaling_from_registry.data.aws_default_tags.current
module.autoscaling_from_registry.data.aws_partition.current
module.autoscaling_from_registry.aws_autoscaling_group.this[0]
module.autoscaling_from_registry.aws_launch_template.this[0]
module.my_server_module.data.aws_ami.ubuntu
module.my_server_module.aws_instance.web_server
```

&nbsp;

Awesome! We have connected Terraform Cloud, added AWS API tokens in a secure manner. Now we can create, modify, and destroy resources using our local terminal with the state store remotely.

&nbsp;

Finally, let's destroy the resources:
```bash 
$ terraform destroy
# output
Apply complete! Resources: 0 added, 0 changed, 25 destroyed.
```

&nbsp;

You can also view the history of commands executed by this backend by clicking on `Runs` tab within your organization:
![History of commands on Terraform Cloud and enhanced backend](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ntgstay3j2e5iz4u5dpe.png)
 
 

&nbsp;

This wraps up this tutorial. Thank you for reading! See you in the next article.