Configuring Terraform backend with AWS S3 and DynamoDB state locking
Tags: terraform, aws, dynamodb, devops
Published: 2023-06-21 13:16:58

In this blog post I have explained how to create a remote Terraform backend using Amazon S3 and DynamoDB services with state locking. Prerequisites are installed and configured [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html) and [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) with some code editor like [VS Code](https://code.visualstudio.com/). For better understanding of importance to create Terraform backend with S3, first it’s necessary to understand what is **state file**, **backend** and what type of backend is a **standard S3 backend**, besides that what is **state file lock** and what purpose does it have within a teamwork - which we will see later in the demo part.

The Terraform **state file** represents a crucial part of Terraform infrastructure that contains information about managed infrastructure and its configuration. When we initially create our infrastructure, the **state file** is saved locally by default as a **terraform.tfstate** file in **JSON** format. Besides local backend, we also have standard backend which essentially is a remote backend on AWS S3 service or some other cloud provider and it’s always defined inside of Terraform **backend configuration code block** - which we will write later.

Why is the standard backend (like AWS S3) so important? It’s important because of team collaboration, security reasons etc., but again it’s not a complete solution, because when two or more team members work on infrastructure simultaneously, they could have certain issues with resource modification or creation because of unpredictable situations like executing other processes before the state file has been finalized. I will include more illustrative information about this issue later in the post. 

To avoid this issue, Terraform can lock our state **(State file locking)** to prevent other team members from overwriting our infrastructure by using the same but modified **state file** simultaneously. In other words - changes that other team members make will be applied in certain order which we will see later with **Acquiring state lock** and **Releasing state lock** prompts while working on our infrastructure. In this case, state file is locked which provides us with protection of our state file from potential changes that could be made by other team members at the same time. In AWS we can use locking feature via DynamoDB table, which we will see in action later.

Terraform backend infrastructure diagram in AWS


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4337td05nvag87bn7u6y.png)

The practical part of this blog post has 3 components:

1. First we will create simple infrastructure using the EC2 Terraform Instance module. 
2. After that we will create a backend with Terraform backend code block which will build S3 bucket in our AWS account and save the state file here.
3. In the third part we will write code for DynamoDB state locking via Terraform and see how it looks in our AWS account.

**DEMO 1 - creating infrastructure using EC2 Terraform instance module.**

Before we start, we should validate if **AWS CLI** & **Terraform** are installed by using following commands: 
	
`$ aws --version`
`$ terraform --version`


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b9nshsqe6eoi36enqx1j.png)

In the previous, introduction part, I added links that point toward official documentation that we can use to install and configure both AWS CLI and Terraform. After we confirmed that we have these two tools installed we can start creating our basic infrastructure. First we will create **config.tf** file in VS Code and write the following code:

```
terraform {
 required_providers {
   aws = {
   source = "hashicorp/aws"
   version = "5.4.0"
   }
 }
}

provider "aws" {
 region = "eu-central-1"
}
```

After that we will create **variables.tf** file where we will write the following variables: 

```
variable "stage_name" {
 description = "The stage name for the environment"
 type        = string
 default     = "production"
}

variable "instance_name" {
 description = "Instance name"
 type        = string
 default     = "web_server"
}

variable "instance_type" {
 description = "EC2 instance type"
 type        = string
 default     = "t2.micro"
}

variable "key_name" {
 description = "The name of the SSH key pair"
 type        = string
 default     = "ec2-key"
}

variable "subnet_id" {
 description = "Subnet ID"
 type        = string
 default = "subnet-xxxxxxxxx" # write your subnet ID
}
```

After that we will create **main.tf** file where we will write code for our infrastructure resources, we will also create two resources for demo purposes - **AWS EC2 instance** as a web server with one **security group**:

```
module "ec2_instance" {
 source  = "terraform-aws-modules/ec2-instance/aws"

 name = "instance-${var.instance_name}"

 instance_type          = var.instance_type
 key_name               = var.key_name
 monitoring             = true
 vpc_security_group_ids = [aws_security_group.web_server_sg.id]
 subnet_id              = var.subnet_id

 tags = {
   Name      = var.instance_name
   Stage     = var.stage_name
   ManagedBy = "Terraform"
 }
}

resource "aws_security_group" "web_server_sg" {
 name        = "web-server-sg"
 description = "Security group for our web server"

 ingress {
   from_port   = 22
   to_port     = 22
   protocol    = "tcp"
   cidr_blocks = ["0.0.0.0/0"]
 }

 ingress {
   from_port   = 80
   to_port     = 80
   protocol    = "tcp"
   cidr_blocks = ["0.0.0.0/0"]
 }

 egress {
   from_port   = 0
   to_port     = 0
   protocol    = "-1"
   cidr_blocks = ["0.0.0.0/0"]
 }
}
```

The next step is to run the following commands in our terminal:

- `$ terraform init` - where we can see local backend initializing.
- `$ terraform plan` - where we can see the infrastructure plan.
- `$ terraform apply` - where we build the planned infrastructure, we should use yes value when prompted, to execute the command and create resources.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eb5ip8ixg30o9mp4qjth.png)

After the command has been executed, besides our **.tf** files Terraform has created **terraform.tfstate** file, and if we execute `$ cat terraform.tfstate` we can see the content of our **terraform.tfstate** file:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ji6py955jq7a0pvgwv2y.png)

This screenshot contains only part of the file where we can see JSON code that provides us with **version** of **terraform.tfstate** file including many other information like terraform version that was used to generate this file and very important part that contains **resources** with attributes (like arn), type, name and other specific resource values. 

After the **terraform.tfstate** file has been generated, we are ready to start with the practical part for our backend creation.

**DEMO 2 - Creating Terraform backend with AWS S3 service** 

When we built our infrastructure, Terraform has created a **state file** that is saved locally on our machine. In [official documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) we can see code which we can write to build our S3 bucket on AWS using Terraform, and then we will migrate our **state file** to that bucket.

Before we proceed with practical part, I think it’s very important to add few paraphrased notes related to **state file** from the book [Terraform: Up and Running, 3rd Edition](https://www.oreilly.com/library/view/terraform-up-and/9781098116736/): 

- If we use Terraform for our personal projects, it’s OK to use **terraform.tfstate** file locally on our machine, but if we want to use Terraform within our team on the real project we can encounter several issues with the approach above: our team members will not be able to access **state file** because our local machine is not a shared location, there would be no **lockin**g option and we could not **isolate state files**.

- We can save our Terraform code on VCS (ie. Github), but we should not save our **state file** in the same way. Why? Because of the fact that code inside our Terraform state file is saved in textual format and can contain confidential information. For example, if we are using `aws_db_instance` resource to create our database, Terraform will save database username and password in **state file** which will be in textual format and it will upload it on GitHub which can represent security risk later. 

- Instead of using VCS for remote backend, the best practice is to use Terraform **code block for remote backend**, in this case we will use this code block later for creating Amazon S3 bucket.

- Remote backend will provide us with a **locking** feature that is useful when several team members are working on a project and running the `$ terraform apply` command. In that case the first team member will have a lock on their side and will be able to apply the changes while other team members will be prompted to wait for a lock to be released. This does not mean that they will not be able to work, it only means that their changes will be applied in some order. 

- Besides this we should mention **encryption in transit** and how much of benefits we have when we are using Terraform with AWS S3 service which is Amazon’s managed file store and in the [book](https://www.oreilly.com/library/view/terraform-up-and/9781098116736/) above it’s mentioned as the recommended way for remote backend. You can read more about this in chapter 3 - [How to Manage Terraform State](https://www.oreilly.com/library/view/terraform-up-and/9781098116736/).

Now we should continue with our practical part, first we will create a **backend.tf** file which we will use to write our code related to bucket and key parameters. We should also add region parameters but we already included them previously in our code.

```
resource "aws_s3_bucket" "terraform-backend-bucket" {
 bucket = "backend-bucket-iac-terraform"


 tags = {
   Name        = "tf-bucket"
   Environment = "dev"
 }
}
```

When it comes to naming convention, we can name our S3 bucket like **backend-bucket-iac-terraform**, and you should provide it with some unique name. According to the [Terraform: Up and Running, 3rd Edition](https://www.oreilly.com/library/view/terraform-up-and/9781098116736/) it’s recommended to add few layers of protection for our S3 bucket that will enable S3 versioning, the default encryption, and block all public access to S3 bucket:

```
resource "aws_kms_key" "mykey" {
 description = "This key is used to encrypt bucket objects"
 deletion_window_in_days = 10
}


resource "aws_s3_bucket_server_side_encryption_configuration" "default" {
 bucket = aws_s3_bucket.terraform-backend-bucket.id


 rule {
   apply_server_side_encryption_by_default {
     kms_master_key_id = aws_kms_key.mykey.arn
     sse_algorithm     = "aws:kms"
   }
 }
}


resource "aws_s3_bucket_versioning" "versioning_bucket_enabled" {
 bucket = aws_s3_bucket.terraform-backend-bucket.id
 versioning_configuration {
   status = "Enabled"
 }
}


resource "aws_s3_bucket_public_access_block" "public_access" {
  bucket                  = aws_s3_bucket.aws_s3_bucket.terraform-backend-bucket.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}
```

After we wrote our code in the backend.tf file, we should run the `$ terraform plan` command to see what will be built from our infrastructure, in the output we should see that Terraform will create an S3 bucket with default encryption and enabled versioning. After that we will apply the `$ terraform apply` command to apply all changes. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pzx3i0iv2im38cerx0wz.png)

We can see that the command has been executed successfully and we can confirm in our AWS S3 service dashboard that S3 bucket has been created.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/piadvenowyj6h7kvl1t2.png)

However, our S3 bucket is empty and now it’s time to write terraform **backend configuration code block** in our **backend.tf** file with values that we used when we were creating the S3 bucket: 

```
terraform {
   backend "s3" {
       bucket = "backend-bucket-iac-terraform"
       key = "terraform.tfstate"
       region = "eu-central-1"
   }
}
```

After we wrote this code block, to store our **state file** in S3 bucket we have to run the `$ terraform init` command to initialize our terraform project again. We will get a prompt this time with a question if we want to copy our **state** to the new “S3” backend, we should write the **yes** value.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wj499gej7c33a2udnm37.png) 

After the command is executed successfully when we open our S3 bucket we can confirm that the **terraform state file** is placed here. At the same time if we open our **terraform.tfstate** locally on our machine, we will see that the file is empty. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xiq1a3x0iqus20r2y2ri.png)

At this moment we have successfully built our Terraform backend where instead of using our **terraform.tfstate** file that we have locally, we are using **terraform.tfstate** file remotely from our S3 bucket, and now it’s time to initialize DynamoDB state locking. 

**DEMO 3 - DynamoDB state locking**

After we created our S3 bucket and migrated our **state file** we should create DynamoDB state **locking** by creating a table for Terraform **state locking** with a simple hash **LockID** key and one **string** attribute. We can write code for this in our **backend.tf file**.

```
resource "aws_dynamodb_table" "terraform_locks" {
   name = "terraform-state-locking"
   billing_mode = "PAY_PER_REQUEST"
   hash_key = "LockID"

   attribute {
       name = "LockID"
       type = "S"
   }
}
```

After that we will run `$ terraform apply` command for Terraform to create our table within the DynamoDB service, and we will get the following output on the screenshot below.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mhsjbcrjf36a9v0kmi7a.png)

Now we will update our **backend code block** within our **backend.tf** file with the name of the recently created database.

```
terraform {
  backend "s3" {
      bucket = "backend-bucket-iac-terraform"
      key = "terraform.tfstate"
      region = "eu-central-1"

      dynamodb_table = "terraform-state-locking"
      encrypt        = true
  }
}
```

After that we will run `$ terraform init -reconfigure` to reconfigure our **state locking**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ws34vuh147b9rwzuuy9n.png)

In the screenshot below we can see our table inside of DynamoDB service.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gzkqusipnkw1092n48zy.png)

With this backend configuration, Terraform will automatically pull the latest changes on the **state file** from S3 bucket and later by running the `$ terraform apply` command it will update and set the latest **state file** changes on the S3 bucket. We can test this by using output variables that we will write in our **variables.tf** file. 

```
output "s3_bucket_arn" {
value       = aws_s3_bucket.terraform-backend-bucket.arn
description = "The ARN of the S3 bucket"
}

output "dynamodb_table_name" {
value       = aws_dynamodb_table.terraform_locks.name
description = "The name of the DynamoDB table"
}
```

These variables will print out the Amazon Resource Name (arn) of our S3 bucket and the name of our DynamoDB table. After we run the `$ terraform apply` command, we can notice **Acquiring state lock** (in the beginning) and **Releasing state lock** (at the end) prompts.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uppx5oy1yk1xtw84hshd.png)

We can also see that we had an entry inside of our DynamoDB table with a corresponding **Digest** value.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/80ndry8sskkkbeo4geq7.png)

**Digest** value is a hash value of our **terraform.tfstate** file, besides integrity check - if the content of our file has been changed or modified, **Digest** hash code will also change which tells us that new change has been made. We can test that if we comment out some existing resource inside of our code, change, modify or add a new one and then run the `$ terraform apply` command. In the screenshot below we can see **Info** that is providing information about **state** infrastructure.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bi6hmc6o6yjpwnwn52y4.png)

When changes have been applied successfully, we can see a new **Digest** hash value.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h2i8a252ncq6k3z4uiwf.png)

Conclusion:

The goal of this blog post was to explain a way to create a remote Terraform backend by using S3 and DynamoDB with state locking. For better understanding of how important it is to create Terraform backend with S3, I explained what is a **state** file, **backend** and what type of backend is a standard S3 backend, what is a **state file lock** and what kind of purpose does it have when it comes to teamwork and collaboration. 

Besides theory, in the practical part I explained how to create infrastructure by using the code from the Terraform: Up and Running book and official documentation. I tried to provide some simple and practical insights and keep the **state** file topic in the scope of the article, without addressing some other important concepts such as **state file isolation** and how to practically apply **terraform_remote_state**, which probably will be explained in some of my future posts. 

To avoid unnecessary costs we can clean our resources by using the `$ terraform destroy` command - just remember to avoid using this command within the production environment.

**Literature:**

- https://www.oreilly.com/library/view/terraform-up-and/9781098116736/

**Official documentation:**

- https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket
- https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration
- https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning
- https://developer.hashicorp.com/terraform/language/settings/backends/s3