Managing Multiple Resources in Terraform: Loops and Key considerations
Tags: terraform, devops, aws, cloud
Published: 2023-07-11 18:43:17

## TL:DR
Terraform is a tool that made it easy, efficient and fast for us to build out our Infrastructure as a Code (IaaC). However, when it came to Day 2 operations, we were running into some issues where Terraform was destroying our resources, causing us headaches that were slowing us down.  Here is a recap of the problems we are facing and how we plan to avoid these issues in the future by effectively using loops and choosing the right keys. 

---

The Problem: Terraform Destroying Resources
-------------------------------------------

Terraform is an Infrastructure as a Code (IaaC) tool widely used to manage infrastructure. It allows you to define required resources in a declarative manner. All our cloud infrastructure such as VPCs, Kubernetes Clusters, RDS, S3 Buckets, Service Accounts are provisioned and maintained by Terraform.

Our product is growing at a rapid pace and Terraform allows us to build our IaaC in a fast and easy manner. But on Day 2, we faced an issue that Terraform was taking unexpected actions that were destroying our resources. We were using Loops incorrectly, and were not able to change existing resources anymore, just add new ones. We refactored our code and would like to share it to help you avoid a headache.

Using Loops in Terraform
------------------------

Sometimes we need to create multiple resources at once. So what do we do?
```
resource "aws_s3_bucket" "bucket0" {
  bucket = "wgk24wt55-demo0"
}
resource "aws_s3_bucket" "bucket1" {
  bucket = "wgk24wt55-demo1"
}
resource "aws_s3_bucket" "bucket2" {
  bucket = "wgk24wt55-demo2"
}
```

I'm kidding, of course, we would not make it like this as it would break the [DRY principle](https://dzone.com/articles/software-design-principles-dry-and-kiss). We use loops instead.

Let’s make a small recap about available loops in Terraform/HCL:
----------------------------------------------------------------

1.  `Count` Loops: The simplest way to make a loop based on predefined or counted numbers to create resources.
    
2.  `For_each` Loops: This loop allows you to iterate over variables in maps and sets based on keys to create resources.
    
3.  `For` Loops: These loops don't allow you to directly create resources, but allow you to filter or transform lists, sets, tuples, or maps.
    

### Count Loop

Let's look for some examples, of how we can create multiple S3 buckets with the simplest count:

```
resource "aws_s3_bucket" "bucket" {
  count = 3
  bucket = "wgk24wt55-demo${count.index}"
}

# Or lets add more controll, and iterate over tuple
locals {
  buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]
}

resource "aws_s3_bucket" "bucket" {
  count = length(local.buckets)
  bucket = "wgk24wt55-${local.buckets[count.index].name}"
}
```

After `terraform apply` our S3 buckets are created. We can see them listed in our state:

```
aws_s3_bucket.bucket[0]
aws_s3_bucket.bucket[1]
aws_s3_bucket.bucket[2]
```

But later, we decided that we don't need the demo1 bucket anymore, let's delete it:

```
locals {
  buckets = [
    { name = "demo0" },
    { name = "demo2" },
  ]
}

resource "aws_s3_bucket" "bucket" {
  count = length(local.buckets)
  bucket = "wgk24wt55-${local.buckets[count.index].name}"
}
```

Running `terraform apply` again, and what do we see?

`Plan: 1 to add, 0 to change, 2 to destroy.`

But wait, why would Terraform to destroy 2 resources when we only deleted 1?

```
# aws_s3_bucket.bucket[1] must be replaced

~ bucket = "wgk24wt55-demo1" -> "wgk24wt55-demo2" # forces replacement
```

So demo1 and demo2 will be deleted, and only after that demo2 will be recreated.

The reasons is key of state objects. When we use `count` it marks `aws_s3_bucket.bucket[X]` with numbered index. And object with index 1 had a different name before.

It might be ok for some stateless resources, but not for S3 buckets with data. So how can this be avoided?

### For\_each Loop

Let’s try switching to another loop: `for_each`

```
locals {
  buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]
}
# !Will not work!
resource "aws_s3_bucket" "bucket" {
  for_each = local.buckets
  bucket = "wgk24wt55-${each.value.name}"
}
```

But this snippet will not work, because `for_each` can only iterate over a map or a set of strings. Here we should use a `for` loop, which can help us transform tuple to map where key would be presented as `bucket.name`.

```
locals {
  buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]
}
resource "aws_s3_bucket" "bucket" {
  for_each = { for bucket in local.buckets : bucket.name => bucket }
  bucket = "wgk24wt55-${each.value.name}"
}
```

The above `for` loop transformed the `buckets` variable:

```
# From:
buckets = [
    { name = "demo0" },
    { name = "demo1" },
    { name = "demo2" },
  ]

# Into:
buckets = {
       demo0 = {
           name = "demo0"
        }
       demo1 = {
           name = "demo1"
        }
       demo2 = {
           name = "demo2"
        }
    }
```

State Migration
---------------

However, if we apply these updates, it still will lead to deleting all our buckets. To avoid it, we need to migrate our state first:

```
terraform state mv "aws_s3_bucket.bucket[0]" "aws_s3_bucket.bucket[\"demo0\"]"
terraform state mv "aws_s3_bucket.bucket[1]" "aws_s3_bucket.bucket[\"demo1\"]"
terraform state mv "aws_s3_bucket.bucket[2]" "aws_s3_bucket.bucket[\"demo2\"]"
```

Let’s check the state list:

```
aws_s3_bucket.bucket["demo0"]
aws_s3_bucket.bucket["demo1"]
aws_s3_bucket.bucket["demo2"]
```

Now if we apply a new code we will get:

```
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
```

Change as expected
------------------

The key right now is a bucket name that we specified via the `for` loop, so if we delete `demo1` bucket now, other buckets will stay untouched as we expected.

```
locals {
  buckets = [
    { name = "demo0" },
    { name = "demo2" },
  ]
}
resource "aws_s3_bucket" "bucket" {
  for_each = { for bucket in local.buckets : bucket.name => bucket }
  bucket = "wgk24wt55-${each.value.name}"
}
```

`Plan: 0 to add, 0 to change, 1 to destroy.`

Conclusion
----------

We can also do more complex keys. For example:

```
locals {
  buckets = [
    { name = "demo0", namespace = "A" },
    { name = "demo2", namespace = "B" },
  ]
}

resource "aws_s3_bucket" "bucket" {
  for_each = { for bucket in local.buckets : "${bucket.namespace}-${bucket.name}" => bucket }
  bucket = "${each.value.namespace}-wgk24wt55-${each.value.name}"
}

# Which will give us:
aws_s3_bucket.bucket["A-demo0"]
aws_s3_bucket.bucket["B-demo2"]
```

We are striving to keep keys simple but unique. Mostly we use a combination of `name + namespace`, but you can also add your domain or environment name. Keep in mind, changing the key requires either deletion or one more state movement, so I don’t recommend including frequently changing parameters such as tags or access policies to your keys.

As you have seen before, choosing the right key can help you avoid state manipulation in the future. Chose them carefully from the beginning.

_P.S. I published this post in our [corporate blog](https://www.perfectscale.io/blog/terraform-loops-and-key-considerations) before, but hope it will be useful for Dev.to community too. Your comments and questions are welcome, would be glad to see your suggestions or help with your issues._