pattern(composition) : terraform nested for loop
Tags: terraform, cloud, aws
Published: 2023-03-25 14:05:49

Recently I have discovered a pattern that has emerged to become extremely handy when wanting to build out resources that are composed together, typically a one-to-many relationship. 

## One-to-Many
![one-to-many](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gxydmrsxerlayrugus6z.png)

Now in terraform/cloud land the above image one can express the above image with something as simple as a security-group. The security group can be expressed as the "parent" and a security-group rule can would be expressed as a child of security-group. Thus, the security group is a composition of many security-group rules.


## Foo Bar Code

```terraform

locals {
  map = {
    "foo" = ["child1", "child4"]
    "bar" = ["child2", "child5"]
    "baz" = ["child3", "child6"]
  }

  out = merge([for parentKey, parentValue in local.map : {
    for child in parentValue : "${parentKey}/${child}" => {
      value = child
    }
    }
  ]...)
}


output "out" {
  value = local.out
}

```
The above code loops through each key-value pair in the map and then it loops through each item in the array (the value of the respective pair).

The for expression in the array will create a map based on the key and item . This is responsible for grouping the maps by the keys of the initial map.

The next for expression in the map is responsible for the creation of the key - values based on the respective value (array of strings ). The key will get namespaced by the respective key (e.g. "foo") and the respective value of the element in the array (e.g. "child1").

Resulting in the composed key : "foo/child1" 


The result of the two for expressions before the merge looks like such: 

```terraform

  array = [
    {
      "foo/child1" = {
        value = "child1"
      },
      "foo/child4" = {
        value = "child4"
      }
    },
    {
      "bar/child2" = {
        value = "child2"
      },
      "bar/child5" = {
        value = "child5"
      }
    },
    {
      "baz/child3" = {
        value = "child3"
      },
      "baz/child6" = {
        value = "child6"
      }
    },
  ]


```

The ellipsis at the end of the array is similar to the spread operator in Javascript , similarly in golang it's a vararg . 

What it does is spread the three maps as args in the merge function like such `merge({...},{...},{...})`.

From there the merge function handles the rest by merge the 3 maps into a single map. Just be careful with this as you need unique keys , otherwise the duplicated key will get overridden with the last respective value. 

```terraform
console output
  + out = {
      + "bar/child2" = {
          + value = "child2"
        }
      + "bar/child5" = {
          + value = "child5"
        }
      + "baz/child3" = {
          + value = "child3"
        }
      + "baz/child6" = {
          + value = "child6"
        }
      + "foo/child1" = {
          + value = "child1"
        }
      + "foo/child4" = {
          + value = "child4"
        }
    }

```


## Example

So obviously the above pattern is pretty nifty, let's find an example of how this pattern can become useful.

Lets say , you need to create a bunch of vpc-endpoints and you want each endpoint to have its own security-group because you need to control what can access these endpoints.

```terraform
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}



locals {
  ## variables
  default_vpc_id = "vpc-id"
  subnet_ids     = ["subnet-1-id", "subnet-2-id"]
  endpoint_services = {
    "s3" = {
      service_name        = "com.amazonaws.eu-west-1.s3",
      private_dns_enabled = true,
      vpce_name_tag       = "s3-interface",

      security_group_ingress_rules = {
        "team-z-vpc-443" = {
          port        = 443,
          cidr_blocks = ["10.0.0.0/24"],
          protocol    = "tcp",
          description = "allow https vpc traffic in ",
        },
        "onprem-443" = {
          port        = 443,
          cidr_blocks = ["192.168.0.0/16"],
          protocol    = "tcp",
          description = "allow https onprem traffic in ",
        },
      },
    },
    "ddb" = {
      service_name        = "com.amazonaws.eu-west-1.dynamodb",
      private_dns_enabled = true,
      vpce_name_tag       = "ddb-interface",

      security_group_ingress_rules = {
        "team-a-vpc-443" = {
          port        = 443,
          cidr_blocks = ["10.0.1.0/24"],
          protocol    = "tcp",
          description = "allow https vpc traffic in ",
        },
      },
    }
  }


  ## nested for loop for sg-rules
  ## not that sg_key_ref is used to reference the sg which is created based on the key-values of local.endpoint_services
  endpoint_sg_ingress_rules = merge([for k, v in local.endpoint_services : {
    for ingressKey, ingressValue in v.security_group_ingress_rules : "${k}/${ingressKey}" => merge({ sg_key_ref = k }, ingressValue)
  }]...)
}

resource "aws_vpc_endpoint" "endpoint_services" {
  for_each = local.endpoint_services

  vpc_id            = local.default_vpc_id
  subnet_ids        = local.subnet_ids
  service_name      = each.value.service_name
  vpc_endpoint_type = "Interface"
  security_group_ids = [
    aws_security_group.this[each.key].id,
  ]
  private_dns_enabled = each.value.private_dns_enabled
  tags = {
    Name = each.value.vpce_name_tag
  }

}



resource "aws_security_group" "this" {
  for_each    = local.endpoint_services
  vpc_id      = local.default_vpc_id
  name        = "${each.value.vpce_name_tag}-sg"
  description = "sg for endpoint service: ${each.value.service_name}"

  tags = {
    Name = "${each.value.vpce_name_tag}-sg"
  }
}

resource "aws_security_group_rule" "ingress" {
  for_each          = local.endpoint_sg_ingress_rules
  type              = "ingress"
  from_port         = each.value.port
  to_port           = each.value.port
  protocol          = each.value.protocol
  cidr_blocks       = each.value.cidr_blocks
  ## referencing the parent ( security group) from which the rule is composed into
  security_group_id = aws_security_group.this[each.value.sg_key_ref].id
}

```

The cool thing is: when you need to create another endpoint , all you do is add another object into the map that meets the interface thats defined by the variable(yes I didn't create a variable for this example).

## Alternatives/Anti-pattern

I think the alternative to this pattern is to rather create a module for the logic that needs to composed and a for_each on the module by passing the respective properties/input variables into the module. The module would be responsible for handling the logic.

```terraform
module "endpoints" {
  for_each = local.endpoint_services
  source   = "../"

  default_vpc_id = "vpc-id"
  subnet_ids     = ["subnet-1-id", "subnet-2-id"]

  service_name        = each.value.service_name
  private_dns_enabled = each.value.private_dns_enabled
  vpce_name_tag       = each.value.vpce_name_tag

  security_group_ingress_rules = each.value.security_group_ingress_rules
}


```


Thanks for taking the time read this article , hopefully it will become useful for you. Lastly, comment below how you would implement this sort of pattern !