Terraform AWS - Dynamic Subnets
Tags: aws, terraform, devops, tutorial
Published: 2019-09-20 12:10:24

**Update** - [17-Oct-2019 changelog](https://github.com/hashicorp/terraform/blob/v0.12.11/CHANGELOG.md#01211-october-17-2019): Terraform released a new function named [cidrsubnets](https://www.terraform.io/docs/configuration/functions/cidrsubnets.html), this function creates a list of cidr-subnets. This function is great, and I recommend using it. Even though this function shortens some parts of this tutorial, you should still read it if you want to learn how to use functions in Terraform.

# Objectives
1. Create sets of subnets dynamically
2. Learn advanced concepts in Terraform
   - map variable and lookup function
   - for loop and conditional for loop
   - index function in a for loop

# Knowledge and assumptions
1. You are familiar with subnetting (a.b.c.d/xx) - a great online tool for calculating subnets - [cidr.xyz](https://cidr.xyz)
2. You are using Terraform v0.12+
3. You have previous experience with:
   - [variables](https://www.terraform.io/docs/configuration/variables.html)
   - [local values](https://www.terraform.io/docs/configuration/locals.html)
   - [modules](https://www.terraform.io/docs/configuration/modules.html)
4. You know what's a [for loop](https://docs.python.org/3/tutorial/controlflow.html#for-statements)
5. I'll be using the module [terraform-aws-modules/vpc/aws](https://github.com/terraform-aws-modules/terraform-aws-vpc) for the VPC. This module requires lists of subnets (private, database, public), which is exactly what we're going to create

# Issue #1: Subnets per environment
Hardcoding subnets per environment can be a significant overhead. By doing so, you might end up writing a lot of lines with hardcoded network prefixes.
Moreover, it takes off the flexibility that infrastructure as code (IaC) has to offer.


## **Set** cidr_ab per environment
cidr_ab to the rescue! Set the start of the VPC CIDR as a map variable, and map each cidr_ab value to the appropriate environment. If you're not using all four, remove it from the mapping.

A subnet pattern (prefix) is: **a.b**.c.d/xx , so this will cover the **a.b** part of all subnets per environment. For example:

```terraform
variable "cidr_ab" {
    type = map
    default = {
        development     = "172.22"
        qa              = "172.24"
        staging         = "172.26"
        production      = "172.28"
    }
}
```



## **Get** cidr_ab per environment
Maps and lookup functions provide great functionality, and in our case, it makes it easy to get the cidr_ab per environment.

Here's the syntax of the [lookup function](https://www.terraform.io/docs/configuration/functions/lookup.html): `lookup(map, key, default)`

To create a local list `private_subnets` with the relevant cidr_ab per environment, use the following:

*Reminder*: To concatenate an expression and text: "${expression} my text"
```terraform
locals {
    private_subnets = [
        "${lookup(var.cidr_ab, var.environment)}.1.0/24",
        "${lookup(var.cidr_ab, var.environment)}.2.0/24",
        "${lookup(var.cidr_ab, var.environment)}.3.0/24"
    ]
}
```

Keep in mind that we need to do that for database and public subnets aswell.

## Issue #1: Full Solution
Assuming we want to create the following subnets: private, database, and public. Here's how our `variables.tf` and `vpc.tf` files should look like:

`variables.tf`
```terraform
variable "cidr_ab" {
    type = map
    default = {
        development     = "172.22"
        qa              = "172.24"
        staging         = "172.26"
        production      = "172.28"
    }
}

locals {
    private_subnets         = [
        "${lookup(var.cidr_ab, var.environment)}.1.0/24",
        "${lookup(var.cidr_ab, var.environment)}.2.0/24",
        "${lookup(var.cidr_ab, var.environment)}.3.0/24"
    ]

    database_subnets        = [
        "${lookup(var.cidr_ab, var.environment)}.11.0/24",
        "${lookup(var.cidr_ab, var.environment)}.12.0/24",
        "${lookup(var.cidr_ab, var.environment)}.13.0/24"
    ]

    public_subnets          = [
        "${lookup(var.cidr_ab, var.environment)}.64.0/24",
        "${lookup(var.cidr_ab, var.environment)}.65.0/24",
        "${lookup(var.cidr_ab, var.environment)}.66.0/24"
    ]
}

variable "environment" {
    type = string
    description = "Options: development, qa, staging, production"
}
```

`vpc.tf`
```terraform
module "vpc" {
    source = "terraform-aws-modules/vpc/aws"
    version = "~>2.0"
    name                 = "my-vpc"
    cidr                 = "${lookup(var.cidr_ab, var.environment)}.0.0/16"
    private_subnets      = local.private_subnets
    database_subnets     = local.database_subnets
    public_subnets       = local.public_subnets

    azs                  = ["us-west-2a", "us-west-2b", "us-west-2c"]

    # omitted arguments for brevity

}
```

# Issue #2: Subnets per Availability Zone
In issue #1, we solved the subnets per environment, but the Availability Zones (azs) were hardcoded.

One might think; "Let's simply create a map variable for the region, and then use the region's name followed by 'a', 'b', 'c'". For example:

```terraform
variable "region" {
    type = map(string)
    default = {
        "development" = "us-west-2"
        "qa"          = "us-east-2"
        "staging"     = "us-east-1"
        "production"  = "ca-central-1"
    }
}

module "vpc" {

    # omitted arguments for brevity

    azs = [
        "${lookup(var.region, var.environment)}a",
        "${lookup(var.region, var.environment)}b",
        "${lookup(var.region, var.environment)}c",
    ]
}
```

If all the regions that you're using have the same amount of availability zones - then the above solution is perfect. Unfortunately, that's not true when it comes to **ca-central-1** (Canada Central) which has only two availability zones. Another example would be **us-east-1** (Virginia) which has six availability zones.

*Note*: Use [AWS Global Infrastructure](https://aws.amazon.com/about-aws/global-infrastructure/) to find out the number of availability zones per region. And  [Regions and Availability Zones](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html) to figure out the region's code (e.g., eu-west-1)

## data aws_availability_zones
We can get the availability zones of the region that is currently taking place, by using [data "availability_zones"](https://www.terraform.io/docs/providers/aws/d/availability_zones.html). 

Here's is how we do it:

```terraform
data "aws_availability_zones" "available" {
    state = "available"
}
```

Some explanations regarding the code above:

1. `data` - get existing data/resources available in your account
2. `aws_availability_zones` - gets the list of availability zones in the current region
3. `available` - a name for that data, it's important to pick a name that reflects the meaning of the data
4. `state = "available"` - filters out availability zones that currently experience outages

Let's save the data in a local value. You might be wondering, "Why are we using local values? We also did it with the subnets, why oh why?"

Rule of thumb - if there's any chance you're going to manipulate a variable/value, use a local value. This trick provides granularity to manage `local.var_name`. It is best to manage variables behind the scenes without touching the infrastructure's code (vpc.tf, rds.tf, s3.tf files). Worst case scenario - we've used a local value even though we haven't manipulated it.

Saving the availability zones [names](https://www.terraform.io/docs/providers/aws/d/availability_zones.html#attributes-reference) list in a local value:
```terraform
locals {
    availability_zones = data.aws_availability_zones.available.names
}
```


# Issue #2: Full solution
`variables.tf`
```terraform
variable "cidr_ab" {
    type = map
    default = {
        development     = "172.22"
        qa              = "172.24"
        staging         = "172.26"
        production      = "172.28"
    }
}

locals {
    private_subnets         = [
        "${lookup(var.cidr_ab, var.environment)}.1.0/24",
        "${lookup(var.cidr_ab, var.environment)}.2.0/24",
        "${lookup(var.cidr_ab, var.environment)}.3.0/24"
    ]

    database_subnets        = [
        "${lookup(var.cidr_ab, var.environment)}.11.0/24",
        "${lookup(var.cidr_ab, var.environment)}.12.0/24",
        "${lookup(var.cidr_ab, var.environment)}.13.0/24"
    ]

    public_subnets          = [
        "${lookup(var.cidr_ab, var.environment)}.64.0/24",
        "${lookup(var.cidr_ab, var.environment)}.65.0/24",
        "${lookup(var.cidr_ab, var.environment)}.66.0/24"
    ]
}

data "aws_availability_zones" "available" {
    state = "available"
}

locals {
    availability_zones = data.aws_availability_zones.available.names
}

variable "environment" {
    type = string
    description = "Options: development, qa, staging, production"
}
```

`vpc.tf`
```terraform
module "vpc" {
   source = "terraform-aws-modules/vpc/aws"
    version = "~>2.0"
    name                 = "my-vpc"
    cidr                 = "${lookup(var.cidr_ab, var.environment)}.0.0/16"
    private_subnets      = local.private_subnets
    database_subnets     = local.database_subnets
    public_subnets       = local.public_subnets

    azs                  = local.availability_zones

    # omitted arguments for brevity

}
```

# Issue #3: Dynamic cidr_c
That's the part where I'm greedy, and I want to populate **cidr_c** according to the number of availability zones. So far we've covered **a.b**.c.d/xx, now we will cover this part a.b.**c**.d/xx

For example (pseudo-code):
```
# define cidr_c per subnet
cidr_c_private_subnets =  1
cidr_c_database_subnets = 11
cidr_c_public_subnets = 64

number_of_azs = 2

# dynamically populate lists of subnets
private_subnets  = ["a.b.1.d/xx", "a.b.2.d/xx"]
database_subnets = ["a.b.11.d/xx", "a.b.12.d/xx"]
public_subnets   = ["a.b.64.d/xx", "a.b.65.d/xx"]
```


## for loop
Dynamically populating sounds like a loop, and in Terraform we got the [for loop](https://www.terraform.io/docs/configuration/expressions.html#for-expressions).

Terraform's for loop reminds me of [Python's list comprehension](https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions), which means - create a new list with a for loop.

Syntax of the for loop statement:
`[for item in list: do_something_on(item)]`

Pseudo-code of the loop that we need:

```terraform
for availability_zone in local.availability_zones:
  "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_private_subnets + index_of_availability_zone}.0/24"
``` 

Some explanations regarding the code above:
1. `availability_zone` - I picked the name of the iterator, it could also be `az` or any other name
2. `index_of_availability_zone`
   - First index of a list is zero (0)
   - Assuming `${lookup(var.cidr_ab, var.environment)` equals to one (1) for private_subnets
   - Adding the current index of the availability zone to `${lookup(var.cidr_ab, var.environment)`, result in:
     - 1 + 0
     - 1 + 1
     - 1 + 2

## get the index of the availability zone
The [index function](https://www.terraform.io/docs/configuration/functions/index.html) to the rescue!

Syntax of index function:
`index(list_to_search_in, value_to_search_for)`

The list we're searching in is `local.availability_zones` and the value we're looking for is the loop's iterator, `availability_zone` in our case.

Replacing `index_of_availability_zone` with `index(local.availability_zones, az)` result in:
```terraform
locals {
    private_subnets = [
        for az in local.availability_zones:
          "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_private_subnets + index(local.availability_zones, az)}.0/24"
    ]
}
```

# Issue #3: Full solution
The iterator's name in each loop is `az`, I picked a shorter name than `availability_zone` to make it more readable. Since we use local values, we only need to change the `variables.tf` file, no need to touch `vpc.tf` file

`variables.tf`
```terraform
variable "cidr_ab" {
    type = map
    default = {
        development     = "172.22"
        qa              = "172.24"
        staging         = "172.26"
        production      = "172.28"
    }
}

locals {
    cidr_c_private_subnets  = 1
    cidr_c_database_subnets = 11
    cidr_c_public_subnets   = 64
}


data "aws_availability_zones" "available" {
    state = "available"
}

locals {
    availability_zones = data.aws_availability_zones.available.names
}

variable "environment" {
    type = string
    description = "Options: development, qa, staging, production"
}

locals {
    private_subnets = [
        for az in local.availability_zones : 
            "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_private_subnets + index(local.availability_zones, az)}.0/24"
        ]
    database_subnets = [
        for az in local.availability_zones : 
            "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_database_subnets + index(local.availability_zones, az)}.0/24"
        ]
    public_subnets = [
        for az in local.availability_zones : 
            "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_public_subnets + index(local.availability_zones, az)}.0/24"
        ]
}
```

# Issue #4: Limit number of subnets
In solution #3 we populated subnets according to the number of availability zones, which is excellent, but this can lead to unwanted behavior when using the module [terraform-aws-modules/vpc/aws](https://github.com/terraform-aws-modules/terraform-aws-vpc).

If you want to have a set of subnets per availability zone, without caring for how many subnets are created per region, you can stop here.
It will be easier to explain with an example:
ca-central-1 (Canada Central) - 2 availability zones, hence **6** subnets
us-west-2 (Oregon) - 4 availability zones, hence **12** subnets

If you wish the number of subnets to be similar across all environments (and regions) - keep on reading. For example:
Let's assume we must use ca-central-1 (Canada Central), and because of that, we are forced to use only two availability zones in each region.
ca-central-1 (Canada Central) - 2 availability zones, hence **6** subnets
us-west-2 (Oregon) - 4 availability zones, forcing maximum of **6** subnets (2 per type of subnet)
In total, we will have six subnets in our VPC (not including the 'default' one that comes with the VPC).

Rule of thumb - We need the lowest common number of availability zones that are available in all the regions we intend to use.

## Maximum number of subnets
We will now initialize three more variables, `subnet_max` per subnet.


`variables.tf`
```terraform
locals {
    cidr_c_private_subnets  = 1
    cidr_c_database_subnets = 11
    cidr_c_public_subnets   = 64

    max_private_subnets     = 2
    max_database_subnets    = 2
    max_public_subnets      = 2
}
```

## Conditional for loop
Luckily the for loop has a built-in keyword [if](https://www.terraform.io/docs/configuration/expressions.html#for-expressions), which still reminds me of Python's list comprehension :)

Syntax of for loop with condition:
`[for item in list: do_something_on(item) if expression]`

Our goal is to iterate over the availability zones until we reach the maximum desired number of subnets.


# Issue #4: Full Solution
And of course, we only need to change the `variables.tf` file.

`variables.tf`
```terraform
variable "cidr_ab" {
    type = map
    default = {
        development     = "172.22"
        qa              = "172.24"
        staging         = "172.26"
        production      = "172.28"
    }
}

locals {
    cidr_c_private_subnets  = 1
    cidr_c_database_subnets = 11
    cidr_c_public_subnets   = 64

    max_private_subnets     = 2
    max_database_subnets    = 2
    max_public_subnets      = 2
}

data "aws_availability_zones" "available" {
    state = "available"
}

locals {
    availability_zones = data.aws_availability_zones.available.names
}

variable "environment" {
    type = string
    description = "Options: development, qa, staging, production"
}

locals {
    private_subnets = [
        for az in local.availability_zones : 
            "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_private_subnets + index(local.availability_zones, az)}.0/24"
            if index(local.availability_zones, az) < local.max_private_subnets
        ]
    database_subnets = [
        for az in local.availability_zones : 
            "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_database_subnets + index(local.availability_zones, az)}.0/24"
            if index(local.availability_zones, az) < local.max_database_subnets
        ]
    public_subnets = [
        for az in local.availability_zones : 
            "${lookup(var.cidr_ab, var.environment)}.${local.cidr_c_public_subnets + index(local.availability_zones, az)}.0/24"
            if index(local.availability_zones, az) < local.max_public_subnets
        ]
}
```

# Summary
1. Some solution architects might prefer hard-coding the subnets prefixes in the `vpc.tf` file, which in most cases will work fine.
The fact that I had only to change the `variables .tf` file each time is priceless
2. The logic behind the order of variables in `variables.tf` file -
   - variables that should be modified, such as:
     - region per environment (map)
     - local cidr_c and max_subnets for each type of subnet
   - static variables/data, such as:
     - data "aws_availability_zones" "available"
     - local.availability_zones
     - local.subnet_list for each type of subnet
3. I find that using local values instead of variables is a good approach, and it provides the flexibility I need when designing the infrastructure
   - The only case where we need variables is when we want to prompt for values, for example: `var.environment`
   - Always use local values; they provide the ability to use functions, unlike variables
   - I used `var.environment` and `var.cidr_ab` - but I should've assigned them to local values. I used variables because I wanted to make this tutorial versatile. Remember- local values are the best
4. AWS keeps adding availability zones (AZs), so the number of availability zones per region in the examples might be outdated. Be sure to check the current number of AZs
   - [AWS Global Infrastructure](https://aws.amazon.com/about-aws/global-infrastructure/)
   - [Regions and Availability Zones](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html)

Did you like this tutorial? Clap/heart/unicorn and share it with your friends and colleagues.
Didn't you like it? Let me know which parts and I'll take care of it.

My next article will be about [Terraform Cloud](https://app.terraform.io/session) and [Terraform's Workspaces](https://www.terraform.io/docs/state/workspaces.html).

---
Originally published at [https://www.prodops.io](https://www.prodops.io/blog/terraform-aws-dynamic-subnets).