Main-module approach for handling multiple environments in Terraform
Tags: cloud, terraform, devops
Published: 2020-02-26 08:57:39

Handling multiple environments in Terraform might be challenging and you can find many approaches and best practices on the Internet. Ths "main-module" approach is one of them using just built-in Terraform concepts (as an alternative to Terraform workspaces with Terraform Cloud/Enterprise or using wrapper tools like Terragrunt).

*But why not just use “tfvars” files?*
Because you cannot use variables in backend and provider configuration blocks.

Goals:
- Have separate backends for Terraform remote state-files per environment
- Be able to use separate system accounts for different environments
- Be able to use different versions of providers and Terraform itself per environment (and upgrade one by one)
- Ensure that all required properties are provided per environment (Terraform validate won't pass if an environmental property is missing)
- Ensure that all resources/modules are always added to all environments. It is not possible to "forget" about a whole module.


Here is a basic structure of a Terraform project with the main-module approach that worked for me and few other teams:
```
- terraform_project
  - env
     - dev01 		<-- Terraform home, run from here 
        - .terraform	<-- git ignored of course
	    - dev01.tf	<-- backend, env config, includes always _only_ the main module
	 - dev02
        - .terraform
	    - dev02.tf 
     - stg01
        - .terraform
	    - stg01.tf
     - prd01
        - .terraform
       	- prd01.tf 
  - main 		<-- main umbrella module
     - main.tf
	 - variables.tf			
  - modules 		<-- submodules of the main module
     - module_a
	 - module_b
	 - module_c
```

And a real-life example screenshot:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/whjkigwair4rktcgizda.png)

**env**
Each env subdir is a Terraform "home". 
Run terraform init/plan/apply in these directories. Change the environment by changing the directory. This prevents an error of merging state files of multiple environments.

Sample **dev01.tf** file (part):
```
provider "azurerm" {
  version = "~>1.42.0"
}
 
terraform {
  backend "azurerm" {
    resource_group_name  = "tradelens-host-rg"
    storage_account_name = "stterraformstate001"
    container_name       = "terraformstate"
    key                  = "dev.terraform.terraformstate"
  }
}
 
module "main" {
  source               = "../../main"
  subscription_id      = "000000000-0000-0000-0000-00000000000"
  project_name         = "tlens"
  environment_name     = "dev"
  resource_group_name  = "tradelens-main-dev"
  tenant_id            = "790fd69f-41a3-4b51-8a42-685767c7d8zz"
  location             = "West Europe"
  developers_object_id = "58968a05-dc52-4b69-a7df-ff99f01e12zz"
  terraform_sp_app_id  = "8afb2166-9168-4919-ba27-6f3f9dfad3ff"
 
  kubernetes_version      = "1.14.8"
  kuberenetes_vm_size     = "Standard_B2ms"
  kuberenetes_nodes_count = 4
 
  enable_ddos_protection = false
  enable_waf             = false
}

```
This gives a possibility to configure backend and provider per environment as well as provide all required parameters to the main module (i.e. the environment)

**main**

This is the main module that is an umbrella module for the whole Terraform project.
 
Having this module ensures that each environment will be as similar as possible to other environments. Adding a new resource or module to the main module will add this to all environments.

In order to implement differences between environments (pricing tiers etc.) use flags and parameters. 

The **variables.tf** file lists the required parameters for an environment. Environment-specific settings. 

Sample **variables.tf** file (part)
```
variable "subscription_id" {
  description = "Azure Subscription ID"
  type        = string
}
 
variable "project_name" {
  description = "Project name"
  type        = string
}
 
variable "environment_name" {
  description = "Environment name"
  type        = string
}
 
variable "resource_group_name" {
  description = "Target resource group name"
  type        = string
}
 
...

```
The main module can include submodules and resources. However, it is a good practice to divide it into a few submodules to introduce some encapsulation, increase readability and limit dependencies between resources.

Any shared remote resources will be included here.

A sample **main.tf** file (part):
```
module "eventhub" {
  source              = "../modules/eventhub"
  environment_name    = var.environment_name
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
}
 
module "dremio" {
  source                   = "../modules/dremio"
  environment_name         = var.environment_name
  resource_group_name      = var.resource_group_name
  project_name             = var.project_name
  linux_admin              = "dremio"
  linux_admin_password     = "1b4d7c79-a143-45e2-8ff1-ee63c0823090"
  dremio_data_name         = module.dremio_storage.dremio_data_name
  dremio_data_id           = module.dremio_storage.dremio_data_id
  dremio_data_disk_size_gb = module.dremio_storage.dremio_data_disk_size_gb
}
 
module "dremio_storage" {
  source              = "../modules/dremio-storage"
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
}
 
 
module "datalake" {
  source              = "../modules/datalake"
  resource_group_name = var.resource_group_name
  project_name        = var.project_name
  environment_name    = var.environment_name
}

```

This should be enough to visualize the concept.