Real-life Terraform Refactoring Guide
Tags: terraform, devops, infrastructureascode, cloud
Published: 2021-08-16 18:06:35

# Table of Contents

1.  [Intro](#orga4c7a97)
2.  [How to break a big ball of mud? STRANGLE IT](#org3702753)
3.  [The mono-repository (monorepo) approach to Legacy](#orgeaee6de)
    1.  [Splitting the `modules` sub-path to its own repository](#orgdd4979a)
    2.  [Let&rsquo;s start strangling the repository](#org5026de0)
        1.  [Import state? Remove state and code from what? Where?](#org5230407)

<a id="orga4c7a97"></a>

# Intro

As reality hits, the unavoidable fact of dealing with a hard-to-manage Terraform [Big ball of mud](https://en.wikipedia.org/wiki/Big_ball_of_mud) code base comes in. There is no way around natural growth and evolution of code bases and the design flaws that come with it. Our Agile mindset is to [&ldquo;move fast and break things&rdquo;](https://www.brainyquote.com/quotes/mark_zuckerberg_453439), implement something as simple as possible and let the design decisions for the next iterations (if any).

Refactoring Terraform code is actually as natural as developing it, time and time again you will be face situation where a better structure or organization can be achieved, maybe you want to upgrade from a home-made module to an open-source/community alternative, maybe you just want to segregate your resources into different states to speed-up development. Regardless of the goal, once you get into it, you will realize that Terraform code refactoring is actually a basic missing step on the development process that no one told you before.

As the [Suffering-Oriented Programming](http://nathanmarz.com/blog/suffering-oriented-programming.html) mantra dictates:

> &ldquo;First make it possible. Then make it beautiful. Then make it fast.&rdquo;

So, time to make the Terraform code beautiful!

<a id="org3702753"></a>

# How to break a big ball of mud? STRANGLE IT

`<joke>`Martin Fowler has already written everything there is to write about (early 2000s) DevOps, Agile, and Software Development. Therefore, we could reference Martin Fowler for virtually anything Software related`</joke>`, but really, the [Refactoring book](https://martinfowler.com/books/refactoring.html) is **THE** reference on this subject.

Martin Fowler shared the [Stangler (Fig) Pattern](https://martinfowler.com/bliki/StranglerFigApplication.html), which describes a strategy to refactor a legacy code base by re-implementing the same features (sometimes even the bugs) on another application.

> [&#x2026;] the huge strangler figs. They seed in the upper branches of a tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host.
>
> This metaphor struck me as a way of describing a way of doing a rewrite of an important system.

In this document we are going to follow the same idea:

1.  implement the same feature on a different [Terraform composition](https://www.terraform-best-practices.com/key-concepts#composition);
2.  migrate the Terraform state;
3.  delete (kill) the previous implementation.

<a id="orgeaee6de"></a>

# The mono-repository (monorepo) approach to Legacy

Let&rsquo;s suppose that your Terraform code base is versioned in a single repository (a.k.a. monorepo), following the random structure displayed below (just to help illustrate)

```
.
├── modules/    # Definition of TF modules used by underlying compositions
├── global/     # Resources that aren't restricted to one environment
│   ├── aws/
├── production/ # Production environment resources
│   └── aws/
└── staging/    # Staging environment resources
    └── aws/
```

In this example, each directory corresponds to a Terraform state. In order to apply changes, you have to walk to a path and execute `terraform`.

The structure on this example repository was created a few hypothetical years ago when the number of existing microservices and resources (DB, message queues, etc) was significantly smaller. At the time, it was feasible to keep Terraform definitions together because it was easier to maintain, Cloud resources were managed with one-shot!

As time went by, the number of Products and the team grew, and engineers started facing concurrency issues: Terraform lock executions on shared storage when someone else is running `terraform apply` as well as a general slowness on **every execution** since the number of data sources to sync is frightening.

A mono-repository approach is not necessarily bad, versioning is actually simpler when performed in one single repository. Ideally, there won&rsquo;t be many changes on the scale of GiB meaning that it is safe to proceed on this one _as long as the Terraform remote states are divided_.

<a id="orgdd4979a"></a>

## Splitting the `modules` sub-path to its own repository

One thing to mention though is the `modules` sub-path, this one could be stored in a different git repository to leverage its own versioning. Since Terraform modules and their implementations don’t always evolve at the same pace, keeping two distinct version trees is beneficial. Additionally, a separated repository for Terraform modules allows the specification of “pinned versions”, e.g.:

```terraform
module "aws_main_vpc" {
    source = "git::https://github.com/terraform-aws-modules/terraform-aws-vpc.git?ref=2ca733d"
    # Note the ref=${GIT_REVISION_DIGEST}
}
```

That reference for a module&rsquo;s version should always be specified, regardless if it comes from an internal/private repository or public. When you specify the version, you are ensuring reproducibility.

Therefore, let&rsquo;s move the `modules` sub-path to another git repository, following instructions from [this StackOverflow answer](https://stackoverflow.com/questions/359424/detach-move-subdirectory-into-separate-git-repository/17864475#17864475) so that the git commit history is preserved:

### 0.

Walk to the monorepo path and create a branch from the commits at `monorepo/modules` path

```bash
MAIN_BIGGER_REPO=/path/to/the/monorepo
cd "${MAIN_BIGGER_REPO}"
git subtree split -P modules -b refact-modules
```

### 1.

Create the new repository

```bash
mkdir /path/to/the/terraform-modules && cd $_
git init
git pull "${MAIN_BIGGER_REPO}" refact-modules
```

### 2.

Link the new repository to your remote Git (server) and push the commits

```bash
git remote add origin <git@git.com:user/terraform-modules.git>
git push -u origin master
```

### 3.

Cleanup the history related to `modules` from `$MAIN_BIGGER_REPO` [OPTIONAL]

```bash
cd ${MAIN_BIGGER_REPO}
git rm -rf modules
git filter-branch --prune-empty \
    --tree-filter "rm -rf modules" -f HEAD
```

<a id="org5026de0"></a>

## Let&rsquo;s start strangling the repository

Now that a substantial piece of code was moved somewhere else, it is time to put the [Stangler (Fig) Pattern](https://martinfowler.com/bliki/StranglerFigApplication.html) in practice.

Move all the existing content as-is to the `legacy` sub-path, keeping the same repository and change history (commits). It also allows applying the `legacy` code as it used to be from one of those paths.

```
.
└── legacy
    ├── global
    │   └── aws
    ├── production
    │   └── aws
    └── staging
        └── aws
```

Once the content is moved to legacy, the idea is to follow the [Boy Scout rule](https://www.oreilly.com/library/view/97-things-every/9780596809515/ch08.html) in order to strangle the `legacy` content little by little (unless you are
really committed to migrating it all at once, which is going to be exhaustive).

The Boy Scout rule goes like this:

1.  every time a task that involves deprecated code appears, we implement it on [the new structure](https://dev.to/macunha/terraform-design-best-practices-1i0h);
2.  import the Terraform state to keep the Cloud resources that a given code represents/describes;
3.  remove the state and the code from `legacy`.

Until there is nothing left inside `legacy` (or there are only unused resources/left-behinds that could be destroyed/garbage collected either way).

<a id="org5230407"></a>

### Import state? Remove state and code from what? Where?

That will depend on the kind of resource we are migrating from the remote state, on the bottom of each `resource` on Terraform&rsquo;s provider documentation you can find a reference command to import existing resources into your Terraform code specification. e.g.: [AWS RDS DB instance](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance#import).

Suppose we want to replace the code of the AWS RDS Aurora defined in `production/aws` and then re-implement the same using [the community module](https://github.com/terraform-aws-modules/terraform-aws-rds-aurora). After creating the corresponding sub-path to the monorepo according to your preference, provisioning the bucket and initializing the Terraform `backend`:

### 1.

Implement the definition of the community module [github.com/terraform-aws-modules/terraform-aws-rds-aurora](https://github.com/terraform-aws-modules/terraform-aws-rds-aurora) with the closest parameters from the existing one; e.g.:

```terraform
module "aws_aurora_main_cluster" {
 source  = "terraform-aws-modules/rds-aurora/aws"
 version = "~> 5.2"

 # ...
}
```

### 2.

Import the Terraform states from the previous (existing) cluster

```bash
terraform import 'aws_aurora_main_cluster.aws_rds_cluster.this[0]' main-database-name
terraform import 'aws_aurora_main_cluster.aws_rds_cluster_instance.this[0]' main-database-instance-name-01
terraform import 'aws_aurora_main_cluster.aws_rds_cluster_instance.this[1]' main-database-instance-name-02

# ...
```

then if you haven&rsquo;t yet and would like to &ldquo;match reality&rdquo; between the existing and the specified resource, run `terraform plan` a few times and adjust the parameters until Terraform reports:

```
No changes. Your infrastructure matches the configuration.
```

### 3.

Last but not least, remove the corresponding resources from the `legacy` Terraform state so that it doesn&rsquo;t try to keep track of the changes and also don&rsquo;t try to destroy once the resource definition is no longer in that code base:

```bash
# Hypothetical name of the resource inside production/aws/main.tf
terraform state rm aws_rds_cluster.default \
    'aws_rds_cluster_instance.default[0]' 'aws_rds_cluster_instance.default[1]'

# ...

```

once that is performed, feel free to remove the corresponding resource&rsquo;s definition from the `legacy` code.

```terraform
resource "aws_rds_cluster" "default" {
    # ...
}

resource "aws_rds_cluster_instance" "default" {
    count = var.number_of_database_instances

    # ...
}
```