Managing AWS SSM Parameters With Terraform And Parzival
Tags: aws, terraform, devops, tutorial
Published: 2021-09-17 10:53:46

Modern applications often use external systems to manage configuration values and secrets. One of those external systems is [AWS Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html), which is part of the [AWS Systems Manager](https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html) service.

There's a great blog about the top [10 Best Secret Management Software for Application Security](https://geekflare.com/secret-management-software/) that covers this subject in a significant way.

This blog post will demonstrate a practical way of managing secrets/parameters in the AWS SSM Parameter Store with Terraform. Wondering what Parzival is? Keep on reading to find out.

## Plot

Managing application values is quite easy for small applications, where only one (1) to three (3) values are fetched from an external system. Let's say my application is small and requires 3 values per live environment (`development`, `staging` and `production`) let's map them and their types:

<table><thead><tr><th>Name</th><th>Type</th></tr></thead><tbody><tr><td>GOOGLE_CLIENT_ID</td><td>Secure String</td></tr><tr><td>GOOGLE_CLIENT_SECRET</td><td>Secure String</td></tr><tr><td>LOG_LEVEL</td><td>String</td></tr></tbody></table>

## Challenge #1: Live Environments Alignment

I logged in to AWS Console and created the three (3) parameters in AWS Parameter Store per environment (9 total). Now, I need to add another parameter, `SLACK_WEBHOOK_URL` of `String` type. Do you realize that each parameter that needs to be added requires doing so per live environment? That's **3x** times, and it gets crazier as the application grows.

There's also a chance that I've added a parameter in `development` and didn't add it in `staging`, following that I rolled out a deployment of my application to `staging` and boom :boom: -  `staging` is dead baby, `staging`'s dead.

![zed-is-dead-baby](https://media.giphy.com/media/xT9KVIGRWBMmpkpNmg/giphy.gif?cid=ecf05e47qqh8m662g1nriy08ekzozsathyyde8tfpoq8uzac&rid=giphy.gif&ct=g)

Luckily it's only `staging`, and `production` didn't suffer any downtime, but it still consumes precious engineering time investigating why the application crashed in `staging`. Also, if the QA engineers are running tests on `staging`, they're now stuck until I figure out what went wrong. Of course, that applies to all cloud resources, such as AWS EC2, S3 Bucket, etc. 

Many sad stories are being told by engineers who maintain their cloud resources for multiple live environments with the AWS Console (GUI), let's not be the ones who tell those stories.

### Live Environments Alignment - Solution

Using [Infrastructure as Code](https://learn.hashicorp.com/tutorials/terraform/infrastructure-as-code) mitigates the risk of having misaligned live environments. And as mentioned in the title, I'll implement IaC for managing SSM Parameters with [Terraform](https://www.terraform.io/).

## Challenge #2: Storing Secrets Safely

For [obvious reasons](https://blog.gitguardian.com/secrets-credentials-api-git/), I don't want to commit sensitive data to my source code (git). But, on the other hand, I do want version control for parameters **values**.

This part is tricky; how do I create an SSM Parameter with Terraform without committing its value to the source code? Terraform's files are part of the source code, so it's kind of a paradox.

![inception-stairs-paradox](https://s3.scoopwhoop.com/anj2/5f1e878560c0604350db577c/16b57f71-6dfd-451a-8c72-d45c54c9ac04.jpg)

### Storing Secrets Safely - Solution

Firstly, I'll create SSM parameters with Terraform; the initial value for each parameter should be a dummy value like `"empty"`. Later on, the Dev team can validate parameter values by checking if it's still `"empty"`, which means humans didn't set it. That makes it easier to track and monitor which parameters weren't set by humans.

Secondly, I'll manage the parameter **values** with AWS Console (GUI). Another paradox is that I'm about to use AWS Console, but in the previous section, I wrote:

> "...Many sad stories are being told by engineers who maintain their **cloud resources** ... with AWS Console (GUI)..."

The bolded text is the answer; I'm about to manage my **values** in the GUI, **not cloud resources**.

Lastly, to address the requirement "...*versioning for parameters values*..." I'll use the version management feature of SSM Parameters, see [Working with SSM Parameters Versioning](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-versions.html). I will not cover SSM Parameters Versioning in this blog post.

Okay, sounds great, but I don't see a real solution here; it sounds like a nice theory, but how do I do it in practice?

## There's A Terraform Module For That

There are many terraform modules out there that help with managing SSM Parameters, here are some examples:

- [cloudposse/terraform-aws-ssm-parameter-store](https://github.com/cloudposse/terraform-aws-ssm-parameter-store)
- [umotif-public/terraform-aws-ssm-parameters](https://github.com/umotif-public/terraform-aws-ssm-parameters)
- [tmknom/terraform-aws-ssm-parameter](https://github.com/tmknom/terraform-aws-ssm-parameter)
- [fxinnovation-public/terraform-module-aws-ssm-parameters](https://scm.dazzlingwrench.fxinnovation.com/fxinnovation-public/terraform-module-aws-ssm-parameters)

I needed something "simpler"; A way to provide a list of parameters and their types, a prefix to be added to each parameter, and ... that's it ¯\_(ツ)_/¯

Meet [unfor19/terraform-aws-ssm-parameters](https://github.com/unfor19/terraform-aws-ssm-parameters), a Terraform module that I've created. Here's what it does:

> "Create AWS SSM Parameter Store parameters with a Terraform module. The creation/deletion (schema) is managed with Terraform, and the values should be maintained via AWS Console."

Getting back to our example, here's how you would create SSM Parameters with my module. To keep it simple, I'm leaving out the previously mentioned parameter `SLACK_WEBHOOK_URL`.

```ruby
module "app_params" {
    source  = "unfor19/ssm-parameters/aws"
    version = "0.0.2"

    prefix = "/myapp/dev/"

    string_parameters = [
        "LOG_LEVEL",
    ]
    securestring_parameters = [
        "GOOGLE_CLIENT_ID",
        "GOOGLE_CLIENT_SECRET"
    ]
}
```

Well? Isn't it incredible? The Terraform module will create all the provided parameters and add `prefix` to their names. For me, having the parameters schema (name+type) in a single place makes it easier to realize which parameters are available per live environment.

Reminding you that the initial value for each parameter is `"empty"`, which can be changed by setting some of the inputs;  check [Terraform Registry Inputs Page](https://registry.terraform.io/modules/unfor19/ssm-parameters/aws/latest?tab=inputs) to see which inputs are available.

Finally, I can log in to AWS Console and manage my SSM Parameter **values** without worrying that the next `terraform apply` will change their values. If you wonder how I did it, I used [Terraform lifecycle Meta-Argument](https://www.terraform.io/docs/language/meta-arguments/lifecycle.html); To see a practical example of how to use `lifecycle` with SSM Parameters values, check my module's [main.tf](https://github.com/unfor19/terraform-aws-ssm-parameters/blob/master/main.tf#L12-L16).

It's also important to mention that the module is tested against various Terraform versions.

- 0.12.31
- 0.13.6
- 0.14.8
- 0.15.0-beta2
- 1.0.6 (latest)

For more details about the test suite, see the [CI/CD logs](https://github.com/unfor19/terraform-aws-ssm-parameters/actions/workflows/tests.yml) and [Terraform Module Local Development](https://github.com/unfor19/terraform-aws-ssm-parameters#local-development).

## Recap

I've created three (3) SSM Parameters with Terraform, managing their values through the AWS Console. The values are detached from Terraform's scope, so using `terraform apply` will **not** override the changes that I've made to values via AWS Console.

## Challenge #3: Managing SSM Parameter Values Programmatically

Setting the values for three (3) parameters via AWS Console per live environment is okayish. However, as the application grows, the number of parameters might grow.

Bear with me on this one; The application grew more prominent and required thirty (30) SSM Parameters. So I've created those thirty (30) parameters in `development`, and set their values by using AWS Console (GUI). Let's assume that twenty-five (25) out of thirty (30) are identical in `development` and `staging`, leaving five (5) unique values per environment.

I wish there were a way to copy the twenty-five (25) parameters' values from `development` to `staging`. I hope you realize how painful it can be to do it manually and go back and forth to copy-paste parameter values.

![psycho-horror](https://i.guim.co.uk/img/static/sys-images/Arts/Arts_/Pictures/2010/3/29/1269881473433/1960-PSYCHO-001.jpg?width=620&quality=45&auto=format&fit=max&dpr=2&s=cf72a0acfe56da8ce2ab5dc6be10c665)

Usually, it's not a pain in `development` because parameter values are set one at a time and not at once. However, the scenario I'm referring to happens when creating a new live environment, like `staging`, where all the initial values are `"empty"`. 

Another possible scenario for needing a "bulk copy of SSM Parameters" is when I need to create a dynamic environment and set the initial values of the dynamic environment to be the same as those in the `development` environment.

And again, it sounds nice, but how would you copy SSM Parameters values in bulk? Is there such a way? 

### Managing SSM Parameter Values Programmatically - Solution

Meet [unfor19/parzival](https://github.com/unfor19/parzival). The reason for the name is ridiculous 😁 - I thought about naming it *paraval* which stands for **para**meter-**val**ues, and it sounded like *[Parzival](https://en.wikipedia.org/wiki/Parzival)* which is way cooler, especially if you watched [Ready Player One](https://readyplayerone.fandom.com/wiki/Wade_Watts).

![ready-player-one](https://imagesvc.meredithcorp.io/v3/mm/image?url=https%3A%2F%2Fstatic.onecms.io%2Fwp-content%2Fuploads%2Fsites%2F6%2F2018%2F03%2Frpo-trlr-0175-2000.jpg)

> **CHALLENGE**: I urge you to check if there's an existing CLI that can get/set [more than 10 SSM Parameters](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetParametersByPath.html#API_GetParametersByPath_RequestSyntax) by path in a single command. The official AWS CLI supports pagination, but it means I need to write code that wraps `aws ssm get-parameters-by-path ...`. The only similar project that I found is called [binxio/aws-ssm-copy](https://github.com/binxio/aws-ssm-copy), which didn't have the capabilities that I need.

Moving on with the plot, Parzival is beneficial for the following scenarios:

1. Copy values from one environment to another - for example, from `development` to `staging`, or `development` to `my-dynamic-env-1`, and so on. Copying values across accounts is also possible because the `get` and `set` commands are separated.
1. Troubleshooting SSM Parameters values - for example, get all parameters by the path and find the ones that a human didn't initialize (contains the value `"empty"`).

To conclude, Parzival is a simple wrapper for [get-parameters-by-path](https://docs.aws.amazon.com/cli/latest/reference/ssm/get-parameters-by-path.html) and [put-parameter](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ssm/put-parameter.html).

Let's see it in action!

#### Get Parameters By Path

```bash
parzival get --region "us-east-1" \
     --output-file-path ".dev_parameters.json" \
     --parameters-path "/myapp/dev/"
```

#### Set Parameters Values By Path

```bash
parzival set --region "us-east-1" \
     --input-file-path ".dev_parameters.json" \
     --prefix-to-replace "/myapp/dev/" \
     --parameters-path "/myapp/stg/"
```

To view the output of the above commands, check the [CI/CD Logs](https://github.com/unfor19/parzival/actions) and look for the **Test** job.

## Final Words

What are you waiting for? Go ahead and play with the Terraform module [unfor19/terraform-aws-ssm-parameters](https://github.com/unfor19/terraform-aws-ssm-parameters), and then manage the values of SSM Parameters with [unfor19/parzival](https://github.com/unfor19/parzival).

Both projects take advantage of [localstack](https://github.com/localstack/localstack), which provides a fully functional local cloud environment, like "local AWS". Using localstack as part of the development process helped me a lot to battle-test my application rapidly; for that reason, you don't even need an AWS account to check the Terraform module and Parzival. So follow the instructions in the `README.md` files and do your magic.

**Happy SSMing!**

## Images Sources

- [Cover Image HackerBoy Emoticon](http://123emoji.com/hacker-boy-sticker-5682/)
- [Zed Is Dead Baby](https://giphy.com/gifs/quentin-tarantino-pulp-fiction-bruce-willis-xT9KVIGRWBMmpkpNmg)
- [Inception Stairs Paradox](https://www.scoopwhoop.com/entertainment/inception-quiz/)
- [Psycho Horror](https://www.theguardian.com/film/2010/mar/29/psycho-body-double-marli-renfro)
- [Ready Player One](https://ew.com/movies/2018/03/26/ready-player-one-review/)