3 ways to handle secrets in AWS ECS tasks
Tags: aws, terraform, devops, docker
Published: 2019-06-15 18:17:43

No matter how much time and effort we spend on application security, it is always less. But simple workflows for things like secret management, key rotation and password expiration go a long way in making our applications and infrastructure resilient against the obvious attack vectors.

If you have used AWS Elastic Container Service (ECS) / Fargate, you’ve definitely stumbled upon the issue of passing secrets to the running container. This post describes 3 methods to pass secrets as environment variables to applications running as AWS ECS Tasks.

**Method 1** — Pass secrets as environment variables in the ECS Task Definition

* Good: Minimum infrastructure needed
* Bad: ECS Task Definition tightly coupled with secrets. A new deployment is needed if you change the value of the secret
    Bad: Secrets visible to anyone with access to the ECS Task Definition

**Method 2** — Fetch latest secrets using an entry-point script

Using a docker entry-point script, you can fetch secrets from wherever you want before the application boots.

* Good: Secrets visible to the application only at runtime and not before that. Meaning that they are not part of the ECS Task Definition
* Bad: The logic to fetch secrets is embedded repeatedly in different applications. So, while the secrets are (hopefully) maintained in your infrastructure configuration, each application need to know how and where to fetch them from. This violates both the [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) and [Single Responsibility principles](https://en.wikipedia.org/wiki/Single_responsibility_principle).

A lot of repositories on Github use this approach. An example of fetching secrets in this way from AWS S3 is also described [here](https://aws.amazon.com/blogs/security/how-to-manage-secrets-for-amazon-ec2-container-service-based-applications-by-using-amazon-s3-and-docker/).

**Method 3** — Use ‘valueFrom’ attribute for ECS task definition

This method puts the burden of grabbing secrets for you on AWS and the application container can expect for the environment variables to exist before the application is run

* Good: Application doesn’t need to know how to fetch secrets — it expects them to be available as environment variables during run-time.
* Good: Supports both AWS SSM Parameter Store and AWS Systems Manager (currently)

**My Recommendation**

The method I would recommend is to use the ‘valueFrom’ attribute to inject KMS encrypted AWS SSM parameters directly into the running task. This way the secrets are neither visible to someone logging into the AWS Console nor getting saved in plain text anywhere.

Standardizing the path you use for parameters that you add to the AWS SSM Parameter Store is a great idea because it’ll help you limit the scope of access to these secrets. The way you’ve designed your infrastructure would definitely determine what path formats you could possibly use. It doesn’t matter what format you decide — what is important is to just standardize ’some format’ rather than allowing each service to specify its own path format. So, overall, here are the 3 steps I recommend:

**Step 1**: Decide a path format of you AWS SSM Parameter Store. Examples are:

    /{environment}/{service}/DATABASE_PASSWORD
    /{service}/{enviroment}.DATABASE_PASSWORD

**Step 2**: Add your key value pairs to AWS SSM Parameter Store and use a KMS key to encrypt them

**Step 3**: Use the ‘secrets/valueFrom’ attribute to inject AWS SSM parameters directly into the running task

And here’s the Terraform configuration to do this. These files only contain the clauses needed for secrets — you still need to fill in the rest of the details to actually create an ECS task.
Terraform file (partial)

```tf
data "template_file" "blog" {
  template = "${file("template.json")}"

  vars {
    app_name    = "blog"
    environment = "staging"
    region      = "us-east-1"
    account     = "12341234"
  }
}

resource "aws_ecs_task_definition" "blog" {
  container_definitions = "${data.template_file.blog.rendered}"

  family             = "blog"
  task_role_arn      = "${var.iam_role_arn}"
  execution_role_arn = "${var.iam_role_arn}"
}
```

Template file:
```json
[
    {
        "secrets": [
            {
                "name": "DATABASE_PASSWORD",
                "valueFrom": "arn:aws:ssm:${region}:${account}:parameter/${environment}/${appName}/DATABASE_PASSWORD"
            }
        ]
    }
]
```

> Note: if you’re using ‘secrets/valueFrom’ in your ECS task definition, you need to also give it a ‘serviceRoleArn’ which has permissions to read SSM params needed for that scope.

**Additional Reading**

* [AWS Documentation » Amazon ECS » Developer Guide » Amazon ECS Task Definitions » Specifying Sensitive Data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html)
* [AWS Documentation » Amazon ECS » Developer Guide » Amazon ECS Task Definitions » Task Definition Parameters](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html)
* [Managing Secrets for Amazon ECS Applications Using Parameter Store and IAM Roles for Tasks](https://aws.amazon.com/blogs/compute/managing-secrets-for-amazon-ecs-applications-using-parameter-store-and-iam-roles-for-tasks/)
* [How to Manage Secrets for Amazon EC2 Container Service–Based Applications by Using Amazon S3 and Docker](https://aws.amazon.com/blogs/security/how-to-manage-secrets-for-amazon-ec2-container-service-based-applications-by-using-amazon-s3-and-docker/)
* [Secrets Management within AWS ECS](https://hackernoon.com/secrets-management-within-aws-ecs-1b6975819ccd)
* [ECS Secrets Done Right](https://hackernoon.com/ecs-secrets-done-right-9e094cfa6200)

*This post was originally published on [Medium](https://medium.com/faun/3-ways-to-handle-secrets-in-aws-ecs-tasks-c3f4c5d688f9)*