Terraforming AWS: a serverless website backend, part 2
Tags: terraform, aws, serverless, backend
Published: 2017-07-19 21:18:49

---
title: "Terraforming AWS: a serverless website backend, part 2"
published: true
description: Setup a serverless backend to your website using Terraform, an infrastructure-as-code tool from HashiCorp
tags: terraform, aws, serverless, backend
cover_image: https://thepracticaldev.s3.amazonaws.com/i/zmkbgpneu3efnm9yzo80.jpg
canonical_url: https://www.olicole.net/blog/2017/07/terraforming-aws-a-serverless-website-backend-part-2/
---
*This is part two of my article series on using Terraform to build a serverless backend in AWS. Check out [part one](terraforming-aws-a-serverless-website-backend-part-1) to get started.*
<!--more-->

## Data sources and encryption

Next up is a new concept: data sources. These allow you to pull in data from other external sources at runtime.
At the end of part one, we added this:

```javascript
data "aws_caller_identity" "current" {}
```
[`aws_caller_identity`](https://www.terraform.io/docs/providers/aws/d/caller_identity.html) allows you to get the AWS user ID of the account, and it's used in constructing policy documents and ARNs, using exactly the same interpolation as before:

```javascript
resource "aws_kms_key" "LambdaBackend_config" {
  description             = "LambdaBackend_config_key"
  deletion_window_in_days = 7

  policy = <<POLICY
{
  "Version" : "2012-10-17",
  "Id" : "key-consolepolicy-3",
  "Statement" : [ {
    "Sid" : "Enable IAM User Permissions",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "arn:aws:iam::${data.aws_caller_identity.current.account_id}:root"
    },
    "Action" : "kms:*",
    "Resource" : "*"
  }, {
    "Sid" : "Allow use of the key",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "${aws_iam_role.LambdaBackend_master_lambda.arn}"
    },
    "Action" : [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ],
    "Resource" : "*"
  }, {
    "Sid" : "Allow attachment of persistent resources",
    "Effect" : "Allow",
    "Principal" : {
      "AWS" : "${aws_iam_role.LambdaBackend_master_lambda.arn}"
    },
    "Action" : [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ],
    "Resource" : "*",
    "Condition" : {
      "Bool" : {
        "kms:GrantIsForAWSResource" : "true"
      }
    }
  } ]
}
POLICY
}

resource "aws_kms_alias" "LambdaBackend_config_alias" {
  name          = "alias/LambdaBackend_config"
  target_key_id = "${aws_kms_key.LambdaBackend_config.key_id}"
}
```

When we create some configuration for the service we'll be encrypting it - these resources allow us to do that!

You can `terraform apply` all of that.

Terraform is pre v1.0 software, and the `aws_kms_key` resource can sometimes throw an error. If this happens, just `terraform apply` again, and keep an eye on [this issue!](https://github.com/terraform-providers/terraform-provider-aws/issues/245)

# Application config

Now we're going to define some configuration for our Lambda function. It's good practice to separate application code and configuration, and it's even better practice not to check the latter into source control. We're going to store our configuration using the AWS EC2 Systems Manager, known as SSM, and in the process going to learn about Terraform variables and modules.

We're going to be storing our configuration in the SSM [Parameter Store](https://aws.amazon.com/ec2/systems-manager/parameter-store/). This is a key-value store useful for simple configuration strings, API keys and passwords - and it's free! You can configure content to be encrypted with the keys we just created, and control access to it with the IAM policies we also created.

[Variables](https://www.terraform.io/language/values/variables) in Terraform work in a similar way to most programming languages. You declare them in the code, possibly with a value, and then you retrieve the value with the `${var.variable_name}` syntax, similar to data sources above. You can also specify these variable values on the command line, or in separate files, meaning you don't have to check your secrets into source control. See [How to Use Terraform Variables](https://spacelift.io/blog/how-to-use-terraform-variables) for more information.

Terraform maps are similar to hashtables, hashmaps, dictionaries or similar data structures in other languages. Go ahead and define it in `terraform.tf`.

```javascript
variable "environment_configs" {
  type = "map"
}
```

Then you can create a new file called `terraform.tfvars` in the same directory as `terraform.tf`:

```javascript
environment_configs = {
  site_callback = "CALLBACK TO RETURN TO FROM LAMBDA",
  email_table_name = "EMAIL DYNAMODB TABLE",
  aes_password = "AES PASSWORD",
  mail_api_key = "MAILGUN KEY",
  mailgun_domain_name = "MAILGUN DOMAIN",
  from_email = "OUTGOING EMAIL ACCOUNT"
}
```

Some tips on the contents of the variables:

  * `site_callback` - An http(s) URL.
  * `email_table_name` - Just a simple table name following the [DynamoDB rules](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html#HowItWorks.NamingRules), like `emails`.
  * `aes_password` - A 32 character AES key. If you don't have one handy, check out [`keygen.js`](https://github.com/OliverCole/serverless-backend/blob/master/keygen.js).
  * `mail_api_key` - From your Mailgun account.
  * `mailgun_domain_name` - A domain or subdomain you control DNS for.
  * `from_email` - An account on that domain. Just a name, no need to set anything up - Mailgun will work regardless!

Terraform supports the concept of [modules](https://www.terraform.io/intro/getting-started/modules.html) - reusable collections of resources intended to work together: for example, a set of web servers and a load balancer. You can use this across multiple projects, even separating it out into separate source control or storage.

We're going to use a simple module to turn the `environment_configs` map into entries in the SSM Parameter Store. In the same directory as `terraform.tf` create a new directory called `ssm_parameter_map`, containing a file called `ssm_parameter_map.tf`, with this content:

```javascript
variable "configs" {
  description = "Key/value pairs to create in the SSM Parameter Store"
  type        = "map"
}

variable "prefix" {
  description = "Prefix to apply to all key names"
}

variable "kms_key_id" {
  description = "ID of KMS key to use to encrypt values"
}

resource "aws_ssm_parameter" "configs" {
  count  = "${length(keys(var.configs))}"
  name   = "/${var.prefix}/${element(keys(var.configs),count.index)}"
  type   = "SecureString"
  value  = "${element(values(var.configs),count.index)}"
  key_id = "${var.kms_key_id}"
}
```

You will notice that this module has variables of its own, and the values get set when you call this module from your own code. For more details on the rest of the code, check out Gruntwork's article on [loops and if statements](https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9).

Call your module from `terraform.tf`:

```javascript
module "parameters" {
  source     = "/ssm_parameter_map"
  configs    = "${var.environment_configs}"
  prefix     = "${terraform.env}"
  kms_key_id = "${aws_kms_key.LambdaBackend_config.key_id}"
}
```

The `source` argument points to the module code, and the other arguments feed into the variables you saw in `ssm_parameter_map.tf`. `terraform.env` is currently the string `default`, but this will change later on when we delve into Terraform state environments.

Because Terraform modules could be located anywhere, you need to run `terraform init` to pull down your copy of the module code. This is the same command you ran earlier to download the AWS provider for Terraform. It should look like this: 

```
λ terraform init 
Initializing modules... 
- module.parameters 
  Getting source "./ssm_parameter_map" 
```

Then `terraform apply` as usual. Remember that you will need to run `terraform get` again if you move between machines or Git working copies.

Terraform is pre v1.0 software, and the `aws_ssm_parameter` resources in the module can sometimes throw a `TooManyUpdates` error. If this happens, just `terraform apply` again, and keep an eye on [this issue!](https://github.com/terraform-providers/terraform-provider-aws/issues/1082)

# Application code

Now we're going to put together code for our Lambda function. Once it's built and packaged for deployment, we'll switch back to Terraform and deploy it.

 1. Create a sub-directory called `email_lambda`
 1. Download [`index.js`](https://github.com/OliverCole/serverless-backend/blob/master/email_lambda/index.js) to this directory.
    * On line 8, change us-west-1 to the same AWS region you configured your `aws` provider with.
    * This is our main app code
 1. Add a file called `email.html` to this directory containing the email content.
    * If you're short on ideas, you can use this [example file](https://github.com/OliverCole/serverless-backend/blob/master/email_lambda/email.html). 
 1. Install the email libraries we will use with npm. Make sure to do this from inside `email_lambda`: 
```
npm install -prefix=./ nodemailer@4.0.1 nodemailer-mailgun-transport@1.3.5 aws-sdk@2.81.0 ssm-params@0.0.6
```

Now zip the whole directory into `email_lambda.zip`, making sure that the root of the zip file contains the files (`index.js` etc) and not a directory called `email_lambda`. It should look like this:

```
email_lambda.zip
├───etc
├───node_modules
├───email.html
└───index.js
```

If you have 7Zip installed on Windows, the command would be `7z.exe a -r email_lambda.zip .\email_lambda\*`.

At this point, your directory structure should look something like this:

```
├───.terraform
│   └───modules
│       └───03f77d1ff66d94c49e171247a4234cd8
├───email_lambda
│   ├───etc
│   └───node_modules
│       ├───nodemailer
│       │   └─── ...
│       ├───nodemailer-mailgun-transport
│       │   ├─── ...
│       └───ssm-params
│           └─── ...
├───ssm_parameter_map
├───email_lambda.zip
├───terraform.tf
├───terraform.tfstate
├───terraform.tfstate.backup
└───terraform.tfvars
```

Now, you can add the Lambda function to `terraform.tf`:

```javascript
resource "aws_lambda_function" "LambdaBackend_lambda" {
  filename         = "email_lambda.zip"
  function_name    = "LambdaBackend"
  role             = "${aws_iam_role.LambdaBackend_master_lambda.arn}"
  handler          = "index.handler"
  source_code_hash = "${base64sha256(file("email_lambda.zip"))}"
  runtime          = "nodejs6.10"
  timeout          = 15
  publish          = true

  environment {
    variables = {
      env = "${terraform.env}"
    }
  }
}
```

Apply that, and carry on to the [final part](terraforming-aws-a-serverless-website-backend-part-3).