Building AWS Infra with Terraform 3
By: Joaquín Menchaca, 智裕
Tags: AWS, Bash, Terraform, Site Reliability Engineer, DevOps

Building AWS Infra with Terraform 3

Creating Web Application and Database Infrastructure

I wanted to get this out quickly as someone besides my friend would want to see the final step to this small series. This is continuation of the series to learn how to provision AWS with Terraform.

In the last article we covered the infrastructure concern. In this article we will cover the web application concern that will include the following:

Instance hosting web application ( EC2 )

) Database used by web application ( MySQL managed by RDS )

managed by ) Web Application itself (simple LAMP Application)

Previous Article

Web Application Concern

We can start to put applications on top of the infrastructure foundation we just created.

In the ~/tf-projects/ directory, we’ll create the following structure:

.

└── webapp/

├── app/

│ ├── main.tf

│ ├── user_data.sh

│ └── variables.tf

├── aws.tf

├── db/

│ ├── main.tf

│ └── variables.tf

├── main.tf

└── variables.tf

Create Structure

Run this under Bash to create the structure and the files we will edit:

cd ~/tf-projects/webapp

touch {.,app,db}/{main.tf,variables.tf} app/user_data.sh

Create WebApp Module

We will create the module for the web application concern with two sub-modules: one for the web application itself and the other for the database.

WebApp Input Variables

First we will create some variables that will be used in this module. We’ll start with the variables for the AWS provider:

cat <<-'WEBAPP_VARIABLES' > ~/tf-projects/webapp/variables.tf variable "profile" {}

variable "region" {} WEBAPP_VARIABLES

We’ll want to add variables that we’ll pull from the infra module (from the previous article) and reuse in the web application:

cat <<-'WEBAPP_VARIABLES' >> ~/tf-projects/webapp/variables.tf # security groups

variable "sg_web" {}

variable "sg_db" {} # subnets

variable "sn_web" {}

variable "sn_db1" {}

variable "sn_db2" {} WEBAPP_VARIABLES

Lastly, we’ll want to add configuration that is unique to our web application, the database user name and password to a newly created database.

cat <<-'WEBAPP_VARIABLES' >> ~/tf-projects/webapp/variables.tf # config artifact

variable "database_name" {}

variable "database_user" {} # secrets artifact

variable "database_password" {} # instance key pair

variable "key_name" {} WEBAPP_VARIABLES

Variables that we configure for an application can be called configuration artifacts, and configuration artifacts that are sensitive are called secrets artifacts.

Ideally, we will want store configuration artifacts somewhere that can be referenced, and secrets artifacts should be stored in encrypted format.

In order to keep things simple for this tutorial, we’ll store these in as ~/tf-projects/db.tfvars to store secrets and configuration:

cat <<-'SECRETS' >> ~/tf-projects/db.tfvars

database_user = "admin"

database_password = " database_name = "webdb"database_user = "admin"database_password = " @U1bO8s $^&GkUAz*l$$@BG87" SECRETS

Never check this file into a code repository because our secret would not be safe. We will want to put *.tfvars into .gitignore file.

WebApp Main

cat <<-'WEBAPP_MODULE' > ~/tf-projects/webapp/main.tf module "instances" {

source = "./app" sg_web = "${var.sg_web}"

sn_web = "${var.sn_web}"

key_name = "${var.key_name}"

} module "db" {

source = "./db" sg_db = "${var.sg_db}"

sn_db1 = "${var.sn_db1}"

sn_db2 = "${var.sn_db2}" database_name = "${var.database_name}"

database_user = "${var.database_user}"

database_password = "${var.database_password}"

} WEBAPP_MODULE

Create Web Application

For this sub-module app , we’ll create an EC2 instance to host the web application and install the web application itself.

Note: this application is not highly available, as it is only installed on a single public subnet that lives on a single AZ (availability zone). Should we want to make it more available, we would create at least two identical web servers installed on subnets in different AZs, and then park these behind an ELB (elastic load balancer) that could send traffic to one of these two web servers. For this exercise, we’re keeping it simple.

App Input Variables

This sub-module takes two inputs, a public subnet and a security group.

cat <<-'APP_VARIABLES' >> ~/tf-projects/webapp/app/variables.tf variable "sg_web" {}

variable "sn_web" {}

variable "key_name" {} APP_VARIABLES

System Image Data Source

We the operating system we wish to use, we’re going to use Amazon Linux, which based from RHEL. We have to find AMI (Amazon Machine Image) for us-east-2 .

The lazy way is to find the ID, but then this make the script only work for us-east-2 , and also invites security vulnerabilities, as these machine images churn often to fix bugs and plug vulnerabilities.

For ameliorate this, we can look up the information using a data source:

cat <<-'APP_MODULE' > ~/tf-projects/webapp/app/main.tf data "aws_ami" "amazon-linux-2" {

most_recent = true filter {

name = "virtualization-type"

values = ["hvm"]

} filter {

name = "architecture"

values = ["x86_64"]

} filter {

name = "name"

values = ["amzn2-ami-hvm-2.0*"]

} owners = ["137112412989"] # Amazon

} APP_MODULE

Now that we have the this, we can reference our target image with:

data.aws_ami.amazon-linux-2.id

User Data Startup Script

We need a script to provision our server with the web service. Amazon provided a small web application that we’ll download and install. We also want to install Apache HTTP server, PHP, and MySQL client that the application needs:

cat <<-'USER_DATA' > ~/tf-projects/webapp/app/user_data.sh #!/bin/bash -ex

yum -y update

yum -y install httpd php mysql php-mysql chkconfig httpd on

service httpd start cd /var/www/html

APP_PATH=

wget https://${S3_HOST}/${APP_PATH} S3_HOST= s3-us-west-2.amazonaws.com APP_PATH= us-west-2-aws-training/awsu-spl/spl-13/scripts/app.tgz tar xvfz app.tgz

chown apache:root /var/www/html/rds.conf.php USER_DATA

Instance Resource

Now the fun starts with our EC2 instance:

cat <<-'APP_MODULE' >> ~/tf-projects/webapp/app/main.tf resource "aws_instance" "my-webserver" {

ami = "${data.aws_ami.amazon-linux-2.id}"

instance_type = "t2.micro"

key_name = "${var.key_name}"

user_data = "${file("${path.module}/user_data.sh")}"

subnet_id = "${var.sn_web}" associate_public_ip_address = true vpc_security_group_ids = [

"${var.sg_web}",

] tags {

"Name" = "my-webserver"

"Site" = "my-web-site"

}

} APP_MODULE

This code will reference the following external bits to build the EC2 instance:

Create Database Application

We can now create a MySQL using Amazon RDS (Relational Database Service). By using RDS, we do not have to manage our own database, but instead allow Amazon to manage it for us.

Input Variables

cat <<-'DB_VARIABLES' > ~/tf-projects/webapp/db/variables.tf variable "sg_db" {}

variable "sn_db1" {}

variable "sn_db2" {} variable "database_name" {}

variable "database_user" {}

variable "database_password" {} DB_VARIABLES

Database Subnet Group

cat <<-'DB_MODULE' > ~/tf-projects/webapp/db/main.tf resource "aws_db_subnet_group" "my-dbsg" {

name = "my-dbsg"

description = "my-dbsg"

subnet_ids = ["${var.sn_db1}", "${var.sn_db2}"] tags {

"Name" = "my-dbsg"

"Site" = "my-web-site"

}

} DB_MODULE

Database Instance

We’ll create a small MySQL 5.6.40 database that has no backup. This is a small throwaway database, so don’t use this code for a production database.

cat <<-'DB_MODULE' >> ~/tf-projects/webapp/db/main.tf resource "aws_db_instance" "my-db" {

identifier = "my-db"

allocated_storage = 20

storage_type = "gp2"

engine = "mysql"

engine_version = "5.6.40"

instance_class = "db.t2.micro" name = "${var.database_name}"

username = "${var.database_user}"

password = "${var.database_password}" parameter_group_name = "default.mysql5.6"

db_subnet_group_name = "${aws_db_subnet_group.my-dbsg.id}"

vpc_security_group_ids = ["${var.sg_db}"] # set these for dev db

backup_retention_period = 0 # required for deleting

skip_final_snapshot = true

final_snapshot_identifier = "Ignore" tags {

"Name" = "my-db"

"Site" = "my-web-site"

}

} DB_MODULE

Creating Main Terraform Script

We need to create a main Terraform script that calls both of our modules together, the infra and webapp modules. This script will take output from the infra module, and pass it to the webapp module.

cat <<-'MAIN' >> ~/tf-projects/main.tf #### VARIABLES

variable "profile" {}

variable "region" {}

variable "database_name" {}

variable "database_user" {}

variable "database_password" {}

variable "key_name" {

default = "deploy-aws"

} #### CALL MDOULES

module "core_infra" {

source = "./infra"

profile = "${var.profile}"

region = "${var.region}"

} module "webapp" {

source = "./webapp"

profile = "${var.profile}"

region = "${var.region}" key_name = "${var.key_name}" # pass web security group and public networks

sg_web = "${module.core_infra.sg_web}"

sn_web = "${module.core_infra.sn_pub1}" # pass database security group and private networks

sg_db = "${module.core_infra.sg_db}"

sn_db1 = "${module.core_infra.sn_priv1}"

sn_db2 = "${module.core_infra.sn_priv2}" # database parameters

database_name = "${var.database_name}"

database_user = "${var.database_user}"

database_password = "${var.database_password}"

} MAIN

Execute the Script to Create the Infrastructure and Web App

To run this altogether, we’d do something like this:

cd ~/tf-projects export AWS_PROFILE=learning

export TF_VAR_region=$(

awk -F'= ' '/region/{print $2}' <(

grep -A1 "\[.*$AWS_PROFILE\]" ~/.aws/config)

) # show changes required (using db variables file)

terraform plan -var-file="db.tfvars" # apply changes required (using db variables file)

terraform apply -var-file="db.tfvars"

Testing the Web Application

First we will need to fetch information. We can get computed values using terraform show command. We first need to get the public IP address so that we can log into web app database:

terraform show | grep -o 'public_ip = .*$'

After navigating to the public IP using a web browser, we should see an interface like this:

The web application has no configuration, so we’ll need to fill in the information manually. Let’s get the database endpoint:

terraform show | grep -o 'endpoint = .*$'

This will give you an endpoint similar to this format:

my-db.cknof0oc3nnn.us-east-2.rds.amazonaws.com:3306

Enter this information, plus the database name, username, and password saved in db.tfvars and hit the Submit button. After you should see see something like this:

Wrapping Up

There you have it: how to create VPC and network infrastructure with front end public subnets and backend private subnets. The webapp is not highly available, as it is installed on a single public subnet.

In order to remedy the low availability, we would create another web application on a different subnet, and then create a load balancer ELB to distribute the traffic between those systems. But that is for a future article…

Additionally, this script is dependent on the infrastructure, so it is not really a separated concern. In order to make the web app module truly independent, we’d need to use data sources to lookup the security groups and subnets we need. The downside to this, if the infrastructure was not created, this would then fail, possibly with a cryptic message. I’m considering a follow up article for these concept, as well as the one above…