Creating your cloud servers with Terraform
Tags: terraform, devops, infrastructureasco, aws
Published: 2018-10-23 18:27:49

---
title: Creating your cloud servers with Terraform
published: true
tags: terraform,devops,infrastructure-as-co,aws
canonical_url: https://thecode.pub/creating-your-cloud-servers-with-terraform-bfa01a499bad
---

![](https://cdn-images-1.medium.com/max/800/1*2QvMO07P4smlK-GE45EROA.jpeg)<figcaption>Terraforming Mars</figcaption>

Sometimes when you handle a lot of servers in the cloud, it is pretty easy to get lost on your infrastructure. “Where is that freaking server that I can’t find?”, or even “Why is this instance for?”.

In this post, I will introduce one tool that I found myself liking a lot: To have a bootstrap of an Infrastructure as Code flow in my applications. [Terraform](https://www.terraform.io/). Infrastructure as Code (IaC) allows us to have a repository with code that describes our infrastructure. This way, we can avoid reminding how rebuild the entire infrastructure for an application. It will be on the code and it can be versioned and tested. And if something goes wrong, we can revert it. It is very useful having a continuous integration of the infrastructure. The whole team knows how it was built and all the pieces. You can apply the same flow that you use in you app code to your infrastructure, i.e: Someone makes a change, open a Pull Request, someone reviews it and after it is approved, you merge and your CI tools apply the changes in your environment.

#### Why Terraform? What is the difference between Chef, Puppet or Ansible?

Chef, Puppet and Ansible are IaC tools too, but they focus on **configuring** operating system and applications. They are called Configuration Management Tools and they also can build infrastructure on the cloud with a help of plugins, but usually it is hard to configure and sometimes it is limited. With Terraform you can build from the services to the networking part. You can use Terraform to create the infrastructure and a configuration management tool to configure the applications. Terraform can’t replace your configuration management tool, but it’s made to work together with it.

![](https://cdn-images-1.medium.com/max/400/1*sS6MVPyxzhn3O8pA3nw0kg.jpeg)

### A practical example

In order to follow this article, you’ll need an AWS account.

Let’s create the servers for our web application. Take a look at this diagram of the resources that we’ll build.

![](https://cdn-images-1.medium.com/max/1024/1*nYWHvlp87BBsE6gI2TbKYA.png)

We will create 2 subnets (one for public access and another private). We have Elastic Load Balancer in the public subnet to handle the traffic to our web servers. Our web servers will be on the private subnet and it will only be accessible through the Load Balancer. This mean that we won’t have direct access to make connections (for example, SSH) on the server. In order to access via SSH an instance on a private subnet, you’ll need a bastion host and connect to the web server through it. Thus, we will create the bastion host on the public subnet.

Before getting your hands dirty, you need to install Terraform. Follow the instructions of [this link](https://www.terraform.io/intro/getting-started/install.html) to install it in your machine.

#### Directory structure

Let’s create a folder to handle our infrastructure code.

```
mkdir ~/terraform
```

I like to follow this pattern when working with Terraform:

```
├── modules
│ ├── networking
│ └── web
├── production
└── staging
```

Let’s create this folder structure

```
cd ~/terraform
mkdir -p modules/{networking,web} production staging
```

Our modules folder contains all the **shared code** to create the pieces of the infrastructure (web servers, app servers, databases, vpc, etc). Each folder inside the modules folder is related to a specific module.   
Next, I have folders for my environments (staging, production, qa, development, etc). Each of this folder contains code to use our shared modules and create a different architecture for each environment (This is my personal approach using Terraform, but feel free to work on a different way).

#### Our first module: Networking

Let’s create our networking module. This will be responsible for creating the networking pieces of our infrastructure, like VPC, subnets, routing table, NAT server and the bastion instance.

Before we get deep in the code, I wanna explain how terraform works:

Terraform will provide us with some commands. Some of them are:

**plan** : Displays all the changes that Terraform makes on our infrastructure  
**apply** : Executes all the changes to the infrastructure  
**destroy** : Destroys everything that was created with Terraform

When you run Terraform inside a directory, it loads ALL .tf files from the directory and execute them (will not load on subfolders). Terraform will first create a graph of the resources to apply only in the final phase, so you don’t need to specify the resources in any specific order. The graph will determine the relations between the resources and ensure that Terraform creates they in the right order.

#### Continuing on our networking module

Enter in the networking module

```
cd modules/networking
```

Let’s create our first tf file. The one that specifies all variables needed for our module.

```
touch variables.tf
```

Insert the following content on the variables.tf file:

{% gist https://gist.github.com/duduribeiro/98f1dc6efc2ed985129e3609ef897f7b %}

These are all variables that our networking module needs in order to create all resources. We need the CIDR for the VPC and the subnets, the AWS region that we will use, the key name and the environment that we are building.  
This is the way that you specify a variable in Terraform

```
variable "variable\_name" {
  description = "The description of the variable"
  default = "A default value if this isn't set
}
```

Ok. Now we have our variables.tf file to specify the interface of our module. Let’s create the file that will create networking stuffs for our module. Create the main.tf file in the networking folder. (you can specify any name that you want. Remember, all tf files will be loaded).

```
touch main.tf
```

Insert the following content on the main.tf file:

{% gist https://gist.github.com/duduribeiro/1dd9abdaec4cacd5595d1cdb8a03c3bf %}

Here we are creating all the networking part of our infrastructure based on the diagram that we saw. A VPC, both subnets (public and private), the Internet Gateway to the public subnet, the NAT server for the private subnet, the bastion host and all security group for the VPC, allowing inbound and outbound inside the VPC, and the security group for the bastion host, allowing the SSH on the Port 22. You can check [this link from AWS](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html) for more details.

This is how we create a resource on Terraform.

```
resource "resource" "name" {
  attribute = "value"
}
```

The resource is the name of the resource that we want to build. Each cloud provider has different resources. For example, [these resources](https://www.terraform.io/docs/providers/aws/index.html) from AWS. We can also concatenate values. Remember that we created the variables file? We use them here with the var.variable\_name . Like in this part of the code, which we use the key\_name variable that we specified in the variables file:

```
resource "aws\_instance" "bastion" {
  ...
  key\_name = "${var.key\_name}"
  ...
}
```

This is how we created the instance and called its bastion. You can also get property of this resource in other parts of the code. Example:

```
resource "someresource" "somename" {
  attribute = "${aws\_instance.bastion.id}"
}
```

We use the same idea of var concatenation. But we specify ${resource\_type.resource\_name.property} .

Our networking module is almost ready. We need to output some variables after the module build the resources, so we can use it in other parts of the code. Terraform has the [output command,](https://www.terraform.io/intro/getting-started/outputs.html) allowing us to expose variables.

Create the output.tf file inside the networking folder.

```
touch output.tf
```

Insert the following content on the output.tf file:

{% gist https://gist.github.com/duduribeiro/fdb975b2e331ba890ac0d4910743a2ef %}

This is how we output a variable from our module:

```
output "variable\_name" {
  value = "variable value"
}
```

This will allow us to get these variables outside the module.

#### Using our module to build the networking from our environment

Now that our networking module is ready, we can use it to build our networking from our environment (i.e, staging).  
This is how our terraform folder looks now:

```
├── modules
│ ├── networking
│ │ ├── main.tf
│ │ ├── output.tf
│ │ └── variables.tf
│ └── web
├── production
└── staging
```

Go into staging folder.

```
cd ~/terraform/staging
```

First, create the public key for the staging servers

```
ssh-keygen -t rsa -C "staging\_key" -f ./staging\_key
```

Let’s create our main file. In this file, we will specify information of the AWS provider.

```
touch \_main.tf
```

(We use \_ at the beginning of the name because since terraform loads all files alphabetically, we need this to be loaded first, since it will create the keypair)

You can specify things like Access and secret key in some ways:

- Specify it directly in the provider (not recommended)

```
provider "aws" {
  region = "us-west-1"
  access\_key = "myaccesskey"
  secret\_key = "mysecretkey"
}
```

- Using the AWS\_ACCESS\_KEY and AWS\_SECRET\_KEY environment variables

```
$ export AWS\_ACCESS\_KEY\_ID="myaccesskey"
$ export AWS\_SECRET\_ACCESS\_KEY="mysecretkey"

terraform plan
```

The second option is recommended because you don’t need to expose your secrets on the file. **Bonus point** : If you have the [AWS cli](https://aws.amazon.com/pt/cli/) you don’t need to export these variables. Only run the aws configure command and terraform will use the variables that you set on it.

Insert the following content on the \_main.tf file:

{% gist https://gist.github.com/duduribeiro/81f19802c03f7b7d90eaaa6369cdf400 %}

We will only specify the region to the provider. Both access and secret key, we will rely on the AWS cli .

Now create the networking.tf file. It will use our module to create the resources.

```
touch networking.tf
```

Insert the following content on the networking.tf file:

{% gist https://gist.github.com/duduribeiro/ef0bbdecf0d2b2441a85f2cc548d68d1 %}

This is how we use [modules on Terraform](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjYr97GxIHTAhVCkJAKHWi8AsQQFggaMAA&url=https%3A%2F%2Fwww.terraform.io%2Fdocs%2Fmodules%2Fusage.html&usg=AFQjCNHaKs5YpQe2IsNwXa9ikVF3N67CRw&bvm=bv.151426398,d.Y2I).

```
module "name" {
  source = "location\_path"
  attribute = "value"
}
```

The module attributes are all variables that we specified before in the variables.tf file from our networking module. Look that we are passing more variables to our module attributes. We need our environment to require these variables too. Create the variables.tf file for our staging environment.

```
touch variables.tf
```

Add the following content to the variables.tf file:

{% gist https://gist.github.com/duduribeiro/bb50269de93c467e01a60110c4e25675 %}

This file follows the same pattern of the module’s variables file. These are all variables that we need to build our staging networking piece.

Ok… I think that we are ready to go. Your terraform’s folder structure should be like this:

```
├── modules
│ ├── networking
│ │ ├── main.tf
│ │ ├── output.tf
│ │ └── variables.tf
│ └── web
├── production
└── staging
    ├── \_main.tf
    ├── networking.tf
    └── variables.tf
```

Run this command on staging folder: (Terraform’s commands should be run on the environments folder).

```
cd ~/terraform/staging
terraform get
terraform plan
```

\*the terraform get command only syncs all modules.

After executing the terraform plan command, it will ask you a lot of informations (our variables). Answer they:

![](https://cdn-images-1.medium.com/max/646/1*CKnvWOjniPmXHpbI8u6oNA.png)

It will output a lot of things. Resources that will be created. You can analyze it to check if everything is ok. terraform plan will output only the planned change of the infrastructure.

Now, let’s apply these modifications.

```
terraform apply
```

But it is asking for all that information again. To avoid this, you can follow 2 ways to automatically inject variable’s value to Terraform

- Using a terraform.tfvars file and remember to ignore this file in your VCS.
- Specifying variables in Environment variables. TF\_VAR\_environment=staging terraform apply (this can be useful when you run through some CI tool)

We will follow the first way. So, create a terraform.tfvars file

```
touch terraform.tfvars
```

Insert the following content on terraform.tfvars :

{% gist https://gist.github.com/duduribeiro/c5c7e58e6b028b898c29effaf892851a %}

Now you can run the apply command without being asked for variable’s value.

```
terraform apply
```

You should receive the following message:

![](https://cdn-images-1.medium.com/max/868/1*Lb4nvDoCX_hOyDckORfLOg.png)

Congratulations..

![](https://cdn-images-1.medium.com/max/500/1*6447MjaXTKhttJ5WlqTVpQ.gif)

This will generate 2 files on the staging folder: terraform.tfstate and terraform.tfstate.backup

Terraform controls all their resources on this terraform.tfstate file. You should **NEVER** delete this file. If you do, terraform will think that it needs to create new resources and will lose tracking with the others that it has been already created.

And how can I make my team in sync with the state?

You have 2 ways to keep the team with the remote in sync. You can commit this .tfstate file to your VCS repository, or use [Terraform Remote State](https://www.terraform.io/docs/state/remote.html). If you rely on terraform’s remote state, I really recommend you to use some wrapper tool for Terraform like [Terragrunt](https://github.com/gruntwork-io/terragrunt). It can handle the remote state locking and initializing it for you.

#### Creating the web servers

This is our folder structure until now.

```
├── modules
│ ├── networking
│ │ ├── main.tf
│ │ ├── output.tf
│ │ └── variables.tf
│ └── web
├── production
└── staging
    ├── \_main.tf
    ├── networking.tf
    ├── staging\_key
    ├── staging\_key.pub
    ├── terraform.tfstate
    ├── terraform.tfstate.backup
    ├── terraform.tfvars
    └── variables.tf
```

Go into our web module

```
cd ~/terrform/modules/web
```

Following the same flow we used to create the networking module, let’s create our variables file to specify the interface to our module:

```
touch variables.tf
```

Insert the following content on variables.tf :

{% gist https://gist.github.com/duduribeiro/420ebd4344578a9ebb01adf5b21b137b %}

Create the main.tf file to handle the creation of the resources.

```
touch main.tf
```

Insert the following content on main.tf :

{% gist https://gist.github.com/duduribeiro/d28a122183158c6352063b610be874cc %}

This file is practically the same from our networking’s main.tf, we are only creating different resources.   
Some differences that you can note:

- In this example we are using the count attribute. It specifies to Terraform, to create N times this resource. If we pass 5 on the value, it will create 5 instances.

```
resource "aws\_instance" "web" {
  count = "${var.web\_instance\_count}"
  ...
}
```

and you can create dynamic names with the counting number using the count.index property, for example:

```
tags {
  Name = "web-server-${count.index + 1}"
}
```

- We are passing a file to the user\_data property. In order to execute some code on the instance initialization, we need to pass the user\_data attribute to our instance and we are specifying a file.

```
user\_data = "${file("${path.module}/files/user\_data.sh")}"
```

This will load the content of the user\_data.sh file and pass to the attribute.

But we haven’t created this file yet, let’s do it. On the web module folder, create the files folder and the user\_data.sh file.

```
mkdir files
touch files/user\_data.sh
```

Insert the following content on the files/user\_data.sh :

{% gist https://gist.github.com/duduribeiro/37dd942c2a610332bce98d83485168ec %}

This will install the nginx when the instance is created.

Now, let’s create the output.tf file to get the load balancer’s DNS after the execution.

```
touch output.tf
```

Insert the following content on the output.tf :

{% gist https://gist.github.com/duduribeiro/01d48af0b5f89ce79d8fef7ab432bf42 %}

Our web module is done.

This is how our terraform structure is now:

```
├── modules
│ ├── networking
│ │ ├── main.tf
│ │ ├── output.tf
│ │ └── variables.tf
│ └── web
│ ├── files
│ │ └── user\_data.sh
│ ├── main.tf
│ ├── output.tf
│ └── variables.tf
├── production
└── staging
    ├── \_main.tf
    ├── networking.tf
    ├── staging\_key
    ├── staging\_key.pub
    ├── terraform.tfstate
    ├── terraform.tfstate.backup
    ├── terraform.tfvars
    └── variables.tf
```

#### Using our web module

Let’s back to our staging folder

```
cd ~/terraform
cd staging
```

Now, we will use our recent created web module. Create a web.tf file

```
touch web.tf
```

Insert the following content on the web.tf :

{% gist https://gist.github.com/duduribeiro/23ca8144a181c13bdb2060bcf9cdeb32 %}

This is pretty much the same way we used in our networking module. We are using almost the same variables that we already specified (except for web\_instance\_count . And some variables, we pass the output from our networking module.

```
"${module.networking.public\_subnet\_id}"
```

This way we get the public\_subnet\_id output created on the networking module.

Let’s add the web\_instance\_count variable to our variables.tf file and terraform.tfvars . This variable represents the number of web instances that we will be created.

Your variables.tf from staging folder should be like this:

{% gist https://gist.github.com/duduribeiro/18dc248b531b37b68e609ce1ea58992c %}

And your terraform.tfvars should be like this:

{% gist https://gist.github.com/duduribeiro/62108ed506daaebce4a178d27162fcc8 %}

Let’s create an output.tf for our staging environment. With this, we can get the ELB hostname from our web module.

```
touch output.tf
```

Add the following content to output.tf :

{% gist https://gist.github.com/duduribeiro/0aafb5203dd66cd2b2d50d4335b08ebf %}

This is the final directory structure that we have:

```
├── modules
│ ├── networking
│ │ ├── main.tf
│ │ ├── output.tf
│ │ └── variables.tf
│ └── web
│ ├── files
│ │ └── user\_data.sh
│ ├── main.tf
│ ├── output.tf
│ └── variables.tf
├── production
└── staging
    ├── \_main.tf
    ├── networking.tf
    ├── output.tf
    ├── staging\_key
    ├── staging\_key.pub
    ├── terraform.tfstate
    ├── terraform.tfstate.backup
    ├── terraform.tfvars
    ├── variables.tf
    └── web.tf
```

You can now run terraform plan to check which resources will be created. (before, run terraform get to update the modules)

```
terraform get
terraform plan
```

![](https://cdn-images-1.medium.com/max/724/1*_3j03t04L3BbcAM9B3t5Tg.png)

Done. Terraform will create 5 new resources from our web module.

Let’s apply it.

```
terraform apply
```

![](https://cdn-images-1.medium.com/max/840/1*zojlSrGMXN93qiHdoZVPhQ.png)

Yay.. our instances was created. Let’s get our Load Balancer DNS and try to open it on a Browser:

```
[terraform](http://%24(terraform) output elb\_hostname
```

This command will return the DNS from our Load Balancer. Open it on a browser:

![](https://cdn-images-1.medium.com/max/756/1*KAFGuGGUKWTWISGD2dqmBQ.png)

![](https://cdn-images-1.medium.com/max/1024/1*X4llUCSctCV1RBczcOPusw.png)

It’s working!!! With this, you finished the creation of your infrastructure.

#### Final Notes:

Your web instances don’t have public ips:

![](https://cdn-images-1.medium.com/max/1024/1*SipX4XhQdT5_Z0zYySLmNg.png)

In order to SSH then, you need to use the bastion host created on the networking module.

Get the bastion host public IP,

![](https://cdn-images-1.medium.com/max/1024/1*HctNFDW39zAagy7iRyiirQ.png)

and SSH on it with the -A flag to enable agent forwarding:

```
chmod 400 staging\_key.pub

ssh-add -K staging\_key

ssh -A ubuntu@52.53.227.241
```

Now, inside the Bastion host, you can connect into your web server private IP:

```
ssh ubuntu@10.0.2.113
```

![](https://cdn-images-1.medium.com/max/1024/1*jxxGXtybRz43Rw47BacdxA.png)

Now, destroy everything

```
terraform destroy
```

You can get the full code of the example [here](https://github.com/duduribeiro/terraform_example).

That’s all.

![](https://cdn-images-1.medium.com/max/499/1*PlGYD6UUSY3wbNzGRAkHSw.gif)

Cheers,

🍻

* * *