Deploying code from GitHub to AWS EC2 with CodePipeline
Tags: aws, devops, git, architecture
Published: 2020-04-30 11:32:41

I’m not that old, but when I built my first website I manually copied code to an Apache server with FTP. That server was also running in my closet. As you can imagine, this deployment process was not exactly fullproof and I experienced frequent outages on my website. Deployments have come a long way in the 10 years since my first website and the general goal today is to automate code deployments as much as possible. Let’s explore how we can deploy code from GitHub to EC2 using CodePipeline.

### Why automate?

Put simply, the more repeatable your deployments are, the less downtime you will have. Automated deployments are easier to test, easier to troubleshoot, lend themselves to scalability, can perform rollbacks on failure, support blue/green and canary deployments. The list goes on.

Ideally, your resources should be treated like cattle, not like pets. If it gets sick, you shoot it and replace it with another one. Automated deployments make that easy.

### First, a Git repo

Before we do anything else, we will need a Git repository to house the code. I’ll clone down a empty repo and create a demo file.

```python
git clone git@github.com:seanziegler/AWSDeployDemo.git
touch demo.txt
echo "This is prod" > demo.txt
git add . 
git commit -m "Init commit"
git push origin master
```
I’m going to create a seperate branch so I can push seperately to test and master.

```python
git branch test
git checkout test
echo "This is test" > demo.txt 
git add .
git commit -m "Create test branch"
git push origin test
```

Obviously, you’d normally be deploying an application, not just a text file.

###Creating deployment targets

Now that we have repositories set up, some instances are needed. I’m going to use the management interface since this is just a demo but you can also use the awscli or Terraform/Cloudformation.

It’s good practice to create a new subnet inside your VPC for each application you deploy.

![Create Subnet](https://i0.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-25-at-1.02.04-PM.png?resize=2048%2C1079&ssl=1)

Okay, next up are the EC2 instances themselves. I’m going to make two: one for test and one for production. Obviously this is just a dummy deployment but CodePipeline is versatile enough to allow deployments across hundreds of resources if needed.

![Create Instances](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-25-at-1.12.21-PM-1.png?w=2040&ssl=1)

I’ll also add some tags to identify the production vs test instance. I’ll be using these later to tell CodePipeline which one is which.

![Add Tags](https://i2.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-25-at-1.12.54-PM-1.png?w=1587&ssl=1)

Okay, we have what we need for instances.

![New instances](https://i0.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-25-at-1.23.27-PM.png?w=1590&ssl=1)

### Installing the CodeDeploy agent

Before we can build a pipeline, we need to configure CodeDeploy and it’s agent. CodeDeploy requires a small agent daemon installed on every machine that is a deployment target.

You can check if you have the CodeDepoy agent installed by connecting to your instance and running (on Amazon Linux or RHEL):

```bash
sudo service codedeploy-agent status
```

You’ll either get an error or a message that the service is stopped. If the service is stopped, simply run:

```bash
sudo service codedeploy-agent start
```

If you get an error, the codedeploy-agent isn’t installed. So it becomes a bit more involved. Here’s the docs page on how to install the CodeDeploy agent. Essentially it boils down to the following commands:

```bash
sudo yum update -y
sudo yum install ruby wget -y
cd /home/ec2-user
wget https://bucket-name.s3.region-identifier.amazonaws.com/latest/install
chmod +x ./install
sudo ./install auto
```

In the wget command above, you’ll need to replace both bucket-name and region-identifier with the values that corresponds to your region.

<table><tbody><tr><td><strong>Region Name</strong></td><td><strong>bucket-name</strong></td><td><strong>region-identifier</strong></td></tr><tr><td>US East (Ohio)</td><td>aws-codedeploy-us-east-2</td><td>us-east-2</td></tr><tr><td>US East (N. Virginia)</td><td>aws-codedeploy-us-east-1</td><td>us-east-1</td></tr><tr><td>US West (N. California)</td><td>aws-codedeploy-us-west-1</td><td>us-west-1</td></tr><tr><td>US West (Oregon)</td><td>aws-codedeploy-us-west-2</td><td>us-west-2</td></tr><tr><td>Canada (Central)</td><td>aws-codedeploy-ca-central-1</td><td>ca-central-1</td></tr><tr><td>Europe (Ireland)</td><td>aws-codedeploy-eu-west-1</td><td>eu-west-1</td></tr><tr><td>Europe (London)</td><td>aws-codedeploy-eu-west-2</td><td>eu-west-2</td></tr><tr><td>Europe (Paris)</td><td>aws-codedeploy-eu-west-3</td><td>eu-west-3</td></tr></tbody></table>

Once installed, check one last time to make sure the agent is running.

```bash
sudo service codedeploy-agent status
```

### CodeDeploy Role

In order for CodeDeploy to work, we need to assign it a service role with the correct permissions. Let’s create a role with the AWSCodeDeployRole policy attached to it.

![Create Role](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-7.10.22-AM.png?w=1988&ssl=1)

![Attach Role](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-7.17.47-AM.png?resize=2048%2C527&ssl=1)

Now that the instance roles are setup, it’s time to turn to CodeDeploy.

### Creating an Application and Deployment Group

Creating an application in CodeDeploy is straightforward. Just give it a name and tell it that you plan on deploying code to EC2 instances.

![Create Application](https://i2.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-7.29.33-AM.png?w=1714&ssl=1)

Creating a deployment group is a bit more complicated but not difficult. Give it a name and assign the CodeDeploy service role you created previously. This will allow CodeDeploy to access the resources specified in this deployment group.

![Create Deployment](https://i0.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-7.42.46-AM.png?w=1654&ssl=1)

Next, choose EC2 instances for deployment. You can also deploy to autoscaling groups but for now I’ll deploy to a single instance. Don’t forget to add the env:test tag. You should see one matched instance that corresponds to the test instance created earlier.

![Environment Config](https://i0.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-7.44.44-AM.png?w=1636&ssl=1)

We don’t need a special deployment setting so just leave the default AllAtOnce setting in place. Uncheck Enable load balancing since we haven’t set that up.

![Deployment Settings](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-7.44.56-AM.png?w=1650&ssl=1)

Repeat this process again but target the instances with the env:prod tag.

### appspec.yml

CodePipeline needs to know what to do with the files in your Git repository when you deploy. A file called appspec.yml is CodePipeline’s way of defining the tasks you want to run when deploying code. There’s too much to cover here, but AWS has examples of how to build out an appspec.yml file.

For this project, I’m just going to copy in some text files from the Git repo to test that the pipeline is working. You’ll need to do your own research on how to build an appspec file to suit your deployment. If you want to follow along with my dummy deployment, put this in a file at the root of your git repo. Don’t forget to name it appspec.yml!

```yaml
version: 0.0
os: linux
files:
  - source: ./demo.txt
    destination: /
```

### Building a pipeline

We need to build two pipelines for this to work. One pipeline that deploys the test branch to test instances and one pipeline that deploys the master branch to production instances.

Creating a pipeline is a straightforward process. I suggest letting CodePipeline create your servie role for you. It’s just easier that way.

![Pipeline Settings](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-8.10.06-AM.png?w=1582&ssl=1)

Create a stage that uses your Github repository and the test branch as the source.

![Source Stage](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-8.11.39-AM.png?w=1618&ssl=1)

Skip the build stage for now. If you want, you can come back later and setup CodeBuild to do integration tests or build binaries if you need that sort of thing.

Select the CodeDeploy provider and choose the application and deployment group for your test instances.

![Deploy Stage](https://i0.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-8.12.32-AM.png?w=1584&ssl=1)

Click next and deploy the pipeline. You’ll see it try to download your source and deploy it to your test instances.

![Deploy Stage](https://i1.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-12.18.18-PM.png?resize=768%2C925&ssl=1)

Great, we deployed to our test instances. But what about production? Click “Edit” at the top of the pipeline page. And then click “Edit” on each stage of the pipeline. From here, you can add another source and deploy action for the master branch and production deployment group.

![Deploy Edit](https://i2.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-11.56.27-AM.png?resize=2048%2C1149&ssl=1)

When you click save, your pipeline will deploy the test branch to the test deployment group and the master branch to the production deployment group.

![Deploy Success](https://i2.wp.com/seanjziegler.com/wp-content/uploads/2020/04/Screen-Shot-2020-04-26-at-12.08.32-PM.png?resize=2048%2C1198&ssl=1)

Now, let’s just check and see if the deployments worked. First, let’s check the test instance.

```bash
ssh ec2-user@test-instance

Last login: Sun Apr 26 16:00:02 2020 from <IP>

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/

[ec2-user@test-instance ~]$ cat /demo.txt
This is test
```

Next, check the production instance.

```bash
ssh ec2-user@prod-instance

Last login: Sun Apr 26 16:00:02 2020 from <IP>

       __|  __|_  )
       _|  (     /   Amazon Linux 2 AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-2/

[ec2-user@prod-instance ~]$ cat /demo.txt
This is prod
```

Awesome! Just what was expected. Your pipeline is setup and ready to go.

###Customising the deployment.

There are several ways you can customize your pipeline.

#### appspec.yml

For one thing, you’ll want to start with building out an appspec.yml file that reflects the structure of your application. Copy over all the files you need, run scripts to setup dependencies, etc.

#### AMI

Make a custom AMI for the instances in each of your group. Include any dependencies your code needs to run (and the codedeploy-agent) in the AMI rather than installing it every deployment. This will increase the relability and speed of your deployments.

#### Autoscaling groups

Instead of creating deployment groups with specific EC2 instances identified, consider deploying to autoscaling groups instead so you can apply scale-in and scale-out rules.