CircleCI + AWS How to create CI/CD pipeline from scratch Part 2 - Setup and update servers
Tags: aws, devops
Published: 2020-07-25 09:35:48

After all previous steps, you've got the images that are stored inside the ECR and the script that automatically build
them. In this part of the tutorial we will: 
* setup ECS clusters for development and production environments;
* add deployment commands to the CircleCI script.

Let's do it!

## Initialize ECS cluster

Creating of cluster consists of three steps:
1. Create an empty cluster with a VPC
2. Define the task that will launch the selected container
3. Add service that will launch and maintain the desired count of ec2 instances with the previously defined task

### Create cluster
Let's start with the definition of [cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html): 

> An Amazon ECS cluster is a logical grouping of tasks or services. 

Roughly saying, clusters define the scope and set of rules for the launched tasks. To create one, follow the steps below:

1. Select EC2 Linux + Networking you need two clusters one for development and one for master branches
2. Select 1 `On Demand` t2.micro instance(or other types of ec2 instances), other configurations by default

    <figure class="image">
      <img src="https://dev-to-uploads.s3.amazonaws.com/i/k7mqxf1g01ymaoi0t2pf.png">
      <figcaption>Create cluster</figcaption>
    </figure>

3. For the networking section, I recommend to use the default parameters too. It will create a new VPC with
[security group](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html) allowing income traffic
to the 80 PORT.

<figure class="image">
  <img src="https://dev-to-uploads.s3.amazonaws.com/i/ynh1k3ba8tuyl3xkznkk.png">
  <figcaption>Configure network during creation</figcaption>
</figure>

Yes, that's all, you should create one for the development and one for the production. 

### Define task

[Tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) are used to run Docker containers in Amazon ECS. 

1. Select EC2 launch type compatibility
2. Choose the name for the task definition(or family, like sometimes it's called)
3. For the as the task role choose the role with the `AmazonECSTaskExecutionRolePolicy` policy that we  previously created 
4. Select the memory limit, for example, 128 MB if your server is not going to handle a lot of complex requests
5. And finally, add a container. On this tab we are interested in the following fields:
   * Standard -> Image - initial image for the task should be copied from the ECR looks like 845992245040.dkr.ecr.us-west-2.amazonaws.com/simple_plug_server:master_latest
   * Standard -> Port mappings - associate host 80 with the port which is using our application and will be defined next
       <figure class="image">
  <img src="https://dev-to-uploads.s3.amazonaws.com/i/lnqwg6i59irnpzis8ed2.png">
        <figcaption>Add container to a task and configure the port mapping</figcaption>
         </figure>

* Advanced container configuration -> ENVIRONMENT -> Environment variables - define the variable with the name `PORT` and desired value for example - 4100. This value must be used in the port mapping as the container port
<figure class="image">
  <img src="https://dev-to-uploads.s3.amazonaws.com/i/htqejj8nruz7azy328iv.png">
  <figcaption>Set up environment variables</figcaption>
</figure>
   
Great! You've created the first revision of the task definition. Of course, these tasks can be launched right inside the cluster, but we will use services to simplify updating tasks revisions. Let's add them. 

### Add service

> An Amazon ECS service enables you to run and maintain a specified number of instances of a task definition simultaneously in an Amazon ECS cluster. 

To create a [service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) just click on the `Create` button on the `Services` and fill the form:
1. Select EC2 in the `Launch type` selector, because we are deploying our tasks on EC2 instances
2. In the `Task Definition` select the task and revision that you are created earlier
3. In the `Cluster` select the cluster where you want to define a service. When you are creating service from a cluster this field will be already selected
4. `Service type` - REPLICA
5. `Number of tasks` - 1 because we do not care about scaling for now.
6. Other setting set by default

<figure class="image">
  <img src="https://dev-to-uploads.s3.amazonaws.com/i/yoeullialw88olce81sf.png">
  <figcaption>Service creation</figcaption>
</figure>

After repeating all steps for the development and production cluster, you've got two EC2 instances with running applications. They are available by Public DNS or IP.

<figure class="image">
  <img src="https://dev-to-uploads.s3.amazonaws.com/i/6pzerqn1gn7iq6dacnrt.png">
  <figcaption>Service creation</figcaption>
</figure>

Now let's go to the final part of this tutorial. 🚀
 
## Add deployment scripts

With an official orb [aws-ecs](https://circleci.com/orbs/registry/orb/circleci/aws-ecs) you can make this very easy. We already added all necessary environment variables in the second part of this tutorial so you should only modify the circleci config. 

Result version of the `.circleci/config.yml`
```yaml
version: 2.1
orbs:
  aws-ecr: circleci/aws-ecr@6.9.1
  aws-ecs: circleci/aws-ecs@1.2.0
jobs:
  test:
    docker:
      - image: elixir:1.10
        environment:
          MIX_ENV: test
    working_directory: ~/repo
    steps:
      - checkout
      - run: mix local.hex --force
      - run: mix local.rebar --force
      - run: mix deps.get
      - run: mix deps.compile
      - run: mix test
      - store_test_results:
          path: _build/test/lib/simple_plug_server
workflows:
  version: 2
  test-build-deploy:
    jobs:
      - test
      - aws-ecr/build-and-push-image:
          repo: "simple_plug_server"
          tag: "${CIRCLE_BRANCH}_${CIRCLE_SHA1},${CIRCLE_BRANCH}_latest"
          requires:
            - test
          filters:
            branches:
              only:
                - master
                - development
      - aws-ecs/deploy-service-update:
          name: deploy-development
          requires:
            - aws-ecr/build-and-push-image 
          family: "simple-plug-server-development"
          cluster-name: "SimplePlugServer-development"
          service-name: "sps-dev-serv"
          container-image-name-updates: "container=simple-plug-server-development,tag=${CIRCLE_BRANCH}_${CIRCLE_SHA1}"
          filters:
            branches:
              only:
                - development
      - approve-deploy:
          type: approval
          requires:
            - aws-ecr/build-and-push-image
          filters:
            branches:
              only:
                - master
      - aws-ecs/deploy-service-update:
          name: deploy-production
          requires:
            - approve-deploy
          family: "simple-plug-server-production"
          cluster-name: "SimplePlugServer-production"
          service-name: "simple-plug-server-production"
          container-image-name-updates: "container=simple-plug-server-production,tag=${CIRCLE_BRANCH}_${CIRCLE_SHA1}"
          filters:
            branches:
              only:
                - master
```

In this file was added three new jobs. Two `aws-ecs/deploy-service-update` respond for the updating respective services in the clusters and `approve-deploy` that's waiting for confirmation before the last step for the master branch. For different branches, flows will be a little different. It can be achieved by using parameter `filters` in job definitions, where you can specify for which branches or git tags launch jobs.

### aws-ecs/deploy-service-update job configuration

I would like to tell you about the parameters for the job `aws-ecs/deploy-service-update`:
* **name** - the name is used to make jobs in a workflow more human-readable. I am sure you would agree that's `deploy-production`
looks much more clearer than `aws-ecs/deploy-service-update`. 
* **requires** - used to define the order of jobs execution, namely the previous job that must be finished successfully.
* **family** - there you should write the name of the task definition([Define task](define-task)) that you used when you 
created the task
* **cluster-name** - it's pretty obvious - the name of the desired cluster where all magic happens
* **service-name** - the name of the service that's managing tasks inside the previously mentioned cluster
* **container-image-name-updates** - updates the Docker image names and/or tag names of existing containers 
                                 that had been defined in the previous task definition
  * **container** - the name of the container that you used when you added the container to the task(circled in blue on the screenshot `Add container to a task and configure the port mapping`)
  * **tag** - one of the tags that you are defined in the `aws-ecr/build-and-push-image` job, in this example it's a `${CIRCLE_BRANCH}_${CIRCLE_SHA1}`

And that's all 🎉. When you push your branch with the new circleci config and start to work you will see something like that.

<figure class="image">
  <img src="https://dev-to-uploads.s3.amazonaws.com/i/f7efsjc1ws5slo40k0mx.png">
  <figcaption>Service creation</figcaption>
</figure>

I hope that this tutorial was helpful and was not wasted your time. If you have any question and problems feel free to ask me about it in the comments. 👋