Deploying Docker containers in AWS Fargate | by Eduardo Vioque | Towards Data Science

Docker is a fantastic tool to encapsulate and deploy applications in an easy and scalable way. Indeed, something I find myself doing very often is wrapping Python libraries into Docker images that I can later use as boilerplates for my projects.
In this post, I will illustrate how to register your Docker images in a container registry and how to deploy the containers in AWS using Fargate, a serverless compute engine designed to run containerized applications. Additionally, we will use Cloud Formation to deploy our stack in a programmatic way. Let’s get started!
Container registries are to Docker images what code repositories are to code. In a registry, you create image repositories to push and register your local images, you can store different versions of the same image, and other users can pull and update the image if they have access to the repo.
Now, let’s say you have developed locally an image that you want to deploy to the cloud. To do so, we would need to store our local image in a container registry from which it can be pulled and deployed. In this article, I will be using a fairly simple image that starts a web Python-Dash application on port 80.
We will use the ECR (Elastic Container Registry) to register our images. ECR is an AWS service, quite similar to DockerHub, to store Docker images. The first thing we have to do is creating a repository in ECR, we can use the AWS CLI as follows:
aws ecr create-repository \--repository-name dash-app \--image-scanning-configuration scanOnPush=true \     --region eu-central-1
You should be able to see the repository in the AWS management console
To push local images to our ECR repository we are required to authenticate our local Docker CLI into AWS:
aws ecr get-login-password --region region | docker login --username AWS --password-stdin acccount_id.dkr.ecr.region.amazonaws.com
Just replace the aws_account_id and region appropriately. You should see the message Login Succeeded in the terminal, which means our local Docker CLI is authenticated to interact with the ECR.
Let’s push now our local image to our brand new repository. To do so we must tag our image to point to the ECR repository:
docker tag image_id account_id.dkr.ecr.eu-central-1.amazonaws.com/dash-app:latest
Now we just have to push it to the ECR:
docker push account_id.dkr.ecr.eu-central-1.amazonaws.com/dash-app:latest
You should see the pushed image in the AWS Console:
With that we come to the end of the section, let’s summarize: (i) we have created an image repository called dash-app in ECR, (ii) we have authorized our local Docker CLI to connect to AWS, and (iii) we have pushed an image to the repository. In the next section, we will cover how to deploy this image in AWS.
Leaving Kubernetes aside, AWS provides several options to deploy containerized applications:
Deploying containers on EC2, usually within an auto-scaling group of instances. In this scenario we are responsible for patching, securing, monitoring, and scaling the EC2 instances.Deploying containers on AWS Fargate. Since Fargate is serverless, there are no EC2 instances to manage or provision. Fargate manages the execution of our tasks providing the right computing power (a task in this context refers to a group of containers that work together as an application).
Deploying containers on EC2, usually within an auto-scaling group of instances. In this scenario we are responsible for patching, securing, monitoring, and scaling the EC2 instances.
Deploying containers on AWS Fargate. Since Fargate is serverless, there are no EC2 instances to manage or provision. Fargate manages the execution of our tasks providing the right computing power (a task in this context refers to a group of containers that work together as an application).
In this section, we will focus on the second option, illustrating how to roll out our web application on AWS Fargate. In addition, we will allocate all the necessary resources with AWS Cloud Formation.
Cloud Formation is an AWS service to provision and deploy resources in a programmatic way, a technique usually referred to as infrastructure as code or IaC. In IaC, instead of allocating resources manually through the management console, we define our stack in a JSON or YAML file. The file is then submitted to Cloud Formation which automatically deploys all the resources specified in it. This has two main advantages: (i) it makes it easy to automate resources provisioning and deployments, and (ii) the files help as documentation of our cloud infrastructure.
Although defining our stack in a JSON/YAML file requires going through a learning curve and forgetting about AWS management console and its truly easy to use wizards, it definitely pays off in the long run. As your infrastructure grows, keeping all the stack as code will be incredibly helpful to scale productively.
Now, let’s list the resources we need to run our application:
Task: It describes the group of Docker containers that shape our application. In our example, we simply have one image.ECS Service: It takes care of running and maintaining the desired number of instances of a task. For example, if a task fails or stops, the ECS Service can automatically launch a new instance to keep the desired number of tasks in service.Fargate: A serverless compute engine for containers. With Fargate there is no need to book and manage servers. It automatically supplies compute power to run our tasks.Network resources: we need as well a VPC, a public subnet connected to an internet gateway through a routing table (don’t forget we are deploying a web application that should be reachable from the internet), and a security group to run our containers securely. You don’t need to set up those resources if you already have a network configuration in place, although I include them in the Cloud Formation script.
Task: It describes the group of Docker containers that shape our application. In our example, we simply have one image.
ECS Service: It takes care of running and maintaining the desired number of instances of a task. For example, if a task fails or stops, the ECS Service can automatically launch a new instance to keep the desired number of tasks in service.
Fargate: A serverless compute engine for containers. With Fargate there is no need to book and manage servers. It automatically supplies compute power to run our tasks.
Network resources: we need as well a VPC, a public subnet connected to an internet gateway through a routing table (don’t forget we are deploying a web application that should be reachable from the internet), and a security group to run our containers securely. You don’t need to set up those resources if you already have a network configuration in place, although I include them in the Cloud Formation script.
Now, without further ado, let’s jump into the stack. The Gist below contains all the resources required. Let’s explain them in details:
VPC-FARGATE: This is the VPC. It is important enabling DNS support and DNS Hostnames to reach the ECR and pull the images.INT-GATEWAY: This is an internet gateway, it is needed to expose the subnet to the internet.ATTACH-IG: This attaches the internet gateway to the VPC.ROUTE-TABLE : This is a routing table, to which we will add the rules to expose the subnet to the internet gateway.ROUTE : This adds a rule to the routing table previously described. It forwards traffic to the internet gateway.SUBNETFARGATE : This is the subnet to host our services. We define the availability zone and the VPC to which it belongs.SUBNETROUTE : This associates the routing table to the subnet.SGFARGATE : This is the security group to be applied to our services. It allows traffic on ports 443 and 80 over HTTPS and HTTP protocols.FARGATECLUSTER : It defines the Fargate cluster to host our services.ECSTASK : Determines the tasks to be executed. It includes the list of Docker images that make the task. For each container, it registers ports mapping, starting commands, and logging options. Images are pulled from the ECR repository set up before.SERVICE : Defines the ECS Service that will launch and maintain our tasks. If you already have a network configuration in place and don’t need to create a new subnet and security group, just reference those existing resources in the NetworkConfiguration section of the ECS Service.
VPC-FARGATE: This is the VPC. It is important enabling DNS support and DNS Hostnames to reach the ECR and pull the images.
INT-GATEWAY: This is an internet gateway, it is needed to expose the subnet to the internet.
ATTACH-IG: This attaches the internet gateway to the VPC.
ROUTE-TABLE : This is a routing table, to which we will add the rules to expose the subnet to the internet gateway.
ROUTE : This adds a rule to the routing table previously described. It forwards traffic to the internet gateway.
SUBNETFARGATE : This is the subnet to host our services. We define the availability zone and the VPC to which it belongs.
SUBNETROUTE : This associates the routing table to the subnet.
SGFARGATE : This is the security group to be applied to our services. It allows traffic on ports 443 and 80 over HTTPS and HTTP protocols.
FARGATECLUSTER : It defines the Fargate cluster to host our services.
ECSTASK : Determines the tasks to be executed. It includes the list of Docker images that make the task. For each container, it registers ports mapping, starting commands, and logging options. Images are pulled from the ECR repository set up before.
SERVICE : Defines the ECS Service that will launch and maintain our tasks. If you already have a network configuration in place and don’t need to create a new subnet and security group, just reference those existing resources in the NetworkConfiguration section of the ECS Service.
Once your file is ready, upload it to Cloud Formation to create your stack:
Follow the steps in the management console to launch the stack. Once finished, Cloud Formation will automatically start provisioning the services. You can follow its progress in the events tab:
And more importantly, when ready, you can access your web application at the public IP address assigned to the running task!
If you are following best practices, you are not creating resources with your AWS root account. Instead, you should be using a non-root user. However, you should note that to pass a role to a service, AWS requires the user who creates the service to have “Pass Role” permissions. In our example, we need our user to pass the role ecsTaskExecutionRole to the TaskDefinition service, and therefore we must grant the user permissions to do so. This is something to be done from the root account in the IAM or any account with IAM privileges. Simply add the policy bellow, and attach it to the user who will allocate all the resources.
{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": "iam:PassRole",            "Resource": "arn:aws:iam::account_id:role/ecsTaskExecutionRole"        }    ]}
We’ve covered a lot in this article. We’ve seen how to create an ECR repository and how to push Docker images to it. We’ve also had a brief introduction to CloudFormation and IaC. I would like to restate the importance of specifying your infrastructure and stack as code. As your infrastructure grows, having the stack defined in JSON or YAML files will make it easier to automate deployments, scale in a productive manner, and will provide certain documentation on your infrastructure. Finally, we used AWS Fargate to deploy docker containers in a serverless way, which spared us the burden of provisioning and managing servers.
I hope you find this article helpful, thank you for reading.
Getting started with Amazon ECR using the AWS CLI
Getting started with Amazon ECR using the AWS CLI