Delivering Fast with AWS
By: Nethmi Hettiarachchi
Tags: Continuous Delivery, Docker, AWS, Continuous Integration

Lean management and continuous delivery practices create the conditions for delivering software faster and sustainably. High performing IT companies, deploy releases more frequently with smaller lead times. Let’s deep dive into the meaning behind this, and how AWS helps.

Continuous Integration

Continuous Integration is a software engineering practice which merges all developer work copies to a shared mainline. It’s the process of automating the building and testing of code every time a developer commits a change to the Version Control Service (e.g. Git Hub, Bitbucket). This kind of system is meant to encourage developers to share their code after each task completion, and once that code is shared via a Revision Control, it triggers an automated build system which grabs the latest code from the Revision Control System and deploys the code.

Continuous Delivery

Continuous delivery is the ability to get the changes of all types (new features, configurations, bug fixes, etc) into the hands of users, safely and in a sustainable way.

Continuous Integration / Continuous Delivery Architecture

When using AWS for CI / CD, some of the resources that you are likely to use are:

AWS CodePipeline — A continuous delivery service you can use to model, visualize, and automate the steps required to release your software.

AWS CodeBuild — A fully managed build service that compiles source code, runs unit test cases, and can be used for SonarQube Analysis as well.

AWS CloudFormation

AWS Elastic Container Service (ECS) and Elastic Container Registry (ECR) — Amazon ECR is a managed AWS Docker registry service. Customers can use this to push, pull, and manage docker images. Amazon ECR provides a secure, scalable, and reliable registry. (Amazon ECS) is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run and scale containerized applications on AWS.

The Architecture will look something like this:

CI/CD architecture with AWS

In this architecture (steps are numbered in the diagram):

Once AWS CodePipeline detects a change in the Github repository, it will grab the change and send it to CodeBuild.

AWS CodeBuild will build a Docker image using the latest code, which it got from the CodePipeline. The image will be tagged and pushed to the Amazon ECR.

AWS CodePipeline initiates an update of the docker image to the AWS Cloud formation. AWS CloudFormation, in turn, defines the task definition and the service.

CloudFormation then revises the task definition in the ECS by referencing the docker image in the ECR, and will then update the ECS.

ECS fetches the new container and replaces the old container.

AWS CodePipeline

AWS CodePipeline is the infrastructure which creates the environment for all other services to work together and carry out continuous deployment. It automates fetching changes from code base and the trigger build, and the testing and deployment process. AWS codePipeline can be integrated with best-of-breed third-party tools such as Jenkins, GitHub, BlazeMeter.

A snapshot of the codePipeline

Action

A single unit of execution. It can be a code build, test suite, run or deployment process. We can add other AWS services and an action. In this architecture, we added the AWS code build and AWS cloud formation templates as actions. Actions can be executed sequentially (one after the other) or parallelly.

An example of parallel actions

An example of sequential actions

Stage

A collection of actions. It can be considered as one level of a continuous deployment process. In our simple pipeline, we have a source stage which is responsible for triggering the pipeline when there is a source repository change, and to get all necessary resources for the code build stage. The Build stage is responsible for building the project, running tests, verifying the code and producing a deployable artifact which can be used in the deployment stage.

The deployment stage is responsible for deploying the artifact produced by the build stage in different environments. All the transaction between stages take place when all the actions in the first stage are successful.

AWS CodeBuild

Aws CodeBuild is an on-demand service (where you only need to pay for what you use) provided by AWS. It has a few in-built environments like docker, java, python, c# core, and node. The user can select the build environment they want.

The service is supposed to scale when the number of running tasks increases, meaning that engineers do not have to worry about the CD server overload or scaling the server when the number of projects increases.

In this architecture, the step is supposed to be used for building the docker image of the latest version and pushing it to an Elastic Container Service (CSR). In addition to this, we can use this step to run unit tests, regression tests, check code quality and verify application security and many more.

AWS CloudFormation

AWS Cloud Formation defines the common language to describe & provision for all infrastructure resources in an automated and secure manner. It allows us to define the project deployment infrastructure in a JSON or YAML file which is called Cloud formation templates (CFT).

This makes it easy to configure the infrastructure, troubleshoot issues and recreate the same deployment setup. Almost all AWS services can be configured using CFTs. You just need to write the CFT and upload it to the cloud formation service, and it will automatically acquire all the resources necessary and build all the deployment infrastructure.

In this architecture, we use the cloud formation service mainly to generate and update the stack and task definition which is associated with AWS Elastic container service (ECS). We can also use CFTs to define the CodePipeline and CodeBuild as well.

AWS Elastic Container Service (ECS) and Elastic Container Registry (ECR)

ECS is a highly scalable, high-performance container orchestration service that supports Docker containers and allows a user to easily run and scale containerized applications on AWS. The service gets the Docker image as input and takes care of all the deployment (according to the deployment process which we define). This provides zero downtime.

The service is also responsible for the high availability of the application, efficient resource allocation and making sure everything runs smoothly.

Elastic Container Registry (ECR) is a Docker repository which is hosted by AWS. We build and push a docker image in the code build step after which we define the task definition in the form of a CFT.

We provide the image tags in the task definition. Another CFT for service needs to be configured. Within that configuration, we can provide the task definitions which need to be defined.

Service is the unit which is used to configure all the deployment architecture. With a service, we can define deployment-related parameters like the number of instances, placement strategies, load balancer etc.

When we run the CFT of task definition and service with the Cloud formation service, it generates tasks using task definitions and services using service CFT. The service is deployed in one cluster. A cluster is a collection of resources. The service is going to use the resources of the cluster to deploy the application.

Amazon ECS with FARGATE

AWS FARGATE is a technology that allows you to run containers without having to manage servers or clusters. It allows focussing on designing and building the applications instead of managing the infrastructure that runs them. You only pay for the resources that you use. Pay for the amount of virtual CPU and memory resources consumed by your containerized applications

Benefits of ECS with FARGATE

No clusters to manage — with Fargate you no longer have to pick the instance types, manage cluster scheduling, or optimize cluster utilization.

Seamless scaling — with Fargate you no longer have to worry about provisioning enough compute resources for your container applications.

Can integrate with Amazon ECS and in future, with EKS — after you package your application into task definitions and upload everything to Amazon ECS, Fargate launches and manages containers for you.

These are only a few ways I have discovered in which AWS helps to maintain fast delivery in a sustainable way.

Hope you enjoyed the article.

cheers..!!

Join our community Slack and read our weekly Faun topics ⬇