Reducing Lead Time for Changes
Tags: dora, cicd, aws, docker
Published: 2022-10-03 05:39:01


![Lead Time vs Cycle Time vs Lead Time for Changes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gxtnfvx9owv1et205q7u.png)

Our previous [post](https://dev.to/aws-builders/dynamic-kubernetes-based-feature-environment-on-aws-2cn6) describes how we have implemented a **Dynamic Feature Environment** that would allow engineers at Prenetics to spin up/down a fully functional independent K8s cluster that runs our full set of microservices, offering an internal **Environment as a Service (EaaS)**. The aim is to provide the necessary tools for our engineers to easily build and test **locally**.

At [Prenetics](https://prenetics.com/) Engineering we also look at improving our delivery efficiency. **Lead Time for Changes (LT)** is a one of the four [DORA metrics](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance) and it measures the amount of time a commit would take to get into production. Reducing LT would mean faster Time to market.

We split our LT improvements into the following phases:

1. Provision
2. Build
3. Deploy
4. Test
5. Release

In the following we look at how we have been optimising the **provision** phase.

## Provision

We use CodeBuild to execute build and test steps, and to deploy our digital assets

- k8s changes via [AWS Cli](https://aws.amazon.com/cli/) and [kubectl](https://kubernetes.io/docs/reference/kubectl/),
- web changes via AWS Cli to S3
- mobile changes via [Expo Cli](https://expo.dev/)
- infrastructure changes via [Terraform](https://www.terraform.io/)

Each of our CodeBuild projects executes the required steps inside its specified Docker container. AWS offers a number of [standard Docker images](https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html). However, these images do not have a number of applications needed in our build, deploy and test steps. We have therefore defined our own CodeBuild Docker image (see https://gist.github.com/peteryhwong/7bbbe461b028f5040c81cdcea2f256af is a snippet of the `Dockerfile`), Our custom Docker image defines from an Linux Alpine based "Docker in Docker" image and pre-installed the tools such as `jq`, `terraform`, `kubectl`, and `AWS Cli` that are needed in our build, deploy and test steps. This custom image is **smaller** in size and it means our steps do not need to download the same external dependencies in **every execution**.

This image is stored in our [AWS ECR](https://aws.amazon.com/ecr/) and is provisioned when CodeBuild projects execute.

Note that unlike the AWS standard images, containers created from our custom image would not the Docker daemon (`dockerd`) running by default, hence we would need to start Docker daemon manually when the container starts. See https://gist.github.com/peteryhwong/a51c39d69647c233808339c62d7b8f28 for details.

On average we see a reduction of up to a minute during the provision phase of our CodeBuild projects and our typical microservice's build phase is normally less than 5 minutes.

In our next post we will look at how we have been optimising the **build** phase.