How to configure CircleCI: what you might not know
Tags: devops, cicd, circleci
Published: 2021-12-04 20:21:25

CircleCI is not the most popular CI/CD platform for open source contributors today, but it is widely used by many big sharks in the industry. So we have to keep in mind how to accurately configure it. 

CircleCI offers several execution environments. Those bold nerds from San Francisco call them _executors_. An `executor` defines the underlying technology or environment [in which to run a job](https://circleci.com/docs/2.0/executor-intro/). Set up your jobs to run in the `docker`, `machine`, `macos` or `windows` executor and specify an image with the tools and packages you need.

![A so-called 'executors' concept](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fe87kdg1kovfbcinytt9.png)

Also, CircleCI provide using [so-called orbs](https://circleci.com/docs/2.0/orb-intro/) with its platform. _Orbs_ are open-source, shareable packages of parametrizable reusable configuration elements, including jobs, commands, and executors. You may involve orbs to reduce configuration complexity and help you integrate with your software and services stack quickly and easily across many projects.

In general, orb concepts are similar to GitHub Actions Marketplace items' ones. This feature enables snippets of code to be re-used to automate repetitive tasks and build lightweight pipelines as code using YAML.

A top hierarchy unit in CircleCI flow is called _workflow_. A `workflow` divided to _jobs_. A `job` may include many _steps_.

If you use orbs to perform some standard actions, for example, to build and publish a docker image, then the system works strictly according [to the template](https://circleci.com/developer/orbs/orb/circleci/aws-ecr). Unfortunately, the documentation is sometimes complicated and confusing. One may get the impression, as happened in my case, that the orbs live their own lives, their behaviour cannot be changed. For example, I needed to achieve activation of [docker layer caching](https://circleci.com/docs/2.0/docker-layer-caching/) (DLC) when building images, so as not to re-use previous layers and thereby save resources and time. 

> ⚠️💰Note: The so-called _Performance plan_ is required to use Docker Layer Caching. If you are on the container-based plan you will need to upgrade the plan.

Basically, a layer, or image layer is a change on an image, or an intermediate image. Every command you specify (`FROM`, `RUN`, `COPY`, etc.) in your `Dockerfile` causes the previous image to change, thus creating a new layer. You can think of it as staging changes when you're using `git`: you add a file's change, then another one, then another one... Re-using unchanged data in the build is what we need.

As it turned out, _it is impossible to use a custom docker environment_ or a `setup_remote_docker` parameter _with orbs_, but you can still enable DLC and achieve a win **up to 35%** in flow runtime! I'll explain below how to do the trick.

There is an option to tweak an executor globally at the start of your flow. CircleCI doesn't allow splitting flow files, so it will be a common setting. 

**I.** First let's describe our custom executor which we'll be able to re-use later in our job:
```yaml
version: 2.1
orbs:
  aws-ecr: circleci/aws-ecr@7.2.0
executors:
  my-customized-executor:
    machine:
      docker_layer_caching: true
      image: 'ubuntu-2004:202101-01'
    resource_class: medium
```

Almost everything in CircleCI runs upon docker, so, as you may see we just turned on DLC for that [runner](https://circleci.com/docs/2.0/configuration-reference/). Do not try to imagine how it works, but trust me, it works.

**II.** Now we may point CircleCI to use our customized machine somewhere we need among our jobs:
```yaml
jobs:
  build-and-push-image:
    executor: my-customized-executor
    steps: ...
```

**III.** Launch your flow. Read the logs carefully. With DLC enabled you should see from docker:
```
...
Step 2/8 : RUN apt-get update && apt-get install -y build-essential && apt-get install -y python
 ---> Using cache
...
``` 
Here it is! 

Not all solutions lie on the surface, but by swimming to the iceberg, without being frightened and having consumed it, you can get, e.g., fresh drinking water. 

We at Otomato wish our colleagues, let the best solutions be found quickly even when it comes to complex and insidious CircleCI. 😎