Optimizing GitLab CI for Readability and Maintainability: From 1K to 600 Lines!
Tags: devops, gitlab, cicd, pipelines
Published: 2024-01-26 10:36:11

This short article aims to **share some tips I learnt from optimizing our GitLab CI file** (`.gitlab-ci.yaml`) for building multiple Docker images.

By implementing these techniques, you’ll not only **make your GitLab CI files more readable** but also set the stage for a more agile and adaptable CI/CD environment.
Let’s delve into the practical insights that can elevate your DevOps experience and contribute to a more efficient and maintainable codebase.

## 1- Use default runner tag and override it when needed:

In case you are using a standard or a generic runner for the majority of your jobs and a a different runner for a few jobs, consider defining a global runner tag instead of defining one for each job.
You can then override the global tag when necessary.

❌ Don’t do:

```yaml
compile:
  script:
    - ...
  tags:
    - myruuner
test:
  script:
    - ...
  tags:
    - testrunner  
deploy:
  script:
    - ...
  tags:
    - myrunner
```

✅ Do:

```yaml
default:
  tags:
    - myrunner

compile:
  script:
    - ...

test:
  script:
    - ...
  tags:
    - testrunner  
deploy:
  script:
    - ...
```

## 2- Use `parallel:matrix`

GitLab has a powerful CI feature to run a matrix of jobs in parallel.
In our case we had multiple jobs to test every customized Docker JDK image and ensure that it does not break a standard maven build.
Taking advantage of the keyword `parallel:matrix`, can save you multiple lines of code and make your CI files more readable and easier to maintain.

Here is an example of refactoring our test jobs:

❌ Instead of configuring multiple job with the same script part, for example:

```yaml
test-mvn-jdk-11:
  image: openjdk-11
  stage: test-image
  script:
    - mvn -V compile ...

test-mvn-jdk-17:
  image: openjdk-17
  stage: test-image
  script:
    - mvn -V compile ...

test-mvn-jdk-21:
  image: openjdk-21
  stage: test-image
  script:
    - mvn -V compile ...
....
```

✅ You can easily replace them with:

```yaml
test-mvn:
  image: $JAVA_IMAGE
  stage: test-image
  script:
    - mvn -V compile ...
  parallel:
    matrix:
      - JAVA_IMAGE: [openjdk-11,openjdk-17,openjdk-21]
```

This will also make it easier to incorporate additional JDK versions in the future, requiring minimal adjustments and eliminating the need for extensive line changes or introducing new jobs.

## 3- Use rules to define variables:

GitLab rules provide a mechanism for specifying conditions to determine when CI/CD jobs run.
Additionally, the use of `if` statements within these rules allows for the dynamic definition of variables based on conditions such as branches or releases, offering flexibility in variable assignment.

For our repository's CI/CD workflow, we start by building docker images and test them if there's a Merge Request.
Only when everything checks out, we create a final version `TAG` for release.
Using rules reduced our build jobs by a **half**.

❌ Instead of defining two jobs, for example:

```yaml
build-mr-jdk-11:
  stage: build
  script:
    - docker build -t jdk-11:mr-validation
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

build-jdk-11:
  image: openjdk-11
  stage: build
  script:
    - docker build -t jdk-11:$CI_COMMIT_TAG
  rules:
    - if: $CI_COMMIT_TAG
```

✅ Use the same job and override the variables depending on the rules conditions:

```yaml
build-jdk-11:
  stage: build
  script:
    - docker build -t jdk-11:$TAG
rules:
    - if: $CI_COMMIT_TAG
      variables:
        TAG: "$CI_COMMIT_TAG"
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      variables:
        TAG: "mr-validation"
```

## Conclusion: Lean back and let the magic happen 🧙‍♀

In summary, applying these tips reduced our GitLab CI file from 1176 to 667 lines, significantly improving efficiency, manageability, and overall workflow.

⭐⭐⭐ Enjoy your learning!!! ⭐⭐⭐