🦊 GitLab CI Optimization: 15+ Tips for Faster Pipelines
Tags: gitlab, devops, pipeline, cicd
Published: 2023-11-06 11:41:29

---
title: "\U0001F98A GitLab CI Optimization: 15+ Tips for Faster Pipelines"
tags:
  - gitlab
  - devops
  - pipeline
  - cicd
license: public-domain
cover_image: 'https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/sport/sport-car-1.jpg'
published: true
id: 1547806
date: '2023-11-06T11:41:29Z'
---

<!-- TODO:
- Mettre du cache uniquement s'il n'y en a pas déjà avec la technique du error code
- Sur le cluster de runners (EKS) de mon client, nous avons amélioré les performances en utilisant les ami bottlerockets.
    La performance varie considérablement en fonction des tâches effectuées par le job et du nombre de vCPU qu'un pod de job peut utiliser. Un élément particulièrement notable est le temps nécessaire à un nœud BottleRocket pour être disponible dans le cluster, soit environ 1 minute, ce qui représente une amélioration significative par rapport aux instances Amazon EKS Linux. Des gains légers ont été observés pour les pods limités à 1 vCPU. Pour les pods limités à 2 vCPU et lors de l'exécution de build Maven, une amélioration d'environ 40 % a été constatée. Le mieux est de faire sa propre expérience.
- pour GCP: https://hackernoon.com/aws-bottlerocket-vs-google-container-optimized-os-which-should-you-use-and-when-8a3q3whl
- https://archive.fosdem.org/2023/schedule/event/container_bottlerocket_os/attachments/slides/5730/export/events/attachments/container_bottlerocket_os/slides/5730/Bottlerocket_FOSDEM2023.pdf
 -->

<!-- TOC start (generated with https://github.com/derlin/bitdowntoc) -->

- [Initial thoughts](#initial-thoughts)
- [CI YAML optimizations](#ci-yaml-optimizations)
  - [1. Parallelize large jobs](#1-parallelize-large-jobs)
  - [2. Use small Linux distributions](#2-use-small-linux-distributions)
  - [3. Configure caching, split cache, and set policy](#3-configure-caching-split-cache-and-set-policy)
  - [4. Populate and upload cache only when necessary](#4-populate-and-upload-cache-only-when-necessary)
  - [5. Download only needed artifacts](#5-download-only-needed-artifacts)
  - [6. Use tuned rules](#6-use-tuned-rules)
  - [7. Define stages wisely and adjust with needs](#7-define-stages-wisely-and-adjust-with-needs)
  - [8. Configure interruptible pipelines](#8-configure-interruptible-pipelines)
  - [9. Rerun automatically jobs that failed](#9-rerun-automatically-jobs-that-failed)
- [Project configuration optimizations](#project-configuration-optimizations)
  - [10. Disable separate cache for protected branches](#10-disable-separate-cache-for-protected-branches)
  - [11. Avoid Docker images rebuild with fast-forward merge](#11-avoid-docker-images-rebuild-with-fast-forward-merge)
  - [12. Configure push rules to avoid pipelines for misconfigured branches](#12-configure-push-rules-to-avoid-pipelines-for-misconfigured-branches)
- [Runner configuration optimizations](#runner-configuration-optimizations)
  - [13. Cache Docker builds](#13-cache-docker-builds)
  - [14. Cache jobs images](#14-cache-jobs-images)
  - [15. Optimize caches and artifacts compression](#15-optimize-caches-and-artifacts-compression)
  - [16. Size runners correctly and tune your maximum parallelized jobs](#16-size-runners-correctly-and-tune-your-maximum-parallelized-jobs)
- [Wrapping up](#wrapping-up)
- [Further reading](#further-reading)

<!-- TOC end -->

<!--

TODO: 10, 12

TODO:
- Parler de +++ https://cpcwood.com/blog/6-how-we-reduced-our-gitlab-ci-pipeline-duration-by-70-at-student-beans
  - Running Jobs in Kubernetes
  - Adding Distributed Caching
  - Container Image Layer Caching
  - Parallelizing Large Jobs
  - Taking Advantage of the needs Keyword
- pas mauvais: https://www.augmentedmind.de/2022/06/26/speed-up-gitlab-ci-pipelines/
  - few broad categories why pipelines are slow
  - Choose the right runner infrastructure (belle comparaison VM/Kubernetes)
- Let's make faster GitLab CI/CD pipelines: pas mauvais sauf qu'il recheck entièrement son node module: https://blog.nimbleways.com/let-s-make-faster-gitlab-ci-cd-pipelines/
  - Caching (policy, fastzip, multiple cache, docker)
  - rules
  - needs (and optional needs)
  - don't use docker
  - interruptible
- Extract greater efficiency from your CI pipelines: très léger. https://about.gitlab.com/blog/2022/03/09/efficient-pipelines/
- 7 ways to speed up your GitLab CI/CD times: Pas mal, mais custom image antipattern: https://blog.sparksuite.com/7-ways-to-speed-up-gitlab-ci-cd-times-29f60aab69f9
  - Host your own GitLab runner
  - Pre-install dependencies (docker)
  - Use tiny Linux distros for your CI images
  - Use the overlay2 storage driver
  - Use a cached Docker image when building
  - Thoughtfully organize your Dockerfiles ++++
  - Cache dynamic dependencies
  - rules
  - needs
- How to make your GitLab CI pipelines faster: pas trop de texte, pas mal: https://www.yotec.net/tips-on-how-to-speed-up-ci-cd-tasks-in-gitlab-pipelines/
  - Pre-Installing Dependencies in a Docker Image
  - Docker Image Optimization
  - Use Docker cache when building images
  - Local caching of Docker images
  - if-not-present
  - cache (and policy)
  - fastzip
  - git clone strategy
  - rules, needs, interruptible
  - Use your own runners
- Learn how to speed up Gitlab CI. pas ouf: https://www.padok.fr/en/blog/gitlab-ci-optimization
- Faster runners (no shit sherlock !?): https://www.hatica.io/blog/ci-cd-with-gitlab/
- single job (antipattern): https://medium.com/nerd-for-tech/speed-up-gitlab-ci-by-using-fewer-steps-in-the-pipeline-7e6819cd0e47

- Quand on se prend trop au sérieux: https://medium.com/agoda-engineering/devops-at-agoda-how-we-move-3-million-ci-pipelines-from-vms-to-kubernetes-c05a59b60fb1

Faire le lien avec les project details quand c'est utilisé ?????

orienté projets, pas grosses entreprises. Sécu moins importante

-->

# Initial thoughts

Are you looking to boost your GitLab pipelines' speed and efficiency? If saving time and increasing productivity is on your agenda, then you're in the right place. This article offers 15+ practical tips to optimize your GitLab CI/CD pipelines and accelerate your development process.

We'll study various aspects of pipeline optimization, focusing on improvements that apply broadly. While you can fine-tune your pipelines further based on specific tools like NPM, PNPM, [Yarn](https://blog.nimbleways.com/let-s-make-faster-gitlab-ci-cd-pipelines/), Maven, or Gradle, those details are beyond the scope of this article.

Assuming you've already covered the fundamentals and best practices as discussed in our previous article, [GitLab CI: 10+ best practices to avoid widespread anti-patterns](https://dev.to/zenika/gitlab-ci-10-best-practices-to-avoid-widespread-anti-patterns-2mb5), let's explore how you can further optimize your GitLab CI pipelines.

# CI YAML optimizations

## 1. Parallelize large jobs

When you have extensive test suites or tasks to execute, parallelization can significantly reduce pipeline duration. GitLab allows you to parallelize jobs using the [parallel](https://docs.gitlab.com/ee/ci/yaml/#parallel) keyword. You can split your work into multiple jobs that run concurrently, improving overall pipeline speed.

For example, you can use predefined variables to split test suites like this:

```yaml
tests-in-parallel:
  parallel: 3
  script:
    - bundle
    - bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL
```

You will find another exemple with Yarn in [this article](https://cpcwood.com/blog/6-how-we-reduced-our-gitlab-ci-pipeline-duration-by-70-at-student-beans).

## 2. Use small Linux distributions

Optimize your pipeline performance by selecting small Linux distributions for your Docker images. Alpine Linux, for instance, is a lightweight option that results in smaller image sizes compared to standard distributions like Ubuntu or Debian.

Here are some Linux distribution types suitable for CI:

- **Standard Docker Image**: These are based on full distributions like Ubuntu or Debian, including various pre-installed packages. They tend to be larger.

- **Alpine Docker Image**: Alpine Linux is lightweight and designed to be small and secure. Images based on Alpine are typically much smaller and faster.

- **Slim Docker Image**: These images aim to be small and efficient, using lightweight distributions like CentOS or Debian but without the bloat.

There are also [distroless](https://bell-sw.com/blog/distroless-containers-for-security-and-size/) Docker images, sacrificing dynamic possibilities for size and security. But by the very nature of job running, these images' fit is very limited in CI, where dynamic tools, like a package manager, are needed.

For CI performance, choose Alpine Docker images when available, else Slim Docker images if you have the choice.

## 3. Configure caching, split cache, and set policy

GitLab optimal caching strategy, between local and shared cache, depends on your runner architecture, among the ones described in [GitLab Runners topologies: pros and cons](https://dev.to/zenika/gitlab-runners-topologies-pros-and-cons-2pb1). Overall, where there are multiple runner servers, we should configure runner shared cache when aiming for pipeline speed.

One thing to consider when defining cache, is splitting in multiple caches kind when suitable, taking advantage of the fact that the [cache](https://docs.gitlab.com/ee/ci/yaml/#cache) keyword can take a list of caches. You will find a detailed example with Yarn in [Let's make faster GitLab CI/CD pipelines](https://blog.nimbleways.com/let-s-make-faster-gitlab-ci-cd-pipelines/).

Something complementary, especially to the multi cache and/or gigabytes caches, is the [cache policy](https://docs.gitlab.com/ee/ci/yaml/#cachepolicy). By default, cache is downloaded, unzipped, and at the end of the job, zipped and uploaded (or stored if cache is local). This is the `pull-push` policy. But you can define it to `push` in cache producer-only jobs, and `pull` in cache consumer-only jobs, saving some precious time on unneeded cache phases.

## 4. Populate and upload cache only when necessary

Using cache and setting a good pull/push policy is a good way to improve pipeline speed. However, there are additional optimizations to consider for further enhancement.

Firstly, ensure to share the cache between multiple pipelines when possible by using a generic `cache:key` or `cache:key:files`. Using `maven-$CI_COMMIT_REF_SLUG` as the `cache:key` will maintain the same cache for the current pipeline. But, most of the time, this type of cache can be shared among all branches, allowing `maven` as a key to share widely without much risk. On the other hand, the `cache:key:files` is already designed for the perfect scope.

Secondly, once you have the right cache, try to avoid rebuilding an already valid cache. Test its presence and/or validity:

```yaml
package-and-test:
  image: node:21.6-alpine
  cache:
    - key:
        files: [yarn.lock]
      paths: [node_modules]
  before_script:
    - if [ ! -d node_modules/.bin ]; then yarn install; fi
    - yarn test
    - yarn build
```

Last but not least: the cache upload, at the end of jobs. By default, jobs with `push-pull` policy will push the cache each time, even if it has not changed for weeks! There is no functionality in GitLab CI like _push cache if not exist_. However, thanks to the community, there is [a nice hack](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3523#note_1269290145): relying on `allow_failure:exit_codes` and `cache:when:on_failure` to push only if the cache has been (re)built.

Here is an example, where the cache is not rebuilt when found and not re-uploaded if unchanged:

```yaml
📦✅-webapp-package-and-test:
  stage: 📦 package ✅ test
  image: node:21.6-alpine
  cache:
    key:
      files: [yarn.lock]
    paths: [node_modules]
    policy: pull-push
    when: on_failure
  variables:
    JOB_EXIT_CODE: 0 # default: OK
  before_script:
    - if [ ! -d node_modules/.bin ]; then yarn install && export JOB_EXIT_CODE=9; fi
  script:
    - yarn test
    - exit $JOB_EXIT_CODE
  allow_failure:
    exit_codes: 9
  artifacts:
    paths: [dist]
```

![<lora:ManyPipesAI:.85> ManyPipesAI orange sport car, ((high speed)), futurism](https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/sport/sport-car-2.jpg)

## 5. Download only needed artifacts

By default, GitLab downloads all artifacts from every job and every previous stage at the start of a job. This can create a significant overhead, especially in complex pipelines.

Imagine a mono-repo pipeline with 5 modules and 4 jobs for each module in different stages: `package`, `test`, `image build`, `deploy`. Lets assume that the modules are independents, and the `package` jobs for each module are the only ones producing artifacts, for jobs in `test` and `image build`.

We then have, for each module, 2 jobs with one artifact download needed, that is 10 strictly useful downloads. By default, there would be... 75 downloads ! (5 artifacts produced x 5 jobs per stage x 3 later stages).

So we'd better define artifacts with [dependencies](https://docs.gitlab.com/ee/ci/yaml/#dependencies) keyword:

```yaml
package-a:
  [...]
  artifacts:
    paths: [a/target/]

test-a:
  dependencies: [package-a]
  [...]

image-build-a:
  dependencies: [package-a]
  [...]

deploy-a:
  dependencies: []
  [...]
```

## 6. Use tuned rules

By default, GitLab runs jobs for every Git event, such as a commit being pushed or a new branch being created. While this is suitable for simple use cases, you may want to avoid running unnecessary jobs on certain pipelines.

You can optimize this by defining rules for when a pipeline should be triggered using the [workflow:rules](https://docs.gitlab.com/ee/ci/yaml/#workflowrules) keyword. To decide when a specific job within a pipeline should be triggered, you can use the [rules](https://docs.gitlab.com/ee/ci/yaml/#rules) keyword. Reducing the number of unnecessary jobs in your pipelines can alleviate the load on your runners.

For instance, you can avoid running jobs in the following situations:

- For unmodified modules in mono-repos, consider using [merge request pipelines with rules:changes](https://docs.gitlab.com/ee/ci/jobs/job_control.html#use-onlychanges-with-merge-request-pipelines).
- Skip tests on pipelines where the Docker image has already been pushed and tested.
- Avoid running security tests on feature branches.
- Selectively run unit testing or end-to-end testing based on your workflow requirements.

## 7. Define stages wisely and adjust with needs

Imagine we have a pipeline deploying 3 modules in separate jobs in a single `deploy` stage, but with one deployment preparation task. We would be tempted to add a `pre-deploy` stage for the single job, like below:

![Stages without optimizations](https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/cicd/stage-optim-1.png)

This is inefficient, because the pre-deploy job is solo in its stage, does not need the Docker images to be built, and can be time consuming, being related to infrastructure.

An easy optimization is to merge some stages:

![Stage removed without needs](https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/cicd/stage-optim-2.png)

That would be a recommended and simple way.

Another one would be to take advantage of the [needs](https://docs.gitlab.com/ee/ci/yaml/#needs) keyword. It allows us to bypass the stages constraint for ordering jobs. We can then put the pre-deploy job in the same stage as other deployment jobs, defining some chains constraints between them:

- `pre-deploy` needs `back-test`, `front-test`, `bff-test` (this way it will run concurrently to Docker build jobs)
- `back-deploy`, `front-deploy`, `bff-deploy` needs `pre-deploy`

This will produce this pipeline, with represented dependencies:

![Stage removed with needs](https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/cicd/stage-optim-3.png)

This way we obtain the exact same workflow the stages merge explained earlier.

We have to be careful with _needs_, real workflow can become hard to read at runtime.

## 8. Configure interruptible pipelines

By default, GitLab allows multiple pipelines to run concurrently for the same branch, even if the associated commit(s) have become obsolete. This can result in unnecessary resource consumption on your runners.

You can reduce this strain on your runners by automatically stopping jobs for obsolete pipelines. GitLab provides the `interruptible` keyword, which can be set to `true` to achieve this. When this attribute is enabled, only the most relevant pipelines continue to execute, reducing the load on your runners.

You can set it by default for every job:

```yaml
default:
  interruptible: true
```

Worth noting: It does stop the obsolete pipeline only if both pipelines are launched automatically. If one of them is manually launched, GitLab will not interfere.

## 9. Rerun automatically jobs that failed

Occasionally, jobs may fail due to rare real-time issues. These failures can block the entire pipeline and require manual intervention to rerun the failed job.

To prevent such inconveniences, you can configure automatic job retries using the [retry](https://docs.gitlab.com/ee/ci/yaml/#retry) keyword in your CI/CD configuration. By specifying the number of times a job should automatically retry in case of failure, you ensure that your pipeline can recover from transient issues without manual intervention.

Here's an example of how to configure job retries:

```yaml
flaky-tests:
  script:
    - ./run-flaky-tests.sh
  retry: 2
```

![<lora:ManyPipesAI:.85> ManyPipesAI orange sport car, ((high speed)), futurism](https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/sport/sport-car-3.jpg)

# Project configuration optimizations

In addition to optimizing CI/CD YAML configurations and runner settings, project-specific configurations can also play a crucial role in streamlining GitLab pipelines. Here, we'll explore some strategies to enhance your project settings, which can further improve pipeline performance.

Now, let's explore specific optimizations you can make at the project (or group) level.

## 10. Disable separate cache for protected branches

By default, GitLab separates caches between [protected branches](https://docs.gitlab.com/ee/user/project/protected_branches.html) and non-protected branches. While this approach offers enhanced security, it may not always be relevant for performance optimization.

In many cases, there's no need to maintain separate caches, and it's more efficient to share the cache across all branches. You can [deactivate this option](https://docs.gitlab.com/ee/ci/caching/#use-the-same-cache-for-all-branches) and use the same cache for all branches. This can significantly improve caching and reduce redundancy in your pipeline.

## 11. Avoid Docker images rebuild with fast-forward merge

When it comes to handling merge requests, GitLab provides [three methods](https://docs.gitlab.com/ee/user/project/merge_requests/methods/): merge commit, merge commit with semi-linear history, and fast-forward merge. The choice of merge method can have an impact on your pipeline performance.

Fast-forward merge, in particular, is an efficient way to incorporate changes from a source branch into a target branch when there are no conflicts. It results in a linear history, and the commit at the tip of a feature branch become the commit at the tip of the main branch, since there are no merge commits.

We can take advantage of this to not rebuild docker images already built for this commit...

<!-- CODE EXAMPLE -->

## 12. Configure push rules to avoid pipelines for misconfigured branches

Paraphrasing the [famous quote by Jeff Atwood](https://medium.com/javarevisited/epic-programmers-quotes-explained-aed933257b93): _The best pipeline is no unnecessary pipeline at all_. Configuring [push rules](https://docs.gitlab.com/ee/user/project/repository/push_rules.html#override-global-push-rules-per-project) in GitLab can serve as gate-keeping mechanisms to prevent pipelines from running on branches with certain misconfigurations. The goal here is to free your runners from unnecessary work by avoiding pipelines on branches that are not properly set up.

You can implement various rules to restrict pipeline creation, such as:

- Reject unverified users, inconsistent user names, and non-GitLab users.
- Define regular expressions for commit messages, branch names, and commit author email.
- Exclude specific file extensions from triggering pipelines.

These rules help ensure that your runners are used efficiently and that pipelines are only created when they are truly needed.

# Runner configuration optimizations

Optimizing your GitLab CI/CD pipeline isn't limited to CI/CD configuration files and project-level settings. Runner configuration plays a vital role in ensuring efficient pipeline execution. Here, we'll explore strategies to fine-tune your GitLab runners for improved performance.

Some of these are accessible through GitLab CI YAML configuration, others need administrator access to runners.

## 13. Cache Docker builds

Building and pushing Docker images can significantly contribute to your pipeline's total duration. To optimize this aspect, you can consider various strategies, but first, make sure you're familiar with [GitLab's documentation on using Docker build](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html).

One common approach for building Docker images is to use a [Docker-in-docker CI service](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker). While this method is prevalent, it has some downsides: it takes time to spin up, requires [privileged mode on the runner](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-with-tls-enabled-in-the-docker-executor), and doesn't offer image layer caching between jobs.

A recommended and efficient alternative is to utilize Docker layer caching, which significantly improves build times. The essential principle here is to pull a similar image before starting the build process. This way, Docker layers are cached, and subsequent jobs can reuse them, saving valuable time. Here's an example configuration:

```yaml
docker-build:
  image: docker:20.10.16
  services:
    - docker:20.10.16-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker pull $CI_REGISTRY_IMAGE:latest || true
    - docker build --cache-from $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:latest .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
```

This technique saves up time, but it adds layer downloads, for each job. this is still far from ideal.

A more efficient way is using [Docker alternatives](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-alternatives), such as Kaniko, Buildah, Img.

This is faster, because they do not need to spin up a service (and there is no privileged mode needed). But they still needs the layers download to become efficient.

The fastest solution is to [bind-mount the docker socket](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-the-docker-executor-with-docker-socket-binding) of the underlying server (where the job is running). No need for a service, and cache is already there. However, the more dynamically provisioned are your servers, the less likely you will hit this precious cache. So this is perfect for Docker runners, but less for AWS and Kubernetes, described in [GitLab Runners topologies: pros and cons](https://dev.to/zenika/gitlab-runners-topologies-pros-and-cons-2pb1).

This solution is controversial because of the security breach it opens to the underlying VM. Designing a runner architecture with this in mind is crucial. But this is still our preferred choice for starting projects.

## 14. Cache jobs images

In GitLab, Docker images for jobs are often downloaded by default, which can consume both time and network bandwidth. To enhance performance, consider configuring your jobs to use the "if-not-present" Docker image pull policy [at runner level](https://docs.gitlab.com/runner/executors/docker.html#configure-how-runners-pull-images). This setting instructs the runner to download the image only if it's not already present on the runner. It's especially useful if you have control over your runner.

You can also set this policy for jobs using the [image:pull_policy](https://docs.gitlab.com/ee/ci/yaml/#imagepull_policy) keyword in your CI/CD configuration. Make sure your GitLab runner admin hasn't disabled this feature. By using this policy, you can save time and prevent unnecessary image downloads during pipeline execution.

Do not use `latest` image tags when using this configuration, or you will have unexpected differences between runners. Do not use `latest` at all for public images with or without this configuration, or you will face unexpected errors when breaking changes occur.

To make GitLab configuration specifically and efficiently working on an autoscaling kubernetes cluster, you will have to add (Kubernetes) tools to [improve container images availability and speed with caching](https://itnext.io/improve-container-image-availability-and-speed-with-caching-in-kubernetes-870fa7bfa1ed).

## 15. Optimize caches and artifacts compression

Cache and artifacts compression plays a crucial role in the overall performance of your GitLab pipelines. GitLab offers some [runner feature flags](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) that allow you to choose the FastZip compression tool, which is more efficient than the default one. Additionally, you can fine-tune the compression level to further improve performance:

```yaml
variables:
  FF_USE_FASTZIP: "true"
  # These can be specified per job or per pipeline (slowest, slow, default, fast, and fastest)
  ARTIFACT_COMPRESSION_LEVEL: "fastest"
  CACHE_COMPRESSION_LEVEL: "fastest"
```

You can safely use these variable in every projects. the `fastest` compression level can be the default, proven to be the best optimization for pipeline speed.

## 16. Size runners correctly and tune your maximum parallelized jobs

Optimizing your GitLab CI/CD pipeline performance involves ensuring that your runners are correctly sized and that the maximum number of parallelized jobs is properly tuned. This is a crucial aspect of managing your GitLab infrastructure efficiently.

If you have the choice you can read our article [GitLab Runners topologies: pros and cons](https://dev.to/zenika/gitlab-runners-topologies-pros-and-cons-2pb1) and choose a topology according to your constraints.

When you're determining the appropriate runner setup for your project, consider both the number of runners you need and their specifications. Conducting local tests while monitoring CPU, RAM, and disk I/O usage can help you make informed decisions.

The maximum number of parallelized jobs, defined with the `parallel` runner parameter, should align with the number of available CPU cores on your runner. For instance, if your pipeline jobs typically utilize two CPU cores, you might initially set the `parallel` value to 10 for a runner equipped with 16 CPU cores. However, it's important to experiment with different configurations and fine-tune this value to best match your specific project's workload and infrastructure.

By accurately sizing your runners and adjusting the parallel job limits, you can significantly boost the performance and efficiency of your GitLab CI/CD pipelines.

# Wrapping up

In your quest for faster GitLab CI/CD pipelines, these performance optimizations can make a substantial difference. Saving valuable minutes on pipeline execution not only enhances your productivity but also reduces waiting times and accelerates your software development process.

While these optimizations can significantly enhance pipeline speed, it's essential to strike a balance between performance improvements and pipeline readability and maintainability. Complexity should be kept to a minimum to ensure that your CI/CD process remains manageable and transparent.

If you have any thoughts or suggestions, please feel free to share them in the comments section 🤓

![<lora:ManyPipesAI:.85> ManyPipesAI orange sport car, ((high speed)), futurism](https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/sport/sport-car-4.jpg)

_Illustrations generated locally by Automatic1111 using RevAnimated model with ManyPipesAI LoRA_

# Further reading

{% embed https://dev.to/zenika/gitlab-a-python-script-displaying-latest-pipelines-in-groups-projects-5b5a %}

{% embed https://dev.to/zenika/gitlab-a-python-script-calculating-dora-metrics-258o %}

{% embed https://dev.to/zenika/gitlab-ci-the-majestic-single-server-runner-1b5b %}

{% embed https://dev.to/zenika/gitlab-ci-yaml-modifications-tackling-the-feedback-loop-problem-4ib1 %}

{% embed https://dev.to/zenika/gitlab-ci-10-best-practices-to-avoid-widespread-anti-patterns-2mb5 %}

{% embed https://dev.to/zenika/gitlab-pages-preview-the-no-compromise-hack-to-serve-per-branch-pages-5599 %}

{% embed https://dev.to/zenika/chatgpt-if-you-please-make-me-a-gitlab-jobs-attributes-sorter-3co3 %}

{% embed https://dev.to/zenika/gitlab-runners-topologies-pros-and-cons-2pb1 %}

_This article was enhanced with the assistance of an AI language model to ensure clarity and accuracy in the content, as English is not my native language._