5 tips to make your Gitlab CI/CD cleaner
Tags: beginners, devops, docker, productivity
Published: 2023-02-27 11:16:36

Gitlab can be a bit of a challenge if you have never written any CI/CD pipelines, forget about your software development mindset and quickly adapt to the pipeline mindset with these 5 easy to remember tips.

<hr>

   1. [Templates!](#templates)
   2. [Abstract Logins](#abstract-logins)
   3. [Keep jobs simple](#keeps-job-simple)
   4. [Keep jobs neatly linked](#keep-jobs-linked)
   5. [Keep important steps manual](#keep-manual)

<hr>

### Templates! <a name=#templates></a>

Pipelines can get messy, their top-down execution means you need to carefully analyze which rules apply to each job before executing them. You can simplify most of these rules with templates.

Templates can be defined in separate repos, and then used in your project's gitlab-ci.yml for extra commodity and abstraction. You can enable this behavior with **include** and **extends**


**ci-cd-templates repo**
templates.yml
```yml
.echo-statement
- before_script:
  - echo "this is an inherited step!"
```

**my-project repo**
gitlab-cy.yml
```yml
include:
  - project: 'ci-cd-templates'
    file: 'templates.yml'
    rev: (tag or branch)

example-job:
  extends:
    - .echo-statement
  script:
    - echo "this is a normal step!"
```

**Important caveat**
- Commands in your job will be defined by the template, if you define them in example-job they will overwrite the template's keys and values. This is my templates normally use a before/after script.

<hr>
### Abstract Logins <a name=abstract-logins></a>

Let's use the freshly acquired knowledge from the last tip to simplify our pipelines. When doing pipelines you will inadvertably reach a point where you will need to login into something, cloud, api, gitlab api, anything really.

Keep your pipelines cleaner by abstracting the logins to your templates. Since logins almost always generate files from which to yield the access tokens, combine this with **artifacts** and abstract your logins.

**ci-cd-templates repo**
templates.yml
```yml
.aws-login
variables:
  role: MY-ROLE
  role_session_name: dev
  account_id: 0000000
before_script:
  - aws sts assume-role --role-arn "arn:aws:iam::${ACCOUNT_ID}:role/${ROLE}" --role-session-name "${ROLE_SESSION_NAME}" > creds.tmp.json
  - echo AWS_ACCESS_KEY_ID=$(cat creds.tmp.json | jq -r ".Credentials.AccessKeyId") > build.env
  - echo AWS_SECRET_ACCESS_KEY=$(cat creds.tmp.json | jq -r ".Credentials.SecretAccessKey") >> build.env
  - echo AWS_SESSION_TOKEN=$(cat creds.tmp.json | jq -r ".Credentials.SessionToken") >> build.env
  - rm creds.tmp.json
artifacts:
  reports:
    dotenv: build.env
```

Attach this login templated job to your deployment jobs.

```yml
deploy-dev:
  extends: .aws-login
  script:
    - docker build -t [...]
    - docker push [...]
```

<hr>
### Keep jobs simple <a name=keep-jobs-simple></a>

When developing jobs, keep them as purpose fit as possible, this will ensure readability and maintainability.

Declare the same stages every time and give them individual stages. Those who have read "The Pragmatic Programmer" will kill me for saying this, but remember: forget about your programmer mindset.

A template we use in almost every project:

gitlab-ci.yml
```yml
stages:
- test
- build
- deploy

make-test:  # executes unittests / integration tests
  stage: test

init-dev-dependencies: # downloads dependencies
  stage: build

build-dev: # builds a docker image / gz / tar / etc
  stage: build

deploy-dev: # deploys / uploads a docker image / gz / tar / etc
  stage: deploy

init-tst-dependencies: # same steps, different enviornment
  stage: build

build-tst:
  stage: build

deploy-tst:
  stage: deploy

[ other environments ] 
```

<hr>
### Keep jobs neatly linked <a name=keep-jobs-linked></a>

Neatly tie jobs in order to form dependent pipelines with **needs** and **dependencies**.

- Needs: Needs a job to run before itself, this will create independent pipelines for each environment, if you define it correctly.
- Dependencies: Depends on the files from the dependent jobs.

gitlab-ci.yml
```yml
make-test:
  stage: test

init-dev-dependencies:  
  stage: build
  needs: 
    - make-test

build-dev:
  stage: build
  needs:
    - init-dev-dependencies
  dependencies: 
    - init-dev-dependecies

deploy-dev:
  stage: deploy
  needs:
    - build-dev
  dependencies:
    - build-dev

init-tst-dependencies:
  stage: build
  needs:
    - make-test

build-tst:
  stage: build
  needs:
    - init-tst-dependencies
  dependencies:
    - init-tst-dependencies

deploy-tst:
  stage: deploy
  needs: 
    - build-tst
  dependencies:
    - build-tst
```
For a clearer view of what **needs** actually does:

**With** needs:

![With needs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wkb6693y7a6zqklknaqx.png)

**Without** needs:


![Without needs](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4x90kwbw5e0cxmw70bp5.png)

So, in essence, it parallelizes.

<hr>
### Keep important steps manual <a name=keep-manual></a>

Lastly, keep the most important or dangerous steps manual, by adding **when**.

When will make sure a user presses the button in the gitlab UI in order for the step to run. Of course jobs that are linked to a manual job via needs will not run until the button is pressed. <u>It is also highly recommended to tie the initial production job to the acceptance / test deployment, for extra security against unwanted deployments.</u>

gitlab-ci.yml
```yml

init-prd-dependencies:  # no need for unit tests when deploying to prod, since it should already be tested in the dev/tst/acc pipelines
  stage: build
  when: manual

build-prd:
  stage: build
  needs:
    - init-prd-dependencies
    - deploy-acc
  dependencies: 
    - init-prd-dependecies

deploy-prd:
  stage: deploy
  needs:
    - build-prd
  dependencies:
    - build-prd
```

These are some tips from things we have learned in the past months, these GitLab features have helped our team create better, safer, readable and robust pipelines.