Gitlab's Deploy Keys and Deploy Tokens for CI/CD
Tags: gitlab, ci, python, poetry
Published: 2021-07-19 03:13:46

> UPDATE: Poetry no longer supports deploy tokens on the package URLs due to a change on how they parse git urls, go read [this other post](https://dev.to/gdledsan/how-to-install-a-package-from-a-private-repository-on-poetry-for-ci-1g5l) for a solution

Scenario: Need to install a requirement that is only available on a private repository, from my CI pipeline.

There are two ways (I found) to do this, and the one you use depends on what you need to do:
1. I have a simple library with no dependencies or with only public dependencies
2. I have a dependency that depends on other private repositories

For context and illustration I used gitlab, python and poetry.

## Simple Dependencies: Use a Deploy Token
A [deploy token] (https://docs.gitlab.com/ee/user/project/deploy_tokens/) allows you to access a repository without your user name and password or ssh keys, and it is setup per repository or group.


### How to:
1. Add the dependency on your requirements file as you would normally
1. Setup the [deploy token] (https://docs.gitlab.com/ee/user/project/deploy_tokens/) on the private repository you want to access
1. Store the user and token on [CI Variables](https://docs.gitlab.com/ee/ci/variables/) on the repository you will access the private repository from
1. On the CI Script, fill in the deploy token using the CI variables.

#### [Example](https://gitlab.com/mundo03/sample-public-project/-/tree/deploy_token/):
Dependency on **pyproject.toml**
```
[tool.poetry.dependencies]
python = "^3.9"
private-project = {git = "ssh://git@gitlab.com/mundo03/sample-private-repo.git", rev = "main"}
```
Note the SSH URL is kept so you don;t have to use a [Personal Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) every time you install your dependencies locally.

**gitlab-ci.yml**
```
image: python:3.9-buster

poetry_token:
  script:
    # Install Poetry
    - pip install poetry
    # setup string replacement
    - export SSH_URL=ssh:\\/\\/git@gitlab.com\\/mundo03
    - export TOKEN_URL=https:\\/\\/$deploy_user:$deploy_token@gitlab.com\\/mundo03
    - sed -i "s/$SSH_URL/$TOKEN_URL/g" pyproject.toml
    # Install repo
    - poetry install
    # Use Clonned Repo
    - poetry run print_something "LOOK AT ME!!"
```
Notice `sed` is being used to replace the SSH URL with an HTTP URL that has the Deploy Token in it.

### Complex Dependencies: Use a deploy key
[Deploy keys](https://docs.gitlab.com/ee/user/project/deploy_keys/) also give you access to a repository with out your user name and password or SSH key, it is an SSH key you generate and can be used on multiple repositories, on multiple groups if necessary.

There are also [Public deploy keys](https://docs.gitlab.com/ee/user/project/deploy_keys/#public-deploy-keys) setup at Gitlab Instance level and can be given access to any of the projects in that instance by an admin.

### How to:
1. Add the dependency on your requirements file as you would normally
1. [Generate](https://docs.gitlab.com/ee/ssh/index.html) an ssh key
1. [Deploy the generated public key](https://docs.gitlab.com/ee/user/project/deploy_keys/#how-to-enable-deploy-keys) on any repo you need
1. Set up ssh-agent on the CI script

### [Example](https://gitlab.com/mundo03/sample-public-project/-/tree/deploy_key)
Dependency on **pyproject.toml**
```
[tool.poetry.dependencies]
python = "^3.9"
private-project = {git = "ssh://git@gitlab.com/mundo03/sample-private-repo.git", rev = "main"}
```


**gitlab-ci.yml**
```
image: python:3.9-buster

poetry_key:
  script:
      # Install OS dependencies
    - apt-get update -y -qq && apt-get install -y -qq make openssh-client python3-pip
    # Start ssh-agent
    - eval $(ssh-agent -s)
    # Add Private key to SSH Agent
    - ssh-add <(echo "$SSH_PK")
    # Add gitlab as known host
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
    # Install Poetry
    - pip install poetry
    # Install repo
    - poetry install
    # Use Clonned Repo
    - poetry run print_something "LOOK AT ME!!"
```

## Enter the Vault
CI Variables are not the best way available to store a secret, for that gitlab has [Secrets](https://docs.gitlab.com/charts/installation/secrets.html), which is meant to store things like SSH keys, Passwords Etc.

Read up:  [Secrets](https://docs.gitlab.com/charts/installation/secrets.html)
Spoilers: You need a Hashicorp's Vault account

## TL;DR
I have a dependency in my project that is hosted on a private repository, I need to install it during my CI/CD, I have two options:
1. **Deploy Token**: Are setup by project or group, I can add the token info in a CI variable and manipulate my requirements file to insert the token.
1. **Deploy Keys**: These are SSH keys that can be added to any project I need or in any group. I need to setup the SSH-Agent on the CI/CD script, the Public Key goes in the repository settings, the Private key goes in a CI variable or Vault, and use the ssh URL when requiring the dependency.

Have a look at the Complete Sample here:
[Use a Deploy Token](https://gitlab.com/mundo03/sample-public-project/-/tree/deploy_token)
[Use a Deploy Key](https://gitlab.com/mundo03/sample-public-project/-/tree/deploy_key)