🚀 GITLAB ON KUBERNETES: The Ultimate Deployment Guide! 🌟
Tags: kubernetes, opensource, devops, gitlab
Published: 2023-11-29 16:43:06

## TL;DR 🔍

Explore the step-by-step guide to deploying GitLab on Kubernetes, focusing on the Omnibus package configuration. Learn to set up PostgreSQL, SMTP, Container Registry, Sidekiq, Prometheus metrics, and backups. Discover an alternative with the Glasskube GitLab Kubernetes Operator, simplifying the process and reducing setup time to just 5 minutes.

---

## We Want Your Feedback! 🫶

Share your thoughts in the comments below! Let us know what topics you'd like more content on. If this guide helps, click on the cat and leave a star to support us in creating more developer-centric content. Your feedback matters!


[![Glasskube Github](https://cms.glasskube.eu/uploads/CTA_51bbe3bb2a.png)
](https://github.com/glasskube/glasskube)

---
## Let´s get started 🏌️

[GitLab](https://glasskube.eu/en/s/kubernetes-operator/gitlab/) is an open source DevSecOps platform for software engineering teams.

There are two flavors available for deploying GitLab on your Kubernetes Cluster:

1. GitLab Cloud native hybrid
2. GitLab package (Omnibus)

## GitLab Cloud native hybrid 

Deploying the GitLab Cloud native hybrid architecture makes only sense in query specific use-cases. For example - as seen in the [GitLab decision tree](https://docs.gitlab.com/ee/administration/reference_architectures/#decision-tree) - if the GitLab instance need to serve at least 3,000 users. For this kind of deployment the GitLab components will be installed separately and in different docker containers. Resulting in minimum requirements of 10 nodes with a total of 19 vCPUs and 60 GB memory, which will result in a couple of thousand dollars cloud costs a month.

<img width="100%" style="width:100%" src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExbHl6eXltNWw0ZDNjbnNqbDdicXBpbzNpdXdkNXp5aDVueG1nbnQ0MiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/2aIZfQdC2V7bBvU5t2/giphy.gif">


So in most cases GitLab Cloud native is not needed and overly complex.

**So, how to deploy GitLab Omnibus on Kubernetes?**

## GitLab Omnibus on Kubernetes

Using the name “omnibus”, GitLab also publishes all-in-one packages, which are available as docker images [`hub.docker.com/r/gitlab/gitlab-ce`](https://hub.docker.com/r/gitlab/gitlab-ce), that can be easily configured via environment variables. Doing the configuration right makes it easy to deploy GitLab on Kubernetes.

Following important components should be configured correctly in order to archive a reasonable Kubernetes setup:

1. PostgreSQL database
2. SMTP configuration
3. GitLab container registry on Kubernetes
4. Sidekiq, Gitaly & Puma configuration
5. Prometheus metrics
6. GitLab backups on Kubernetes

### GitLab on Kubernetes: Configure an external PostgreSQL database

Creating a PostgreSQL database can be done with the [CloudNativePG PostgreSQL Operator](https://cloudnative-pg.io/). After the installation of the operator a new Postgres cluster can be deployed by applying a Kubernetes CR:


```yaml
kind: Cluster
apiVersion: postgresql.cnpg.io/v1
metadata:
  name: gitlab
spec:
  enableSuperuserAccess: false
  instances: 2
  bootstrap:
    initdb:
      database: gitlabhq_production
      owner: gitlab
  storage:
    size: 20Gi
```

It is important that the database is called `gitlabhq_production` and the configured `gitlab` database user is the owner of the database. In this case, we created a cluster consisting of two PostgreSQL replicas to ensure that the database remains available, even in case the node running the primary replica fails. This is called high availability (HA).
Now, that we created our very own PostgreSQL cluster, the database included in the omnibus image must be disabled in the `gitlab.rb` file.


```ruby
postgresql['enable'] = false

gitlab_rails['db_adapter'] = 'postgresql'
gitlab_rails['db_encoding'] = 'unicode'
gitlab_rails['db_host'] = 'gitlab-pg-rw'
gitlab_rails['db_password'] = '<your-password>'
```

Nice, we already have cloud native database for our GitLab installation.

<img width="25%" style="width:25%" src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExc2VydjJoN3BsN2RycDY5N3prZHhoaW8xdTYxenhkbHdtazBpM3A5YSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/SVH9y2LQUVVCRcqD7o/giphy.gif">

#### GitLab on Kubernetes: Configure SMTP credentials

In order to make sure your GitLab instance is able to send emails, you need to configure an SMTP server. This also can be done in the `gitlab.rb` file.


```ruby
gitlab_rails['smtp_enable'] = ...
gitlab_rails['smtp_address'] = ...
gitlab_rails['smtp_port'] = ...
gitlab_rails['smtp_user_name'] = ...
gitlab_rails['smtp_password'] = ...
gitlab_rails['smtp_tls'] = ...
gitlab_rails['gitlab_email_from'] = ...
```

A free smtp server for transactional emails can be created for example on the Brevo platform.

<img width="25%" style="width:25%" src="https://media.giphy.com/media/0IR3vO2bWY1AQPAsAn/giphy.gif">

#### GitLab on Kubernetes: Container registry

Among it's many features, GitLab supports serving as a container registry. This is achieved by bundling the reference implementation of the docker container registry, but it is disabled by default as it is quite a hassle to set it up correctly.

> **Important**
> It is important to understand that all requests in the GitLab omnibus container will first be processed by an internal nginx server and then distributed to components.

The container registry only works with TLS-encrypted connections, so we need to disable TLS termination by the ingress load balancer and send encrypted traffic directly to the GitLab container. If the NGINX ingress controller is used, this can be done by adding the following annotation to the ingress: `nginx.ingress.kubernetes.io/ssl-passthrough: true`.

After that, the following `gitlab.rb` configurations must be applied:

```ruby
registry['enable'] = true
gitlab_rails['registry_enabled'] = true
registry['token_realm'] = "https://" + ENV['GITLAB_HOST']
gitlab_rails['registry_enabled'] = true
gitlab_rails['registry_host'] = ENV['GITLAB_REGISTRY_HOST']
gitlab_rails['registry_api_url'] = "http://localhost:5000"

registry_external_url 'https://' + ENV['GITLAB_REGISTRY_HOST']
registry_nginx['redirect_http_to_https'] = true
registry_nginx['listen_port'] = 5443
registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/tls.crt"
registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/tls.key"
registry_nginx['real_ip_trusted_addresses'] = ['10.0.0.0/8', '172.16.0.0/12', '192.168.0.0/16']
registry_nginx['server_names_hash_bucket_size'] = 128
```

Additionally, an S3 bucket (or compatible cloud storage endpoint can be configured so all image layers will not be stored inside a volume, but in a scalable S3


### GitLab on Kubernetes: Sidekiq, Gitaly & Puma configuration

Puma (HTTP server for ruby), Sidekiq (job scheduler) and Gitaly (GitLabs Git bridge) can all be started with a custom amount of workers / threads. The best configuration heavily depends on your use-case and the amount of users that need to be supported. A reasonable minimal configuration would be:


```ruby
puma['worker_processes'] = 2
sidekiq['max_concurrency'] = 9
```

### GitLab on Kubernetes: Prometheus Metrics

The GitLab omnibus package ships with its own prometheus instance inside the image. As a prometheus instance is already present in most Kubernetes clusters the included prometheus can safely be disabled.


```ruby
prometheus_monitoring['enable'] = false
```

A [`ServiceMonitor`](https://prometheus-operator.dev/docs/operator/design/#servicemonitor) can be used to tell a running Prometheus instance to monitor the exposed metrics endpoints of GitLabs component on the following ports:


- 8082 (sidekiq)
- 9168 (gitlab)
- 9236 (gitaly)

### GitLab on Kubernetes: Backups

Well, backing up GitLab on Kubernetes could be tech guide on its own. The easiest way would be to back up the complete namespace using a backup solution like [Velero](https://velero.io/).


## Glasskube GitLab Kubernetes Operator

Meet the Glasskube GitLab [Kubernetes Operator](https://glasskube.eu/en/r/glossary/kubernetes-operator/) - a product of our developer team's frustration with the tedious configuration process, time-consuming setup, and constant troubleshooting associated with GitLab deployment. In response to these challenges, we crafted a solution that simplifies the entire experience. The Glasskube GitLab Kubernetes Operator deploys a fully configured GitLab instance with all features neatly abstracted through a Custom Resource Definition (CRD). Thanks to the Operator, the days of spending excessive time on setup and updates are over. Now, you can effortlessly spin up new GitLab instances within a mere 5 minutes. Try it out and give us feedback!

### Install the Glasskube operator

The first step is to install Glasskube via its helm chart:

```
helm repo add glasskube https://charts.glasskube.eu/
helm repo update
helm install my-glasskube-operator glasskube/glasskube-operator
```

The full documentation can be found in our [Getting started](https://glasskube.eu/docs/getting-started/install/) documentation.

### Deploy GitLab

> **Important**
> A DNS entry must be set on the `LoadBalancer` or the `Ingress Host`. SSL certificates are
> automatically generated by `LoadBalancer` or `cert-manager` if a `ClusterIssuer` is configured.

**gitlab.yaml**

```yaml
apiVersion: "v1"
kind: "Secret"
metadata:
  name: "gitlab-smtp"
stringData:
  username: "..."
  password: "..."
---
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-registry-secret
stringData:
  accessKey: "..."
  secretKey: "..."
---
apiVersion: "glasskube.eu/v1alpha1"
kind: "Gitlab"
metadata:
  name: "gitlab"
spec:
  host: "gitlab.mycompany.eu"
  sshEnabled: true
  sshHost: "ssh.gitlab.mycompany.eu"
  smtp:
    host: "..."
    port: 465
    fromAddress: "noreply+gitlab@mycompany.eu"
    authSecret:
      name: "gitlab-smtp"
    tlsEnabled: true
  registry:
    host: "registry.gitlab.mycompany.eu"
    storage:
      s3:
        bucket: gitlab-registry
        accessKeySecret:
          name: gitlab-registry-secret
          key: accessKey
        secretKeySecret:
          name: gitlab-registry-secret
          key: secretKey
        region: ...
```

```shell
kubectl apply -f gitlab.yaml
```

<img width="25%" style="width:25%" src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExa2RldHpiYnMzOWdlZTgwdWtqOHhvN3N3eG9yamI0NjVyd2l3eTdlYSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/YRhUem7n2UaF9EK2PH/giphy.gif">
That's it!
The complete custom resource documentation can be found in the Glasskube documentation
about [GitLab](https://glasskube.eu/docs/crd-reference/gitlab/)

### Deploy GitLab Runner on Kubernetes
<img width="25%" style="width:25%" src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExZDJsN2x6NTI0dWdhZW55MjBzMjFobHdtbDRtaHEycXlid2t6Z2ttZiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/945jGDodvZCDe/giphy.gif">

Runners on GitLab are execution agents that power continuous integration and continuous delivery (CI/CD) pipelines.
They're responsible for running jobs, which are the individual steps or tasks within a pipeline.

The Glasskube Gitlab Kubernetes Operator makes it as simple as it gets to add runners to your Gitlab. First a new runner needs to be created via `https://{{host}}/admin/runners/new`. After that these runner tokens can just be added to `gitlab.yaml` spec and the Glasskube Kubernetes Operator will automatically spawn and connect these runners with the Gitlab instance.


```yaml
  runners:
    - token: glrt-xxxxXX-xxxxxXXXXX # can be generated at https://{{host}}/admin/runners/new
```

The complete custom resource documentation can be found in the Glasskube documentation about [GitLab runner](https://glasskube.eu/docs/crd-reference/gitlab/runner/)

Now the installation is complete. Updates to the Kubernetes operator will automatically update GitLab.

## Conclusion

Deploying a GitLab instance on Kubernetes for less than a thousand users shouldn't be done with GitLabs cloud native hybrid helm charts but rather with a special purposed installation of the GitLab Omnibus package in combination with cloud native infrastructure. The Glasskube Kubernetes operator takes care of that.

---

[![Glasskube Github](
https://cms.glasskube.eu/uploads/CTA_51bbe3bb2a.png)
](https://github.com/glasskube/glasskube)

---

Star Glasskube:

# [`glasskube/glasskube`](https://github.com/glasskube/glasskube)