Private CI using private docker registry with Drone.io (almost free) 
Tags: docker, drone, rails, testing
Published: 2019-08-05 21:48:57

In previous post I have described how you can set up your own private docker registry for your secret projects, now we want to add CI to this setup.


Currently I host both on single 5$ VPS, which is decent price, considering amount I would need to pay DockerHub for same number of private docker images and for private TravisCI which costs 69$/month for 1 Concurrent job, which they advertise as "IDEAL FOR HOBBY PROJECTS", well not for 69$ for sure.




# Prerequisites
Consider you have a domain `example.com`, and want to have CI on `drone.example.com` and private registry on `registry.example.com`

# Setting up Drone

Drone requires that you use some external authentication provider, which is actually convenient, I would use GitHub as authentication provider. Also this way drone will import my private repositories on GitHub.

You should check official docs on how to get required credentials if you want to use GitHub as well:
https://docs.drone.io/installation/github/single-machine/

The official doc shows only example how to start Drone with `docker run`, but I prefer to put it in the `docker-compose.yml` file, so it is easier to start/restart.

So the combined `docker-compose.yml`, including Drone and private docker registry would look like:

```yml
version: "3"
services:
  drone:
    image: drone/drone
    ports:
      - "1080:80"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/drone:/data
    environment:
      - DRONE_USER_FILTER=your_user
      - DRONE_GITHUB_SERVER=https://github.com
      - DRONE_GITHUB_CLIENT_ID=<your id>
      - DRONE_GITHUB_CLIENT_SECRET=<your secret>
      - DRONE_RUNNER_CAPACITY=2
      - DRONE_SERVER_HOST=drone.example.com
      - DRONE_SERVER_PROTO=http
      - DRONE_TLS_AUTOCERT=true
  registry:
    restart: always
    image: 'registry:2'
    ports:
      - "5000:5000"
    environment:
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
    volumes:
      - /root/docker-volumes/registry/registry:/var/lib/registry
      - /root/docker-volumes/auth:/auth
```

Note important environment variable:
`DRONE_USER_FILTER` - only this user will be able to use your drone, you should limit it to your user so no other person can use your Drone.
`DRONE_SERVER_HOST` - the domain you have used to create OAuth authentication


# Running specs

In order to make Drone run specs for your repo you need to add special file to your project `.drone.yml`

```yml
kind: pipeline
name: default

steps:
- name: test
  image: registry.example.com/rails_api
  commands:
  - bundle exec rspec
  environment:
    RAILS_ENV: 'test'
    DATABASE_CLEANER_ALLOW_REMOTE_DATABASE_URL: 'true'
    REDIS_URL: redis://redis:6379/
    DATABASE_URL: postgres://postgres:@db/ourcodestyle_test
services:
- name: db
  image: 'postgres:11.1-alpine'
  ports:
  - 5432
  environment:
    POSTGRES_USER: postgres
    POSTGRES_DB: rails_api_test
- name: redis
  image: 'redis:3.2-alpine'
  ports:
  - 6379

image_pull_secrets:
- dockerconfigjson

```

There is several interesting parts in this file:

`image: registry.example.com/rails_api`
This way you specify which private docker image you would want your build to start. I have this image with all ruby gems already installed, so it dramatically reduces time before tests start to run.

Also note `services` section. Drone configuration mimics docker compose syntax so it is easy to configure.

The most important config line is the last one:

```yml
image_pull_secrets:
- dockerconfigjson
```

And it is not well described in the official docs. This setting says how Drone should get credentials for you private repository. You need to have a file `/root/.docker/config.json` on the machine where you run Drone. I did not have this file in place. You can create this file by yourself, or there is much easier way, just run: 

```shell
docker login registry.example.com -u testuser
```
 
on the machine which would run Drone and it would create that file.


# Domain for Drone

In order to share same server, your drone should have its subdomain, and also it is convenient and nice.

It can be done by adding nginx configuration like this:

```nginx
upstream drone {
  server 127.0.0.1:1080;
}

server {
    server_name drone.example.com;

    access_log /var/log/nginx/drone.example.com.access.log combined;
    error_log  /var/log/nginx/drone.example.com.error.log crit;

    server_name_in_redirect off;

    location / {
      proxy_pass                          http://drone;
      proxy_set_header  Host              $http_host;   # required for docker client's sake
      proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_read_timeout                  900;
    }

    listen 80;
}

```

Then don't forget to run CertBot in order to get `https`
```shell
certbot --nginx -d drone.example.com
```