Waiting for Docker Compose services in CircleCI - some options
Published: 2020-05-31 04:12:53

I've used [CircleCI](https://circleci.com/) a fair bit over the past couple of years - I love how you can run the tests locally before you push - it's a real safety net (although it still doesn't guarantee I didn't do something silly).

Because my builds tend to be [Docker Compose](https://docs.docker.com/compose/) stacks, there's usually a bit of service ordering to do - or more precisely, service waiting.  You can't install into a database that isn't ready yet.

Jason Wilder's [Dockerize](https://github.com/jwilder/dockerize) has always been the go-to for ensuring that correct order is restored.

My first exposure to this involved including dockerize into a docker image used specifically to build out the tests - it can then be easily called via docker-compose
```
      - run:
          name: check if mariadb is responding
          command: docker-compose exec test dockerize -wait tcp://mariadb:3306 -timeout 1
```

This is a great first step, but surely we shouldn't need to add a binary to an image so we can check if a different container is up and running?

In CircleCI, the most common [pattern](https://github.com/search?q=dockerize+in%3Afile+path%3A.circleci) is actually to install it into the CircleCI environment every time
```
      - run:
          name: Install dockerize
          command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
          environment:
            DOCKERIZE_VERSION: v0.6.1
      - run:
          name: Wait for db
          command: dockerize -wait tcp://HOST:PORT -timeout 1m
```

This works (why else would hundreds of sites use it?), and indeed, now CircleCI is onto it, and now [includes dockerize](https://circleci.com/docs/2.0/circleci-images/#pre-installed-tools) in their provided base images alongside docker and docker-compose - nifty huh!
```
      - run:
          name: Wait for db
          command: dockerize -wait tcp://HOST:PORT -timeout 1m
```

However, both these CircleCI based solutions have one catch when used with docker-compose - trying to get to the database endpoint to check it - because we use `setup_remote_docker`, it's abstracted away from the container building the images.

In my case, the mariadb exposes its 3306 port to a 32xxx one, but only on the bridge network created by docker-compose.  We can still use this to get to the port, but it's _complex_ to achieve, even locally (I've installed dockerize here from homebrew to demo).
```
toby@pop-os:~/sites/example$ docker-compose port mariadb 3306
0.0.0.0:32781
toby@pop-os:~/sites/example$ docker network inspect example_default | grep Gateway
                    "Gateway": "192.168.144.1"
toby@pop-os:~/sites/example$ dockerize -wait tcp://192.168.144.1:32781 -timeout 1m
2020/05/31 02:43:15 Waiting for: tcp://192.168.144.1:32781
2020/05/31 02:43:15 Connected to tcp://192.168.144.1:32781
2020/05/31 02:43:15 Command finished successfully.
```

Victory - now all we need to do is artisan code some bash script to retrieve the port, find the network gateway and craft a command to dockerize, make a cup of tea, and sit back in satisfaction (which will probably last until you realise that CircleCI generates a random name for the network it creates...)

But wait - surely there's a better, more Docker-ish way?

There's three key pieces of information to this puzzle:
1. Docker containers can be used to run single purpose applications (d'oh)
2. Those standalone containers can be joined to a named docker-compose network
3. You can control the name of CircleCI's (and docker-compose) default network - using `COMPOSE_PROJECT_NAME` as an [environment variable](https://docs.docker.com/compose/reference/envvars/#compose_project_name)

So, if we know what the network is called, surely we can join a container to that network, run dockerize (or something else...) inside that network, then dispose of it?

We make sure to add the COMPOSE_PROJECT_NAME env var to the CircleCI config (this has the benefit of naming your stack logically - which also makes it easier to control the stack independently)
```
  docker-compose:
    docker:
      - image: circleci/php:cli
        user: root
        environment:
          COMPOSE_PROJECT_NAME: circleci
...
      - run:
          name: Check mariadb is responsive with dockerize
          command: docker run --rm --net circleci_default jwilder/dockerize dockerize -wait tcp://mariadb:3306 -timeout 1m
```

Dockerize is available on [dockerhub](https://hub.docker.com/r/jwilder/dockerize/) as an image - so we can run it (and remove afterwards), attach it to the predictably named network, and use the docker-compose service name and internal port.
```
toby@pop-os:~/sites/example$ docker run --rm --net example_default jwilder/dockerize dockerize -wait tcp://mariadb:3306 -timeout 1m
2020/05/31 03:54:58 Waiting for: tcp://mariadb:3306
2020/05/31 03:54:58 Problem with dial: dial tcp: lookup mariadb on 127.0.0.11:53: no such host. Sleeping 1s
2020/05/31 03:54:59 Problem with dial: dial tcp: lookup mariadb on 127.0.0.11:53: no such host. Sleeping 1s
2020/05/31 03:55:00 Problem with dial: dial tcp: lookup mariadb on 127.0.0.11:53: no such host. Sleeping 1s
2020/05/31 03:55:01 Connected to tcp://mariadb:3306
2020/05/31 03:55:01 Command finished successfully.
```

There is another image out there that does a similar thing - [dokku/wait](https://hub.docker.com/r/dokku/wait/) - their documentation recommends the use of `--link` to join it to running containers, but given --link is possibly [being deprecated](https://docs.docker.com/network/links/), we can use the same `--net` strategy as before:
```
toby@pop-os:~/sites/example-$ docker run --net example_default --rm dokku/wait -c mariadb:3306
Waiting for mariadb:3306  .nc: getaddrinfo: Name does not resolve
.nc: getaddrinfo: Name does not resolve
.nc: getaddrinfo: Name does not resolve
.nc: getaddrinfo: Name does not resolve
..  up!
Everything is up
```

Both these packages allow for multiple endpoints to be checked - so for more complex stacks, you can run multiple stages if needed.

I personally really like using the COMPOSE_PROJECT_NAME in my CircleCI projects - given that I like to run them up locally, I also want to be able to control and view them once running - you can use the `-p` switch to docker-compose to specify a different project to the folder you're currently in - e.g. to clear it all away once you're done!

```
toby@pop-os:~/sites/example$ docker-compose -p circleci down -v --remove-orphans
Stopping circleci_php_1     ... done
Stopping circleci_mariadb_1 ... done
Removing circleci_php_1     ... done
Removing circleci_mariadb_1 ... done
Removing network circleci_default
Removing volume circleci_files
```

Hopefully you've found this useful - please let me know if I've missed any obvious options (I've deliberately not veered into log tailing here!)