Docker Compose update on GitHub release webhook
Tags: docker, automation, devops, git
Published: 2019-09-02 17:46:52

---
title: Docker Compose update on GitHub release webhook
published: true
description: Simple use case - pulling and deploying latest docker-compose images on tagged release
tags: docker, automation, devops, git
cover: https://thepracticaldev.s3.amazonaws.com/i/6w0sncnp3fck4ppuvhtj.png
---

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/6w0sncnp3fck4ppuvhtj.png)

Last year I wrote a blog post about [combining several tools](/blog/2018/07/17/auto-deploy-on-git-push) to automate simple NodeJS app updates on git push. Many users were solving similar problems by writing local web servers in Ruby, Python or PHP to receive webhooks and then do async processing. I am happy to announce that we finally decided to add this feature to the [relay CLI](/v1/installation/cli). Now, to execute a bash script, you can:

```shell
relay forward --bucket my-bucket-name --relayer exec --command bash update.sh
```

And to launch a Python app on webhook:

```shell
relay forward --bucket my-bucket-name --relayer exec --command python my-app.py
```

This opens up some interesting posibilities to create pipelines that can react to pretty much anything that emits webhooks. In this article I will show you how to build a GitOps style pipeline that keeps a Docker Compose deployment in sync with a docker-compose.yaml hosted on a git repository. 

<!-- more -->

**Prerequisites**

- Docker & [Docker Compose](https://docs.docker.com/compose/install/)
- [Webhook Relay account](https://my.webhookrelay.com/login) with configured [relay CLI](https://webhookrelay.com/v1/installation/cli)
- [Github account](https://github.com)

Repository with scripts that I used for this article can be found here: [https://github.com/webhookrelay/docker-compose-update-on-git-push](https://github.com/webhookrelay/docker-compose-update-on-git-push).

## Step 1: Deploying containers through Docker Compose

First step is to do the initial deployment. We will create a simple dockerized Python application that you can find [here](https://github.com/webhookrelay/docker-compose-update-on-git-push) that connects to Redis and deploy it:

```yaml
version: '3'
services:
  web:
    image: "karolisr/python-counter:0.1.0"
    ports:
      - "5000:5000"
  redis:
    image: "redis:alpine"
```

## Step 2: Setting up updates on Github tag

Since we only want to update on git tags and not just any pushes, let's configure a webhook and analyse the payload.

To achieve that, let's first create a bucket with an internal output:

```shell
$ relay forward --bucket docker-compose-update-on-git-push http://localhost:4000
Forwarding:
https://my.webhookrelay.com/v1/webhooks/a956a9f7-2260-4bc2-a54b-3d896acf4206 -> http://localhost:4000
Starting webhook relay agent...
2019-08-28 23:14:41.773	INFO	using standard transport...
2019-08-28 23:14:41.928	INFO	webhook relay ready...	{"host": "my.webhookrelay.com:8080", "buckets": ["8e977e70-09a6-464c-ad30-855e1cd5d9f9"]}
```

Here, bucket will be used later to subscribe to github requests while destination is just a mandatory argument that we don't have to use in this case.

Grab that https://my.webhookrelay.com/v1/webhooks/*** URL and go to your repository's settings -> webhooks section. Once there, set:

* Payload URL to your unique https://my.webhookrelay.com/v1/webhooks/*** URL
* Content type to *application/json*
* Secret to a random secret name, for the sake of this example my secret will be 'webhooksecret'
* Click `Let me select individual events.` and select *Releases*.

Now, go to your repository's releases page (for example https://github.com/webhookrelay/docker-compose-update-on-git-push/releases) and make a new release `1.0.0`. Then, if you visit bucket details page or [logs page](https://my.webhookrelay.com/logs) - you should see webhook from Github. Open it and let's inspect the payload. It's quite lengthy but we should be able to see 

```json
 "action": "released",
```

in the top. To ensure that we only react on these events, create a rule:

![release filter](https://thepracticaldev.s3.amazonaws.com/i/h81b4f6kj0p40gju4mbs.png)

If you tag another release, you should see now that only one webhook was forwarder

![rule stopped webhook](https://thepracticaldev.s3.amazonaws.com/i/vglu9whudofjj376jq84.png)

## Step 3: Update script and starting relay background service

Our update script is:

```bash
#!/bin/bash

git pull
docker-compose up -d
```

It will pull the latest compose file and update containers. Now, let's update configuration in `relay.yml` file ([access key & secret can be generated here](https://my.webhookrelay.com/tokens)):

```yaml
version: v1
key: xxx     # your access key
secret: xxx  # your access secret
buckets:
- docker-compose-update-on-git-push # your bucket name where github webhooks are sent
relayer:
  type: exec
  command: bash
  commandArgs:
  - /full/path/to/docker-compose-update-on-git-push/update.sh # <-- should be full path to your update script
  timeout: 300
```

To start the relay, run: 

```shell
relay run -c relay.yml
```

This will be running it through your terminal. For production use cases, please use [background service mode](/v1/installation/background-service). It will ensure that the daemon is launched on OS startup.

## Let's try it out

Launch docker-compose:

```shell
docker-compose up -d
```

Check containers:

```shell
$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                    NAMES
26cd2219e18b        redis:alpine                    "docker-entrypoint.s…"   20 seconds ago      Up 3 seconds        6379/tcp                 docker-compose-update-on-git-push_redis_1
63c8cd1ae7bb        karolisr/python-counter:0.1.0   "flask run"              20 seconds ago      Up 18 seconds       0.0.0.0:5000->5000/tcp   docker-compose-update-on-git-push_web_1
$ curl http://localhost:5000
I have been seen 1 times.
$ curl http://localhost:5000
I have been seen 2 times.
```

Next step would be to build a new image `0.2.0` and push it to the registry. Once it's available, we can update our github repository `docker-compose.yml` and make a new release. For the sake of this example, let's do this through the Github UI. 

In a few seconds you should see a new container running:

```shell
$ docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                    NAMES
27b2542423ec        karolisr/python-counter:0.2.0   "flask run"              9 seconds ago       Up 7 seconds        0.0.0.0:5000->5000/tcp   docker-compose-update-on-git-push_web_1
26cd2219e18b        redis:alpine                    "docker-entrypoint.s…"   10 minutes ago      Up 9 minutes        6379/tcp                 docker-compose-update-on-git-push_redis_1
```

## Optional: Validating Github secret

Webhook Relay output rules can also validate Github signature:

![webhook filter signature](https://thepracticaldev.s3.amazonaws.com/i/5k36auilqr3xk3yzn7ur.png)

This will ensure that only webhooks signed by Github will be processed.

## Alternative solutions

I have somewhat successfully used these tools to update containers:

- https://github.com/pyouroboros/ouroboros
- https://github.com/containrrr/watchtower

Both tools work if you have a simple setup that just needs to track `latest` tags. However, if you have multiple containers with semver (or any other versioning mechanism) - those solutions do not work. That's why it's useful to have an either .env or docker-compose.yml config with image tags committed into git and synchronized into the host that runs the services.

## Conclusion

As with any code executed on your machine - you have to be careful when automating tasks. Webhook Relay will provide you with a unidirectional flow of webhooks into the machine. Your script/applications are on your machine and cannot be modified remotely through Webhook Relay. Coupled with authenticated webhook endpoints (you can configure it on a bucket level) or webhook payload checksum validation - you can build a secure update mechanism.


*Originally released on https://webhookrelay.com/blog/2019/09/02/docker-compose-update-on-github-webhooks/*