Improving our github actions runner image
Tags: github, docker, bash
Published: 2020-02-02 16:15:17

---
title: Improving our github actions runner image
published: true
description: 
tags: github, docker, bash
series: Ephemeral Github Actions
---

Table of Contents
=================

   * [Table of Contents](#table-of-contents)
   * [Undocumented parameters for config.sh](#undocumented-parameters-for-configsh)
      * [Diving into the source](#diving-into-the-source)
   * [Automated token retrieval](#automated-token-retrieval)
      * [Generating a personal access token](#generating-a-personal-access-token)
      * [Automatically generating a registration token](#automatically-generating-a-registration-token)
   * [Automated runner removal](#automated-runner-removal)
   * [Conclusion](#conclusion)

In the last post we built a docker image which can launch an actions runner. The image definition up to the end of that post is [here](https://github.com/wayofthepie/ephemeral-actions-blog/tree/dd7929733d9eca77a3791773e787d15c2ce99d81).

There are a few issues with the setup in that image. 

* We have a few hacks in the `entrypoint.sh` script for configuring the runner - e.g. passing the name via an `echo`.
* Anytime you want to start a new runner you need to get the latest token from the UI.
* You need to manually unregister runners which you want to remove.


I'm going to tackle these three issues now. To help, we should dive into the actions runner source code.

# Undocumented parameters for config.sh
I was pretty sure the help section for `config.sh` was missing some flags. All it prints is:

```bash
$ ./config.sh  --help

Commands:,
 ./config.sh          Configures the runner
 ./config.sh remove   Unconfigures the runner
 ./run.sh             Runs the runner interactively. Does not require any options.

Options:
 --version  Prints the runner version
 --commit   Prints the runner commit
 --help     Prints the help for each command
```

To configure a runner we need the following four things at least:

* The url to the repo to register the runner against, this has a `--url` option.
* The token this has a `--token` option.
* The name of the runner, we cheat in our [entrypoint.sh](https://github.com/wayofthepie/ephemeral-actions-blog/blob/dd7929733d9eca77a3791773e787d15c2ce99d81/entrypoint.sh#L8) by echoing a name into the execution of `config.sh`. This sets the first prompt as the value we echo.
* The work directory, here we also cheat with `echo` and because we echo a single value this defaults to `_work`. 

`./config.sh` calls out to a `.net` binary called Runner.Listener, the source code can be found [here](https://github.com/actions/runner/tree/a0a590fb48fa2db92dacdef5d9b94294ab7940b4). What can we find in this?

## Diving into the source
Our aim is to find any flags which are not documented in `./config --help` that may help us automate configuration. The [source code](https://github.com/actions/runner/tree/a0a590fb48fa2db92dacdef5d9b94294ab7940b4) is large enough that blindly searching through would be tedious, so what information do we have that can narrow down our search?

When we run `config.sh` it defaults the work folder to *_work*:

```bash
$ ./config.sh --url https://github.com/${OWNER}/${REPO} --token ${TOKEN}

...
Enter name of work folder: [press Enter for _work]
...
```

So, their must be a string in the code with the value `_work`. If we search in the [actions-runner repo](https://github.com/actions/runner/tree/a0a590fb48fa2db92dacdef5d9b94294ab7940b4), we find it in a few places. The most promising is [Runner.Common Constants.cs](https://github.com/actions/runner/blob/c59c0e2ded040f151ccd8845856def6b4cd7b58a/src/Runner.Common/Constants.cs#L174). 

This file contains quite a few interesting constants which may be of use later on! For our current goal, there is the [CommandLine Args](https://github.com/actions/runner/blob/c59c0e2ded040f151ccd8845856def6b4cd7b58a/src/Runner.Common/Constants.cs#L85-L105) class.

```csharp
public static class Args
{
    public static readonly string Auth = "auth";
    public static readonly string MonitorSocketAddress = "monitorsocketaddress";
    public static readonly string Name = "name";
    public static readonly string Pool = "pool";
    public static readonly string StartupType = "startuptype";
    public static readonly string Url = "url";
    public static readonly string UserName = "username";
    public static readonly string WindowsLogonAccount = "windowslogonaccount";
    public static readonly string Work = "work";

    // Secret args. Must be added to the "Secrets" getter as well.
    public static readonly string Token = "token";
    public static readonly string WindowsLogonPassword = "windowslogonpassword";
    public static string[] Secrets => new[]
    {
        Token,
        WindowsLogonPassword,
    };
}
```

This looks promising. It mentions two args we already use - `--token` and `--url`. Let's see if we can set name and the work dir similarly:

```bash
$ ./config.sh --url https://github.com/${OWNER}/${REPO} \
    --token ${TOKEN} \
    --name my-runner \
    --work _work

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication


√ Connected to GitHub

# Runner Registration


√ Runner successfully added
√ Runner connection is good

# Runner settings


√ Settings Saved.
```

It works! Now there are no prompts. We can update our docker image's `entrypoint.sh` from the previous post to take this into account:


```bash
#!/usr/bin/env bash

OWNER=$1
REPO=$2
TOKEN=$3
NAME=$4

./config.sh \
    --url https://github.com/${OWNER}/${REPO} \
    --token ${TOKEN} \
    --name ${NAME} \
    --work _work

./run.sh
```

See [here](https://github.com/wayofthepie/ephemeral-actions-blog/tree/5fa9582a97bba3be000ece0cc3da2eb185462133) for the full code up to this point including the Dockerfile.

# Automated token retrieval
Currently anytime we want to run a new action we need to login to the UI and retrieve a new token. Recently github release an API to generate this token, see the [self hosted runners API docs](https://developer.github.com/v3/actions/self_hosted_runners/). We can take advantage of this in our `entrypoint.sh` script.

## Generating a personal access token
To generate an actions registration token we first need to generate a Personal Access Token that we can use to call the github API. I'll refer to these as PAT's from here on in so as not to confuse them with the registration token, as they are both tokens of a different type!

To generate a PAT, go to *Settings* in your account:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/q3smyupy13mjry777922.png)

Then *Developer Settings* in the left panel:

 ![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/veozmpanpvojt78e8m3r.png)

Go to *Personal Access Tokens*:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9psv0e5dncl4a9hwf93s.png)

Finally click the *Generate a new token* button:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/3oqextxeht3kwary1fkh.png)

Give your token a name that associates it with what it's doing. I've called it `actions-runner-registration`. I am not 100% sure the exact scopes that it needs, setting all `repo` scopes works however so we can do this for now. 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/pdantf1ziferdb06xelu.png)

Scroll to the bottom and click *Generate Token*.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ujlxpha8du0vipz9tkhl.png)

Make sure to copy the token on the next page and store it in a safe place! This is the only chance you get to see it, and if you do not you will have to generate a new one.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/is377cgtord3k1gkdzgw.png)

## Automatically generating a registration token
Now we have our PAT, we can use the API to generate an actions runner registration token. The docs for the different methods of auth with the github API can be found [here](https://developer.github.com/v3/auth/#authenticating-for-saml-sso). Let's keep his simple and use `curl`.

```bash
$ curl -XPOST \
    -H "authorization: token ${YOUR_PAT}" \ # ${YOUR_PAT} is the PAT we generated above
    https://api.github.com/repos/${OWNER}/${REPO}/actions/runners/registration-token
{
  "token": "*******",
  "expires_at": "2020-02-02T16:21:21.410+00:00"
}
```

It works! With this we can update our `entrypoint.sh` script again.

```bash
#!/usr/bin/env bash

OWNER=$1
REPO=$2
PAT=$3
NAME=$4

token=$(curl -s -XPOST \
    -H "authorization: token ${PAT}" \
    https://api.github.com/repos/wayofthepie/gh-app-test/actions/runners/registration-token |\
    jq -r .token)

./config.sh \
    --url https://github.com/${OWNER}/${REPO} \
    --token ${token} \
    --name ${NAME} \
    --work _work

./run.sh
```

If you have never used the `jq` command before, `jq .token` means take the token field out of the given json. The json returned from the `curl` call has two fields, token and expires_at. If we used `jq` without the `-r` flag it would return the token wrapped in quotes, we want it without quotes. So the full command is `jq -r .token`. 

We need to use `curl` and [jq](https://github.com/stedolan/jq) now, neither of these are in our docker image so we need to also update our *Dockerfile*.

```docker
FROM ubuntu

ENV RUNNER_VERSION=2.164.0

RUN useradd -m actions \
    && apt-get update \
    && apt-get install -y \
    wget \
    # add curl and jq
    curl \
    jq

RUN cd /home/actions && mkdir actions-runner && cd actions-runner \
    && wget https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
    && tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
WORKDIR /home/actions/actions-runner

RUN chown -R actions ~actions && /home/actions/actions-runner/bin/installdependencies.sh

USER actions

COPY entrypoint.sh .
ENTRYPOINT ["./entrypoint.sh"]
```

The source for the image up to this point can be found [here](https://github.com/wayofthepie/ephemeral-actions-blog/tree/bbf1478e158c03eba4ac0cff1e88f21c0869296c).

Let's build it and run, see if it works:

```bash
$ docker build -t actions-image .
Sending build context to Docker daemon  81.92kB
Step 1/9 : FROM ubuntu
 ---> 775349758637
...
Step 9/9 : ENTRYPOINT ["./entrypoint.sh"]
 ---> Using cache
 ---> 0dff3c9bac89
Successfully built 0dff3c9bac89
Successfully tagged actions-image:latest

$ docker run -ti --rm actions-image ${OWNER} ${REPO} ${PAT} my-runner

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication


√ Connected to GitHub

# Runner Registration


A runner exists with the same name
Would you like to replace the existing runner? (Y/N) [press Enter for N]
```

Ah an issue! We registered a runner called *my-runner* previously in this post and never removed it. We can manually remove for now and try again:

```bash
$ docker run -ti --rm actions-image ${OWNER} ${REPO} ${PAT} my-runner

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication


√ Connected to GitHub

# Runner Registration


√ Runner successfully added
√ Runner connection is good

# Runner settings


√ Settings Saved.


√ Connected to GitHub

2020-02-02 15:42:22Z: Listening for Jobs
```

It works! Now we don't have to generate a token manually each time we want to start a new runner. One major issue remains, the fact you have to manually remove runners.

# Automated runner removal
When we kill the container running an actions runner it should automatically unregister itself. To manually unregister we use the `./config remove` command. Let's add some small changes to our entrypoint so it will do this automatically.

```bash
#!/usr/bin/env bash

OWNER=$1
REPO=$2
PAT=$3
NAME=$4

cleanup() {
    token=$(curl -s -XPOST -H "authorization: token ${PAT}" \
        https://api.github.com/repos/${OWNER}/${REPO}/actions/runners/registration-token |\
        jq -r .token)
    ./config.sh remove --token $token
}

token=$(curl -s -XPOST \
    -H "authorization: token ${PAT}" \
    https://api.github.com/repos/wayofthepie/gh-app-test/actions/runners/registration-token |\
    jq -r .token)

./config.sh \
    --url https://github.com/${OWNER}/${REPO} \
    --token ${token} \
    --name ${NAME} \
    --work _work

./run.sh

cleanup
```

The cleanup function is run at the end of the script, it gets a new registration token and called `./config.sh remove --token ${TOKEN}` to unregister the runner. The code to this point can be found [here](https://github.com/wayofthepie/ephemeral-actions-blog/tree/31fa3a14dcd0e0b8c7c270ff21cce4d8c9a439d4). Does it all work?

```bash
$ docker build -t actions-image .
Sending build context to Docker daemon  88.06kB
Step 1/9 : FROM ubuntu
 ---> 775349758637
...
Successfully built 1de80d7b74d6
Successfully tagged actions-image:latest

$ docker run -ti --rm actions-image ${OWNER} ${REPO} ${PAT} my-runner

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication


√ Connected to GitHub

# Runner Registration


√ Runner successfully added
√ Runner connection is good

# Runner settings


√ Settings Saved.


√ Connected to GitHub

2020-02-02 16:05:31Z: Listening for Jobs
^CExiting...

# Runner removal


√ Runner removed successfully
√ Removed .credentials
√ Removed .runner
```
I used CTRL-C to cancel and it ran the cleanup! 

# Conclusion
We can now easily register multiple actions runners on the fly and have the automatically clean up when we are finished with them. However, we still need to launch them manually. What if we could launch a runner per commit, have them run a single build, and disappear? That's the dream. We'll star working towards that in the next post.