Creating an s2i builder for Go (and a runtime image)
Tags: docker, devops, openshift, s2i
Published: 2020-07-23 01:03:37


This post is a continuation of:

{% post jromero/building-an-app-image-using-s2i-5gcp %}

# Objective

I will try to go through the process of creating an s2i builder for... `go`.

The goals of this builder are the following:
- The builder should compile `go` programs.
- The app image should not contain the source code.
- The app image should not contain the `go` compiler.
- The app image should not run as `root`.

---

![aye aye](https://media.giphy.com/media/l4FGkfXrTl9qxV24U/giphy.gif)

---

## App

First, we need an app to use as our test subject.

This is a simple http server app that uses [mux](https://github.com/gorilla/mux) for routing. This server could be written without the use of this library but I wanted to show how dependencies would be handled.

Let's look at the app contents...

```
$ tree test-app

test-app/
├── app.go    # app source
├── go.mod    # dependency declarations
└── go.sum    # dependency checksums
```

**app.go:**

```go
package main

import (
	"fmt"
	"log"
	"net/http"
	"os"

	"github.com/gorilla/mux"
)

const port = "8080"

func main() {
	log.Println("Starting app on port:", port)
	r := mux.NewRouter()
	r.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		_, _ = fmt.Fprintln(w, "This is a test app!")
	})
	r.HandleFunc("/host", func(w http.ResponseWriter, r *http.Request) {
		var name, _ = os.Hostname()
		_, _ = fmt.Fprintf(w, "This request was processed by host: %s\n", name)
	})
	r.HandleFunc("/hello/{object}", func(w http.ResponseWriter, r *http.Request) {
		vars := mux.Vars(r)
		_, _ = fmt.Fprintf(w, "Hello %v!\n", vars["object"])
	})
	http.Handle("/", r)
	if err := http.ListenAndServe(":"+port, nil); err != nil {
		log.Fatal(err)
	}
	log.Println("Shutting down")
}

```

**go.mod:**

```
module github.com/jromero/learning-s2i/s2i-golang/test/test-app

go 1.14

require github.com/gorilla/mux v1.7.4
```

**go.sum:**

```
github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
```

## Builder

The builder as we learned [previously](https://dev.to/jromero/building-an-app-image-using-s2i-5gcp#builder) is an image that knows how to build a specific type of application. In this case, we will write a builder for a `go` app.

In order to do this we'll need the following files:

```
$ tree builder/

builder/
├── Dockerfile               # docker instructions to create builder
└── s2i
    └── bin
        ├── assemble         # script to build application
        ├── run              # script to run application
        ├── save-artifacts   # script to package cached items
        └── usage            # script that displays usage
```

Let's go into more detail about each one...

### `s2i/bin/usage`

The `usage` script is very straight forward. We will use it as a way to assist users when they try to run the builder using `docker run`.

##### Contents

```bash
#!/bin/bash -e
cat <<EOF
This is the s2i-golang S2I image:
To use it, install S2I: https://github.com/openshift/source-to-image

Sample invocation:

s2i build <source code path/URL> s2i-golang <application image>

You can then run the resulting image via:
docker run <application image>
EOF
```

### `s2i/bin/save-artifacts`

The `save-artifacts` script is a special script that is called as part of the build process directly into a tar stream. We will use it to specify what data we want cached so that users can leverage `--incremental` builds.

In `go`, we care about caching the following directories:

* `${GOPATH}/src`
* `${GOPATH}/pkg`

> **Important Note:** Due to the way this script is used, no output other than the cache contents should be present.

##### Contents

```bash
#!/bin/sh -e
pushd ${GOPATH} >/dev/null
if [ -d src ]; then
    chmod -R +w src
    tar cf - src
fi
if [ -d pkg ]; then
    chmod -R +w pkg
    tar cf - pkg
fi
popd >/dev/null
```

### `s2i/bin/assemble`

The `assemble` script is responsible for the following:

1. restoring cache
2. compiling code
3. relocating contents

##### Contents

```bash
# If the 's2i-golang' assemble script is executed with the '-h' flag, print the usage.
if [[ "$1" == "-h" ]]; then
	exec /usr/libexec/s2i/usage
fi

# Restore artifacts from the previous build (if they exist).
echo
echo "---> Checking for cache..."
if [ "$(ls /tmp/artifacts 2>/dev/null)" ]; then
  pushd /tmp/artifacts >/dev/null
  echo "-----> Pulling cache..."
  shopt -s dotglob
  if [ -d src ]; then
    echo "Restoring cache ${GOPATH}/src/..."
    mv src ${GOPATH}/src
  fi
  if [ -d pkg ]; then
    echo "Restoring cache ${GOPATH}/pkg/..."
    mv pkg ${GOPATH}/pkg 
  fi
  shopt -u dotglob
  popd >/dev/null
fi

# Compile app to final location
echo
echo "---> Building application from source..."
pushd /tmp/src/ >/dev/null
go build -o ${APP_ROOT}/bin/app
popd >/dev/null
```

### `s2i/bin/run`

The `run` script will be what is executed on the produced app image. In our case it will simply call the compiled executable.

##### Contents

```bash
#!/bin/sh -e
${APP_ROOT}/bin/app
```

### `Dockerfile`

Finally, we'll put it all together by composing our builder image using a standard `Dockerfile`.

##### Contents

```Dockerfile
# s2i-golang
FROM openshift/base-centos7

      # the maintainer
LABEL maintainer="Javier Romero <root@jromero.codes>" \
      # specify where s2i scripts are located
      io.openshift.s2i.scripts-url="image:///usr/libexec/s2i/bin"

# configuration
ARG GO_VERSION=1.14
ARG GO_INSTALL_DIR=/usr/local/

# env vars
ENV APP_ROOT=/opt/app-root
ENV BUILDER_VERSION 1.0
ENV GOPATH ${APP_ROOT}/src/go
ENV PATH=${PATH}:${GOPATH}/bin:${GO_INSTALL_DIR}/go/bin

# build dependencies
RUN curl -sSL https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o go${GO_VERSION}.linux-amd64.tar.gz && \
    tar -C ${GO_INSTALL_DIR} -xzf go${GO_VERSION}.linux-amd64.tar.gz && \
    rm -f go${GO_VERSION}.linux-amd64.tar.gz && \
    mkdir -p ${GOPATH}

# s2i scripts
COPY s2i /usr/libexec/s2i

# set permissions
RUN chown -R 1001:1001 ${APP_ROOT} && \
    chown -R 1001:1001 /usr/libexec/s2i

# default user
USER 1001

# default CMD for the image
CMD ["/usr/libexec/s2i/usage"]
```

> Note: user `1001` and group `1001` have been created by the base image `openshift/base-centos7`.

## Build!

Now that we've got all the pieces together let's test out our builder.

#### Build our builder image

Building the image using docker should be as easy as:

```shell
docker build -t s2i-golang ./builder
```

#### Build our app using the builder

To build our app we will use our builder:

```shell
$ s2i build --copy test-app/ s2i-golang my-go-app --incremental

---> Checking for cache...

---> Building application from source...
go: downloading github.com/gorilla/mux v1.7.4
Build completed successfully
```

> Note: Because we are caching `go` dependencies, if we re-ran the same build command (with `--incremental`) we should see that `mux` is not downloaded again.

#### Run our app

With the app image built, we can run it using `docker`:

```
docker run --rm -it -p 8080:8080 my-go-app
```

... and verify that it works by going to http://localhost:8080

```
$ curl -s http://localhost:8080

This is a test app!
```

## App Image

Let's take a look at what we've ended up with.

Checking what user we are running as yields `default`. 👍

```shell
$ docker run --rm my-go-app /bin/bash -c "whoami"

default
```

Our app ended up being **7.5MBs**.

```shell
$ docker run --rm my-go-app /bin/bash -c "ls -lh /opt/app-root/bin/app"

-rwxr-xr-x 1 default root 7.5M Jul 22 15:53 /opt/app-root/bin/app
```

And our image... is **725MBs**!!!

```shell
$ (docker images | head -1 && docker images | grep my-go-app) | awk '{print $1, $NF}'

REPOSITORY SIZE
my-go-app 725MB
```

If we [`dive`](https://github.com/wagoodman/dive) into our app image we can see that this is because of a few things:

1. The base image of the app image is the builder image which in itself has a lot of stuff our app doesn't care about.
2. The `go` compiler and build tools are present in the final app image.
3. All the intermittent artifacts such as cache and source code are on the final app image.

In addition to the size, all this _extra_ stuff has the additional negative side-effect of increasing our attack surface. 😈

![dive](https://dev-to-uploads.s3.amazonaws.com/i/nidclb2rnf87myprm9wz.png)

If we review our goals, it's clear that we haven't satisfied them all.

- ✅ The builder should compile `go` programs.
- ❌ The app image should not contain the source code.
- ❌ The app image should not contain the `go` compiler.
- ✅ The app image should not run as `root`.

---

![wat](https://media.giphy.com/media/3o7qDWoeN6PaJ6Z6RW/giphy.gif)

---

## Runtime Image

We can try to optimize our app image by using a `runtime` image to meet our initial goals.

A `runtime` image is an image that will be used _instead_ of the builder as the base for the app image.

In order to make this runtime image work with `s2i` we need the following:

```
$ tree runtime/

runtime/
├── Dockerfile                 # docker instructions to create image
└── s2i
    └── bin
        ├── assemble-runtime   # script to relocate assets
        └── run                # script to run our application
```

### `s2i/bin/run`

Just like the `run` script for the builder this script will simply execute our binary. In this case we are executing it the current working directory since that's where we'll be placing it.

```shell
#!/bin/sh -e
./app
```

### `s2i/bin/assemble-runtime`

The `assemble-runtime` is analogous to `assemble` but for the `runtime` image. In our case, there is nothing additional to do so we'll just `echo`. Unfortunately this script is required even if we don't actually need it.

```shell
#!/bin/sh -e
echo "Nothing special to do here..."
```

### `Dockerfile`

Lastly, but most importantly, we'll define our image with the right permissions on the minimal base image we need. In this case we are going to go with the base cento7 image.

```Dockerfile
# s2i-golang-runtime
FROM centos:7

      # the maintainer
LABEL maintainer="Javier Romero <root@jromero.codes>" \
      # specify where s2i scripts are located
      io.openshift.s2i.scripts-url="image:///usr/libexec/s2i/bin"

# s2i scripts
COPY s2i /usr/libexec/s2i

# create user/group
RUN groupadd -g 1001 app && \
    adduser -u 1001 -g 1001 default

# default user
USER 1001
```

## Build! (w Runtime Image)

Let's build our app again but this time providing a separate runtime image in an effort to improve the final app image.

#### Build our runtime image

Again, building the image using docker should be as easy as:

```shell
docker build -t s2i-golang-runtime ./runtime
```

#### Build our app using the builder and runtime

To build our app we will use our builder and newly build runtime image:

```shell
$ s2i build --copy test-app/ s2i-golang my-go-app --runtime-image s2i-golang-runtime --runtime-artifact /opt/app-root/bin/app

---> Checking for cache...

---> Building application from source...
go: downloading github.com/gorilla/mux v1.7.4
Build completed successfully
```

> A few things to note:
> 
> 1. We can no longer use `--incremental` 😞. See [issue #824](https://github.com/openshift/source-to-image/issues/824).
> 2. We must provide the location of what artifacts we want to "transfer" over to the runtime image. This requires the end user to know more than they should about the internals of the builder. 😥

#### Run our app

Now that we've built our app (again) we can run it:

```
docker run --rm -it -p 8080:8080 my-go-app
```

Cool it still works!

## App Image (Redux)

Let's hope we have a _better_ app image. 🤞

Checking the running user...

```shell
$ docker run --rm my-go-app /bin/bash -c "whoami"

default
```

Still good.

Image size?

```shell
$ (docker images | head -1 && docker images | grep my-go-app) | awk '{print $1, $NF}'

REPOSITORY SIZE
my-go-app 212MB
```

Nice! We've saved **>500MBs**!!! This is data that we don't have to store or transfer. 🙌

Plus, much less of an attack surface. 👿

A [`dive`](https://github.com/wagoodman/dive) into the image:

![dive](https://dev-to-uploads.s3.amazonaws.com/i/f9kgga5nmxcii75maba6.png)

This looks _much_ better!

... and we've met all our original goals.

- ✅ The builder should compile `go` programs.
- ✅ The app image should not contain the source code.
- ✅ The app image should not contain the `go` compiler.
- ✅ The app image should not run as `root`.

---

![awesome](https://media.giphy.com/media/3o7qDON1FYhi5eoqgU/giphy.gif)

---

## Conclusion

So what did we learn? 

1. Builders are fairly easy to construct and can be generally used with multiple applications (unlike pure `Dockerfiles`).
2. Built-in caching can be used to improve local development via `--incremental`.
3. `--incremental` does not currently work with `--runtime-image`.
4. To use `--runtime-image` you must have intimate knowledge about what artifacts will be produced.
5. We gain a performance and security boost from using a seperate runtime image.

If you'd like to look at the source used in this tutorial, you may find it here: https://github.com/jromero/learning-s2i

---

![cheers](https://media.giphy.com/media/3o6ozomjwcQJpdz5p6/giphy.gif)

---

#### Additional Reading

- [OpenShift "create s2i builder image" post](https://www.openshift.com/blog/create-s2i-builder-image)
- [Runtime Image docs](https://github.com/openshift/source-to-image/blob/master/docs/runtime_image.md)