Creating a Docker Image for a Twemproxy Server
Tags: googlecloud, docker, redis, twemproxy
Published: 2021-02-27 14:01:45

[Twemproxy](https://github.com/twitter/twemproxy) is an open source proxy server created by Twitter and is [one of the methods recommended by Google](https://cloud.google.com/blog/products/databases/running-redis-on-gcp-four-deployment-scenarios) for managing a cluster of Memorystore instances in GCP (Google Cloud Platform). [Memorystore](https://cloud.google.com/memorystore) is the managed service provided by GCP for application caching with either Redis or Memcached. Twemproxy provides a way to evenly distribute cached data between multiple Redis instances to improve the performance, reliability, and resilience of a distributed system.

![diagram of cloud architecture with Proxy server between backend services and redis instances](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i55jewq37h8s2cnfn5jd.png)

This blog post is the start of a series that will provide you with the foundational information you need to go from creating a Docker image of a Twemproxy server to deploying a managed instance group of containers created from that image to serve as a proxy for several instances of Memorystore in GCP.

The first step is to download a copy of the distribution tarball using the link provided by Twitter in the [Twemproxy](https://github.com/twitter/twemproxy) Github repo. The link can be found under the "Build" section. This example uses version 0.4.1 of Twemproxy, (also referred to as *nutcracker*).

Create a directory for your Twemproxy server project and place the `nutcracker-0.4.1.tar.gz` file in it along with an empty Dockerfile and an empty `nutcracker.yml` file. The YAML file is where you will write your configuration for the server.

Next, copy and paste the following code in your Dockerfile.

```dockerfile
FROM debian:buster as builder
​
COPY nutcracker-0.4.1.tar.gz /
​
RUN apt-get update && apt-get -qy install \
    automake \
    libtool \
    make \
    && rm -rf /var/lib/apt/lists/*
    
RUN tar xzf /nutcracker-0.4.1.tar.gz
RUN cd nutcracker-0.4.1 && CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full && make && make install
​
​
# Main
​
FROM debian:buster
​
COPY --from=builder /usr/local/sbin/nutcracker /usr/local/sbin/nutcracker
COPY nutcracker.yml /etc/nutcracker.yml
​
EXPOSE 6380
CMD ["nutcracker", "-c", "/etc/nutcracker.yml"]
```

There are two stages to this Dockerfile. The first stage installs the tools required to build Twemproxy, unzips the `nutcracker-0.4.1.tar.gz` file, and then uses the `make` tool to build it. 

The second stage copies over the nutcracker directory that was created by the build process as well as the configuration file, `nutcracker.yml`. Lastly, this stage defines the command that will start the server once the container is running. `nutcracker` is the name of the executable and the `-c` flag followed by `/etc/nutcracker.yml` directs the application to your customized configuration file. 

There are a lot of options for customizing the Twemproxy server based on your requirements, most of which can be defined in the configuration file. Other customization options, like changing the size of the mbufs used to ferry requests and responses, can be changed through optional parameters passed to the `nutcracker` command. Changes to logging must be done at the compilation step as illustrated in the Dockerfile with the `--enable-debug=full` parameter passed to the `./configure` command.

You can find a complete list of configuration options in the Twemproxy [README](https://github.com/twitter/twemproxy) file.

As a starting point, you can use the configuration below in your `nutcracker.yml` file and adjust as needed. You will need to make sure that the port in the  `listen` field matches the exposed port in your Dockerfile and you will need to change the IP addresses and port numbers listed under the `servers` property to match your Memorystore instances once they have been created.

```yaml
pool:
    listen: 0.0.0.0:6380
    hash: fnv1a_64
    distribution: ketama
    redis: true
    auto_eject_hosts: true
    timeout: 2000
    server_retry_timeout: 10000
    server_failure_limit: 3
    server_connections: 40
    preconnect: true
    servers:
        - 127.0.0.1:6379:1
        - 127.0.0.2:6379:1
        - 127.0.0.3:6379:1
```

To make sure the Dockerfile builds and that the image runs locally in a container, run the following commands from within the same directory as your Dockerfile.

```powershell
docker build . -t twemproxy-test
docker run --name twemproxy -it --rm twemproxy-test
```

You will see the following message once the server has successfully started running along with a couple of error messages due to the fact that the proxy is unable to connect with Redis instances that do not exist yet.

```
nutcracker-0.4.1 build for Linux 4.19.121-linuxkit x86_64 started on pid 1
run, rabbit run / dig that hole, forget the sun / and when at last the work is done / don't sit down / it's time to dig another one
```

In the next blog post we will use Terraform to define all of the resources required to deploy a managed instance group of Twemproxy servers and a cluster of three Memorystore instances. Once the cloud resources have been provisioned you will be able to replace the arbitrary server IP addresses and ports in the example configuration file above with actual address information for your deployed Redis instances.