Serverless Docker Patterns
Tags: docker, serverless, lambda
Published: 2020-10-24 21:42:35

---
series: Serverless & Containers
---

ℹ️ Please read our new post [New Amazon Linux Dev Container Features](https://dev.to/aws-heroes/new-amazon-linux-dev-container-features-5db9) for our latest patterns.

This is the first part in a series on how we use Docker for Rails on AWS Lambda at [Custom Ink](https://www.customink.com). After reading, drop a comment and share how you or your teams leverage Docker & Serverless together. Did you find any of these tips useful? Perhaps you use the [serverless](https://www.serverless.com) framework instead of the [AWS SAM CLI](https://aws.amazon.com/serverless/sam/) tooling? If so, how does that work for you?

This post will focus on three abstract patterns derived from our usage of the [docker-lambda](https://github.com/lambci/docker-lambda) images with AWS SAM for Rails projects. My hope is that you can apply these to your own specific Docker work.

  1. [Normalized Script Conventions](#normalized-script-conventions)
  2. [Cross-Platform SSH Agent for CI/CD](#crossplatform-ssh-agent-for-cicd)
  3. [Mac OS Filesystem Performance](#mac-os-filesystem-performance)

## Normalized Script Conventions

 Long before our adoption of Docker, every Custom Ink project required the [Scripts to Rule Them All](https://github.blog/2015-06-30-scripts-to-rule-them-all/) pattern which allows any engineer to get "up and running" with any project quickly. Our more modern Docker usage compliments this pattern with a few small changes that make sense for us:

* We use the `bin` directory vs the `script` directory.
* Because we use Docker/Compose, there is no need for `ci` specific scripts.
* Since we are primarily a Rails shop, we have a `bin/console` for easy [REPL](https://en.wikipedia.org/wiki/Read–eval–print_loop) access.
* We extended this pattern to include `bin/deploy` for cloud native projects like Lambda.
* A `bin/update` script that will tear down any previous resources then bootstrap & setup.

Because we make heavy use of [docker-compose](https://docs.docker.com/compose/) which helps us abstract out common environment variables, shared volumes, and services, all of these scripts will use `docker-compose run` vs `docker run`. This avoids duplicating CLI args across every `bin` script, but it also has the added benefit of making your usage of Docker mirror real local development. A few details and outcomes include:

1. Every `bin/${name}` script is a simple `docker-compose run bin/_${name}` wrapper. Avoids tons of compose args and escaping issues. Helps longer scripts like setup to be more maintainable.
2. We have essentially commoditized CI/CD. Long gone are the special "how do I get this project running" on TravisCI, CircleCI, or GitHub Actions. Any system with Docker & Git will "just work", promise!
3. No `ENTRYPOINT` or `CMD` means `bin/server` takes on the responsibility of passing ports and delegating to your projects native local server. This allows other bin script like running one off tasks or console access all avoid starting an superfluous server process.

Curious what this looks like? Our [Lamby Quick Start](https://lamby.custominktech.com/docs/quick_start) leverages an AWS Lambda cookiecutter project that adopts both these script conventions and docker-compose patterns. Give it a go and deploy a starter Rails application to Lambda or have a [look at the code](https://github.com/customink/lamby-cookiecutter/tree/master/%7B%7Bcookiecutter.project_name%7D%7D) if you want.

## Cross-Platform SSH Agent for CI/CD

Remember that promise I made on the commoditization of CI/CD because all local development & testing happens with Docker? There is one gotcha. Access to private Ruby gems or Node packages hosted on GitHub. A very common use case for us at Custom Ink and I suspect many large companies. 

For a long time we used GitHub access tokens. But this required special platform-specific tooling for both local development and our CI/CD pipelines. For Node this often required crazy hacks to the package.json file too. The internet is full of posts on how to solve this. All of them however speak to a specific platform or language's package manager. **Could there be a unified SSH pattern?** Yes! Here is what I found works. 

In your `docker-compose.yml` file under your application or service, add or merge these environment/volume settings below.

```yaml
environment:
  - SSH_AUTH_SOCK=${SSH_AUTH_SOCK}
volumes:
  - ${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK}
  - ~/.ssh/known_hosts:/root/.ssh/known_hosts
```

In order for it to work on both Linux & Mac, you must setup the `SSH_AUTH_SOCK` environment variable to the specific value required by OS X. Since your `bin/_setup` script will most likely be calling either bundle or yarn install, we add this bash condition right before our docker compose run command.

```bash
if [[ "$OSTYPE" == *"darwin"* ]]; then
  export SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock
fi

docker-compose run myapp ./bin/_setup
```

We love [GitHub Actions](https://technology.customink.com/blog/2019/09/02/from-travis-ci-to-github-actions/) at Custom Ink and leverage it for all of our AWS Lambda CI/CD pipelines. Thanks to [organization secrets](https://github.blog/changelog/2020-05-14-organization-secrets/) and great marketplace additions like [webfactory/ssh-agent](https://github.com/webfactory/ssh-agent), we found the above patterns worked with no fuss by adding this to our workflow before the setup step.

```yaml
- name: SSH Agent
  uses: webfactory/ssh-agent@v0.4.0
  with:
    ssh-private-key: ${{ secrets.MYORG_GITHUB_SSH_KEY }}
```

Remember, that this pattern should work for any language's package manager that can leverage SSH access to private GitHub packages. If you are using Ruby's bundler, make sure your use this github source format in your `Gemfile`.

```ruby
git_source(:github) { |repo| "git@github.com:#{repo}.git" }
```

For Node's yarn (maybe npm too), your dependencies would look like this in `package.json`.

```json
"image_changer": "github:myorg/myproject.git"
```

## Mac OS Filesystem Performance

I never like to speak poorly of anyone's or company's work. But I think we can all admit that Docker Desktop for Mac with large projects has downright horrible filesystem performance. It has been this way for years and as such the internet is littered with solutions and hacks.

Part of me would like to believe that a [Twitter ❤️](https://twitter.com/metaskills/status/1316820193268715520/likes) by the VP of Product for Docker might mean they can be trusted to reverse course on trying to, [yet again](https://github.com/docker/roadmap/issues/7), write their own file system. We can all hope. But in the meantime we need Docker unlocked for development for our predominantly MacBook driven engineering teams. But how can we do this with the following value questions: 

1. Embrace hope and plan for the [spacklepunch](https://faasandfurious.com/122)?
2. Be minimally invasive and easy to delete? 
3. Work cross-platform from local development to CI/CD pipelines?

There are tons of solutions out there from [Docker Machine](https://github.com/docker/machine) to [dinghy](https://github.com/codekitchen/dinghy). After some careful research I found [docker-sync](http://docker-sync.io) was the easiest to adopt while addressing the concerns above. Here is how we added docker-sync to both our Lambda & Container Rails projects. 

First, we created or added this line to our projects `.env` file which leverages [compose's overlay](https://docker-sync.readthedocs.io/en/latest/getting-started/configuration.html#docker-compose-yml) capability. By doing this we ensure every bin script that uses `docker-compose run` remains untouched.

```bash
COMPOSE_FILE=docker-compose.yml:docker-compose-dev.yml
```

Assuming your `docker-compose.yml` has a service named `myapp`, your newly created `docker-compose-dev.yml` overlay file will look like this. Remember, our `bin/server` takes responsibility for our entry point. Since technically compose up needs a service, we have added a non operation tail command here. If your service already has an entry point, you can omit this line.

```yaml
version: '3.7'
services:
  myapp:
    command: tail -f /dev/null
    volumes:
      - myapp-sync:/var/task:nocopy
volumes:
  myapp-sync:
    external: true
```

Finally we add our simple docker-sync config file that connects up the `myapp` sync along with setting our Docker scope to the present working directory. Here is our `docker-sync.yml` file:

```yaml
version: '2'
syncs:
  myapp-sync:
    src: '.'
```

Finally we need every other bin script like console, server, and friends to just work. To do this we need to start our services so each `docker-compose run myapp` command works as it did before without docker-sync. We also need to do this in a platform agnostic way. Thankfully, our `bin/setup` bash check for `OSTYPE` is the perfect place to add our new docker-sync logic.

```bash
if [[ "$OSTYPE" == *"darwin"* ]]; then
  export SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock
  gem install docker-sync
  docker-sync start
  docker-compose up --detach
fi

docker-compose run myapp ./bin/_setup
```

That's all 🎉🎉🎉. With only a few additions everything else in our Docker project remains untouched. If you are on a Mac, you will get close to native speed after the initial [file sync setup](https://docker-sync.readthedocs.io/en/latest/advanced/sync-strategies.html) has run. If the Docker team (bless their hearts) ever ships a usable and performant filesystem, we can delete a few files and call it a day. Here are some additional things to consider when adopting this pattern:

1. Make sure different projects use a unique `myapp` naming convention. Failing to do so could mean two projects share the same docker-sync file system. Don't cross the streams!
2. For your team members that use Linux or for your CI/CD system which presumable also uses Linux, remove the `.env` file or override the `COMPOSE_FILE` environment variable to only have the single `docker-compose.yml` file. Hence, avoid the overlay.
3. Remember to add `.docker-sync*` to your `.gitignore` file.
4. Consider adding a few helpers to your `bin/update` script to cleanup docker-sync. Example:

```shell
docker-compose down
docker-sync stop
docker-sync clean

./bin/bootstrap
./bin/setup
```

## Thanks for Reading!

As they become available please check out my other posts on this series. And remember, I would love to hear how you are using Docker for Serverless or Containers on your projects. Thanks!!!