Docker-compose local development with nginx-proxy
By: Victor Rodriguez
Tags: Docker Compose, Docker, Nginx Proxy, Nginx, Rails

Context

As your business grew, you and your team might have dockerized your services to easily deploy your production builds into cloud providers such as AWS or GCP.

Here at Webdox CLM, we have an amazing team building the next big thing, with a very diverse stack (Typescript, Ruby, Go, Java and Python mainly) running monoliths and micro-services, so being able to rapidly start developing is a must.

When I first joined the team, I struggled to feel productive due to the complexity of our ecosystem. Sometimes I felt overwhelmed because we had a bunch of guides to set up all the services and some of them were hard to follow. Also, all of these services were running on the host machine and it was fine when you were working with one or two services, but when you had to run a lot of them at the same time it became a little tedious. To avoid collisions between services and applications, each one had a different port we had to set in a large exports file to have it working nicely, but it was growing and, I did not felt it was the right way to do things. There should be an easier way, right?

Step 1: Creating a docker-compose.dev.yml

As you may already know, docker already solves a big part of our problems to set up our services. Our team weighted the mental and performance overhead that containers bring to the table in a local development environment versus all the effort and time that setting everything up in a native environment, and decided in the end that we could create a container’s configuration that could suit both workflows, as some experienced teammates would rather prefer a smooth, lag-free environment.

We started creating a docker-compose.dev.yml file for this project to avoid colliding with existing configurations and it looked like this:

version: "3.9" x-config: &backend

image: app_backend

volumes:

- .:/home/app:cached

- bundle:/usr/local/bundle

- .dockerdev/.psqlrc:/root/.psqlrc:ro

env_file:

- .env services:

runner:

<<: *backend

build:

context: .

dockerfile: Dockerfile.dev

args:

- PG_MAJOR=11

- BUNDLER_VERSION=2.2.27

depends_on:

- redis

- db

- es rails:

<<: *backend

stdin_open: true

tty: true

entrypoint: entrypoints/backend-dev.sh

depends_on:

- runner

- redis

- db

- es

- webdav sidekiq:

<<: *backend

stdin_open: true

tty: true

entrypoint: entrypoints/sidekiq-dev.sh

depends_on:

- runner

- redis

- db

- es db:

image: postgres:11

volumes:

- db:/var/lib/postgresql/data

ports:

- "5432:5432"

env_file:

- .env redis:

image: redis

volumes:

- redis:/data

ports:

- 6379 es:

image: elasticsearch:5-alpine

volumes:

- elastic:/usr/share/elasticsearch/data

ports:

- "9200:9200"

- "9300:9300" webdav:

image: bytemark/webdav

volumes:

- webdav:/var/lib/dav

ports:

- "8080:8080"

environment:

- USERNAME=myapp

- PASSWORD=mysupersecretpassword volumes:

db:

elastic:

webdav:

redis:

bundle:

With this simple configuration, our bootstrapping process went from requiring 20 steps to roughly 4. The build process is slow only in the first clean build.

Also, we are sharing the same image for our Rails back-end, Sidekiq server, and Runner container, so getting the whole stack up and running is pretty fast. In case you are wondering, our Runner container exists just to build our base image and run rake commands easily without executing into a running container.

To avoid rebuilding the image after gem installations or code changes, we mounted volumes for persisted data like our main app code, database information, and WebDAV server.

However, as some of you may notice, we are still exposing the ports to the host and it still may conflict with other projects/services.

Step 2: Adding nginx-proxy

To avoid exposing all the ports to the host, we need to create a private docker network and use a reverse proxy to route all the applications to the exterior. However, that would increase the burden to maintain the entire ecosystem configuration, right? Because setting up a reverse proxy configuration can be complicated when containers are started and stopped. Typically the configuration needs to be updated manually which is error-prone and time-consuming.

Luckily, there’s this tool called nginx-proxy that uses docker APIs to expose container meta-data to templates and convert them to nginx config files automatically. You can read more about it here.

In summary, at a high level, our services need to run in their own private network, and there should be a reverse proxy service that is going to sit in front of them to intercept the request and route them to their respective application.

So how are we going to do that?

2.1 Create the nginx service with the private network

Create a new project to hold the nginx configuration for the containers:

nginx-proxy/docker-compose.yml

version: "3.9" services:

nginx-proxy:

build: .

container_name: nginx-proxy

restart: always

ports:

- "80:80"

- "443:443"

volumes:

- /var/run/docker.sock:/tmp/docker.sock:ro networks:

default:

name: myapp-private

First, we need to define our service and expose ports 80 and 443. This is the only service that will be exposed to the host in our private network. Of course, you can expose each container to the host but that’s not going to solve our port collision problem.

It is crucial that we bind the docker.sock since this utility is going to listen to container changes using the docker API.

Also, it’s very likely that you are going to face CORS issues when you try to connect applications at a client level. To fix this, we created a template to add headers to each request from our virtual host.

nginx-proxy/nginx/vhost.d/default_location

if ($http_origin ~* (.*\\.myapp\\.local)) {

add_header 'Access-Control-Allow-Origin' "$http_origin" always;

add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, DELETE, PATCH, PUT';

add_header 'Access-Control-Allow-Credentials' 'true' always;

add_header 'Access-Control-Allow-Headers' 'Keep-Alive,Content-Type,Authorization';

add_header 'Access-Control-Expose-Headers' 'Authorization';

}

if ($request_method = OPTIONS) {

add_header 'Access-Control-Allow-Origin' "$http_origin";

add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, DELETE, PATCH, PUT';

add_header 'Access-Control-Allow-Credentials' 'true';

add_header 'Access-Control-Allow-Headers' 'Keep-Alive,Content-Type,Authorization';

add_header 'Access-Control-Expose-Headers' 'Authorization';

return 204;

}

In this example, our virtual host will be myapp.local , and since the TLD is invalid, we are just going to redirect all requests made from our host to localhost or 127.0.0.1 . To achieve this, we need to edit our /etc/hosts file and append our virtual subdomains like this:

/etc/hosts → In your host machine

127.0.0.1 frontend.myapp.local backend.myapp.local webdav.myapp.local ...

In addition, you can add a custom nginx configuration :

nginx-proxy/nginx/conf.d/custom_proxy.conf

client_max_body_size 20m;

proxy_read_timeout 5m;

proxy_connect_timeout 600;

proxy_send_timeout 600;

These configuration files need to be copied to the container. We can do it in our Dockerfile file:

nginx-proxy/Dockerfile

FROM jwilder/nginx-proxy

COPY nginx /etc/nginx/

Finally, we can start our nginx reverse proxy server with just a docker-compose up . In case you do not want to start with service every time at boot, you could add restart: always in the compose file.

2.2 Add our services to the private network

After setting up our reverse proxy, it’s time to include our services in our new private network.

Simply add at the end of your compose files:

networks:

default:

name: myapp-private

Also, we need to remove the exposed ports. So, instead of declaring

ports:

- "9200:9200"

It should be now:

ports:

- 9200

Additionally, we need to make sure that the applications we are going to expose to the client are running on port 80, in order to be routed automatically by our nginx reverse proxy (you can use other ports too, do not forget to check the nginx-proxy documentation).

In Rails, there are several ways of doing it. To achieve the compatibility we expect from this setup, we opted for setting up an environment variable in puma.rb .

port ENV.fetch("RAILS_PORT") { 8080 }

This variable will be defined in a .env file, alongside all other values used by our services.

Ultimately, we need to export the variable VIRTUAL_HOST=xxxx.myapp.local on the services that are going to be exposed and reachable from the host.

So, our final docker-compose.dev.yml will look like this:

version: "3.9" x-config: &backend

image: app_backend

volumes:

- .:/home/app:cached

- bundle:/usr/local/bundle

- .dockerdev/.psqlrc:/root/.psqlrc:ro

env_file:

- .env services:

runner:

<<: *backend

build:

context: .

dockerfile: Dockerfile.dev

args:

- PG_MAJOR=11

- BUNDLER_VERSION=1.17.3

depends_on:

- redis

- db

- es rails:

<<: *backend

stdin_open: true

tty: true

entrypoint: entrypoints/backend-dev.sh

environment:

- VIRTUAL_HOST=backend.myapp.local

depends_on:

- runner

- redis

- db

- es

- webdav sidekiq:

<<: *backend

stdin_open: true

tty: true

entrypoint: entrypoints/sidekiq-dev.sh

depends_on:

- runner

- redis

- db

- es db:

image: postgres:11

volumes:

- db:/var/lib/postgresql/data

ports:

- 5432

env_file:

- .env redis:

image: redis

volumes:

- redis:/data

ports:

- 6379 es:

image: elasticsearch:5-alpine

volumes:

- elastic:/usr/share/elasticsearch/data

ports:

- 9200

- 9300 webdav:

image: bytemark/webdav

volumes:

- webdav:/var/lib/dav

ports:

- 80

environment:

- VIRTUAL_HOST=webdav.myapp.local

- USERNAME=myapp

- PASSWORD=mysupersecretpassword volumes:

db:

elastic:

webdav:

redis:

bundle: networks:

default:

name: myapp-private

Rebuild your containers and you are all set! Head out to http://frontend.myapp.local/ or whatever virtual host you have defined 🥳🎉.