Start a Production-Ready Dockerized Django Project
Tags: python, django, docker, webdev
Published: 2022-11-01 13:00:00

Making a Django app production-ready inside Docker is quite useful for developers. It minimizes the hassle of setup and deployment. This allows developers to focus on what’s important i.e. development and business logic.

## Table of Contents

1. [Prerequisite](#prerequisite)
2. [Introduction](#introduction)
3. [Project Configuration](#project)
4. [Split Settings for Different Environments](#settings)
5. [Environment Variables](#env)
6. [Postgres Configuration](#postgres)
    1. [Ensure Postgres is healthy before Django is started](#healthy)
7. [Celery and Redis Configuration](#celeryredis)
8. [Tweak Docker Compose for Production](#prod)
9. [Conclusion](#conclusion)

## Prerequisite <a name="prerequisite"></a>

This guide assumes that you are familiar with the following technologies:

- Intermediate Django
- Beginner to Intermediate Docker
- Familiarity with Postgres, Celery, Redis, Nginx

## Introduction <a name="introduction"></a>

This guide is aimed at helping you start and organize your Django project to work in different environments mainly, development and production. You can then take this template, modify it to fit your specific requirements, and finally deploy it on your choice of a cloud service provider like AWS, Azure, or Digital Ocean to name a few. 

_**Note:-** If you encounter any issues throughout the tutorial, you can check out the code in the [GitHub repository](https://github.com/earthcomfy/django-docker-template)_

## **Project Configuration** <a name="project"></a>

First, create a repo on GitHub. Initialize the repository with a `README` file and `.gitignore` template for Python.

![GH Repo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5wfijtdnap8elu7u8f6u.png)

Now, on your machine, open up a terminal and run the following commands to set up and open your project.

```bash
mkdir django-docker-template
cd django-docker-template
git clone <link-to-repo> .
code .
```

In the root directory of your project, create a file named `requirements.txt` 

```bash
touch requirements.txt
```

and add the following dependencies:

```
celery==5.2.7
Django==4.1.2
gunicorn==20.1.0
psycopg2-binary==2.9.5
python-decouple==3.6
redis==4.3.4
```

Then, create the `Dockerfile`

```bash
touch Dockerfile
```

and add the following snippet:

```docker
FROM python:3.10.2-slim-bullseye

ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

WORKDIR /code

COPY ./requirements.txt .
RUN pip install -r requirements.txt

COPY . .
```

Then, create a `docker-compose.yml` file:

```bash
touch docker-compose.yml
```

and add a `web` service inside it:

```yaml
version: "3.9"

services:
  web:
    build: .
    volumes:
      - .:/code
    ports:
      - 8000:8000
```

Finally, create a `.dockerignore` file so that Docker will ignore some files thus speeding up the build process of your image.

```bash
touch .dockerignore
```

Add the following inside it:

```bash
.venv
.git
.gitignore
```

Great, build your image by running the following command:

```bash
docker-compose build
```

This will take some time. You can now use this image to create the Django project.

```bash
docker-compose run --rm web django-admin startproject config .
```

## Split Settings for Different Environments <a name="settings"></a>

It is important to take into account the different environments/modes your project will be running on: usually, these are development and production. However, you can apply a similar logic for other environments you may need to include.

You can split your settings to dictate which environment your project is running on, similar to what is presented below:

```
config
│
└───settings
│   │   __init__.py
│   │   base.py
│   │   development.py
│   │   production.py
```

`base.py` will contain the common settings used regardless of the environment. Hence, copy all the content of `settings.py` which Django created by default into `settings/base.py` and delete `settings.py` as it is no longer needed.

Then, import `base.py` in both environments. Environment-specific settings will be updated later.

```python
# import this in development.py, production.py
from .base import *
```

## Environment Variables <a name="env"></a>

Using environment variables allows you to describe different environments. `python decouple` is one of the most commonly used packages to strictly separate settings from your source code. This package is added earlier in `requirements.txt` so just create a `.env` file in the root directory of your project:

```bash
touch .env
```

And add the following variables:

```
SECRET_KEY=

ALLOWED_HOSTS=.localhost, .herokuapp.com, .0.0.0.0
DEBUG=True

DJANGO_SETTINGS_MODULE=config.settings.development
```

Update your settings accordingly:

```python
# base.py

from decouple import config, Csv

SECRET_KEY = config("SECRET_KEY")

DEBUG = config("DEBUG", default=False, cast=bool)

ALLOWED_HOSTS = config("ALLOWED_HOSTS", cast=Csv())
```

`DJANGO_SETTINGS_MODULE` tells Django which setting to use. By providing its value in an environment variable, `manage.py` will be able to automatically use the appropriate setting for different environments. Therefore update `manage.py` as follows:

```python
# manage.py

from decouple import config

os.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))
```

Also update the `web` service in `docker-compose.yml` file to pull in environment variables from the `.env` file:

```yaml
version: "3.9"

services:
  web:
    build: .
    volumes:
      - .:/code
    env_file:
      - ./.env
    ports:
      - 8000:8000
```

Now, what are some of the potential environment-specific settings? Here are some of them:

1) **Email**

You can use the console backend in development mode to write emails to the standard output.

```python
# development.py

from .base import *

EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"
```

While in production mode, use an SMTP backend like SendGrid, Mailgun, etc…

```python
# production.py

from .base import *

EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_HOST = "'smtp.mailgun.org'"
EMAIL_PORT = 587
EMAIL_HOST_USER = config("EMAIL_USER")
EMAIL_HOST_PASSWORD = config("EMAIL_PASSWORD")
EMAIL_USE_TLS = True
```

2) **Media and Static files**

In production mode, you may want to use services like AWS S3 to serve your static and media files. Having multiple settings comes in handy in such scenarios.

```python
# development.py

MEDIA_URL = "/media/"
MEDIA_ROOT = os.path.join(BASE_DIR, "../", "mediafiles")

STATIC_URL = "static/"
STATIC_ROOT = os.path.join(BASE_DIR, "../", "staticfiles")
```

And then you can add AWS-related configs in `production.py`

3) **Caching**

Ideally, you don’t need to cache your site in development so you can separately add a cache server like Redis in `production.py` file

```python
# production.py

# Redis Cache
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": config("REDIS_BACKEND"),
    },
}
```

In addition, you can add apps, middleware, etc. separately for your environments.

## Postgres Configuration <a name="postgres"></a>

To configure Postgres, you first need to add a new service to the `docker-compose.yml` file:

```yaml
version: "3.9"

services:
  web:
    build: .
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - .:/code
    env_file:
      - ./.env
    ports:
      - 8000:8000
    depends_on:
      - db
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=${DB_USERNAME}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}

volumes:
  postgres_data:
```

Next, update the `.env` file to include database-related variables:

```yaml
# Database
DB_NAME=
DB_USERNAME=
DB_PASSWORD=
DB_HOSTNAME=db
DB_PORT=5432
```

Finally, update the settings to use Postgres RDBMS instead of the SQLite engine Django uses by default.

```python
# base.py

# Remove the sqlite engine and add this
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": config("DB_NAME"),
        "USER": config("DB_USERNAME"),
        "PASSWORD": config("DB_PASSWORD"),
        "HOST": config("DB_HOSTNAME"),
        "PORT": config("DB_PORT", cast=int),
    }
}
```

_**Note:-** You may want to use different databases for development and production. If that’s the case, you can remove the `DATABASES` setting from `base.py` and add different databases for development and production._

Great! Now, re-build the container to ensure what you have so far is working.

```bash
docker-compose build
```

Also, ensure that the migrations are applied:

```bash
docker-compose run --rm web python manage.py migrate
```

### Ensure Postgres is healthy before Django is started <a name="healthy"></a>

Usually, when working with Postgres and Django in Docker, the `web` service (Django) tries to connect to the `db` service even when `db` service is not ready to accept connections. To solve this issue, you can create a short bash script that will be used in the Docker [ENTRYPOINT](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#entrypoint) command.

In the root directory of your project create a file named `entrypoint.sh`

```bash
touch entrypoint.sh
```

Add the following script to listen to the Postgres database port until it is ready to accept connections and then apply migrations and collect static files. 

```bash
#!/bin/sh

echo 'Waiting for postgres...'

while ! nc -z $DB_HOSTNAME $DB_PORT; do
    sleep 0.1
done

echo 'PostgreSQL started'

echo 'Running migrations...'
python manage.py migrate

echo 'Collecting static files...'
python manage.py collectstatic --no-input

exec "$@"
```

Update the file permissions locally

```bash
chmod +x entrypoint.sh
```

Now, to use this script, you need to have `Netcat` installed on your image. Therefore, update `Dockerfile` to install this networking utility and use the bash script as a Docker entrypoint command.

```docker
FROM python:3.10.2-slim-bullseye

ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

WORKDIR /code

COPY ./requirements.txt .

RUN apt-get update -y && \
    apt-get install -y netcat && \
    pip install --upgrade pip && \
    pip install -r requirements.txt

COPY ./entrypoint.sh .
RUN chmod +x /code/entrypoint.sh

COPY . .

ENTRYPOINT ["/code/entrypoint.sh"]
```

Rebuild the image and spin up the containers.

```bash
docker-compose up --build
```

Go to [http://localhost:8000/](http://localhost:8000/)

## Celery and Redis Configuration <a name="celeryredis"></a>

Celery does time-intensive tasks asynchronously in the background so that your web app can continue to respond quickly to users’ requests. Use Redis together with Celery since it can serve as both a message broker and a database back end at the same time.

Add Redis and Celery services to `docker-compose.yml`

```yaml
redis:
    image: redis:7
  
celery:
  build: .
  command: celery -A config worker -l info
  volumes:
    - .:/code
  env_file:
    - ./.env
  depends_on:
    - db
    - redis
    - web
```

While at it, update the `web` service as well:

```yaml
depends_on:
    - redis
    - db
```

Once that is set up, navigate to the config folder and create a file named `celery.py`

```bash
cd config
touch celery.py
```

Then, add the following snippet inside it:

```python
# config/celery.py

import os

from decouple import config
from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))
app = Celery("config")
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()
```

Next, head over to `base.py` and add the following configuration at the bottom:

```python
# settings/base.py

# Celery
CELERY_BROKER_URL = config("CELERY_BROKER_URL")
CELERY_RESULT_BACKEND = config("REDIS_BACKEND")
```

Update `.env` to include the above environment variables:

```
# Celery
CELERY_BROKER_URL=redis://redis:6379/0

# Redis
REDIS_BACKEND=redis://redis:6379/0
```

The final update goes into the `__init__.py` file of the config folder:

```python
# config/__init__.py

from .celery import app as celery_app

__all__ = ('celery_app',)
```

Test it out again:

```python
docker-compose up --build
```

## Tweak Docker Compose for Production <a name="prod"></a>

Django’s built-in server is not suitable for production so you should be using a production-grade WSGI server like Gunicorn in a production environment.

In addition, you should also consider adding Nginx to act as a reverse proxy for Gunicorn and serve static files.

Therefore, create a file named `docker-compose.prod.yml` at the root of your project and add/update the following services:

```yaml
version: "3.9"

services:
  web:
    build: .
    restart: always
    command: gunicorn config.wsgi:application --bind 0.0.0.0:8000
    env_file:
      - ./.env
    expose:
      - 8000
    volumes:
      - static_volume:/code/staticfiles
      - media_volume:/code/mediafiles
    depends_on:
      - redis
      - db
  db:
    image: postgres:13
    restart: always
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=${DB_USERNAME}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_NAME}
  
  redis:
    image: redis:7
  
  celery:
    build: .
    restart: always
    command: celery -A config worker -l info
    volumes:
      - .:/code
    env_file:
      - ./.env
    depends_on:
      - db
      - redis
      - web
  
  nginx:
    build: ./nginx
    restart: always
    ports:
      - ${NGINX_PORT}:80
    volumes:
      - static_volume:/code/staticfiles
      - media_volume:/code/mediafiles
    depends_on:
      - web

volumes:
  postgres_data:
  static_volume:
  media_volume:
```

There are a couple of things worth noting from the above file:

- The use of `expose` instead of `ports`. This allows the `web` service to be exposed to other services inside Docker but not to the host machine.
- Static and media volumes to persist data generated by and used by `web` and `nginx` services.

Don’t forget to update `.env` to include the `NGINX_PORT` environment variable:

```
# NGINX
NGINX_PORT=80
```

Then, in your project root directory, create the following folder and files:

```bash
mkdir nginx
cd nginx
touch Dockerfile
touch nginx.conf
```

Update the respective files:

```docker
# nginx/Dockerfile

FROM nginx:stable-alpine

RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d

EXPOSE 80
```

- The above file pulls the base Nginx image, removes the default configuration, and copies the one that you created i.e. `nginx.conf` with the following content:

```
# nginx/nginx.conf

upstream web_app {
    server web:8000;
}

server {

    listen 80;

    location /static/ {
        alias /code/staticfiles/;
    }

    location /media/ {
        alias /code/mediafiles/;
    }

    location / {
        proxy_pass http://web_app;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

}
```

- Worth noting in the above configuration is that static and media file requests are routed to the static files and media files folders respectively.

Test your production setup locally:

```bash
docker-compose -f docker-compose.prod.yml up --build
```

Go to [http://localhost/](http://localhost/) The static files should be loaded correctly as well.

## Conclusion <a name="conclusion"></a>

This tutorial has walked you through containerizing your Django application both for local development and production. In addition to the ease of containerized deployment, working inside Docker locally is also time-saving because it minimizes the setup you need to configure on your machine.

If you got lost somewhere throughout the guide, check out the project on [GitHub](https://github.com/earthcomfy/django-docker-template)

Happy coding! 🖤