Django (DRF) 12 Factor App with examples
Tags: django, rest
Published: 2020-04-18 19:36:56

I've been working with Django (especially with DRF) for a while now. Deploying Django apps is the one thing
that bothers me the most. It's hard and does not have a pattern. So I decided to build a guide here to keep notes for
myself and may help anyone that is struggling with the lack of patterns out there.

### What is and why 12 Factors App?

[The Twelve-Factor App](https://12factor.net/) is a methodology for building SaaS apps. The main concern is to keep
everything as isolated and as secure as possible. It was first presented by Heroku in 2011 and is still referenced as a
good guide for best practices in building and deploying apps.

### Tools used on this guide:

- Gitlab CI
- VIM (my main editor)
- Django and friends (special mention to python-decouple)
- Assumes the bellow project structure (that I try to keep for all my projects)

    <pre>
    ├── Dockerfile
    ├── apps
    │   ├── app1
    │   └── app2
    ├── config
    │   ├── settings.py
    │   ├── urls.py
    │   └── wsgi.py
    ├── docker-compose.yml
    ├── docker-entrypoint.sh
    ├── manage.py
    ├── requirements
    │   ├── dev-requirements.in
    │   └── requirements.in
    └── requirements.txt
    </pre>

So, without further ado, to the guide!
> Each factor title has a link to its explanation on the original site. Remember: all things explained here can be
> ported to other CI/CD environments and to other languages.

### [1. Codebase - One codebase tracked in revision control, many deploys](https://12factor.net/codebase)

My approach is to use [Gitflow](https://nvie.com/posts/a-successful-git-branching-model/). There are some variations on
this but, since Gitflow envisions different branches for different phases of your app, it's easy to create `git hooks`
to start a build and/or a deploy.

Since I'm using Gitlab CI, hooks are "translated" to the `only` tag on `.glitlab-ci.yml` configuration file. Examining
the example file bellow, notice the `only` tag. It informs Gitlab CI to only execute this task on commits to the
branches list:

    image: gitlab/dind

    stages:
      - build
      - deploy

    run_build:
      stage: build
      only:
        - master
        - develop

    run_deploy:
      stage: deploy
      only:
        - master

### [2. Dependencies - Explicitly declare and isolate dependencies](https://12factor.net/dependencies)

Recently I migrated my projects to `pip-tools`. Before the migration I was using `pip` but the workflow for managing
development/production dependencies was not working for me. So now, I have a `/requirements` directory on my project
root with `dev-requirements.in` and `requirements.in`.

The first line of my `dev-requirements.in` file includes `requirements.in`.

    -r requirements.in
    flake8
    isort
    ...

So, for local development I can run:

    pip-compile requirements/dev-requirements.in -o ./requirements.txt
    pip-sync

And have both dev/production dependencies. In production, my pipeline has a slightly different `pip-compile`:

    pip-compile requirements/requirements.in -o requirements.txt
    pip-sync

And there you have: no development dependencies.

### [3. Config - Store config in the environment](https://12factor.net/config)

Most of the "juice" of this workflow relies on this item. I have struggled for a while to find some kind of solution for
Django `settings.py` environment separation. The great majority involves different files for each environment. But those
solutions always ends up with sensitive information (`SECRET_KEYs`, database passwords, third-party app keys, etc.) in your version
control system.

As I mentioned in the begging of this guide, `python-decouple` comes to the rescue! 

> There are other packages that does similar job - it's just a personal preference to use `python-decouple`.

The way Decouple works is simple: try to find the declared `KEYS` on the following places in this order:
1. Environment variables
2. `ini` or `.env` files
3. Default argument passed to config

So, to illustrate, take this `config/settings.py` file excerpt:

    import os

    from decouple import Csv, config

    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

    DEBUG = config('DEBUG', default=False, cast=bool)
    ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=Csv())
    SECRET_KEY = SECRET_KEY = config('SECRET_KEY')

If you have a `DEBUG=True` in your environment (exported on your Linux/macOS system or declared in your Windows preferences), `python-decouple` will read it, parse it to `boolean` and inject it
in your `settings.py`. If it cannot find the `DEBUG` key in you environment, it will try to look for it in a `.env` (or `.ini`) file.

In development, I keep an `.env` file in the root of my project. The `.env` file format is just key/value pairs:

    DEBUG=True
    ALLOWED_HOSTS=localhost,127.0.0.1
    SECRET_KEY=secret

In production, I take advantage of Gitlab's CI environment settings. It allows me to declare the same key/value pairs
and make it available to the build environment:

![Gitlab's CI environments variable example]({static}/images/gitlab_env_variables.png)

This way, no database passwords, secrets or app keys ends up on your source control (to avoid deploying your `.env`
files do production, add it to your `.gitignore` - even though it wouldn't be a big deal as `python-decouple` gives precedence to
environment variables).

### [4. Backing services - Treat backing services as attached resources](https://12factor.net/backing-services)

Implementing #4 is easy if you implement #3. Since every external configuration, URL address, port binding, etc. should
be in environment variables (or `.env` file, for that matter), it's already treated as an attached resource. It's quite
simple to verify that, in my case. I just push code to my remote master branch and wait the Gitlab CI do its job. No need to change
anything in code or config files. Just push code.

### [5. Build, release, run - Strictly separate build and run stages](https://12factor.net/build-release-run)

As described in factors #1 and #3, part of this problem is already solved. Still, there's an inherent problem since I'am
using Docker/docker-compose to deply my apps. Due to Docker way of doing things, passing environment variable values to
the build process is a little tricky.

The secret is knowing how Dockerfile and docker-compose `environment` tag works.

In production, I don't use `virtualenv`. It wouldn't make much sense since, by definition, a container is aleady an isolated
"environment". Given that, my `Dockerfile` is usually pretty straight forward:

    FROM python:3.8

    ENV PYTHONUNBUFFERED 1
    ENV PYTHONDONTWRITEBYTECODE 1

    RUN mkdir /code
    WORKDIR /code/
    ADD . /code/

    RUN pip install --upgrade pip && \
        pip install pip-tools && \
        pip-compile -q /code/requirements/requirements.in --output-file /code/requirements.txt && \
        pip install --no-cache-dir -r /code/requirements.txt

    EXPOSE 8001

For everything to work right, my `docker-compose.yml` file looks like the following:

    version: '3'

    services:
      api:
        image: image_name
        build: .
        ports:
          - "8001:8001"
        volumes:
          - /var/www/sample_project/media:/code/media
        entrypoint: sh docker-entrypoint.sh
        environment:
          - DJANGO_SETTINGS_MODULE=config.settings
          - SECRET_KEY=${S_KEY}
          - DEBUG=${DEBUG}
          - ALLOWED_HOSTS=${ALLOWED_HOSTS}
          - DATABASE_NAME=${DB_NAME}
          - DATABASE_USER=${DB_USER}
          - DATABASE_PASSWORD=${DB_PASSWORD}
          - DATABASE_HOST=${DB_HOST}
          - DATABASE_PORT=${DB_PORT}

That means: docker-compose reads from the environment variables set in Gitlab CI the value of, for example,
`${S_KEY}` and sets it on the Docker build environment under the key `SECRET_KEY`. It happens that, on my
`settings.py` file, its the exact key `python-decouple` looks for.

To simplify even more:
> CI environment variable --> Docker enviroment variable --> `settings.py`

To complete the process and follow the factor #5 premisses, your build pipeline should make a TAG on your source control
before cloning/checkout the code and build it. This way you could track how and when a release was generated.

### [6. Processes - The app is executed in the execution environment as one or more processes.](https://12factor.net/processes)

This one has a little to do on how we build the app and much more on how we architect it. As I mentioned before, I
mainly work with DRF (Django REST Framework) to build REST APIs for web and mobile clients.

This usually means that I rely on JWT tokens to keep "sessions" on my apps. Since no sticky sessions are needed and
my APIs don't have a meaninful state, I'm covered on this item. But, in case I needed some kind of server-side state, I
once relied on Redis to do that. Just remember to keep Redis server address, port and credentials on your configuration
environment (`.env` file and/or environment variables).

That all means that, if needed, this whole setup can scale horizontaly by just spanning more processes (containers in
this case).

### [7. Port binding - Export services via port binding](https://12factor.net/port-binding)

As seen on my `Dockerfile` and my `docker-compose.yml` files, I export the `8001` port to the operational system. That
means my services are accesible through that port. Although its possible and valid keep things this way, its usual to
have a proxy (a reverse proxy) in front of my services.

For that I have to configure 2 things: a gunicorn WSGI server and a Nginx proxy.

The first one, the Gunicorn server, is configured by the `docker-entrypoint.sh` script (notice that this script is
called by the `docker-compose.yml` on `entrypoint` tag):

    #!/bin/bash
    python manage.py collectstatic --noinput
    python manage.py migrate

    gunicorn -b 0.0.0.0:8001 -w 4 config.wsgi:application

This means that, on execution, docker-compose builds the container and run this script to start the container execution.
Gunicorn in binding on port `8001` which is exposed to the host operational system on the same port. If needed, we could
change that by changing the `ports` on `docker-compose.yml` file.

The second step is to configure a proxy server. My setup already have a Nginx server running on an VPS instance and its
not containerized. That's by design since I can move the API container to a cloud provider and have the another kind of
reverse proxy pointing to my API container (and that's why my docker-compose don't start a proxy service inside the
container).

Configuring a Nginx as a reverse proxy is very straight forward:

    server {
       listen 80;
       listen [::]:80;
       server_name api.server.com;

       location / {
           proxy_read_timeout 360;
           proxy_set_header Host $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
           proxy_pass http://127.0.0.1:8001;
       }
    }

And that's it. Now requests made to port `80` gets proxied to `8001` (the container!).

### [8. Concurrency - Scale out via the process model](https://12factor.net/concurrency)

As stated before, this whole setup is based on Docker and is stateless. That means its scalable via processes. I'm not
an expert on cloud but, I bet that with minimal changes to port bindings (and network definitions on containers), this setup can run on Kubernetes or some sort
of container manager.

### [9. Disposability - Maximize robustness with fast startup and graceful shutdown](https://12factor.net/disposability)

Again, since factor #8 is covered, this one is easy. Kill and restart the process or any container process and you still have a
new container ready to receive requests. Of course this spawning of containers must be managed by some kind of container
manager.

Just have to check how Gunicorn manages SIGTERM graceful finalizations.

### [10. Dev/prod parity - Keep development, staging, and production as similar as possible](https://12factor.net/dev-prod-parity)

I don't use Docker on my development machine. But I could as easily as in production. In fact, during the process of
building this setup, I had to validate my ideas and configurations. And, for that, I relied on running Docker and
docker-compose locally.

Notice that my database is not started inside the container as well. That's also by design. My production setup has a
dedicated database server machine. And to keep the environment parity, I also have a database installation on my local
machinei (but this could be another local container - I just happen to have an old installation already running). This 
keeps things as similar as possible to my production environment and my configuration "compliant" with factor #4.

### [11. Logs - Treat logs as event streams](https://12factor.net/logs)

That's one missing piece on my setup. I just don't log. I mean, on my development machine I log some stuff for
debugging purposes. In production, I don't have any kind of application logs. I can, in fact, peek inside de container
and see the container and Gunicorn logs, but that's all.

Ideally I should use a third-party log service but, for now, it's still too expensive for my budget.

### [12. Admin processes - Run admin/management tasks as one-off processes](https://12factor.net/admin-processes)

This one I achieve with the help of Docker. We can easily run REPL commands inside the container, e.g.:

    docker exec -it <container id> python manage.py makemigrations

It may vary depending on the chosen Docker image but the idea is that, even with a leaner image (like an *-alpine one),
you can install the needed tools changing the `Dockerfile` file.

### Conclusion

I find it nice to have some kind of best practices already defined and battle tested. Applying it to my stack was a
great way to learn some stuff new and useful. Its definitly not the end of it. Sure I can evolve this to something
better.

And by putting it together I hope I can help, or give some ideas, to someone it trouble trying to figure some of this
out.