Deploying Django Rest Framework with Postgres on fly.io
Tags: django, flyio, programming, python
Published: 2021-12-30 22:45:18

## Background
In this post, I will be demonstrating how to deploy a [Django Rest Framework](https://www.django-rest-framework.org/) (DRF) application on [fly.io](https://fly.io). DRF is built on top of Django and is my choice when it comes to building small to massive APIs. I have used it extensively professionally and personally, and as a cybersecurity researcher, it's been my favorite to build as a backend for some complex applications that track malicious actors and their infrastructure. I noticed the fly website did not have a Django example, so I wanted to provide the community with a template to get started to avoid the headaches that I endured building this :) 

You should have an understanding of Docker, docker compose, DRF and postgres for this post. Although it's possible to use this template without too much knowledge of these concepts, you'd get a ton of knowledge going through the following tutorials:

- https://www.django-rest-framework.org/tutorial/quickstart/
- https://docs.docker.com/samples/django/
- https://fly.io/docs/getting-started/python/

### Why DRF? 

From the DRF website:

```
Some reasons you might want to use REST framework:

-  The Web browsable API is a huge usability win for your developers.
-  Authentication policies including packages for OAuth1a and OAuth2.
-  Serialization that supports both ORM and non-ORM data sources.
-  Customizable all the way down - just use regular function-based views if you don't need the more powerful features.
-  Extensive documentation, and great community support.
-  Used and trusted by internationally recognised companies including Mozilla, Red Hat, Heroku, and Eventbrite.
```

The biggest issue with DRF is that it can be somewhat challenging to get up and running (can be lots of up front work). My experience has been that once you get a good template for an API up, it's the fastest to build, most scalable and most intuitive ORM. 

### Why fly.io?

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z8cg503lfumrmni4rfdp.png)
 

I've used Heroku for one-off apps for a few years. Fly came across my Twitter feed and I've been following it closely. The company has a great section on why you [should use fly](https://fly.io/docs/introduction/), and this section caught my eye and has kept me interested ever since I read it:

> Despite the benefits of location-smart, time-agile and cloud-clever applications, there’s been no good platform for building applications that work like this. This is what Fly has set out to fix. In the process we want to make application distribution platforms as ubiquitous as CDNs.

You can think of fly as a Heroku competitor, although some folks might disagree with me. I like it because it does what it says it does well, is focused, and isn't as bloated as the Heroku stack.

## Getting started

### Bookkeeping

We'll be making a DRF app, the Silly Simple API, or `ss-api` for short, on fly. This will have the following features:

- Postgres backend, [courtesy of fly](https://fly.io/docs/reference/postgres/)
- No session authentication, only using Tokens (you can use some tricks from `fly` to manage this). This is especially nice for APIs and simplifies authentication to a token, which I prefer for microservices
- Swagger and OpenAPI capabilities using [drf-yasg](https://drf-yasg.readthedocs.io/en/stable/), where you can only see endpoints and Swagger docs if you have a valid Token
- TCP & HTTP health checks using fly. The HTTP health check will be somewhat _useful_ by issuing a query to our DRF app under `/ping/`, which connects to the DB and issues an innocuous `select 1` statement to make sure things are working
- Using `docker` with a `Dockerfile` and `gunicorn` to launch the app. The cool thing about fly is that you can give it a `Dockerfile` and a `fly.toml` and you have a full-fledged app running on their infrastructure

This app will only have 1 endpoint, `users`, that you can use to manage your users. You must be authenticated to see it. 

I will leave the following for later blog posts:

- Metrics exposure and app tracing via Datadog (p.s., we're hiring https://www.datadoghq.com/careers/)
- Tables for an app (this isn't a tutorial on building DRF apps, rather, to get you a template to get started)
- Scaling primitives in DRF (avoiding n+1, indexing, replicas for postgres)

If this all sounds interesting for you still, let's get started :D 

### Clone and run locally

Make sure to have the following installed locally:

- Latest Docker (with `docker compose`)
- `git`
- fly via [here](https://fly.io/docs/hands-on/installing/)

then clone from the [repo](https://github.com/zmallen/ss-api) here:

`git clone git@github.com:zmallen/ss-api.git`

Run `docker compose up` and connect locally by navigating to:

`http://localhost:8000`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/63dih5ixmamlncj88osj.png)

The `docker-compose.yml` file overrides the `RUN` command in the `Dockerfile` by issuing the following command on every `docker compose up`:

```yaml
command: bash -c "python manage.py migrate && python manage.py runserver 0.0.0.0:8000"
```

This will differ from when we deploy on fly.io, where we use `gunicorn` to serve the app and we run migrations manually.

### Token Auth & Authentication in DRF

Unauthenticated users can only see the `/ping/` endpoint on Swagger. This is by design - the app will render endpoints based on permission, and under `/ping/views.py` on Line 16, the permission for this endpoint is:

`permission_classes = (AllowAny,)`
 
Compare this to `ssapi/views.py`, under the `UserViewSet` on Line 17:

`authentication_classes = (authentication.TokenAuthentication,)`

This is achieved via some magic in `settings.py` Lines 88-111:

```json
# DRF settings
REST_FRAMEWORK = {
    "DEFAULT_PERMISSION_CLASSES": ("rest_framework.permissions.IsAuthenticated",),
    "DEFAULT_AUTHENTICATION_CLASSES": (
        "rest_framework.authentication.TokenAuthentication",
    ),
    "DEFAULT_RENDERER_CLASSES": ("rest_framework.renderers.JSONRenderer",),
}

# SWAGGER_SETTINGS
SWAGGER_SETTINGS = {
    "USE_SESSION_AUTH": False,
    "LOGIN_URL": "rest_framework:login",
    "LOGOUT_URL": "rest_framework:logout",
    "VALIDATOR_URL": None,
    "SECURITY_DEFINITIONS": {
        "api_key": {
            "type": "apiKey",
            "name": "Authorization",
            "in": "header",
        },
    },
    "REFETCH_SCHEMA_WITH_AUTH": True,
}
```

Under `DRF_SETTINGS`, I forced `TokenAuthentication` and `isAuthenticated` for viewing endpoints, so no more sessions! 🎉🎉

So how do you get an API token if you can't authenticate? This is where some magic with `manage.py` comes into play.

### Authenticating - `getdevtoken` and `/users/`

To get a local API key, run the following command in a separate tab, in the same directory as `docker-compose.yml`:

```bash
└> docker compose run api python manage.py getdevtoken
[+] Running 1/0
 ⠿ Container ss-api-db-1  Running                                          0.0s
Looking for superuser..
superuser doesn't exist, creating!
superuser created!
Use the following key for dev:
(｡◕‿‿◕｡)☞☞  Token 9884a551be31b80a61b49becf7c3640224a9ec42  ☜☜(｡◕‿‿◕｡)
```

* Note, this Token is for my local deployment :) 

Copy the `Token abc` and navigate over to your browser, then click 'Authorize', paste and press submit. You should get the `/users/` endpoint to return in the Swagger frontend, and issuing a GET request will list the `superuser`!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2rb2p7ojeg0qsl9kd2u5.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zkrywmdaiklmkbjs0ixm.png)

You can do a lot more with Swagger documentation than just the defaults, I suggest checking out these resources to learn about Swagger docs in Django:

- https://django-rest-swagger.readthedocs.io/en/stable-0.3.x/examples.html
- https://swagger.io/tools/open-source/ 
 
### Deploying to fly

Everything is running smoothly in your local environment, now let's get it to a prod environment!

First, we need to create a `toml` file for fly. This is a configuration file used by fly to deploy your app. I generated one within the Github repo, but you can explore how to generate and configure other `toml` files on fly [here](https://fly.io/docs/reference/configuration/).

A few things need to happen to finish our "deploy to prod" for ss-api:

1. Create a fly app
2. Create a postgres db with fly, and retrieve the `DATABASE_URL` string 
3. Set a `fly secret` with the `DATABASE_URL` from Step 1 so our app can dynamically render the secret in a fly environment and use the db created in step 1
4. Deploy the app on fly, make sure TCP & HTTP health checks pass (they wont on first pass :D)
5. Create a database and run a migration using `fly ssh`
6. Get a `devtoken` for prod

#### Step 1: Create your fly app

Simply run `fly create` and name your app!

```bash
└> fly create
? App Name: ssapiblog
automatically selected personal organization: Zack Allen
New app created: ssapiblog
```

#### Step 2: postgres 

Launch a new postgres instance via `fly` with `fly postgres create`. Accept all the defaults (minimal DB settings, aka the cheapest!) and wait for fly to give you the `DATABASE_URL`.

```bash
└> fly postgres create
? App Name: ssapidb
Automatically selected personal organization: Zack Allen
? Select region: iad (Ashburn, Virginia (US))
? Select VM size: shared-cpu-1x - 256
? Volume size (GB): 10
Creating postgres cluster ssapidb in organization personal
Postgres cluster ssapidb created
  Username:    postgres
  Password:   SECRETPASSWORD
  Hostname:    ssapidb.internal
  Proxy Port:  5432
  PG Port: 5433
Save your credentials in a secure place, you won't be able to see them again!

Monitoring Deployment
...
...
2 desired, 2 placed, 2 healthy, 0 unhealthy [health checks: 6 total, 6 passing]
--> v0 deployed successfully

Connect to postgres
Any app within the personal organization can connect to postgres using the above credentials and the hostname "ssapidb.internal."
For example: postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432

See the postgres docs for more information on next steps, managing postgres, connecting from outside fly:  https://fly.io/docs/reference/postgres/
```

You want the [12factor](https://www.12factor.net/backing-services) string after `For example:`, which in this example is:
`postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432`

#### Step 3: Set DATABASE_URL as a `fly secret`

You want to set the DATABASE_URL with a 12factor string from before, as well as a database name (which we will create). 

Note the `/ssapidb` at the end of the `DATABASE_URL` 

`fly secrets set DATABASE_URL="postgres://postgres:SECRETPASSWORD@ssapidb.internal:5432/ssapidb"`

#### Step 4: Deploy your app

Change the following line in `fly.toml` to whatever you want:

```toml
app = "ssapiblog"
```

Run `fly deploy`:

```bash
└> fly deploy 
Deploying ssapiblog
==> Validating app configuration
--> Validating app configuration done
Services
TCP 80/443 ⇢ 8000
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.8 linux x86_64
Sending build context to Docker daemon  153.1kB
...
==> A bunch of Docker output

You can detach the terminal anytime without stopping the deployment
Monitoring Deployment

v0 is being deployed
2021-12-30T22:26:21.000 [info] 145.40.89.203 - - [30/Dec/2021:22:26:21 +0000] "GET /ping/ HTTP/1.1" 500 114326 "http://172.19.10.66:8000/ping" "Consul Health Check"

1 desired, 1 placed, 0 healthy, 0 unhealthy [health checks: 2 total, 1 passing, 1 critical]
```

#### Step 5: Build a DB then migrate

Notice how 1 health check is passing (tcp), and 1 is critical. This is because we did not do a database migration and our `/ping` healthcheck is failing. This will most likely fail after a certain amount of time, so in a separate tab navigate to the project directory to run a few `fly ssh` commands.

First, make the `ssapidb` database by running a handy dandy bash script I added into the repo. We can use this via `fly ssh console -C` command:

```bash
└> fly ssh console -C 'bash /app/provision_db.sh'
Connecting to ssapiblog.internal... complete
Database does not exist. Creating now..
CREATE DATABASE
```

If you run `fly logs` in a separate tab, you should see Consul health checks returning 200, which is healthy \o/:

```bash
2021-12-30T22:31:27.116 app[3d7bc4b5] iad [info] 145.40.89.203 - - [30/Dec/2021:22:31:27 +0000] "GET /ping HTTP/1.1" 301 0 "-" "Consul Health Check"
2021-12-30T22:31:27.119 app[3d7bc4b5] iad [info] 145.40.89.203 - - [30/Dec/2021:22:31:27 +0000] "GET /ping/ HTTP/1.1" 200 2 "http://172.19.10.66:8000/ping" "Consul Health Check"
```

Let's migrate and get a devtoken:

```bash
└> fly ssh console -C 'python /app/manage.py migrate'
Connecting to ssapiblog.internal... complete
Operations to perform:
  Apply all migrations: admin, auth, authtoken, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying authtoken.0001_initial... OK
  Applying authtoken.0002_auto_20160226_1747... OK
  Applying authtoken.0003_tokenproxy... OK
  Applying sessions.0001_initial... OK
```

#### Step 6: Get your devtoken and open the app

```bash
└> fly ssh console -C 'python /app/manage.py getdevtoken'
Connecting to ssapiblog.internal... complete
Looking for superuser..
superuser doesn't exist, creating!
superuser created!
Use the following key for dev:
(｡◕‿‿◕｡)☞☞  Token TOKEN  ☜☜(｡◕‿‿◕｡)
```

Woot! Run `fly open` and go through the same workflow as your local deployment: put `Token TOKEN` into the Authorize panel, and you can now see the authenticated users endpoint, and issue a GET request to get your token!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lcivi3mditvg02d6vne1.png)

If you want to add more users, just use the `POST` request endpoint here to create a new user. 

## Conclusion

I enjoyed writing this app and this blog post! Fly is a cool concept and I will definitely play with it more. There are some sharp edges with DRF, so I tried to simplify it, but please study the DRF tutorials and the `api/settings.py` file for other configuration options I used. 

For my next posts, Im looking to develop an app to do some basic cybersecurity threat intelligence tracking and correlation. If you have ideas for other apps, or have a question on this app, please leave a comment or open an issue on the `ss-api` repo here:

https://github.com/zmallen/ss-api