Django Authentication system - GitHub Actions, Automated testing, Static Analysis And Deployment on Vercel
Tags: githubhack23, python, githubactions, webdev
Published: 2023-05-20 08:14:01

## Introduction

We have so far built some awesome API endpoints for authenticating and authorizing users securely and in a performant way. However, apart from testing with Postman (or Thunder Client on VS Code) and &mdash; for those who went through building the frontend from the [previous series][1] &mdash; via the frontend application, we haven't made our app simple enough for anyone to just issue a simple command that runs through the app and reports whether or not the app works as expected. Also, Python is forgiving though better in that regard than JavaScript. We are bound to write our codes in styles alien to the accepted styles adopted by the community. Since Python is a dynamically typed language, we need a way to ENFORCE types on all variables used so that we won't assign a string to an integer variable. We also need to deploy our application so that everyone else can access the beauty we have built. All these are what we will address in this article.

## Assumption and Recommendation

It is assumed that you are familiar with Django. I also recommend you go through previous articles in this series so you can keep up to speed with this one.

## Source code

The source code for this series is hosted on GitHub via:

{% github Sirneij/django-auth-backend %}

## Implementation

### Step 1: Static analysis and testing setup

> You need to install `pytest-cov`, `pytest-django`, `pytest-bdd`, `pyflakes`, `pylint`, `pylint-celery`,`pytest-xdist`,`django-stubs`, and other packages. To relieve you of that burden, I have installed them and made them available in the project's `requirements_dev.txt`.

First off, let's create some bash scripts for running our tests and static analysis automatically. One will run the tests, another will enforce static analysis, and the last will help delete all test databases so that they won't cluster our machines:

At the very root of our project, create a `scripts` folder and in it create `test.sh`, `drop_test_dbs.sh` and `static_validation.sh`:

```sh
# scripts/tests.sh
#!/usr/bin/env bash
py.test -n auto --nomigrations --reuse-db -W error::RuntimeWarning --cov=src --cov-report=html tests/
```

This command uses `pytest-xdist` to allow distributed testing. We used `auto` here to denote the number of workers that will be spawned for test. `auto` equals to the number of available CPUs on your machine. Instead of auto, you can use a number such as 2, 4 or any integer. Just ensure that the number is less than or equal to the number of CPUs your machine possesses. `--nomigrations` uses `pytest-django` to disable running migrations for our tests. This makes the test suites faster. Also, `--reuse-db` uses `pytest-django` to create databases without deleting them after the tests ran. Hence the reason we need `drop_test_dbs.sh`. `--cov=src --cov-report=html` uses `pytest-cov` to help report our test stats. `-W error::RuntimeWarning` turns our runtime warnings into errors. Next is `drop_test_dbs.sh`:

```sh
# src/drop_test_dbs.sh

#!/bin/bash

PREFIX='test' || '_sqlx_test'
export PGPASSWORD=<your_db_password>
export PGUSER=<your_db_user>
export PGHOST=<your_db_host>
export PGPORT=<your_db_port>

TEST_DB_LIST=$(psql -l | awk '{ print $1 }' | grep '^[a-z]' | grep -v template | grep -v postgres)
for TEST_DB in $TEST_DB_LIST ; do
    if [ $(echo $TEST_DB | sed "s%^$PREFIX%%") != $TEST_DB ]
    then
        echo "Dropping $TEST_DB"
        dropdb --if-exists $TEST_DB
    fi
done
```

It uses your database credentials to delete all DBs that start with "test".

Next:

```sh
# scripts/static_validation.sh


#!/usr/bin/env bash

# checks whether or not the source files conform with black and isort formatting
black --skip-string-normalization --check tests
black --skip-string-normalization --check src
isort --atomic --profile black -c src
isort --atomic --profile black -c tests

cd src

# Exits with non-zero code if there is any model without a corresponding migration file
python manage.py makemigrations --check --dry-run

# Uses prospector to ensure that the source code conforms with Python best practices
prospector  --profile=../.prospector.yml --path=. --ignore-patterns=static

# Analysis and checks whether or not we have common security issues in our Python code. 
bandit -r . -ll

# Checks for correct annotations
mypy .
```

It is well commented. To ensure that your code passes them, you must run the following after each code modification:

```zsh
black --skip-string-normalization  src tests

isort --atomic --profile black src tests
```

`--skip-string-normalization` prevents black from replacing `''` with `""` or vice versa.

The repo has other important files. Moving on, we can't afford to use S3 for testing. We prefer to use the filesystem or better still, an in-memory storage. Therefore, we'll be overriding the `STORAGES` settings and others during tests. A convenient way to do this is to create a `test_settings.py` file in `src/django_auth`:

```python
# src/django_auth/test_settings.py
from django.test import override_settings

common_settings = override_settings(
    STORAGES={
        'default': {
            'BACKEND': 'django.core.files.storage.InMemoryStorage',
        },
        'staticfiles': {
            'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage',
        },
    },
    DEFAULT_FROM_EMAIL='admin@example.com',
    PASSWORD_HASHERS=[
        'django.contrib.auth.hashers.MD5PasswordHasher',
    ],
)
```

We used Django's `override_settings` to set faster storage, a faster password hasher and a default `DEFAULT_FROM_EMAIL`. We'll use this next.

### Step 2: Testing

Let's start with our `models.py`. In the `tests` package, create a `users` package and in it, `test_models.py`:

```python
from tempfile import NamedTemporaryFile

import pytest
from django.test import TestCase
from factory.django import DjangoModelFactory

from django_auth.test_settings import common_settings
from users.models import Articles, Series, User, UserProfile


class UserFactory(DjangoModelFactory):
    first_name = 'John'
    last_name = 'Doe'
    is_active = True

    class Meta:
        model = User
        django_get_or_create = ('email',)


class UserProfileFactory(DjangoModelFactory):
    class Meta:
        model = UserProfile
        django_get_or_create = ('user',)


class SeriesFactory(DjangoModelFactory):
    name = 'Some title'
    image = NamedTemporaryFile(suffix='.jpg').name

    class Meta:
        model = Series


class ArticlesFactory(DjangoModelFactory):
    title = 'Some article title'
    url = 'https://dev.to/sirneij/authentication-system-using-python-django-and-sveltekit-23e1'

    class Meta:
        model = Articles
        django_get_or_create = ('series',)


@common_settings
class UserModelTests(TestCase):
    def setUp(self):
        """Test Setup."""
        self.user = UserFactory.create(email='john@example.com')

    def test_str_representation(self):
        """Test __str__ of user."""
        self.assertEqual(str(self.user), f'{self.user.id} {self.user.email}')


@common_settings
class UserProfileModelTests(TestCase):
    def setUp(self):
        """Test Setup."""
        self.user = UserFactory.create(email='john@example.com')
        self.user_profile = UserProfileFactory.create(user=self.user)

    def test_str_representation(self):
        """Test __str__ of user."""
        self.assertEqual(
            str(self.user_profile),
            f'<UserProfile {self.user_profile.id} {self.user_profile.user.email}>',
        )

    def test_create_user_success(self):
        """Test create_user method."""
        user = User.objects.create_user(email='nelson@example.com', password='123456Data')
        self.assertEqual(user.email, 'nelson@example.com')

    def test_create_user_failure(self):
        """Test create_user method fails."""

        with pytest.raises(ValueError, match='The Email must be set'):
            User.objects.create_user(email='', password='123456Data')

    def test_create_super_user_success(self):
        """Test create_user method."""
        user = User.objects.create_superuser(email='nelson@example.com', password='123456Data')
        self.assertEqual(user.email, 'nelson@example.com')

    def test_create_super_user_failure(self):
        """Test create_user method fails."""

        with pytest.raises(TypeError, match='Superusers must have a password.'):
            User.objects.create_superuser(email='nelson@example.com', password=None)


@common_settings
class SeriesAndArticlesModelTests(TestCase):
    def setUp(self):
        """Test Setup."""
        self.series = SeriesFactory.create()
        self.articles = ArticlesFactory.create(series=self.series)

    def test_str_representation(self):
        """Test __str__ of series and articles."""
        self.assertEqual(str(self.series), self.series.name)
        self.assertEqual(str(self.articles), self.articles.title)
```

We are using Factoryboy to initialize our models. With that, we can use `ModelName.create()` to create a model instance. If we want one or more fields of the model to be supplied at creation, we use `django_get_or_create = (<tuple_of_the_fields>)` in the `Meta` class. I added some other models, `Series` and `Articles`, to help hold my articles for this project. To provide a default to an image field, I used `NamedTemporaryFile` which does exactly that. On each of the test cases, I added the `@common_settings` decorator which we imported from `test_settings.py` so that the tests will use the faster settings variables. In each test case, we tested all the important things &mdash; `__str__` of the models and other ones. We also tested our custom `UserManager`.

Next, let's test our celery task:

```python
# tests/users/test_tasks.py

from unittest.mock import patch

from django.test import TestCase

from django_auth.test_settings import common_settings
from tests.users.test_models import UserFactory
from users.tasks import send_email_message


@common_settings
class SendMessageTests(TestCase):
    @patch('users.tasks.send_mail')
    def test_success(self, send_mail):
        user = UserFactory.create(email='john@example.com')

        send_email_message(
            subject='Some subject',
            template_name='test.html',
            user_id=user.id,
            ctx={'a': 'b'},
        )
        send_mail.assert_called_with(
            subject='Some subject',
            message='',
            from_email='admin@example.com',
            recipient_list=[user.email],
            fail_silently=False,
            html_message='',
        )
```

We used `patch` from `unittest.mock` to mock `send_mail` from Django so that during testing, we don't really send any mail by mimicking sending it. This is a nice approach to make your tests predictable. We also test part of our `validate_email` util:

```python
# tests/users/test_utils.py

from django.test import TestCase

from users.utils import validate_email


class ValidateEmailTests(TestCase):
    def test_email_empty(self):
        """Test when even is empty."""
        is_valid, message = validate_email('')
        self.assertFalse(is_valid)
        self.assertEqual(message, 'Enter a valid email address.')
```

We won't talk about other tests aside from the profile_update tests:

```python
# tests/users/views/test_profile_update.py
from shutil import rmtree
from tempfile import NamedTemporaryFile, mkdtemp

from django.test import Client, TestCase
from django.test.client import BOUNDARY, MULTIPART_CONTENT, encode_multipart
from django.test.utils import override_settings
from django.urls import reverse
from django.utils import timezone

from django_auth.test_settings import common_settings
from tests.users.test_models import UserFactory


@common_settings
class UserUpdateViewTests(TestCase):
    def setUp(self) -> None:
        """Set up."""
        self.url = reverse('users:profile_update')
        self.client = Client()
        self.media_folder = mkdtemp()

    def tearDown(self):
        rmtree(self.media_folder)

    def test_update_user_not_authenticated(self):
        """Test when user is not authenticated."""
        response = self.client.patch(self.url)

        self.assertEqual(response.status_code, 401)
        self.assertEqual(
            response.json()['error'],
            'You are not logged in. Kindly ensure you are logged in and try again',
        )

    def test_update_user_success_first_name(self):
        """Test update user success with first_name."""
        # First login
        user = UserFactory.create(email='john@example.com')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'first_name': 'Owolabi'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )

        self.assertEqual(response.status_code, 200)

        user.refresh_from_db()
        self.assertEqual(user.first_name, data['first_name'])

    def test_update_user_success_last_name(self):
        """Test update user success with last_name."""
        # First login
        user = UserFactory.create(email='john@example.com')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'last_name': 'Idogun'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )

        self.assertEqual(response.status_code, 200)

        user.refresh_from_db()

        self.assertEqual(user.last_name, data['last_name'])

    def test_update_user_success_thumbnail(self):
        """Test update user success with thumbnail."""
        # First login
        user = UserFactory.create(email='john@example.com')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # Update user
        with override_settings(MEDIA_ROOT=self.media_folder):
            with NamedTemporaryFile() as f:
                f.write(b'some file data')
                f.seek(0)
                data = {'thumbnail': f}
                encoded_data = encode_multipart(BOUNDARY, data)
                response = self.client.patch(
                    self.url, encoded_data, content_type=MULTIPART_CONTENT
                )
                self.assertEqual(response.status_code, 200)

        user.refresh_from_db()

        self.assertIsNotNone(user.thumbnail)

    def test_update_user_success_phone_number(self):
        """Test update user success with phone_number."""
        # First login
        user = UserFactory.create(email='john@example.com')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'phone_number': '+2348145359073'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

        user.userprofile.refresh_from_db()

        self.assertEqual(user.userprofile.phone_number, data['phone_number'])

    def test_update_user_success_birth_date(self):
        """Test update user success with birth_date."""
        # First login
        user = UserFactory.create(email='john@example.com')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'birth_date': timezone.localdate()}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

        user.userprofile.refresh_from_db()

        self.assertEqual(user.userprofile.birth_date, data['birth_date'])

    def test_update_user_success_github_link(self):
        """Test update user success with github_link."""
        # First login
        user = UserFactory.create(email='john@example.com')
        user.set_password('12345SomeData')
        user.save()
        url_login = reverse('users:login')
        login_data = {'email': user.email, 'password': '12345SomeData'}
        response = self.client.post(
            path=url_login, data=login_data, content_type='application/json'
        )
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.json()['email'], user.email)

        # User update
        data = {'github_link': 'https://github.com/Sirneij'}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)

        user.userprofile.refresh_from_db()

        self.assertEqual(user.userprofile.github_link, data['github_link'])
```

We used some nifty tricks here. At `setUp`, we created a temporary `media_folder` to hold the uploaded test image. The folder gets deleted immediately after the tests finish running using the `tearDown` method. Since this endpoint expects a `FormData` from the request, we used Django's `encode_multipart` to encode our data. It's important to use the corresponding `BOUNDARY` from the same Django model so that the `FormData` is properly encoded. Else, our endpoint will have issues parsing the form properly and the input data will not be what is stored in the DB. For uploading an image, we did this:

```python
...
# Update user
with override_settings(MEDIA_ROOT=self.media_folder):
    with NamedTemporaryFile() as f:
        f.write(b'some file data')
        f.seek(0)
        data = {'thumbnail': f}
        encoded_data = encode_multipart(BOUNDARY, data)
        response = self.client.patch(
            self.url, encoded_data, content_type=MULTIPART_CONTENT
        )
        self.assertEqual(response.status_code, 200)
```

Again, `NamedTemporaryFile` was used to generate a temporary file and we overrode our app's `MEDIA_ROOT` to be the temporary folder we created in the `setUp` method. For each request, we ensured that we were logged in.

The testing concepts discussed here are enough to take a look at the final code test suite and not be lost.

### Step 3: Setting GitHub Actions for testing and static analysis

I assume you have a GitHub account and have been pushing your codes so far to the platform. Let's add an action to our project that runs every time we create a pull request or push to the `main` branch. You can also set it so that until a pull request passes, you cannot merge such a request to the `main` branch. Let's create a flow. To do that, create a `.github/workflows/django.yml` file:

```yaml
# .github/workflows/django.yml
name: Django-auth-backend CI

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      max-parallel: 4
      matrix:
        python-version: [3.9.13, 3.10.11, 3.11]

    services:
      postgres:
        image: postgres:latest
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: github_actions
        ports:
          - 5432:5432
        # needed because the postgres container does not provide a healthcheck
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

      redis:
        image: redis:7
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v3
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements_dev.txt
      - name: Run static analysis
        run: chmod +x ./scripts/static_validation.sh && ./scripts/static_validation.sh
      - name: Run tests
        run: chmod +x ./scripts/test.sh && ./scripts/test.sh
```

We gave it a name and we want it to run when there is a push to the `main` branch or when there is a pull request. Then, we specified our `build` jobs which use the latest version of Ubuntu to build our application against three main versions of Python, `[3.9.13, 3.10.11, 3.11]`. To run our jobs, we need a PostgreSQL database and a redis instance. Those were properly configured. Notice that we specified our database credentials.

Next, we specified the build steps which use the important `actions/checkout@v3`. The steps involve setting up Python, installing our project's dependencies, and running static analysis and tests.

### Step 4: Deployment on Vercel

We used to use Heroku for free and hobby deployments until they stopped it in October 2022. Vercel has come to the rescue and we'll deploy our Django application there. You have two options:

- Install Vercel CLI and deploy using it
- Connect your repository to Vercel and allow it to automatically deploy after each push to your repo's `main` branch or any branch of your choice.

You're at liberty to choose any method but ensure you create a file, `vercel.json`, in, for our app structure, `src`:

```json
// src/vercel.json
{
  "version": 2,
  "builds": [
    {
      "src": "django_auth/wsgi.py",
      "use": "@vercel/python",
      "config": { "maxLambdaSize": "15mb" }
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "django_auth/wsgi.py"
    }
  ]
}
```

We are using `wsgi` but you can use `asgi` as well. Ensure you have `requirements.txt` in the folder too. You can check the repo for details.

Since our application needs a database, you can use [Railway][2] to provision free PostgreSQL and Redis instances. Ensure you get their credentials and set them accordingly as your application's environment variables on Vercel.

If you use the CLI, you can run migrations and create a super user using the following steps:

- SSH into your Vercel instance using the command `vercel ssh`.
- Navigate to your app's directory and run the command `python manage.py migrate` to apply any pending migrations.
- Create a superuser by running the command `python manage.py createsuperuser` and following the prompts.

For the last step, you can set `DJANGO_SUPERUSER_PASSWORD` and `DJANGO_SUPERUSER_EMAIL` environment variables and then issue `python manage.py createsuperuser --no-input` instead.

If you decide to use Vercel UI instead, you can create a `build_files.sh` script with this content:

```shell
# build_files.sh
pip install -r requirements.txt

python3.9 manage.py migrate
python3.9 manage.py createsuperuser --no-input
```

and then modify `vercel.json`:

```json
{
  "version": 2,
  "builds": [
    {
      "src": "django_auth/wsgi.py",
      "use": "@vercel/python",
      "config": { "maxLambdaSize": "15mb" }
    },
    {
      "src": "build_files.sh",
      "use": "@vercel/static-build",
      "config": {
        "distDir": "staticfiles_build"
      }
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "django_auth/wsgi.py"
    }
  ]
}
```

This is some hack that will lead to deployment failure because we don't use file storage for our static files but those commands will run. You can then remove that segment and redeploy it.

Using the CLI allows you to incorporate the deployment process as one of the build processes of our GitHub workflow.

That's it for this series. Don't hesitate to drop your reactions, comments and contributions. You may also like me to write about anything and if I can, I will definitely oblige. I am also available for gigs.

## Outro

Enjoyed this article? Consider [contacting me for a job, something worthwhile or buying a coffee ☕](mailto:sirneij@gmail.com). You can also connect with/follow me on [LinkedIn](https://www.linkedin.com/in/idogun-john-nelson/) and [Twitter](https://twitter.com/Sirneij). It isn't bad if you help share this article for wider coverage. I will appreciate it...

[1]: https://dev.to/sirneij/series/22690 "Secure and performant full-stack authentication system using rust (actix-web) and sveltekit Series' Articles"
[2]: https://railway.app/ "Railway"