Understand Django: Command Your App
Tags: python, django, commands
Published: 2021-11-04 14:35:19

In the last [Understand Django](https://www.mattlayman.com/understand-django/) article, we dug into file management. We saw how Django handles user uploaded files and how to deal with them safely.

With this article, you'll learn about commands. Commands are the way to execute scripts that interact with your Django app. We'll see built-in commands and how to build your own.



## Why Commands?

Django makes it possible to run code from a terminal with `./manage.py`, but why is this helpful or needed? Consider this script:

```python
# product_stat.py

from application.models import Product

print(Product.objects.count())
```

which you could try running with:

```bash
$ python product_stat.py
```

The problem with this script is that Django is not ready to run yet. If you tried to run this kind of code, you would get an `ImproperlyConfigured` exception. There are a couple of modifications that you could make to get the script to run.

* Call `django.setup()`.
* Specify the `DJANGO_SETTINGS_MODULE`.

```python
# product_stat.py
import django

django.setup()

from application.models import Product

print(Product.objects.count())
```

Note that `django.setup()` must be before your Django related imports (like the `Product` model in this example). Now the script can run if you supply where the settings are located too.

```bash
$ DJANGO_SETTING_MODULE=project.settings python product_stat.py
```

This arrangement is less than ideal, but why else might we want a way to run commands through Django?

Try running `./manage.py -h`.

What you'll likely see is more commands than what Django provides alone. This is where we begin to see more value from the command system. Because Django provides a standard way to run scripts, other Django applications can bundle useful commands and make them easily accessible to you.

Now that you've had a chance to see why commands exist to run scripts for Django apps, let's back up and see *what* commands are.

## We Hereby Command

Django gives us a tool to run commands before we've even started our project. That tool is the `django-admin` script. We saw it all the way back in the first article where I provided a short set of setup instructions to get you started if you've never used Django before.

After you've started a project, your code will have a `manage.py` file, and the commands you've seen in most articles are in the form of:

```bash
$ ./manage.py some_command
```

What's the difference between `django-admin` and `manage.py`? In truth, **not much!**

`django-admin` comes from Django's Python packaging. In Python packages, package developers can create scripts by defining an entry point in the packaging configuration. In Django, this configuration looks like:

```ini
[options.entry_points]
console_scripts =
    django-admin = django.core.management:execute_from_command_line
```

Meanwhile, the entire `manage.py` of one of my projects is:

```python
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys


def main():
    os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == '__main__':
    main()
```

If you look closely, you can see that the different scripts are both ways to invoke the `execute_from_command_line` function to run a command. The primary difference is that the latter script will attempt to set the `DJANGO_SETTINGS_MODULE` environment variable automatically. Since Django needs to have `DJANGO_SETTINGS_MODULE` defined for most commands (note: `startproject` does *not* require that variable), `manage.py` is a more convenient way to run commands.

`execute_from_command_line` is able to present what commands are available to a project, whether a command comes from Django itself, an installed app, or is a custom command that you created yourself. How are the commands discovered? *The command system does discovery by following some packaging conventions.*

Let's say your project has an app named `application`. Django can find the command if you have the following packaging structure.

```text
application
├── __init__.py
├── management
│   ├── __init__.py
│   └── commands
│       ├── __init__.py
│       └── custom_command.py
├── models.py
└── views.py
... Other typical Django app files
```

With this structure, you could run:

```bash
$ ./manage.py custom_command
```

Notes:

* Django will create a command for a module found in `<app>/management/commands/<command name>.py`.
* Don't forget the `__init__.py` files! Django can only discover the commands if `management` and `commands` are proper Python package directories.
* The example uses `custom_command`, but you can name your command with whatever valid Python module name that you want.

Unfortunately, we can't slap some Python code into `custom_command.py` and assume that Django will know how to run it. Within the `custom_command.py` module, Django needs to find a `Command` class that subclasses a `BaseCommand` class that is provided by the framework. Django requires this structure to give command authors a consistent way to access features of the command system.

With the `Command` class, you can add a `help` class attribute. Adding help can gives users a description of what your command does when running `./manage.py custom_command -h`.

The `Command` class will also help you with handling arguments. If your command needs to work with user input, you'll need to parse that data. Thankfully, the class integrates with Python's built-in `argparse` module. By including an `add_arguments` method, a command can parse the data and pass the results to the command's handler method in a structured way. If you've had to write Python scripts before, then you may understand how much time this kind of parsing can save you (and for those who haven't, the answer is "a lot of time!").

Other smaller features exist within the `Command` class too. Perhaps you only want your command to run if your project has satisified certain pre-conditions. Commands can use the `requires_migration_checks` or `requires_system_checks` to ensure that the system is in the correct state before running.

I hope it's clear that the goal of the `Command` class is to help you with common actions that many commands will need to use. There is a small API to learn, but the system is a boon to making scripts quickly.

## Command By Example

Let's consider a powerful use case to see a command in action. When you initially start a Django app, all of your app's interaction will probably be through web pages. After all, you were trying to use Django to make a web app, right? What do you when you need to do something that doesn't involve a browser?

This kind of work for your app is often considered *background* work. Background work is a pretty deep topic and will often involve special background task software like [Celery](https://docs.celeryproject.org/en/stable/getting-started/introduction.html). When your app is an early stage, Celery or similar software can be overkill and far more than you need.

A simpler alternative for some background tasks could be a command paired with a scheduling tool like [cron](https://en.wikipedia.org/wiki/Cron).

On one of my projects, I offer free trials for accounts. After 60 days, the free trial ends and users either need to pay for the service or discontinue using it. By using a command and pairing it with the [Heroku Scheduler](https://devcenter.heroku.com/articles/scheduler), I can move accounts from their trial status to expired with a daily check.

The following code is very close to what my `expire_trials` command looks like in my app. I've simplified things a bit, so that you can ignore the details that are specific to my service.

```python
# application/management/commands/expire_trials.py

import datetime

from django.core.management.base import BaseCommand
from django.utils import timezone

from application.models import Account


class Command(BaseCommand):
    help = "Expire any accounts that are TRIALING beyond the trial days limit"

    def handle(self, *args, **options):
        self.stdout.write("Search for old trial accounts...")
        # Give an extra day to be gracious and avoid customer complaints.
        cutoff_days = 61
        trial_cutoff = timezone.now() - datetime.timedelta(days=cutoff_days)
        expired_trials = Account.objects.filter(
            status=Account.TRIALING, created__lt=trial_cutoff
        )
        count = expired_trials.update(status=Account.TRIAL_EXPIRED)
        self.stdout.write(f"Expired {count} trial(s)")
```

I configured the scheduler to run `python manage.py expire_trials` every day in the early morning. The command checks the current time and looks for `Account` records in the trialing state that were created before the cutoff time. From that query set, the affected accounts are set to the expired account state.

How can you test this command? There are a couple of approaches you can take when testing a command.

If you need to simulate calling the command with command line arguments, then you can use `call_command` from `django.core.management`. Since the example command doesn't require arguments, I didn't take that approach.

Generally, my preference is to create a command object and invoke the `handle` method directly. In my example above, you can see that the command uses `self.stdout` instead of calling `print`. Django does this so that you could check your output if desired.

Here is a test for this command:

```python
# application/tests/test_commands.py

from io import StringIO

from application.models import Account
from application.tests.factories import AccountFactory

def test_expires_trials():
    """Old trials are marked as expired."""
    stdout = StringIO()
    account = AccountFactory(
        status=Account.TRIALING,
        created=timezone.now() - datetime.timedelta(days=65),
    )
    command = Command(stdout=stdout)

    command.handle()

    account.refresh_from_db()
    assert account.status == Account.TRIAL_EXPIRED
    assert "Expired 1 trial(s)" in stdout.getvalue()
```

In this test, I constructed a command instance and checked the account state after the command invocation. Also, observe that the `StringIO` instance is injected into the `Command` constructor. By building the command this way, checking the output becomes a very achievable task via the `getvalue` method.

Overall, this scheme of making a command and running it on a schedule avoids all the work of setting up a background worker process. I've been extremely satisfied with how this technique has worked for me, and I think it's a great pattern when your app doesn't have to do a lot of complex background processing.

## Useful Commands

Django is full of [useful commands](https://docs.djangoproject.com/en/3.2/ref/django-admin/) that you can use for all kinds of purposes. Thus far in this series, we've discussed a bunch of them, including:

* `check` - Checks that your project is in good shape.
* `collectstatic` - Collects static files into a single directory.
* `createsuperuser` - Creates a super user record.
* `makemigrations` - Makes new migration files based on model changes.
* `migrate` - Runs any unapplied migrations to your database.
* `runserver` - Runs a development web server to check your app.
* `shell` - Starts a Django shell that allows you to use Django code on the command line.
* `startapp` - Makes a new Django app from a template.
* `startproject` - Makes a new Django project from a template.
* `test` - Executes tests that checks the validity of your app.

Here is a sampling of other commands that I find useful when working with Django projects.

### `dbshell`

The `dbshell` command starts a different kind of shell. The shell is a database program that will connect to the same database that your Django app uses. This shell will vary based on your choice of database.

For instance, when using PostgreSQL, `./manage.py dbshell` will start `psql`. From this shell, you can execute SQL statements directly to inspect the state of your database. I don't reach for this command often, but I find it very useful to connect to my database without having to remember database credentials.

### `showmigrations`

The `showmigrations` command has a simple job. The command shows all the migrations for each Django app in your project. Next to each migration is an indicator of whether the migration is applied to your database.

Here is an example of the `users` app from one of my Django projects:

```bash
$ ./manage.py showmigrations users
users
 [X] 0001_initial
 [X] 0002_first_name_to_150_max
 [ ] 0003_profile
```

In my real project, I've applied all the migrations, but for this example, I'm showing the third migration as it would appear if the migration wasn't applied yet.

`showmigrations` is a good way to show the state of your database from Django's point of view.

### `sqlmigrate`

The `sqlmigrate` command is *very* handy. The command will show you what SQL statements Django would run for an individual migration file.

Let's see an example. In Django 3.1, the team changed the `AbstractUser` model so that the `first_name` field could have a maximum length of 150 characters. Anyone using the `AbstractUser` model (which includes me) had to generate a migration to apply that change.

From my `showmigrations` output above, you can see that the second migration of my `users` app applied this particular framework change.

To see the Postgres SQL statements that made the change, I can run:

```bash
$ ./manage.py sqlmigrate users 0002
BEGIN;
--
-- Alter field first_name on user
--
ALTER TABLE "users_user" ALTER COLUMN "first_name" TYPE varchar(150);
COMMIT;
```

From this, we can tell that Postgres executed an `ALTER COLUMN` [DDL](https://en.wikipedia.org/wiki/Data_definition_language) statement to modify the length of the `first_name` field.

### `squashmigrations`

Django migrations are a stack of separate database changes that produce a final desired schema state in your database. Over time, your Django apps will accumulate migration files, but those files have a shelf life. The `squashmigrations` command is designed to let you tidy up an app's set of migration files.

By running `squashmigrations`, you can condense an app's migrations into a significantly smaller number. The reduced migrations can accurately represent your database schema, and make it easier to reason about what changes happened in the app's history. As a side benefit, migration squashing can make Django's migration handling faster because Django gets to process fewer files.

## Even More Useful Commands

The commands above come with the standard Django install. There's even more cool stuff out there to help with your project development!

A package that I often reach for with my Django projects is the [django-extensions](https://django-extensions.readthedocs.io/en/latest/index.html) package. This package is fully of goodies, including some great optional commands that you can use!

A couple of my favorites include:

### `shell_plus`

How often do you fire up a Django shell, import a model, then do some ORM queries to see the current state of the database? This is something I do *quite* often.

The `shell_plus` command is like the regular shell, but the command will import all your models *automatically*. For the five extra characters of `_plus`, you can save your fingers a lot of typing to import your models and get directly to whatever you needed the shell for.

The command will also import some commonly used Django functions and features like `reverse`, `settings,` `timezone`, and more.

### `graph_models`

When I'm live streaming my side projects on [my Twitch channel](https://www.twitch.tv/mblayman), I will often want to show the model relationships of my Django project. With the `graph_models` command, I can create an image of all my models and how those models relate to each other (using UML syntax). This is a great way to:

* Remind myself of the data modeling choices in my apps.
* Orient others to what I'm doing with my project.

This particular command requires some extra setup to install the right tools to create images, but the setup is manageable and the results are worth it.

Aside from `shell_plus` and `graph_models`, there are 20 other commands that you can use that may be very useful to you. You should definitely check out django-extensions.

## Summary

In this article, you saw Django commands. We covered:

* Why commands exist in the Django framework
* How commands work
* How to create your own custom command and how to test it
* Useful commands from the core framework and the django-extensions package

In the next article, I'm going to describe internationalization. Internationalization is the process of making your app work for multiple spoken languages. You'll learn about:

* Enabling internationalization for your app
* Tools to help manage an internationalized app
* Extra considerations like handling time formatting

If you'd like to follow along with the series, please feel free to sign up for my newsletter where I announce all of my new content. If you have other questions, you can reach me online on Twitter where I am [@mblayman](https://twitter.com/mblayman).