Understand Django: Making Sense Of Settings
Tags: python, django, settings
Published: 2021-07-15 14:18:42

In the last [Understand Django](https://www.mattlayman.com/understand-django/) article, we looked at a storage concept in Django called sessions. Sessions help us answer questions like "How does Django know when a user is logged in?" or "Where can the framework store data for a visitor on your app?"

With this article, you'll learn about Django settings and how to manage the configuration of your application. We'll also look at tools to help you to be extra effective with settings.



## How Is Django Configured?

To run properly, Django needs to be configured. We need to understand where this configuration comes from. Django has the ability to use default configuration values or values set by developers like yourself, but where does it get those from?

Early in the process of starting a Django application, Django will internally import the following:

```python
from django.conf import settings
```

This `settings` import is a module level object created in `django/conf/__init__.py`. The `settings` object has attributes added to it from two primary sources.

The first source is a set of global default settings that come from the framework. These global settings are from `django/conf/global_settings.py` and provide a set of initial values for configuration that Django needs to operate.

The second source of configuration settings comes from user defined values. Django will accept a Python module and apply its module level attributes to the `settings` object. To find the user module, Django searches for a `DJANGO_SETTINGS_MODULE` environment variable.

### Sidebar: Environment Variables

Environment variables are not a Django concept. When any program runs on a computer, the operating system makes certain data available to the running program. This set of data is called the program's "environment," and each piece of data in that set is an environment variable.

If you're starting Django from a terminal, you can view the environment variables that Django will receive from the operating system by running the `env` command on macOS or Linux, or the `set` command on Windows.

We can add our own environment variables to the environment with the `export` command on macOS or Linux, or the `set` command on Windows. Environment variables are typically named in all capital letters.

```bash
$ export HELLO=world
```

Now that we have a base understanding of environment variables, let's return to the `DJANGO_SETTINGS_MODULE` variable. The variable's value should be the location of a Python module containing any settings that a developer wants to change from Django's default values.

If you create a Django project with `startproject` and use `project` as the name, then you will find a generated file called `project/settings.py` in the output. When Django runs, you could explicitly instruct Django with:

```bash
$ export DJANGO_SETTINGS_MODULE=project.settings
```

Instead of supplying the file path, the `DJANGO_SETTINGS_MODULE` should be in a Python module dotted notation.

You may not actually need to set `DJANGO_SETTINGS_MODULE` explicitly. If you stick with the same settings file that is created by `startproject`, you can find a line in `wsgi.py` that looks like:

```python
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
```

Because of this line, Django will attempt to read from `project.settings` (or whatever you named your project) without the need to explicitly set `DJANGO_SETTINGS_MODULE`.

Once Django reads the global settings and any user defined settings, we can get any configuration from the `settings` object via attribute access. This convention of keeping all configuration in the `settings` object is a convenient pattern that the framework, third party library ecosystem, and *you* can depend on.

```python
$ ./manage.py shell
>>> from django.conf import settings
>>> settings.SECRET_KEY
'a secret to everybody'
```

The `settings` object is a shared item so it is generally thought to be a Really Bad Idea™ to edit and assign to the object directly. Keep your settings in your settings module!

That's the core of Django configuration. We're ready to focus in on the user defined settings and our responsibilities as Django app developers.

## Settings Module Patterns

There are multiple ways to deal with settings modules and how to populate those modules with the appropriate values for different environments. Let's look at some popular patterns.

### Multiple Modules Per Environment

A Django settings module is a Python module. Nothing is stopping us from using the full power of Python to configure that module the way we want.

Minimally, you will probably have at least two environments where your Django app runs:

* On your local machine while developing
* On the internet for your live site

We should know by now that setting `DEBUG = True` is a terrible idea for a live Django site, so how can we get the benefits of the debug mode without having `DEBUG` set to `True` in our module?

One technique is to use separate settings modules. With this strategy, you can pick which environment your Django app should run for by switching the `DJANGO_SETTINGS_MODULE` value to pick a different environment. You might have modules like:

* `project.settings.dev`
* `project.settings.stage`
* `project.settings.production`

These examples would be for a local development environment on your laptop, a staging environment (which is a commonly used pattern for testing a site that is as similar to the live site as possible without *being* the live site), and a production environment. As a reminder from the deployment article, the software industry like to call the primary site for customers "production."

This strategy has certain challenges to consider. Should you replicate settings in each file or use some common module between them?

If you decide to replicate the settings across modules, you'll have the advantage that the settings module shows *all* of the settings in a single place for that environment. The disadvantage is that keeping the common settings the same could be a challenge if you forget to update one of the modules.

On the other hand, you could use a common module. The advantage to this form is that the common settings can be in a single location. The environment specific files only need to record the *differences* between the environments. The disadvantage is that it is harder to get a clear picture of all the settings of that environment.

If you decide to use a common module, this style is often implemented with a `*` import. I can probably count on one hand the number of places where I'm ok with a `*` import, and this is one of them. In most cases the Python community prefers explicit over implicit, and the idea extends to the treatment of imports. Explicit imports make it clear what a module is actually using. The `*` import is very implicit, and it makes it unclear what a module uses. For the case of a common settings module, a `*` import is actually positive because we want to use *everything* in the common module.

Let's make this more concrete. Assume that you have a `project.settings.base` module. This module would hold your common settings for your app. I'd recommend that you try to make your settings safe and secure by default. For instance, use `DEBUG = False` in the base settings and force other settings modules to opt-in to the more unsafe behavior.

For your local development environment on your laptop, you could use `project.settings.dev`. This settings module would look like:

```python
# project/settings/dev.py

from project.settings.base import *

DEBUG = True

# Define any other settings that you want to override.
...
```

By using the `*` import in the `dev.py` file, all the settings from `base.py` are pulled into the module level scope. Where you want a setting to be different, you set the value in `dev.py`. When Django starts using `DJANGO_SETTINGS_MODULE` of `project.settings.dev`, all the values from `base.py` will be used via `dev.py`.

This scheme gives you control to define common things once, but there is still a big challenge with this. What do we do about settings that need to be kept secret (e.g., API keys)?

*Don't commit secret data to your code repository!* Adding secrets to your source control tool like Git is usually not a good idea. This is especially true if you have a public repository on GitHub. Think no one is paying attention to your repo? Think again! There are tools out there that scan *every public commit* made to GitHub. These tools are specifically looking for secret data to exploit.

If you can't safely add secrets to your code repo, where can we add them instead? You can use environment variables! Let's look at another scheme for managing settings with environment variables.

### Settings Via Environment Variables

In Python, you can access environment variables through the `os` module. The module contains the `environ` attribute, which functions like a dictionary.

By using environment variables, your settings module can get configuration settings from the external environment that is running the Django app. This is a solid pattern because it can accomplish two things:

* Secret data can be kept out of your code
* Configuration differences between environments can be managed by changing environment variable values

Here's an example of secret data management:

```python
# project/settings.py

import os

SECRET_KEY = os.environ['SECRET_KEY']

...
```

Django needs a secret key for a variety of safe hashing purposes. There is a warning in the default `startproject` output that reads:

```python
# SECURITY WARNING: keep the secret key used in production secret!
```

By moving the secret key value to an environment variable that happens to have a matching name of `SECRET_KEY`, we won't be committing the value to source control for some nefarious actor to discover.

This pattern works really well for secrets, but it can also work well for *any* configuration that we want to vary between environments.

For instance, on one of my projects, I use the excellent [Anymail](https://anymail.readthedocs.io/en/stable/) package to send emails via an email service provider (of the ESPs, I happen to use [SendGrid](https://sendgrid.com/)). When I'm working with my development environment, I don't want to send real email. Because of that, I use an environment variable to set Django's `EMAIL_BACKEND` setting. This let's me switch between the Anymail backend and Django's built-in `django.core.mail.backends.console.EmailBackend` that prints emails to the terminal instead.

If I did this email configuration with `os.environ`, it would look like:

```python
# project/settings.py

import os

EMAIL_BACKEND = os.environ.get(
    'EMAIL_BACKEND', "anymail.backends.sendgrid.EmailBackend")

...
```

I prefer to make my default settings closer to the live site context. This not only leads to safer behavior (because I have to explicitly opt-out of safer settings like switching to `DEBUG = False`), but it also means that my live site has less to configure. That's good because there are fewer chances to make configuration mistakes on the site that matters most: the one where my customers are.

We need to be aware of a big gotcha with using environment variables. *Environment variables* are only available as a `str` type. This is something to be aware because there will be times when you want a boolean settings value or some other *type* of data. In a situation where you need a different type, you have to coerce a `str` into the type you need. In other words, don't forget that every string except the empty string is truthy in Python:

```python
>>> not_false = "False"
>>> bool(not_false)
True
```

In the next section, we will see tools that help alleviate this typing problem.

Note: As you learn more about settings, you will probably encounter advice that says to avoid using environment variables. This is well intentioned advice that highlights that there *is* some risk with using environment variables. With this kind of advice, you may read a recommendation for secrets management tools like [HashiCorp Vault](https://www.vaultproject.io/). These are good tools, but consider them a more advanced topic. In my opinion, using environment variables for secrets management is a reasonably low risk storage mechanism.

## Settings Management Tools

We can focus on two categories of tools that can help you manage your settings in Django: built-in tools and third party libraries.

The built-in tool that is available to you is the `diffsettings` command. This tool makes it easy to see the computed settings of your module. Since settings can come from multiple files (including Django's `global_settings.py`) or environment variables, inspecting the settings output of `diffsettings` is more convenient than thinking through how a setting is set.

By default, `diffsettings` will show a comparison of the settings module to the default Django settings. Settings that aren't in the defaults are marked with `###` after the value to indicate that they are different.

I find that the default output is not the most useful mode. Instead, you can instruct `diffsettings` to output in a "unified" format. This format looks a lot more like a code diff. In addition, Django will colorize that output so that it's easier to see. Here's an example of some of the security settings by running `./manage.py diffsettings --output unified` for one of my projects.

```diff
- SECURE_HSTS_INCLUDE_SUBDOMAINS = False
+ SECURE_HSTS_INCLUDE_SUBDOMAINS = True
- SECURE_PROXY_SSL_HEADER = None
+ SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
```

Finally, I'll note that you can actually compare two separate settings modules. Let's say you wanted to compare settings between your development mode and your live site. Assuming your settings files are names like I described earlier, you could run something like:

```bash
$ ./manage.py diffsettings \
    --default project.settings.dev \
    --settings project.settings.prod \
    --output unified
```

By using the `--default` flag, we instruct Django that `project.settings.dev` is the baseline for comparison. This version of the command will show where the two settings modules are different.

Django only includes this single tool for working with settings, but I hope you can see that it's really handy. Now let's talk about a useful third party library that can help you with settings.

Earlier in the article, I noted that dealing with environment variables has the pitfall of working with string data for everything. Thankfully, there is a package that can help you work with environment variables. The project is called [django-environ](https://django-environ.readthedocs.io/en/latest/). django-environ primarily does two important things that I value:

* The package allows you coerce strings into a desired data type.
* The package will read from a file to load environment variables into your environment.

What does type coercion look like? With `django-environ`, you start with `Env` object.

```python
# project/settings.py

import environ

env = environ.Env()
```

The keyword arguments to `Env` describe the different environment variables that you expect the app to process. The key is the name of the environment variable. The value is a two element tuple. The first tuple element is the type you want, and the second element is a default value if the environment variable doesn't exist.

If you want to be able to control `DEBUG` from an environment variable, the settings would be:

```python
# project/settings.py

import environ

env = environ.Env(
    DEBUG=(bool, False),
)

DEBUG = env("DEBUG")
```

With this setup, your app will be safe by default with `DEBUG` set to `False`, but you'll be able to override that via the environment. `django-environ` works with a handful of strings that it will accept as `True` such as "on", "yes", "true", and others (see the documentation for more details).

Once you start using environment variables, you'll want an convenient way to set them when your app runs. Manually calling `export` for all your variables before running your app is a totally unsustainable way to run apps.

The `Env` class comes with a handy class method named `read_env`. With this method, your app can read environment variables into `os.environ` from a file. Conventionally, this file is named `.env`, and the file contains a list of key/value pairs that you want as environment variables. Following our earlier example, here's how we could set our app to be in debug mode:

```env
# .env
DEBUG=on
```

Back in the settings file, you'd include `read_env`:

```python
# project/settings.py

import environ

environ.Env.read_env()
env = environ.Env(
    DEBUG=(bool, False),
)

DEBUG = env("DEBUG")
```

If you use a `.env` file, you will occasionally find a need to put secrets into this file for testing. Since the file can be a source for secrets, you should add this to `.gitignore` or ignore it in whatever version control system you use. As time goes on, the list of variables and settings will likely grow, so it's also a common pattern to create a `.env.example` file that you can use as a template in case you ever need to start with a fresh clone of your repository.

## My Preferred Settings Setup

Now we've looked at multiple strategies and tools for managing settings. I've used many of these schemes on various Django projects, so what is my preferred setup?

For the majority of uses, I find that working with `django-environ` in a single file is the best pattern in my experience.

When I use this approach, I make sure that all of my settings favor a safe default configuration. This minimizes the configuration that I have to do for a live site.

I like the flexibility of the pattern, and I find that I can quickly set certain configurations when developing. For instance, when I want to do certain kinds of testing like checking email rendering, I'll call something like:

```bash
$ EMAIL_TESTING=on ./manage.py runserver
```

My settings file has a small amount of configuration to alter the email settings to point emails to a local SMTP server tool called [MailHog](https://github.com/mailhog/MailHog). Because I set an environment variable directly on my command line call, I can easily switch into a mode that sends email to MailHog for quick review.

Overall, I like the environment variable approach, but I do use more than one settings file for one important scenario: testing.

When I run my unit tests, I want to guarantee that certain conditions are always true. There are things that a test suite should never do in the vast majority of cases. Sending real emails is a good example. If I happen to configure my `.env` to test real emails for the local environment, I don't want my tests to send out an emails accidentally.

Thus, I create a separate testing settings file and configure my test runner (pytest) to use those settings. This settings file *does* mostly use the base environment, but I'll override some settings with explicit values. Here's how I protect myself from accidental live emails:

```python
# project/testing_settings.py

from .settings import *

# Make sure that tests are never sending real emails.
EMAIL_BACKEND = "django.core.mail.backends.locmem.EmailBackend"
```

Even though my `Env` will look for an `EMAIL_BACKEND` environment variable to configure that setting dynamically, the testing setting is hardcoded to make email sending accidents impossible.

The combination of a single file for most settings sprinkled with a testing settings file for safety is the approach that has worked the best for me.

## Summary

In this article, you learned about Django settings and how to manage the configuration of your application. We covered:

* How Django is configured
* Patterns for working with settings in your projects
* Tools that help you observe and manage settings

In the next article, we will look at how to handle files and media provided by users (e.g., profile pictures). You'll learn about:

* How Django models maintain references to files
* How the files are managed in Django
* Packages that can store files in various cloud services

If you'd like to follow along with the series, please feel free to sign up for my newsletter where I announce all of my new content. If you have other questions, you can reach me online on Twitter where I am [@mblayman](https://twitter.com/mblayman).