From Project to Productionized with Python
Tags: python, webdev, tutorial
Published: 2020-07-16 18:20:20

---
title: From Project to Productionized with Python
published: true
date: 2020-06-22 16:00:00 UTC
tags: python, webdev, tutorial
canonical_url: https://blog.heroku.com/from-project-to-productionized-python
---

We hope that you and your loved ones are staying safe from the COVID-19 pandemic. As a result of its effect on large gatherings, [PyCon 2020 was ~~cancelled~~ changed to an online event](https://pycon.blogspot.com/2020/03/pycon-us-2020-in-pittsburgh.html). Although not being able to gather in person was disheartening for organizers, speakers, and attendees, the Python community shared virtual high-fives and hugs with [PyCon US 2020 Online.](https://us.pycon.org/2020/online/) We [recorded our planned Heroku workshop for the event](https://www.youtube.com/watch?v=1923eduj0Gg), on which this blog post is based.

<iframe title="From Project to Productionized on Heroku" src="https://www.youtube-nocookie.com/embed/1923eduj0Gg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

Imagine that you've just spent the last two weeks pouring all your energy into an application. It's magnificent, and you're finally ready to share it on the Internet. How do you do it? In this post, we're going to walk through the hands-on process aimed at Python developers deploying their local application to Heroku.

An application running on Heroku works best as [a 12-factor application](https://12factor.net/). This is actually a concept that Heroku championed over 10 years ago. It's the idea that you build an application with robust redeployments in mind. Most of this workshop is actually not specific to Heroku, but rather, about taking a regular Django application and making it meet the 12 factor app methodology, which has become a standard that most cloud deployment providers not only support but recommend.

## [Prerequisites](#prerequisites)

Before completing this workshop, we're going to make a few assumptions about you, dear reader. First, this is not going to be a Django tutorial. If you're looking for an introduction to Django, [their documentation has some excellent tutorials to follow](https://www.djangoproject.com/start/). You will also need a little bit of [Git](https://git-scm.com) familiarity, and have it installed on your machine.

In order to complete this workshop, you'll need a few things:

1. [An account on Heroku](https://signup.heroku.com/). This is completely free and doesn't require any payment information.
2. [The Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli#download-and-install). Once your application is on Heroku, this will make managing it much easier.
3. You'll need to [clone the repository for this workship](https://github.com/heroku-python/PyCon2020), and be able to open it in a text editor.

With all that sorted, it's time to begin!

## [Look around you](#look-around-you)

With the project cloned and available on your computer, take a moment to explore its structure. We'll be modifying the `manage.py` and `requirements.txt` files, as well as `settings.py` and `wsgi.py` in the `gettingstarted` folder.

### [Updating `.gitignore`](#updating-code-gitignore-code)

To begin with, we'll be updating the gitignore file. [A gitignore file excludes files](https://git-scm.com/docs/gitignore) which you don't want to check into your repository. In order to deploy to Heroku, you don't technically need a gitignore file. You can deploy successfully without one, but it's highly recommended to always have one (and not just for Heroku). A gitignore can be essential for keeping out passwords and credentials keys, large binary files, local configurations, or anything else that you don't want to expose to the public.

Copy the following block of code and paste it into the gitignore file in the root of your project:

```
/venv
__pycache__
db.sqlite3 # not needed if you're using Postgres locally
gettingstarted/static/

```

The `venv` directory contains [a virtual environment](https://docs.python.org/3/tutorial/venv.html) with the packages necessary for your local Python version. Similarly, the ` __pycache__ ` directory contains [precompiled modules unique to your system](https://docs.python.org/3/tutorial/modules.html#compiled-python-files). We don't want to check in our database (`db.sqlite3`), as we don't want to expose any local data. Last, the static files will be automatically generated for us during the build and deploy process to Heroku, so we'll exclude the `gettingstarted/static/` directory.

Go ahead and run `git status` on your terminal to make sure that gitignore is the only file that's been modified. After that, call `git add`, then `git commit -m "step 1 add git ignore"`.

### [Modularize your settings](#modularize-your-settings)

Next up, we want to modularize our Django settings. To do that, add a new folder within `gettingstarted` called `settings`. Then, move the `settings.py` file into that directory. Since this naming scheme is a bit confusing, let's go ahead and rename that file to `base.py`. We'll call it that because it will serve as the base (or default) configuration that all the other configurations are going to pull from. If something like `dev.py` or `local.py` makes more sense to you, feel free to use that instead!

Local projects only have one environment to keep track of: your local machine. But once you want to deploy to different places, it's important to keep track of what settings go where. Nesting our settings files this way makes it easy for us to keep track of where those settings are, as well as take advantage of Heroku's continuous delivery tool pipelines.

By moving and renaming the settings file, our Django application now has two broken references. Let's fix them before we move on.

The first is in the `wsgi.py` in your `gettingstarted` folder. Open it up, and on [line 12](https://github.com/heroku-python/PyCon2020/blob/1a4cf7eabfcc994f60f4b8efeed1f0d9a245e768/gettingstarted/wsgi.py#L12) you'll see that a default Django settings module is being set to `gettingstarted.settings`, a file which no longer exists:

```python
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "gettingstarted.settings")

```

To fix this, append the name of the file you just created in the settings subfolder. For example, since we called ours `base.py`, the line should now look like this:

```python
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "gettingstarted.settings.base")

```

After saving that, navigate up one directory to `manage.py`. On [line 6](https://github.com/heroku-python/PyCon2020/blob/1a4cf7eabfcc994f60f4b8efeed1f0d9a245e768/manage.py#L6), you'll see the same default being set for the Django settings module. Once again, append `.base` to the end of this line, then commit both of them to Git.

## [Continuous delivery pipelines](#continuous-delivery-pipelines)

In an application's deployment lifecycle, there are typically four stages:

1. You build your app in the development stage on your local machine to make sure it works.
2. Next comes the review stage, where you check to see if your changes pass with the full test suite of your code base.
3. If that goes well, you merge your changes to staging. This is where you have conditions as close to public as possible, perhaps with some dummy data available, in order to more accurately predict how the change will impact your users.
4. Lastly, if all that goes well, you push to production, where the change is now live for your customers.

Continuous delivery (CD) workflows are designed to test your change in conditions progressively closer and closer to production and with more and more detail. Continuous delivery is a powerful workflow that can make all of the difference in your experience as a developer once you've productionized your application. Heroku can save you a lot of time here, as we've already built the tools for you to have a continuous delivery workflow. From your dashboard on Heroku, you can—with the mere click of a button!–[set up a pipeline](https://devcenter.heroku.com/articles/pipelines), add applications to staging and production, and deploy them.

If you [connect your GitHub repository](https://devcenter.heroku.com/articles/github-integration), pipelines can also automatically deploy and test new PRs opened on your repo. By providing the tooling and automating these processes, Heroku's continuous delivery workflow is powerful enough to help you keep up with your development cycle.

## [Adding new middleware to `base.py`](#adding-new-middleware-to-code-base-py-code)

Modularizing your Django settings is a great way to take advantage of this continuous delivery workflow by splitting up your settings, whether you're deploying to Heroku or elsewhere, but there's one more change we have to make to `base.py`.

Django static assets work best when you also use the [whitenoise](http://whitenoise.evans.io/en/stable/django.html) package to manage your static assets. It's really easy to add to your project.

In your `base.py` file, scroll down to about line 43, and you should see an array of package names like this:

```python
MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    # Whitenoise goes here
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
]

```

This is your list of [Django middleware](https://docs.djangoproject.com/en/3.0/topics/http/middleware/), which are sort of like plugins for your server. Django loads your middleware in the order that it's listed, so you always want your security middleware first, but it's important to add whitenoise as the second step in this base file.

Copy the following line of code and replace the line that says `Whitenoise goes here` with this:

```python
"whitenoise.middleware.WhiteNoiseMiddleware",

```

We've loaded whitenoise as middleware, but to actually _use_ the whitenoise compression, we need to set one more variable. Copy the following code and paste it right at the end of your `base.py` file:

```python
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"

```

With that, we're done with `base.py`. Congratulations! Save your work and commit it to Git.

## [Setting up `heroku.py`](#setting-up-code-heroku-py-code)

Our base settings are complete, but now we need our Heroku-specific settings. Create a new file under `gettingstarted/settings` called `heroku.py` and paste the following block of code:

```python
"""
Production Settings for Heroku
"""

import environ

# If using in your own project, update the project namespace below
from gettingstarted.settings.base import *

env = environ.Env(
    # set casting, default value
    DEBUG=(bool, False)
)

# False if not in os.environ
DEBUG = env('DEBUG')

# Raises django's ImproperlyConfigured exception if SECRET_KEY not in os.environ
SECRET_KEY = env('SECRET_KEY')

ALLOWED_HOSTS = env.list('ALLOWED_HOSTS')

# Parse database connection url strings like psql://user:pass@127.0.0.1:8458/db
DATABASES = {
    # read os.environ['DATABASE_URL'] and raises ImproperlyConfigured exception if not found
    'default': env.db(),
}

```

You can see in this file the values that we're listing here are the ones that we're overriding from our base settings, so these are the settings that will be different and unique for Heroku.

To do this, we're using one of my favorite packages, [Django-environ](https://github.com/joke2k/django-environ). This allows us to quickly and easily interface with the operating system environment without knowing much about it. It has built-in type conversions, and in particular it has automatic database parsing. This is all we need in order to parse our Heroku Postgres database URL that we will be given. It's just really convenient.

## [Heroku-specific files](#heroku-specific-files)

That's all the work we need to do to get our application into 12 factored shape, but there are three more files we need in order to deploy to Heroku.

### [`requirements.txt`](#code-requirements-txt-code)

In addition to the packages your project already uses, there are a few more you need to deploy to Heroku. If we take a look at the provided `requirements.txt` file, you can see these required packages here. We've already talked about Django, Django-environ, and whitenoise, and we've already configured those for use. But the other two are also important and needed for deployment.

The first one is called [Gunicorn](https://gunicorn.org/). This is the recommended WSGI server for Heroku. We'll take a look at configuring this in just a bit. The next one is [psychopg2](https://www.psycopg.org/). This is a Postgres database adapter. You need it in your `requirements.txt` file to deploy, but you don't need any code changes in order to activate it.

A quick side note: we're keeping our discussion on packages simple for the purpose of this demo, but when you're ready to deploy a real project to Heroku, consider freezing your dependencies. You can do this with the [`pip freeze`](https://pip.pypa.io/en/stable/reference/pip_freeze/) command. This will make your build a little bit more predictable by locking your exact dependency versions into your Git repo. If your dependencies aren't locked, you might find yourself deploying one version of Django one day and a new one the next.

### [`runtime.txt`](#code-runtime-txt-code)

Heroku will install a default Python version if you don't specify one, but if you want to pick your Python version, you'll need a `runtime.txt` file. Create one in the root directory, next to your `requirements.txt`, `manage.py`, `.gitignore` and the rest. Specify your Python version with the prefix `python-`, followed by the major, minor, and patch version that you want your application to run on:

```
python-3.8.2

```

### [`Procfile`](#code-procfile-code)

The last file we need to add is a file specific to Heroku: [the `Procfile`](https://devcenter.heroku.com/articles/procfile). This is what we use to specify the processes our application should run. The processes specified in this file will automatically boot on deploy to Heroku. Create a file named `Procfile` in the root level directory, right next to your `requirements.txt` and `runtime.txt` files. (Make sure to capitalize the P of Procfile otherwise Heroku might not recognize it!) Copy-paste the following lines into it:

```
release: python3 manage.py migrate
web: gunicorn gettingstarted.wsgi --preload --log-file -

```

[The `release` phase](https://devcenter.heroku.com/articles/release-phase) of a Heroku deployment is the best place to run tasks, like migrations or updates. The command we will run during this phase is to simply run the `migrate` task defined in `manage.py`.

The other process is the `web` process, which is very important, if not outright essential, for any web application. This is where we pass our Gunicorn config, the same things we need when running the server locally. We pass it our WSGI file, which is located in the `gettingstarted` directory, and then we pass a few more flags to add it a bit more configuration. The `--preload` flag ensures that the app can receive requests just a little bit faster; the `--logfile` just specifies that the log file should get routed to Heroku.

## [Readying for deployment](#readying-for-deployment)

Take a second before moving on and just double check that you've saved and committed all of your changes to Git. Remember, we need those changes in the Git repo in order for them to successfully deploy. After that, let's get ready to make an app!

### [Creating an app with `heroku create`](#creating-an-app-with-code-heroku-create-code)

Since we have the Heroku CLI installed, we can [call `heroku create` on the command line to have an app generated](https://devcenter.heroku.com/articles/creating-apps) for us:

```shell
$ heroku create
Creating app... done, ⬢ mystic-wind-83
Created http://mystic-wind-83.herokuapp.com/ | git@heroku.com:mystic-wind-83.git

```

Your app will be assigned a random name—in this example, it's `mystic-wind-83`—as well as a publicly accessible URL.

### [Setting environment variables on Heroku](#setting-environment-variables-on-heroku)

When we created our `heroku.py` settings file, we used Django-environ to load environment variables into our settings config. [Those environment variables also need to be present in our Heroku environment](https://devcenter.heroku.com/articles/config-vars), so let's set those now.

The Heroku CLI command we'll be using for this is `heroku config:set`. This will take in key-value pairs as arguments and set them in your Heroku runtime environment. First, let's configure our allowed hosts. Type the following line, and replace `YOUR_UNIQUE_URL` with the URL generated by `heroku create`:

```shell
$ heroku config:set ALLOWED_HOSTS=<YOUR_UNIQUE_URL>

```

Next, let's set our Django settings module. This is what determines what settings configuration we use on this platform. Instead of using the default of `base`, we want the Heroku-specific settings:

```shell
$ heroku config:set DJANGO_SETTINGS_MODULE=gettingstarted.settings.heroku

```

Lastly, we'll need to create a `SECRET_KEY`. For this demo, it doesn't matter what its value is. You can use a secure hash generator like `md5`, or a password manager's generator. Just be sure to keep this value secure, don't reuse it, and NEVER check it into source code! You can set it using the same CLI command:

```shell
$ heroku config:set SECRET_KEY=<gobbledygook>

```

## [Provisioning our database](#provisioning-our-database)

Locally, Django is configured to use a SQLite database but we're productionizing. We need something a little bit more robust. Let's provision a Postgres database for production.

First, let's check if we have a database already. The [`heroku addons`](https://devcenter.heroku.com/articles/heroku-cli-commands#heroku-addons-all-app-app) command will tell us if one exists:

```shell
$ heroku addons
No add-ons for app mystic-wind-83.

```

No add-ons exist for our app, which makes sense—we just created it! To add a Postgres database, we can use the `addons:create` command like this:

```shell
$ heroku addons:create heroku-postgresql:hobby-dev

```

Heroku offers several tiers of Postgres databases. `hobby-dev` is the free tier, so you can play around with this without paying a dime.

## [Going live](#going-live)

It is time. Your code is ready, your Heroku app is configured, you are ready to deploy. This is the easy part!

Just type out

```shell
$ git push heroku master

```

And we'll take care of the rest! You'll see your build logs scrolling through your terminal. This will show you what we're installing on your behalf and where you are in the build process. You'll also see the `release` phase as well that we specified earlier.

## [Scaling up](#scaling-up)

The last step is to scale up our web process. This creates new dynos, or, in other words, copies of your code on Heroku servers to handle more web traffic. You can do this using the following command:

```shell
$ heroku ps:scale web=1

```

To see your app online, enter `heroku open` on the terminal. This should pop open a web browser with the site you just built.

## [Debugging](#debugging)

If you hit some snags, don't worry, we have some tips that might help:

- Are all of your changes saved and checked into Git?
- Are your changes on the `master` branch or are they on a different branch? Make sure that whatever you're deploying, all of your changes are in that Git branch.
- Did you deploy from the root directory of your project? Did you also call `heroku create` from the root directory of your project? If not, this could absolutely cause a trip up.
- Did you remove anything from the code in the provided demo that we didn't discuss?

### [Logging](#logging)

If you've run through this list and still have issues, take a look at your log files. In addition to your build logs—which will tell you whether your application successfully deployed or not—you have access to all logs produced by Heroku and by your application. You can get to these through a couple of different ways, but the quickest way is just to run the following command:

```shell
$ heroku logs --tail

```

### [Remote console](#remote-console)

Another tool you have is the `heroku run bash` command. This provides you with direct access from your terminal to a Heroku dyno with your code deployed to it. If you type `ls`, you can see that this is your deployed application. It can be useful to check that what is up here matches what is locally on your machine. If not, you might see some issues.

## [Wrapping up](#wrapping-up)

Congratulations on successfully deploying your productionized app onto Heroku!

To help you learn about Heroku, we also have a wealth of technical documentation. Our [Dev Center](https://devcenter.heroku.com) is where you'll find most of our technical how-to and supported technologies information. If you're having a technical issue, chances are someone else has asked the same question and it's been answered on our help docs. Use these resources to solve your problems as well as to learn about best practices when deploying to Heroku.