The Right Way™ to do Serverless in Python (Part 2)
Tags: python, django, aws, serverless
Published: 2018-07-10 19:58:03

---
title: The Right Way™ to do Serverless in Python (Part 2)
published: true
description: How to build serverless apps in Python using Zappa and Chalice.
tags: Python, Django, AWS, Serverless
---
_Note: This article was authored by [Michael Lavers](https://github.com/kolanos)_

In [Part 1](https://read.iopipe.com/the-right-way-to-do-serverless-in-python-e99535574454) we covered the serverless basics and got our feet wet with the Serverless Framework. If you haven't read that part yet, it's highly encouraged that you start there. Assuming you've followed along in Part 1, you now know how to create and deploy a simple AWS Lambda function in Python; including a web API using AWS API Gateway. But we've only just scratched the surface of what you can do in serverless with Python. In this post, we'll introduce a library that allows you to plug any web framework that speaks WSGI into serverless (pretty much all of them); and we'll take a look at AWS' own Python web framework for creating web APIs.

## Zappa

[Zappa](https://www.zappa.io), named after the legendary Frank Zappa, is the original Python serverelss library/framework. It was originally conceived as django-zappa, a library to allow you to deploy Django web applications to AWS Lambda. It has since grown in leaps and bounds and now supports any WSGI compatible Python web framework. It has also expanded it's feature set and is very much a "batteries included" tool for creating, deploying and managing serverless web applications written in Python.

### What is WSGI?

If you've done Python web development before, you've likely heard of WSGI. But what is it? WSGI stands for "Web Server Gateway Interface" and it's an implementation agnostic interface between web servers and web applications/frameworks. It was [created by the Python community](https://www.python.org/dev/peps/pep-3333/) back in 2003 because there wasn't a standard way to develop a web application in Python back then. Instead you first had to pick a web server, which could use CGI, FastCGI, mod_python or something else entirely and develop against that. Which meant your web application was very much coupled to the web server in which it was built upon (remember Zope?). Now we have WSGI and all modern Python web frameworks support it, which means you can use whatever web server you like -- or no server at all? Yes, thanks to Zappa, WSGI is supported on serverless, too.

The reason I refer to Zappa as a "library/framework" is that it kind of lives in both worlds. Zappa at it's core is a compatibility/translation layer between serverless and WSGI. For example, with AWS Lambda, Zappa takes the HTTP event we covered in Part 1 and converts it into a WSGI compatible equivalent representation that all WSGI web frameworks can use. This is where Zappa feels like a "library", but then Zappa packs a ton of useful features on top and starts to feel more like a "framework". Although, like WSGI, Zappa is very much agnostic when it comes to the actual web framework you use.

### Which web framework should I use?

If you're asking this question, then I would recommend you start with either [Django](https://www.djangoproject.com) or [Flask](http://flask.pocoo.org). Django is the most popular web framework in Python and is very much a "batteries included" option. Flask is arguably the second most popular web framework in Python and calls itself a "micro framework"; it has a simpler API and doesn't come with all the bells and whistles that Django does. Both are excellent options and are very well documented. In this post, we'll focus on these two web frameworks when it comes to Zappa, but Zappa should work with pretty much [any Python web framework](https://wiki.python.org/moin/WebFrameworks). Suffice to say, if you're using Bottle, Pyramid, Tornado, Falcon or something else -- it should be easy to get your web application running on serverless using Zappa.

### Django Meets Zappa

As we mentioned in Part 1, the rule of thumb for new Python serverless projects is to use Python 3.6, and true to form, Django has [dropped Python 2.7 support](https://docs.djangoproject.com/en/2.0/releases/2.0/) in Django 2.0. This may seem like a sad day for all the Python 2.7 loyalists out there, but the pros greatly outweigh the cons here. Think about it: no more [six](https://pythonhosted.org/six/). That alone should put a smile on your face -- at least until Python 4 is released.

Now before we dive into Zappa, we first need to create a Django web app. Django makes this really easy:

```bash
pip install -U django
django-admin startproject mydjangoapp
cd mydjangoapp
```

This will install the latest version of `django` using `pip` and the `django-admin startproject` command will create a directory called `mydjangoapp` with a preconfigured Django web app ready to go. Now if you've done Python web development before, you're likely asking "what about the virtualenv?" If you don't know what a "virtualenv" is, it's a best practice in Python to create a "virtual environment" for each project so that all your dependencies remain in one place and don't get mixed up with other projects. There are a number of tools out there to make managing virtual environments easy. for example, the one gaining a lot of traction recently is Kenneth Reitz' [Pipenv](https://docs.pipenv.org). If you don't know who that is, he's the guy behind [requests](http://docs.python-requests.org/en/master/), you likely know his work. But I'm still partial to [virtualenvwrapper](http://virtualenvwrapper.readthedocs.io/en/latest/) I'm going to treat virtual environments as a personal preference. It's still a best practice, though, so pick whichever tool fits your workflow the best.

Now that we have a Django project created and we're in our `mydjangoapp` directory let's fire up Zappa:

```bash
pip install -U zappa
zappa init
```

Zappa is going to ask a series of questions about your web app. For now, you can stick with the defaults that Zappa suggests. As we mentioned in Part 1, like with the Serverless Framework, Zappa is going to check that your AWS credentials [are properly configured](https://aws.amazon.com/blogs/security/a-new-and-standardized-way-to-manage-credentials-in-the-aws-sdks/). If you have multiple AWS profiles configured, Zappa wil ask which one to use, but if you only have one, then the default is what you want. After running `zappa init` and answering the questions, you'll find that Zappa has created a `zappa_settings.json` file in your project directory. This file serves the same purpose as the `serverless.yml` in Part 1, but because it is JSON there are no comments. Thankfully, Zappa [documents](https://github.com/Miserlou/Zappa#advanced-settings) the available settings. 

Now to deploy:

```bash
zappa deploy dev
```

Zappa will perform steps similar to Serverless Framework covered in Part 1, but out-of-the-box it will do some handy Python-specific things like sort out dependencies for us. Once it is done deploying, you should see something like this:

```bash
Deployment complete!: https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
```

If you've read Part 1 this is going to look familiar. It's a URL provided by AWS API Gateway. Let's open it up in a browser and see what we get:

```
DisallowedHost at /

Invalid HTTP_HOST header: 'xxxxxxxxxx.execute-api.us-east-1.amazonaws.com'. You may need to add 'xxxxxxxxxx.execute-api.us-east-1.amazonaws.com' to ALLOWED_HOSTS.
```

Huh. Ok, let's do that. Open up `mydjangoapp/settings.py` and look for the `ALLOWED_HOSTS` setting and change it to the following:

```python
ALLOWED_HOSTS = [".execute-api.us-east-1.amazonaws.com"]
```

Why aren't we including the `xxxxxxxxxx` part? Well, this part of the hostname is prone to change. For example, if you decide to deploy a staging or production environment with Zappa, then this part of the hostname is going to be different. Setting your `ALLOWED_HOSTS` this way allows you to support every environment you deploy with Zappa in AWS `us-east-1`. If you decide to deploy in another AWS region, then you'll need to update your `ALLOWED_HOSTS` accordingly. Let's save and re-deploy:

```bash
zappa update dev
```

Notice here the command is `zappa update` and not `zappa deploy`. This is because most of your serverless web app is already deployed. All we need to do now is replace our package in the S3 bucket Zappa created for us. You should notice that `zappa update` runs noticeably faster. Once done, let's reload our `https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev` URL in a browser:

```
Page not found  (404)
```

Ok, we're making progress. The reason we're getting a "Page Not Found" error is that Django doesn't configure a page for `/` by default. Let's fix that. First, we'll need to create a Django app in our project. A Django web application is comprised of "apps". Say your web application had a blog and a store, each of those would be an "app" in your Django project. To create an app:

```bash
python manage.py startapp myapp
```

This command will create a `myapp` directory in your project. Inside that directory are a handful of modules to support an app. But before we start working on our app we must first register it within our Django project. To do this, we need to open our `mydjangoapp/settings.py` again and look for the `INSTALLED_APPS` setting. There we want to add our app to the end of the list, so it should look something like this:

```python
INSTALLED_APPS = [
    ...
    'myapp',
]
```

Note that `INSTALLED_APPS` will have several Django apps already installed. We want to keep those and just add a new app to the list. So the `....` represents all the Django apps that are already configured. Save this file and open up `mydjangoapp/urls.py`. This file is where Django configures all the routes of our web application. In fact, if you notice, there's already a route configured:

```python
urlpatterns = [
    path('admin/', admin.site.urls),
]
```

Interesting. What if we open `https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev/admin/` in a browser? How about that. The Django admin app is already there running. Unfortunately we haven't configured any users in our Django project, so we won't be able to get beyond this login page. Back to what we were originally doing. Let's add a route for `/` and serve up an HTML template. First we'll need to import Django's `TemplateView`. After the `from django.urls import path` line, add the following:

```python
from django.views.generic import TemplateView
```

And then update our `urlpatterns` like so:

```python
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', TemplateView.as_view(template_name='index.html')),
]
```

Here we're configuring an empty route, which is equivalent to `/` which is to be served by `TemplateView` using a `template_name` of `index.html`. Now to create the template, we need to go back to our app. From our project directory, we create a `templates` directory in our app and add a template to it:

```bash
mkdir myapp/templates
echo "<h1>Hello Django</h1>" > myapp/templates/index.html
```

Note that the template's name is `index.html` like we specified in our `urls.py`. Django will automatically load any templates it finds in any of the `INSTALLED_APPS`, assuming a `templates` directory is present for that app. Now re-deploy:

```bash
zappa update dev
```

And reload `https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev` in a web browser:

```
Hello Django
```

And  just like that you have a serverless Django web app. Zappa truly couldn't make it any easier. Deploying the same app to a regular web server would actually be more work. Aren't you glad you ditched those things?

Now as cool as a stock Django app running on serverless is, if we refer back to Part 1, we're really looking to build a web API and workers, right? With Django, we're in luck, as it has one of the most powerful and well-documented REST frameworks out there called [Django REST Framework](http://www.django-rest-framework.org). And if you follow the [installation instructions](http://www.django-rest-framework.org/#installation) you'll have a fully-featured REST API running on serverless in no time. Or if you're building a GraphQL API, then the excellent [Graphene](https://github.com/graphql-python/graphene-django) integrates with Django.

As for workers, Zappa offers [a solution](https://github.com/Miserlou/Zappa#scheduling) not unlike the one we described in Part 1. If you had a function called `scheduled_job` in `myapp/tasks.py` that you wanted to execute every ten minutes, you would open your `zappa_settings.json` and add the following:

```json
{
    "dev": {
       ...
       "events": [{
           "function": "myapp.tasks.scheduled_job",
           "expression": "rate(10 minutes)"
       }],
       ...
    }
}```

And then to deploy it:

```bash
zappa schedule dev
```

And you're good to go. And what about custom domains? Zappa [has you covered](https://github.com/Miserlou/Zappa#deploying-to-a-domain-with-aws-certificate-manager) there, too. Before we continue, assuming you don't need this app, Zappa makes it easy to clean up:

```bash
zappa unschedule dev
zappa undeploy dev
```

Want to learn more about Django? Check out the [Django documentation](https://docs.djangoproject.com/en/2.0/). Or even better, check out these [Django tutorials](https://docs.djangoproject.com/en/2.0/intro/).

### Flask Meets Zappa

Now if the Django setup wasn't straightforward enough, Flask might be even easier. Assuming we're starting from a new virtualenv (see Django section for suggested tools for creating a virtualenv). We first install Flask and Zappa:

```bash
pip install -U flask zappa
```

Then create a project directory:

```bash
mkdir myflaskapp
cd myflaskapp
```

And within that directory add a `app.py` with the following:

```python
from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello():
    return {'message': 'hi there'}

if __name__ == '__main__':
    app.run()
```

Then fire up Zappa:

```bash
zappa init
```

Like with our Django app, Zappa will auto-detect that our app is using Flask. And to deploy:

```bash
zappa deploy dev
```

And we should get:

```bash
Deployment complete!: https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
```

And to test it:

```bas
curl https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
```

Which should return:

```
{"message": "hi there"}
```

Talk about an instant web API. So what's happening here? Zappa probably feels like magic, but it's really just clever. In Django, the WSGI is exposed in `mydjangoapp/wsgi.py`. So Zappa creates a AWS Lambda handler for us that accepts HTTP requests. This handler transforms these requests into equivalent WSGI requests and passes them onto `mydjangoapp/wsgi.py`. With Flask, it's pretty much the same idea. The only difference is that Flask exposes it's WSGI via the `app.run()` method in our `app.py` above.

The Zappa topics we've covered already with Django are applicable to Flask as well. From scheduled jobs to custom domains, they're all supported.

Want to learn more about Flask? Check out the [Flask documentation](http://flask.pocoo.org/docs/1.0/). Or even better, check out this [Flask mega tutorial](https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world).

Oh, and don't forget to clean up:

```bash
zappa undeploy dev
```

### More Zappa?

In this post, we've only scratched the surface of what you can do with Zappa. Check out the [Zappa documentation](https://github.com/Miserlou/Zappa) for more, including how to rollback deployments, tail logs, keep your functions warm, fire asynchronous jobs, etc.

## Chalice

If Zappa didn't inspire you to start building a serverless Python web app, I don't know what will, but it also appears to have inspired (at least in part) a team at AWS to start their own web framework specifically built for AWS Lambda. They call it [Chalice](https://github.com/aws/chalice).

This project is interesting for a couple reasons. First is that it is one of the few Python web frameworks out there that doesn't implement WSGI. While I've been waxing poetic on the virtues of WSGI, Chalice is built specifically for AWS LAmbda, and as such doesn't need an abstraction layer like WSGI. The trade off here is simplicity for lock in: the Chalice implementation is simplified by only having to target AWS LAmbda, but that also means you can only use Chalice with AWS Lambda. Second, Chalice's API is clearly influenced by another web framework we've already covered in this post, which helps with the learning curve. Which one you ask? Let's take a look.

Assuming we're starting with a fresh virtualenv, install Chalice with:

```bash
pip install -U chalice
```

And to create a project:

```bash
chalice new-project mychaliceapp
```

This command creates a directory called `mychaliceapp` with a pre-configured Chalice project. Inside there's a `.chalice` hidden directory that contains config files, a `requirements.txt` and an `app.py`. There's our first clue, if you recall, in our Flask app we created a `app.py` as well, which is a common Flask idiom. But if we take a look at the `app.py` it will become clear:

```python
from chalice import Chalice

app = Chalice(app_name='mychaliceapp')

@app.route('/')
def index():
    return {'hello': 'world'}
```

Look familiar? They say imitation is the sincerest form of flattery. And if I'm Armin Ronacher (creator of Flask), I'm feeling pretty flattered. And to deploy:

```bash
chalice deploy
```

When the deploy finishes, Chalice will return a URL that should be familiar by now: `https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/api`

You guessed it, there's our old friend, API Gateway. And to test it:

```bash
curl https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/api
```

Which should return:

```
{"hello": "world"}
```

We didn't even have to write any code for that one. Does that make it codeless as well?

Chalice offers a Flask-esque API and is very much designed for JSON REST APIs. It also comes bundled with some nice integrations with other AWS services, including authentication with AWS Cognito. It is clear that the project is focused on the problem they're trying to solve, as such, as long as your problem is JSON REST APIs then Chalice can help you get up and running on AWS Lambda quickly. Anything else? You're going to want to look at the other options we've covered so far.

## In Summation

In this post, we've taken a look a two Python tools looking to make developing, deploying and managing serverless apps easier. One that is agnostic as to what web framework you use, the other is very much opinionated. One thing is clear, the Python community is getting a lot right when it comes to serverless.

In Part 3 we're going to compare and contrast all the options available in the serverless space for Python web apps. We'll also introduce you to a serverless observability tool that will give you visibility into your serverless web apps like never before. Stay tuned.