Queueing Emails With Python, Redis Queue And Twilio SendGrid
Tags: python, email, twilio, redis
Published: 2020-01-06 18:28:49

---
canonical_url: https://www.twilio.com/blog/queueing-emails-python-redis-queue-twilio-sendgrid
---

If you’re sending email from within a web application, you’ll want to do so asynchronously so you don’t block requests from completing. Especially if it takes additional (potentially slow) requests to construct the email content. There are several Python libraries available to help with asynchronous tasks, such as [celery](http://www.celeryproject.org/). Alternately, you can use [asyncio](https://github.com/sendgrid/sendgrid-python/blob/master/use_cases/asynchronous_mail_send.md) if you’re on Python 3.4 or above.

Redis Queue is the friendliest to get started with. Today I’ll show you how to queue emails to be sent asynchronously with [Redis Queue](https://python-rq.org/) and [Twilio SendGrid](https://sendgrid.com/).

To keep things spicy, let’s use the [Taco Fancy API](http://taco-randomizer.herokuapp.com/random/) to email a randomly generated taco recipe.

## Getting Started

To follow along, you’ll need:

- [Python 3.5](https://www.fullstackpython.com/python-2-or-3.html) or above installed
- A free SendGrid account - sign up [here](https://signup.sendgrid.com/)
- An email address, to test things out and make sure they’re working

First, create a new Python project. In case you need it, here’s a guide on [general Python environment setup](https://www.twilio.com/docs/usage/tutorials/how-to-set-up-your-python-and-flask-development-environment).

Create a file, `build_mail.py`, and open it in your editor of choice.

## Diving Into The Tacolicious API

If you go to the [taco API URL](http://taco-randomizer.herokuapp.com/random/) in the browser, you get some output that starts like this:

```json
{"mixin": {"recipe": "### Green Chile Cabbage Salad with Seared Corn\nThis isn't a tradition, or even particularly traditional -- except in my apartment in Oakland, where I make this for myself every time I make pork tacos.
...
```

I’ve truncated the example output here as it’s quite long, but you get the idea.

We’re going to make an HTTP request to that API from our Python code, and then generate an email based on the response. Run `pip install requests` to locally install the `requests` library, which we’ll be using to make the API call.

The API returns the recipe in markdown format. We’re going to use a library called `markdown2` to convert the markdown to HTML so it’s formatted correctly in the email. Run `pip install markdown2` on the command line to install. Define a function, `get_html_content`, in the `build_email` file:

```python
import requests
from markdown2 import Markdown

def get_html_content():
    url = 'http://taco-randomizer.herokuapp.com/random/'
    response = requests.get(url).json()

    taco_components = ['base_layer', 'mixin',
                       'condiment', 'seasoning', 'shell']
    html_content = f"<strong> Here's a taco recipe! </strong>\n"
    markdowner = Markdown()

    for component in taco_components:
       html_component = markdowner.convert(response.get(component).get('recipe'))
       html_content += f"<p>{html_component}</p>\n"
    return html_content

print(get_html_content())
```

Run this code by running `python build_email.py` on the command line. Your output will look something like this:

```html
<strong> Here's a taco recipe! </strong>
<p><h1>Crock Pot Pulled Pork</h1>

<p>This is the base of my very favorite tacos. Added bonus is by cooking these all day in a slow cooker, your house smells <em>amazing</em>.</p>
...
```

Is your stomach growling yet?

It would be nice if our recipe had a title, so it didn’t feel like a random collection of components. Let’s make another function to build the title. Copy this code into `build_email.py`.

```python
    title = ''
    taco_components_to_names = {'base_layer': ' with ',
                                'mixin': ' garnished with ',
                                'condiment': ' topped off with ',
                                'seasoning': ' and wrapped in delicious ',
                                'shell': '!'}
    for key, value in taco_components_to_names.items():
        title += f"{response.get(key).get('name')}{value}"
    
    return title
```

Now we’ll refactor our get_html_content function to call build_title:

```python
   url = 'http://taco-randomizer.herokuapp.com/random/'
    response = requests.get(url).json()

    taco_components = ['base_layer', 'mixin',
                       'condiment', 'seasoning', 'shell']
    html_content = f"<strong> Here's a taco recipe! {build_title(response)} </strong>\n"
    markdowner = Markdown()

    for component in taco_components:
       html_component = markdowner.convert(response.get(component).get('recipe'))
       html_content += f"<p>{html_component}</p>\n"
    return html_content

  print(get_html_content())
```

Run `python build_email.py` script again from the command line. Check our your awesome new title.

```html
<strong> Here's a taco recipe!  Baja Beer Battered Fish with Shredded Brussels Sprouts garnished with Mango Avocado Pico topped off with Kathy's Taco and Fajita Seasoning and wrapped in delicious bad-ass tortillas! </strong>

```

Next, we’ll add in some code to construct and send the email. Since we’re done with the recipe fetching part feel free to remove the print statements.

## Setting up SendGrid

Create your API key from the [SendGrid dashboard](https://app.sendgrid.com/settings/api_keys). Let’s call it “bulk email.”

![Screenshot of the UI for creating a SendGrid API key. The key is named "bulk email" and the permissions selected are "Full Access."](https://thepracticaldev.s3.amazonaws.com/i/rr2lr919b4fhtcpbbbws.png)

After clicking “Create & View”, you’ll see your key. Before you close this dialog box, save the key in an [environment variable](https://www.twilio.com/blog/2017/01/how-to-set-environment-variables.html) named `SENDGRID_API_KEY` since you won’t be able to get that same key from the SendGrid dashboard again for security reasons.

Install the Sendgrid Python helper library with `pip install sendgrid`.

## Generating the email content

Now we’ll write a function to actually send the email, using the SendGrid API.

```python
import os
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail

def send_message(sendgrid_client, email):
    pass
    message = Mail(
        from_email=('tacobot@delicious.ceo', 'Taco Bot'),
        to_emails=email,
        html_content=get_html_content(),
        subject='Delicious tacos 🌮',
    )

    try:
        response = sendgrid_client.send(message)
        print(response.status_code)
        print(response.body)
        print(response.headers)
    except Exception as e:
        print(e)


sendgrid_client = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))
email = ['test@example.com'] # your email goes here
send_message(sendgrid_client, email)
```

We only need to instantiate the SendGrid client once. We’ll enqueue a separate job for each email we want to send mail to. Thus we pass in `sendgrid_client` and `email` as parameters to our `send_message` function.

Run the script again from the command line:

```bash
202
b''
Server: nginx
Date: Tue, 17 Dec 2019 23:37:54 GMT
Content-Length: 0
Connection: close
X-Message-Id: yeFWKbW1R9Kf_8BeXSvFeg
Access-Control-Allow-Origin: https://sendgrid.api-docs.io
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl
Access-Control-Max-Age: 600
X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html
```

In your inbox, you should receive a mouthwatering recipe.

![Screenshot of an email from "Taco Bot." The subject line is "Delicious tacos 🌮" and the body is "Here's a taco recipe! Chopped pork with drunken green beans garnished with charred corn topped off with basic dry rub and wrapped in delicious bad-ass tortillas!"](https://thepracticaldev.s3.amazonaws.com/i/vtkb62mstenlof3k982w.png)

## Adding Redis Queue

[Redis](https://redis.io/) is *“an open source (BSD licensed), in-memory data structure store.”*  Before you install Redis Queue, you must install `redis` on the machine where your code will be running. Download the latest stable version by following the instructions [here](https://redis.io/topics/quickstart). Then start the Redis server by running `src/redis-server` in the directory where you downloaded Redis. You won’t need to do anything else with the Redis server, it just needs to stay running in the background.

Install Redis Queue by running `pip install rq` from the terminal in your Python project directory. Create another file in the same folder where the `build_email` script lives. Call it `enqueue.py`. We need another file because otherwise Redis Queue will throw an error message. We’ll migrate some of our code over to the new file.

```python
from redis import Redis
from rq import Queue
import os

from build_email import send_message
from sendgrid import SendGridAPIClient

sendgrid_client = SendGridAPIClient(os.environ.get('SENDGRID_API_KEY'))

q = Queue(connection=Redis())

emails = ['example1@mail.com', 'example2@mail.com', 'example3@mail.com'] # replace these with your email addresses

for email in emails:
    q.enqueue(send_message, sendgrid_client, email)
```

You don’t need to use three email addresses here - you can use as many or as few as you’d like.

Before you run this code, you need to run the `rqworker` command in another terminal tab in the same folder where your project code lives. Don’t forget to activate your Python virtualenv on the new tab first.

After you start `rqworker`, try running `python enqueue.py` from the command line. Look on the terminal tab where you started `rqworker` and you should see some output like so:

```bash
17:48:16 default: Job OK (9c9d8195-0297-4528-8c33-7ada97ecf114)
17:48:16 Result is kept for 500 seconds
17:48:16 default: send_mail.send_message(<sendgrid.sendgrid.SendGridAPIClient object at 0x1019e7748>, 'tilde@thuryism.net') (98f72e19-f75d-4664-b4f5-c6c04db3edb3)
202
b''
Server: nginx
Date: Tue, 17 Dec 2019 01:48:17 GMT
Content-Length: 0
Connection: close
X-Message-Id: ZAceAObvT96H2ZDW3aJImQ
Access-Control-Allow-Origin: https://sendgrid.api-docs.io
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl
Access-Control-Max-Age: 600
X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html
```

Every email address on the list should receive a unique and delicious recipe. That’s it! You can check out the example code for this whole project on [GitHub](https://github.com/annthurium/sendgrid-queue).

## Conclusion

Today we learned how to queue emails for asynchronous sending with Twilio SendGrid and Redis Queue.

What else can you do with Redis Queue?

You can [receive SMS notifications when the International Space Station flies above you](https://www.twilio.com/blog/2015/11/international-space-station-notifications-with-python-redis-queue-and-twilio-copilot.html).
You can [print photos from the Mars Rover](https://www.twilio.com/blog/asynchronous-tasks-in-python-with-redis-queue).
You can even [generate Nintendo music](https://www.twilio.com/blog/generating-nintendo-music-over-the-phone-with-magenta-and-twilio).

Hungry? Go forth and queue some delicious tacos into your face. 🌮