Send Email With Django Using Gmail SMTP
Tags: Celery, Asynchronous, Django, Redis

SMTP Introduction

First of all, let’s start with a brief intro about SMTP. SMTP, an abbreviation for Simple Mail Transfer Protocol is a standard protocol that is being used for sending emails, one of killer application that brought internet to the spotlight. It mainly uses TLS (Transfer Layer Security) or SSL (Secure Sockets Layer) as its security protocol and port 587 as its standard port, according to RFC 2476 in 1999. SSL and TLS, with the later is the recent one, are a way for web client and web server to authenticate each other, either by adding SSL certificate in SSL, or adding encryption algorithm and separation of handshake and record protocol in TLS. To send an email, SMTP server must recognize and communicate with each other by exchanging command and responses between themselves. This is commonly referred as “SMTP conversation”. Feeling stuffed with this whole lot of definition? Well, let’s move on to the next part and we will talking about how SMTP works. In short, this is how SMTP works:

User writes a piece of email for a target email, an example of this will be design@abc.company.com User send this email. In turn, it will send email from your email client/webmail, such as outlook and Gmail, to outbound server. Outbound server will authenticate it first, before agreeing to send an email to recipient’s target Server will determine which SMTP server they must connect by querying recipient’s domain Outbound server and receiving server will introduce each other and begin SMTP conversation between themselves. Email will be stored in SMTP server and will be accessed by recipient, either by receiving it in email client, either through POP3 or IMAP protocol, or accessing it directly from webmail.

Setup Gmail SMTP

Now, we’re done with theories, let’s move on into practical part of this writing. To send an email, you must specify your SMTP server first. There are many SMTP services out there, such as Mailgun and Amazon SES. However, for this tutorial I will use Gmail SMTP server. Gmail SMTP server setup is not as hard as others, and it is free, a blessing for us, developers who just want to test things out :D.

First, you need to allow less secure apps option first in your google account. Sending email through SMTP in Gmail use password, without any secure sign in technology that Google not allowed by default. However, we can turn it on by accessing this link. However, due to this safety reasons, I usually move on to another approach, which is by enabling 2-step authentication. To do that, you can go to your google account settings > security > 2 step auth. Selecting those will guide you to setup your 2 step auth. After doing those required steps, your 2 step authentication setting will look like this.

2 Step Authentication Result

Our next step will be setting up our application password. To do that, we have to navigate through our google account settings, and went with this navigation: Account > Security > Application Password. There, a menu about application password should appear, and you can set up your application password for your application or device. Because we want to send our email with Django in this tutorial, we will create our application password based on app, and give it our custom name (for this example I will give it my web’s application name). Google will create an application password for you, and you need to COPY IT RIGHT AWAY. That is because an application password will only be showed upon its creation with no way of retaining it later. You have to create new application password if you forget it. Your application password menu will looks like this.

Application Password Menu

Finally, we have finished configuring our Gmail account. Now, let’s move to the next part and learn how to send it with django!

Sending Email With Django

Django functionality also include sending emails. To do that, Django use smtplib module that is provided by Python. There are several ways to implement email sending in Django:

Use light thin wrapper function that Django provide such as send_mail(), send_mass_mail(), and mail_admins() Create a class that extends EmailMessage class, or use EmailMessage class instead.

In most basic cases, sending an email with wrapper function is sufficient, however, you will need EmailMessage class if you want to use advanced features such as BCC’ed recipients, file attachments, or multi-part email. Django docs recommend use of EmailMessage class to embrace object-oriented design further. However, I will use wrapper function instead for this tutorial. This is mainly due to wrapper function is sufficient enough for my use case and there aren’t many differences between them in sending email beside advanced features that EmailMessage provided, so I think that our learning curve will not take an extreme upper trajectory.

To send an email with send_mail function, there are several things that you need to do first:

1. Setup your email configuration in your project’s settings.py. There are 10 configurations that you can add to configure your email configuration. Email_Backend specify which backend that you will be using to send your email. By default, SMTP backend will be used and this is the one you must use in production environment. To specify which SMTP server you will use, you can add EMAIL_HOST variable in your settings.py. EMAIL_HOST_USER and EMAIL_HOST_PASSWORD will be used to authenticate to the SMTP server, while selecting email security protocol can be done with EMAIL_USE_TLS and EMAIL_USE_SSL configuration. Optionall, you can also add timeout, ssl_keyfile, and ssl_certfile by specifying EMAIL_TIMEOUT, EMAIL_SSL_KEYFILE, and EMAIL_SSL_CERTFILE respectively. An example of email configuration should look like this.

# Email setup EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"

EMAIL_HOST = "smtp.gmail.com"

EMAIL_HOST_USER = os.environ.get("EMAIL_HOST_USER")

EMAIL_HOST_PASSWORD = os.environ.get("EMAIL_HOST_PASSWORD")

EMAIL_PORT = 587

EMAIL_USE_TLS = True

2. To send an email, with send_mail() function, you should import it first from django.core.mail module. There are 6 arguments needed, assuming that you follow my settings above. These arguments in sequence are subject, message, recipient_list, fail_silently, and html_message. Subject, message, and recipient_list are self-explanatory. Meanwhile, fail_silently is a configuration to raise SMTPException when an error occurs. If you have HTML template that you want to use for your message, you can use specify it in html_message, instead of message. An example of send_mail function should look like this.

Example of send_mail usage

Now, you have learnt how to send an email with Django. However, most of the time, especially when you were using REST API, this will not be enough. Why? My reasoning will be explained in the next part.

Asynchronous Task, Celery & Redis

Previous tutorial that I have shown you has one major problem. It run synchronously! Huh..? So what effect does it have when we run it synchronously? Well, as we all know Django is a web framework, in which they need to communicate with its client’s browser upon request. However, most of client’s browser has timeout mechanism that determine whether a request is successful or not. Sending an email synchronously took a long time and most of the time, you will encounter timeout error in your client browser, due to your request still being processed in web server. My personal solution for this, will be asynchronous task. Asynchronous allow your program to run and return its result request to your web’s browser while run task, in this case send_mail function separately. Doing this will avoid timeout error due to send_mail and even help you handle error that occurs in sending email. To enable asynchronous task, our infrastructure should look like this.

As we have seen above, we need to add message brokers and celery workers to our infrastructure. Message broker is a software for applications/ systems/ services to communicate with each other. By using this, we can assign tasks that need to be done from Django to our message brokers. Our workers will queue its task, run it when they haven’t got a job, and return its result to our database. We can see similarity in producer customer problem here, although worker act as producer and consumer at the same time.

For this writing, I will use Celery as my worker, and Redis for our message broker. Our first step will be installing celery and Redis first. To install celery and redis, we need to run this command in our command line.

pip install celery

docker run -d -p 6379:6379 redis # Use this if you use docker

pip install redis

If you did not use docker, you can install it directly through your system by reading this tutorial for Linux, or this for Windows.

Next, you should specify Celery and Redis configuration in Django settings. There are several configuration that you can add in Django. However, I will only add this configuration in my settings.py

REDIS_DEFAULT = "redis://localhost:6379/0"

CELERY_BROKER_TRANSPORT_OPTIONS = {"visibility_timeout": 3600}

BROKER_URL = os.environ.get("REDIS_URL", REDIS_DEFAULT)

CELERY_RESULT_BACKEND = os.environ.get("REDIS_URL", REDIS_DEFAULT)

REDIS_DEFAULT is where our redis url is, while BROKER_URL specify where our broker’s url located. Optionally, we can add transport options with CELERY_BROKER_TRANSPORT_OPTIONS, and write our workers result to url that is specified in CELERY_RESULT_BACKEND.

Our next step will be creating our celery app and loaded it when Django start. To achieve this, first, you need to create celery.py in your project module and create an app there, like this.

import os

from celery import Celery os.environ.setdefault("DJANGO_SETTINGS_MODULE", "kum_backend.settings")

app = Celery("kum_backend")

app.config_from_object("django.conf:settings", namespace="CELERY")

app.autodiscover_tasks()

Setting our django environment default is needed to save our time from passing our settings module to the celery program. While creating app instance is done by initializing Celery class. We can specify our location of celery settings in app.config_from_object(), and autodiscover_tasks will help us in determining which task celery can run. Then, we need to make sure that our celery app is loaded when django started. For that, in our __init__.py, we need to import our app.

from .celery import app as celery_app

__all__ = ('celery_app',)

Next, I will follow celery name convention and specify our tasks in app’s directory with the name of tasks.py. To tell celery that this function is a task that celery needs to run, we can add @app.task decorator above our function.

Inside of tasks.py

I add logger for debug purposes, you can just exempt it if you feel that you did not need it. I also add autoretry_for argument in my @app.task decorator. autoretry_for will instruct our celery worker to repeat this task if they met certain error, in this case SMTPException error. And… voila! You have implement email sending asynchronously.