Breaking Down Celery ≥4.x With Python and Django
By: Timothy Mugayi
Tags: Software Development, Python, Django, Data Science, Programming

Adding Celery to your Django ≥ 3.0 Application

Let's see how we can configure the same celery task into our Django project. Ideally, you should create a new virtual environment for your new Django project. Ensure you run the pip install Celery and Redis command you ran earlier on as well within your Django project.

$ pip install django

We’ll be using the default Django admin start project to autogenerate a simple HelloWorld Django application so we can retrofit it with the celery task we created earlier.

$ django-admin startproject django_celery_site

Once your Django application has been created your folder structure should resemble this:

cd into the inner django_celery_site folder, the outer is not important you may rename it as you see fit.

$ cd django_celery_site

Let's add celery to our Django project. The first thing we need to do is create the following file django _celery_site/django_celery_site/celery.py and add the following code below, consisting of the celery context that is used to register our tasks:

celery.py

The Celery() object contains the Celery tasks and configurations. By defining config_from_object and namespace , we’re going to set Celery configs inside Django’s settings.py with any variables that starts with 'CELERY_' . I call this “Namespace Configuration”.

Take note, configurations do not have to be specified within your Django settings.py file. You may opt to add all your Celery configurations within the same file you initialize the Celery app context.

Your settings.py will consist of the following celery namespace configurations of everything that starts with CELERY :

Django celery settings.py

If USE_TZ = True (timezone is active), then ensure you set the corresponding CELERY_TIMEZONE .

The last step is to ensure Django loads the celery app when it gets initialized add below code snippet inside your django_celery_site/django_celery_site/__init__.py :

Django Celery __init__.py

The next step is to create your django_celery_site/django_celery_site/tasks.py where all your Python tasks will be invoked by Celery:

from __future__ import absolute_import, unicode_literals

from celery import task, shared_task





@task()

def latest_bitcoin_price(currency_code: str = None) -> object:

# Copy same bitcoin logic here our task will virtually peform the same logic as within our standalone celery example

pass

In order to start your Celery, use the project name that starts Celery. Since your celery.py is located inside django_celery_site , that’s the directory you need to run the worker.

$ celery -A django_celery_site worker -l info --concurrency=1 -n wkr1@hostname

e $

Periodical Tasks the Django Way

Running Django periodical tasks can be achieved by enabling a few configurations inside your Django project settings.py. Since periodical tasks do not need to return a result, we can create a copy of our original task, call it latest_bitcoin_price_notify_task , and make some minor modifications so that results can be pushed out to a third party service pushsafe.com.

Usage of pushsafe is for illustrative purposes — for production deployment, there are more scalable solutions, like SNS with Firebase Cloud Messaging.

Let's define a new entry, CELERY_BEAT_SCHEDULE , inside our settings.py as illustrated:

CELERY_BEAT_SCHEDULE = {

'latest_bitcoin_price_notify_task': {

'task': 'latest_bitcoin_price_notify',

'schedule': crontab(minute='*/1'),

'args': ('USD')

},

}

Add PUSHSAFER_PRIVATE_KEY to your environment variables:

$ export PUSHSAFER_PRIVATE_KEY=yourprivatekeyhere

Lets put it all together:

Celery task push notification

Since we’re within our development environment we can use an all in one command for convenience: