Understanding celery and creating simple task in django
Tags: python, django
Published: 2021-01-30 14:55:11

## Aim of this post
1. Understand what is celery 
2. Create a simple task in Django project (celery)


#### What is celery?
Celery is an `asynchronous` processing and `scheduling` tool in Django (not limited to Django).

`So what is asynchronous?`
When a client sends a request to the server, normally client cannot do anything until it receives a response from the server. That said, when request is time-consuming work, I mean more than 1 minute to handle, 
user will start clicking the refresh button couple of times or click on the back button.

To prevent this situation, we process APIs asynchronously. Asynchronous processing allows clients to send requests and do other tasks immediately. When the response returns, the processing of the response continues.

This asynchronous processing is done by celery.

`What about scheduling?`
You can make a task which runs at 8 o'clock every morning. Or you might also want to send e-mails to users every Friday at 4:00 p.m. This is possible with celery-beat.

`How does celery work?`
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/swry6zpwn3b28roeg1rf.png)
The unit of work that Celery has to deal with is called a task. Producer (celery) creates a task and the task is added to the Task Queue before it is executed. The broker then forwards the task in the task queue to the appropriate worker node which is the consumer (celery) where the task is executed. Celery can be seen as a larger concept, including producer, broker, and consumer. 

#### Let's create a simple task!

1. Prepare a django project (any project is fine - you can just use https://docs.djangoproject.com/en/3.1/intro/tutorial01/ if you don't know where to start)

2. Install and run rabbitmq. (I used homebrew) 
Try https://www.rabbitmq.com/install-homebrew.html if you don't have rabbitmq yet.
`brew services start rabbitmq` to start up rabbitmq

3. Install celery in virtual environment
`pip install celery`

4. create new celery app
`python manage.py startapp djangocelery` <- you can name your app anything. I called mine `djangocelery`

5. Settings and more
5-1. create `celery.py` `djangocelery` app(directory) and add following code
```
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'betweak.settings') 
app = Celery('djangocelery') 

# You can add celery settings in settings.py starting with CELERY_ 
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks(lambda : settings.INSTALLED_APPS)
``` 

5-2. Edit djangocelery > __init__.py file
```
from .celery import app as celery_app
# celery app is automatically imported when django starts
__all__ = ['celery_app']
```

5-3. Edit settings.py
```
INSTALLED_APPS = [
 
   ...
   'celery'
   'djangocelery' # newly created app
]

# set CELERY settings with CELERY_{}
CELERY_BROKER_URL = amqp://guest:guest@localhost:5672/ # rabbitmq setting
CELERY_TIMEZONE = 'Asia/Seoul'
CELERY_ENABLE_UTC=False
CELERY_TASK_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT = ['json']

```

6 . Migrate database
`python mange.py migrate`

7 . Create task

Create task.py file in djangocelery and add following code
```
from djangocelery.celery import app

@app.task(name="mod", bind=True, default_retry_delay=10, max_retries=5)
def mod(self, x, y):
    try:
        z = x % y
        print(f'{x} % {y} = {x%y}')
        return z
    except :
        mod.retry()
        print(f'Error with mod')

```

8 . Add views so we can access task with API

djangocelery > views.py
```
from django.http import HttpResponse
from rest_framework.decorators import api_view
from djangocelery.tasks import mod

@api_view(['GET'])
def get_test_add(request):
    x, y = int(request.query_params.get('x')), int(request.query_params.get('y'))
    mod.apply_async(args=(x, y), countdown=20)
    return HttpResponse(f'Success !')
```

djangocelery > urls.py 
```
from django.urls import path

from djangocelery import views

urlpatterns = [
    path('mod', views.get_test_add)
]

```

betweak > urls.py (project urls)
```
urlpatterns = [
    path('celery/', include('djangocelery.urls')),
    path('admin/', admin.site.urls),
]

```

Ok so settings are ready. Let's test it out!

#### Run celery
`celery -A djangocelery worker loglevel-info`

#### Run django (in another terminal)
`python manage.py runserver`


![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9n3qklt5v04g7ibwra6d.png)

Let's check celery log to see what's happening.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/e7gcbx4upcv0wlbdwq2f.png)

We can see that at 22:32:33 the task is received. Approximately 20 seconds later at 22:32:54, the task succeeded.

If we see djangocelery > views.py again
````
ffrom djangocelery.tasks import mod
@api_view(['GET'])
def get_test_add(request):

    # so we get the x and y values from the url
    x, y = int(request.query_params.get('x')), int(request.query_params.get('y'))
    
    # Here countdown=20 means executes in 20 seconds from now 
    mod.apply_async(args=(x, y), countdown=20)
    return HttpResponse(f'Success ! ')

```

Well that was easy?

Let's try to make an error. We will pass 0 value for y. Python raises when we try to use 0 for modulo.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/jrsp78p46hagpqt0rh26.png)


Let's check the log out
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/0dekbl3tza3atqtta6u8.png)

There's a lot going on there. We see there was a ZeroDivisionError, and the task retried 5 times!

djangocelery > task.py 
```
# Here are the settings for retries!!
@app.task(name="mod", bind=True, default_retry_delay=10, max_retries=5)
def mod(self, x, y):
    try:
        z = x % y
        print(f'{x} % {y} = {x%y}')
        return z
    except :
        mod.retry()
        print(f'Error with add')

```


So that's it for getting started with celery in django :)