Database Scheduler with celery beat and manage schedule in Django admin
Tags: django, python, database, todayilearned
Published: 2021-02-20 14:29:03




> Must read previous post first before reading this one :D
[Previous Post about celery] (https://dev.to/paige0701/understanding-celery-and-creating-simple-task-in-django-3gdk)


#Aim of this post
1. Install celery beat in django project and understand how it is used
2. Settings for sending emails in django project using gmail
3. Create a task that sends email
4. Create schedule using django admin and integrate it with email task
Eg) Send notification email to users every Fridays at 4 a.m.

###What is celery beat?
celery beat is a scheduler. When it's time to run the task, it delivers the entry to the worker node. (Periodic task execution)

Install celery beat
`pip install django-celery-beat`

Migrate - Tables will be created to store tasks and schedules
`python manage.py migrate`

settings.py
```
INSTALLED_APPS = (
    ...,
    'django_celery_beat',
)
```


###Settings for emails
1 . Click `Manage your Google Account`
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/adw1tscyb7l7mv8j9w7h.png)

<br/>
2 . Make sure `2 step verification` is on in Security > 2 step verification
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cijmkmugcsguhght2v1n.png)

<br/>
3 . Create `app password`. App passwords > select app > other > Name it whatever! 

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/61hbd1ldipfl7p4f6r6f.png)


<br/>
4 . Add `settings.py`
```
# e-mail settings
EMAIL_USE_TLS = True
EMAIL_PORT = 587
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = {YOUR EMAIL ADDRESS}
EMAIL_HOST_PASSWORD = {YOUR PASSWORD FROM NO.3}
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
```

Email setting is finished!

<br/>

### Where does celery beat store tasks and schedules?
####  1. Default entries are taken from the `beat_schedule` setting

task.py
```
app.conf.beat_schedule = {
  'add-every-30-seconds': {
        'task': 'tasks.add',
        'schedule': 30.0,
        'args': (16, 16)
    },
}
app.conf.timezone = 'UTC'
```

<br/>
Or if you want more control over your schedules use `crontab`
```
from celery.schedules import crontab

app.conf.beat_schedule = {
    # Executes every Monday morning at 7:30 a.m.
    'add-every-monday-morning': {
        'task': 'tasks.add',
        'schedule': crontab(hour=7, minute=30, day_of_week=1),
        'args': (16, 16),
    },
}
```

Start celery beat service 
`celery -A <project name> beat`

<br/>
#### 2 . Use custom stores - I will store them in database

1. Let's create a task that sends email to users
```
@app.task(name="send_notification", bind=True, default_retry_delay=300, max_retries=5)
def send_notification(self, subject, message):
    from django.core.mail import send_mail as sm
    
    # Fetch all users except superuser
    users = User.objects.exclude(is_superuser=True).all()
    user_emails = [user.email for user in users]

    # try sending email
    try:
        res = sm(
            subject=subject,
            html_message=message,
            from_email=EMAIL_HOST_USER,
            recipient_list=user_emails,
            fail_silently=False,
            message=None)
        print(f'Email send to {len(user_emails)} users')
    except Exception:
        
        # retry when fail
        send_notification.retry()

```
So here I created a task called `send_notification`.
`default_retry_delay` : retry after 300 seconds when fail
`max_retries` : only retry 5 more times then STOP.
Here I am also getting `subject` and `message` as arguments. This means I can pass `subject` and `message` from django admin. I will show you this later.


2. Let's go to django admin
`localhost:8080/admin`.  

**You will see new `Periodic Tasks` Menu**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3dd9pno3z8miartnckan.png)


**Let's make a `crontab` Crontab > Add crontab**
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p9plewa00ay3yf9tz2so.png)



**Let's create a crontab .Here I created one.  `Every Saturday @ 10:47 p.m.`**
 If you are not sure about crontab, check this out [Crontab examples] (https://crontab.guru/examples.html)
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bojci16hsrh89fnko09z.png)

**Let's create a periodic task**
Click `Periodic tasks` > `Add`. Create a new name of your choice. Find `send_notification` task that we have created up there
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r7mppp6d2owprmlrarnk.png)


`Schedule` > `Crontab Schedule` > Find the `crontab` you just created.
Add the `Start Datetime`
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zpac15g3tsefqgvh8csp.png)


This is where you pass the arguments `subject` and `message`
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gy2gy4z6m5gb3r8tv4t2.png)



So we are all set and ready to go.

Start django server, start rabbitmq

Start celery
`celery -A djangocelery(app name) worker --loglevel=info`

Start celery beat in different terminal
`celery -A djangocelery(app name) beat -l INFO --scheduler django_celery_beat.schedulers:DatabaseScheduler`


Let's check out the terminal of celery beat

@10:47 p.m. celery beat scheduler is sending `send_notification` task that we have created!! to celery!!
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h47zq65i4rujz2wx5or6.png)



Check celery terminal. Celery has received the task and email is sent.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mm193y1n7beyaaixsoud.png)


And I have received the email at 10:47 p.m. on Saturday with the title and content that I have set up in django admin!
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mqip5axzexn9e157q3xt.png)



###Conclusion
So `celery beat` is a `scheduler` and `celery` is the one that executes the `task`. `Database scheduler` is one of the method of celery beat where tasks and schedules are stored in database which means you can manage them in `django admin`.

<br/>
That's it for django celery beat!
The end.