django-cronman 3.0.0
Cron management app for Django

# django-cronman

## Overview

Django app to define and manage periodic tasks at Python level.

## Installation

`django-cronman` can be installed directly from PyPI using `pip`:

```bash
pip install django-cronman
```

You can also install it with additional dependencies to be able to use Cron Remote Manager.

```bash
pip install django-cronman[redis]
```

## Define a new cron job

Cron job definition is inspired by Django Admin configuration. To add a new job, you have to create `cron_job.py`
file inside an app, create `BaseCronJob` subclass inside and register it:

```python
from cronman.job import BaseCronJob, cron_job_registry

class HelloWorld(BaseCronJob):
    """Demo Cron Job class"""

    def run(self):
        """Main logic"""
        pass

cron_job_registry.register(HelloWorld)
```

Cron job classes are registered (and referred to) by name, which may be customized on registration:
```python
cron_job_registry.register(HelloWorld, name='Hello')
```
It's also possible to retrieve or unregister a class (e.g. while testing):
```python
cron_job_registry.get('HelloWorld')
cron_job_registry.unregister('HelloWorld')
```
If there is more than 1 cron job in given app, it's recommended to create a package instead of single `cron_jobs` module, create one submodule per class and do the imports and registration in package's  `__init__.py`.

## Configure cron jobs

To ensure that a cron job is executed periodically, you have add an entry to `CRON_JOBS`:

```python
CRON_JOBS = (
    ...
    # (<time spec>, <job spec>)
    # 'HelloWorld' will be executed a 5:15AM every day:
    ('   15   5   *   *   *', 'HelloWorld'),
)
```

Set ```CRONMAN_JOBS_MODULE``` to the dotted path name of the module where cron jobs are specified. Remember, this module MUST have a ```CRON_JOBS``` attribute. ```CRONMAN_JOBS_MODULE``` is ```None``` by default. For example:

```python
# settings_local.py

CRONMAN_JOBS_MODULE = 'app.cron_jobs.name'
```

## Run the scheduler

Cron jobs defined in ```settings.CRONMAN_JOBS_MODULE``` are started by `cron_scheduler` command from `cron` app.
This command constructs a list of jobs that should be executed in current period (now +/- 1 minute)
and creates **a new subprocess** for each job.
```
python manage.py cron_scheduler run
```
This command should be added to system's crontab on server responsible for running periodic tasks
and executed every 2 minutes.

## Run single cron job

Command `cron_worker run <job spec>` is responsible for executing cron jobs:
```
python manage.py cron_worker run HelloWorld
```

## Cron job parameters

Cron job classes can accept parameters which are passed to `run` method as positional or named arguments:
```python
class HelloWorld(BaseCronJob):
    """Demo Cron Job class"""

    def run(self, what, sleep=None):
        """Main logic"""
        print "Hello {}".format(what)
        if sleep:
            time.sleep(int(sleep))
    ...
```
```
python manage.py cron_worker run HelloWorld:world,sleep=5
```
Parameters are passed as string values, any type casts should be made in `run` method.
Quoted string with spaces are supported, but comma can be used only as argument separator:
```
python manage.py cron_worker run HelloWorld:"big world",sleep=5
```
There are utility functions for extracting lists and boolean values in `cronman.utils` module.

## Configure Cronitor support

`cron_worker` command can notify Cronitor when a job is started, finished or it has failed.
To enable this you have to:
1. Enable Cronitor support in settings `CRONMAN_CRONITOR_ENABLED = True`
2. Configure you cron job class:
```python
class HelloWorld(BaseCronJob):
    """Demo Cron Job class"""
    cronitor_id = 'ab12z'  # ID is assigned in Cronitor's dashboard
```

We may disable sending optional "RUN" and "FAIL" pings to Cronitor when cron job starts by setting `cronitor_ping_run = False` or `cronitor_ping_fail = False` but this doesn't seem to be necessary.

**Important note:**
When adding a new monitor in Cronitor dashboard, please use type **heartbeat**. Avoid using **cron job** monitors, as they're sending false-positive alerts "Has not run on schedule".


## Configure lock

Tasks can acquire locks to prevent concurrent calls. Locks have form of PIDfiles located in `settings.CRONMAN_DATA_DIR`. To modify lock behavior for given cron job class you can set `lock_type` attribute:

```python
from cronman.taxonomies import LockType

class HelloWorld(BaseCronJob):
    """Demo Cron Job class"""
    lock_type = LockType.PARAMS
```
The following values are supported:
* `None` - no lock, concurrency is allowed
* `LockType.CLASS` (default) - only one instance of given cron job class can be run at the same time (e.g. `Foo:p=1` and `Foo:p=2` can't work concurrently)
* `LockType.PARAMS` - only one combination of class and params can be run at the same time (e.g. `Foo:p=1` and `Foo:p=2` can work concurrently, but another call to `Foo:p=1` will be prohibited)
Locks acquired/released by `cron_worker` command.

We can configure a shared lock for several cron job classes to make sure only one of them is running:

```python
from cronman.taxonomies import LockType

class HelloWorld1(BaseCronJob):
    """Demo Cron Job class (1)"""
    lock_type = LockType.CLASS
    lock_name = 'HelloWorld'

class HelloWorld2(BaseCronJob):
    """Demo Cron Job class (2)"""
    lock_type = LockType.CLASS
    lock_name = 'HelloWorld'
```

## Configure CPU and IO priority

We can assign CPU priority (`nice`) to a cron job class by using `worker_cpu_priority` attribute:
```python
from cronman.taxonomies import CPUPriority

class NiceHellowWorld(BaseCronJob):
    """Hello World running through `nice -n 19`"""
    worker_cpu_priority = CPUPriority.LOWEST
```
We can also customize IO priority (`ionice`) by assigning one of values from `cronman.taxonomies.IOPriority` to `worker_io_priority` attribute, but this is not necessary in most cases, as `nice` changes IO priority as well.

Commands `cron_scheduler run`, `cron_worker resume`, and cron job `RunCronTasks` will spawn worker processes with respect to CPU and IO priorities assigned to cron job classes. These settings **are not** enforced when running `cron_worker run` so you have to prepend `nice`/`ionice` to such calls manually.

## List and kill running cron jobs

Command `cron_worker status` shows currently running cron jobs - PIDfile name, PID and status (`ALIVE`, `DEAD`).
Search results can be limited by `job spec` (cron job name, parameters):
```
python manage.py cron_worker status
python manage.py cron_worker status Foo
python manage.py cron_worker status Foo:bar=1
```

Command `cron_worker kill` kills active cron jobs, gracefully (`SIGTERM`) or by force when process refuses to die (`SIGKILL`). List of tasks can be limited by `job spec`:
```
python manage.py cron_worker kill
python manage.py cron_worker kill Foo
python manage.py cron_worker kill Foo:bar=1
```
Single process can be killed also using PID:
```
python manage.py cron_worker kill 39078
```

## Resuming cron jobs

Subset of cron jobs can be resumed after being killed:

```python

class ResumableHelloWorld(BaseCronJob):
    """Demo Cron Job class"""
    can_resume = True
```

Command `cron_worker resume` starts all killed cron jobs with `can_resume` capability:
```
python manage.py cron_worker resume
```

To remove all entries about dead cron jobs and make sure they won't be resumed we can run `cron_worker clean` command:
```
python manage.py cron_worker clean
```

Command `cron_worker suspend` cleans all previous entries about dead cron jobs and then kills all running ones to make sure that next `resume` will raise only recently killed jobs:
```
python manage.py cron_worker suspend
```


## List available cron jobs

Command `cron_worker info` shows list of all available cron jobs:
```
python manage.py cron_worker info
```
When cron job name is passed to this command, system displays docstring and parameters of given cron job:
```
python manage.py cron_worker info Foo
```

## Disable the scheduler

Scheduler command can be disabled temporarily:
```
python manage.py cron_scheduler disable
```
and re-enabled later:
```
python manage.py cron_scheduler enable
```
Calls to `cron_scheduler run` will not spawn worker processes while scheduler is disabled.

## Send errors to sentry

Errors in cron job classes are intercepted by `cron_worker` and sent to Sentry using the same config as other Django commands (`settings.RAVEN_MANAGEMENT_COMMAND_CONFIG`).
If `settings.CRONMAN_RAVEN_CMD` is defined, the scheduler will use it as execution script for worker processes, e.g.
`python manage.py cron_worker run Foo:bar=1` will be converted to `{CRONMAN_RAVEN_CMD} -c "python manage.py cron_worker run Foo:bar=1"`

## Cron Tasks - running cron jobs from Admin area

Some cron jobs can be requested to start from Admin area: **Admin > Cron > Cron Tasks**
To add a cron job class to the list in Admin we need to set `ALLOWED_CRON_TASKS` setting:

```python

ALLOWED_CRON_TASKS = (
    'HelloWorld',
)
```
To request another run of given cron job we can just create a new `CronTask` record in Admin.
Cron job `RunCronTasks` started every 4 minutes by the scheduler will spawn a separate worker process for each pending Cron Task.

## Changelog

2021-01-20 - 3.0.0 Remove Python 2 checks. Fix issue with suspend all.
2020-05-25 - 2.1.0 Python 3 + Django 2 compatibility.
2020-04-30 - 2.0.1 Fix for sentry-sdk.
2020-04-23 - 2.0.0 Replace raven with sentry-sdk.
2020-01-09 - 1.2.0 Django 2 compatibility.
2019-04-30 - 1.1.1 Pre-commit.com hooks support. Docs update
2019-03-13 - 1.1.0 Add support for cronitor ping for cron_scheduler
2019-02-25 - 1.0.0 Initial version released