Package: celery-redbeat-sempr
RedBeatis aCelery Beat Schedulerthat stores the scheduled tasks and runtime metadata inRedis.Why RedBeatDynamic live task creation and modification, without lengthy downtimeExternally manage tasks from any language with Redis bindingsShared data store; Beat isn’t tied to a single drive or machineFast startup even with a large task countPrevent accidentally running multiple Beat serversGetting StartedInstall with pip:pip install celery-redbeatConfigure RedBeat settings in your Celery configuration file:redbeat_redis_url="redis://localhost:6379/1"Then specify the scheduler when running Celery Beat:celery beat -S redbeat.RedBeatSchedulerRedBeat uses a distributed lock to prevent multiple instances running.
To disable this feature, set:redbeat_lock_key=NoneConfigurationYou can add any of the following parameters to your Celery configuration
(see Celery 3.x compatible configuration value names in below).redbeat_redis_urlURL to redis server used to store the schedule, defaults to value ofbroker_url.redbeat_key_prefixA prefix for all keys created by RedBeat, defaults to'redbeat'.redbeat_lock_keyKey used to ensure only a single beat instance runs at a time,
defaults to'<redbeat_key_prefix>:lock'.redbeat_lock_timeoutUnless refreshed the lock will expire after this time, in seconds.Defaults to five times of the default scheduler’s loop interval
(300seconds), so1500seconds (25minutes).See thebeat_max_loop_intervalCelery docs about for more information.Celery 3.x config namesHere are the old names of the configuration values for use with
Celery 3.x.Celery 4.xCelery 3.xredbeat_redis_urlREDBEAT_REDIS_URLredbeat_key_prefixREDBEAT_KEY_PREFIXredbeat_lock_keyREDBEAT_LOCK_KEYredbeat_lock_timeoutREDBEAT_LOCK_TIMEOUTDesignAt its core RedBeat uses a Sorted Set to store the schedule as a priority queue.
It stores task details using a hash key with the task definition and metadata.The schedule set contains the task keys sorted by the next scheduled run time.For each tick of Beatget list of due keys and due next tickretrieve definitions and metadata for all keys from previous stepupdate task metadata and reschedule with next run time of taskcall due tasks using async_applycalculate time to sleep until start of next tick using remaining tasksCreating TasksYou can use Celery’s usual way to define static tasks or you can insert tasks
directly into Redis. The config options is calledbeat_schedule, e.g.:app.conf.beat_schedule={'add-every-30-seconds':{'task':'tasks.add','schedule':30.0,'args':(16,16)},}On Celery 3.x the config option was calledCELERYBEAT_SCHEDULE.The easiest way to insert tasks from Python is it useRedBeatSchedulerEntry():interval = celery.schedules.schedule(run_every=60)  # seconds
entry = RedBeatSchedulerEntry('task-name', 'tasks.some_task', interval, args=['arg1', 2])
entry.save()Alternatively, you can insert directly into Redis by creating a new hash with
a key of<redbeat_key_prefix>:task-name. It should contain a single keydefinitionwhich is a JSON blob with the task details.IntervalAn interval task is defined with the JSON like:{
    "name" : "interval example",
    "task" : "tasks.every_5_seconds",
    "schedule": {
        "__type__": "interval",
        "every" : 5, # seconds
        "relative": false, # optional
    },
    "args" : [  # optional
        "param1",
        "param2"
    ],
    "kwargs" : {  # optional
        "max_targets" : 100
    },
    "enabled" : true,  # optional
}CrontabAn crontab task is defined with the JSON like:{
    "name" : "crontab example",
    "task" : "tasks.daily",
    "schedule": {
        "__type__": "crontab",
        "minute" : "5", # optional, defaults to *
        "hour" : "*", # optional, defaults to *
        "day_of_week" : "monday", # optional, defaults to *
        "day_of_month" : "*/7", # optional, defaults to *
        "month_of_year" : "[1-12]", # optional, defaults to *
    },
    "args" : [  # optional
        "param1",
        "param2"
    ],
    "kwargs" : {  # optional
        "max_targets" : 100
    },
    "enabled" : true,  # optional
}SchedulingAssuming yourredbeat_key_prefixconfig values is set to‘redbeat:’(default) you will also need to insert the new task into the schedule with:zadd redbeat::schedule 0 new-task-nameThe score is the next time the task should run formatted as a UNIX timestamp.MetadataApplications may also want to manipulate the task metadata to have more control over when a task runs.
The meta key contains a JSON blob as follows:{
    'last_run_at': {
        '__type__': 'datetime',
        'year': 2015,
        'month': 12,
        'day': 29,
        'hour': 16,
        'minute': 45,
        'microsecond': 231
    },
    'total_run_count'; 23
}For instance by default`last_run_at`corresponds to when Beat dispatched the task, but depending on queue latency it might not run immediately, but the application could update the metadata with
the actual run time, allowing intervals to be relative to last execution rather than last dispatch.Sentinel supportThe redis connexion can use a Redis/Sentinel cluster. The
configuration syntax is inspired fromcelery-redis-sentinel# celeryconfig.py
BROKER_URL = 'redis-sentinel://redis-sentinel:26379/0'
BROKER_TRANSPORT_OPTIONS = {
    'sentinels': [('192.168.1.1', 26379),
                  ('192.168.1.2', 26379),
                  ('192.168.1.3', 26379)],
    'service_name': 'master',
    'socket_timeout': 0.1,
}

CELERY_RESULT_BACKEND = 'redis-sentinel://redis-sentinel:26379/1'
CELERY_RESULT_BACKEND_TRANSPORT_OPTIONS = BROKER_TRANSPORT_OPTIONSSome notes about the configuration:note the use ofredis-sentinelschema within the URL for broker and results
backend.hostname and port are ignored within the actual URL. Sentinel uses transport optionssentinelssetting to create aSentinel()instead of configuration URL.DevelopmentRedBeat is available onGitHubOnce you have the source you can run the tests with the following commands:pip install -r requirements.dev.txt
py.test testsYou can also quickly fire up a sample Beat instance with:celery beat --config exampleconf