Package: celery-sqlalchemy-scheduler-hotstar
A Scheduler Based Sqlalchemy for Celery.Getting StartedEnglish中文文档PrerequisitesPython 3celery >= 4.2.0sqlalchemyFirst you must installceleryandsqlalchemy, andceleryshould be >=4.2.0.$pipinstallcelery$pipinstallsqlalchemyInstallingInstall from PyPi$pipinstallcelery-sqlalchemy-schedulerInstall from source by cloning this repository$gitclonegit@github.com:AngelLiang/celery-sqlalchemy-scheduler.git$cdcelery-sqlalchemy-scheduler$pythonsetup.pyinstallUsageAfter you have installedcelery_sqlalchemy_scheduler, you can easily start with following steps:This is a demo for exmaple, you can check the code inexamplesdirectorystart celery worker$celeryworker-Atasks-linfostart the celery beat withDatabaseScheduleras scheduler$celerybeat-Atasks-Scelery_sqlalchemy_scheduler.schedulers:DatabaseScheduler-linfoDescriptionAfter the celery beat is started, by default it create a sqlite database(schedule.db) in current folder.
You can useSQLiteStudio.exeto inspect it.When you want to update scheduler, you can update the data inschedule.db.
Butcelery_sqlalchemy_schedulerdon’t update the scheduler immediately.
Then you shoule be change the first column’slast_updatefield in thecelery_periodic_task_changedto now datetime. Finally the celery beat
will update scheduler at next wake-up time.Database ConfigurationYou can configure sqlalchemy db uri when you configure the celery, example as:fromceleryimportCelerycelery=Celery('tasks')beat_dburi='sqlite:///schedule.db'celery.conf.update({'beat_dburi':beat_dburi})Also, you can use MySQL or PostgreSQL.# MySQL: `pip install mysql-connector`beat_dburi='mysql+mysqlconnector://root:root@127.0.0.1:3306/celery-schedule'# PostgreSQL: `pip install psycopg2`beat_dburi='postgresql+psycopg2://postgres:postgres@127.0.0.1:5432/celery-schedule'Example CodeExample creating interval-based periodic taskTo create a periodic task executing at an interval you must first
create the interval object:>>>fromcelery_sqlalchemy_scheduler.modelsimportPeriodicTask,IntervalSchedule>>>fromcelery_sqlalchemy_scheduler.sessionimportSessionManager>>>fromceleryconfigimportbeat_dburi>>>session_manager=SessionManager()>>>session=session_manager.session_factory(dburi=beat_dburi)# executes every 10 seconds.>>>schedule=session.query(IntervalSchedule).filter_by(every=10,period=IntervalSchedule.SECONDS).first()>>>ifnotschedule:...schedule=IntervalSchedule(every=10,period=IntervalSchedule.SECONDS)...session.add(schedule)...session.commit()That’s all the fields you need: a period type and the frequency.You can choose between a specific set of periods:IntervalSchedule.DAYSIntervalSchedule.HOURSIntervalSchedule.MINUTESIntervalSchedule.SECONDSIntervalSchedule.MICROSECONDSnote:If you have multiple periodic tasks executing every 10 seconds,
then they should all point to the same schedule object.Now that we have defined the schedule object, we can create the periodic task
entry:>>>task=PeriodicTask(...interval=schedule,# we created this above....name='Importing contacts',# simply describes this periodic task....task='proj.tasks.import_contacts',# name of task....)>>>session.add(task)>>>session.commit()Note that this is a very basic example, you can also specify the
arguments and keyword arguments used to execute the task, thequeueto
send it to[*], and set an expiry time.Here's an example specifying the arguments, note how JSON serialization
is required:>>>importjson>>>fromdatetimeimportdatetime,timedelta>>>periodic_task=PeriodicTask(...interval=schedule,# we created this above....name='Importing contacts',# simply describes this periodic task....task='proj.tasks.import_contacts',# name of task....args=json.dumps(['arg1','arg2']),...kwargs=json.dumps({'be_careful':True}),...expires=datetime.utcnow()+timedelta(seconds=30)...)>>>session.add(periodic_task)>>>session.commit()Example creating crontab-based periodic taskA crontab schedule has the fields:minute,hour,day_of_week,day_of_monthandmonth_of_year, so if you want the equivalent of a30 * * * *(execute every 30 minutes) crontab entry you specify:>>>fromcelery_sqlalchemy_scheduler.modelsimportPeriodicTask,CrontabSchedule>>>importpytz>>>schedule=CrontabSchedule(...minute='30',...hour='*',...day_of_week='*',...day_of_month='*',...month_of_year='*',...timezone=pytz.timezone('Asia/Shanghai')...)The crontab schedule is linked to a specific timezone using the
‘timezone’ input parameter.Then to create a periodic task using this schedule, use the same
approach as the interval-based periodic task earlier in this document,
but instead ofinterval=schedule, specifycrontab=schedule>>>periodic_task=PeriodicTask(...crontab=schedule,...name='Importing contacts',...task='proj.tasks.import_contacts',...)>>>session.add(periodic_task)>>>session.commit()Temporarily disable a periodic taskYou can use theenabledflag to temporarily disable a periodic task:>>>periodic_task.enabled=False>>>session.add(periodic_task)>>>session.commit()Example running periodic tasksThe periodic tasks still need ‘workers’ to execute them. So make sure
the defaultCelerypackage is installed. (If not installed, please
follow the installation instructions here:
<https://github.com/celery/celery>)Both the worker and beat services need to be running at the same time.Start a Celery worker service (specify your project name):$celery-A[project-name]worker--loglevel=infoAs a separate process, start the beat service (specify the
scheduler):$celery-A[project-name]beat-linfo--schedulercelery_sqlalchemy_scheduler.schedulers:DatabaseSchedulerAcknowledgmentsdjango-celery-beatcelerybeatrediscelery