Package: django-toosimple-q
Django Too Simple QueueThis packages provides a simplistic task queue and scheduler for Django.If execution of your tasks is mission critical, do not use this library, and turn to more complex solutions such as Celery, as this package doesn't guarantee task execution nor unique execution.It is geared towards basic apps, where simplicity primes over reliability. The package offers simple decorator syntax, including cron-like schedules.Features :no celery/redis/rabbitmq/whatever... just Django !clean decorator syntax to register tasks and schedulessimple queuing syntaxcron-like schedulingtasks.py autodiscoverydjango admin integrationLimitations :probably not extremely reliable because of race conditionsno multithreading yet (but running multiple workers should work)InstallationInstall the library :$pipinstalldjango-toosimple-qEnable the app insettings.py:INSTALLED_APPS=[...'django_toosimple_q',...]QuickstartTasks need to be registered using the@register_task()decorator. Once registered, they can be added to the queue by calling the.queue()function.fromdjango_toosimple_q.decoratorsimportregister_task# Register a task@register_task()defmy_task(name):returnf"Hello{name}!"# Enqueue tasksmy_task.queue("John")my_task.queue("Peter")Registered tasks can be scheduled from code using this cron-like syntax :fromdjango_toosimple_q.decoratorsimportregister_task,schedule# Register and schedule tasks@schedule(cron="30 8 * * *",args=['John'])@register_task()defmorning_routine(name):returnf"Good morning{name}!"To consume the tasks, you need to run at least one worker :$pythonmanage.pyworkerThe workers will take care of adding scheduled tasks to the queue when needed, and will execute the tasks.The package autoloadstasks.pyfrom all installed apps. While this is the recommended place to define your tasks, you can do so from anywhere in your code.Advanced usageTasksYou can optionnaly give a custom name to your tasks. This is required when your task is defined in a local scope.@register_task("my_favourite_task")defmy_task(name):returnf"Good morning{name}!"You can set task priorities.@register_task(priority=0)defmy_favourite_task(name):returnf"Good bye{name}!"@register_task(priority=1)defmy_other_task(name):returnf"Hello{name}!"# Enqueue tasksmy_other_task.queue("John")my_favourite_task.queue("Peter")# will be executed before the other oneYou can defineretries=Nandretry_delay=Sto retry the task in case of failure. The delay (in second) will double on each failure.@register_task(retries=10,retry_delay=60)defsend_email():...You can mark a task asunique=Trueif the task shouldn't be queued again if already queued with the same arguments. This is usefull for tasks such as cleaning or refreshing.@register_task(unique=True)defcleanup():...cleanup.queue()cleanup.queue()# this will be ignored as long as the first one is still queuedYou can assign tasks to specific queues, and then have your worker only consume tasks from specific queues using--queue myqueueor--exclude_queue myqueue. By default, workers consume all tasks.@register_task(queue='long_running')deflong_task():...@register_task()defshort_task():...# Then run those with these workers, so that long# running tasks don't prevent short running tasks# from being run :# manage.py worker --exclude_queue long_running# manage.py workerSchedulesBy default,last_checkis set tonow()on schedule creation. This means they will only run on next cron occurence. If you need your schedules to be run as soon as possible after initialisation, you can specifylast_check=None.@schedule(cron="30 8 * * *",last_check=None)@register_task()defmy_task(name):returnf"Good morning{name}!"By default, if some crons where missed (e.g. after a server shutdown or if the workers can't keep up with all tasks), the missed tasks will be lost. If you need the tasks to catch up, setcatch_up=True.@schedule(cron="30 8 * * *",catch_up=True)@register_task()defmy_task(name):...You may define multiple schedules for the same task. In this case, it is mandatory to specify a unique name :@schedule(name="morning_routine",cron="30 16 * * *",args=['morning'])@schedule(name="afternoon_routine",cron="30 8 * * *",args=['afternoon'])@register_task()defmy_task(time):returnf"Good{time}John !"You may get the schedule's cron datetime provided as a keyword argument to the task using thedatetime_kwargargument :@schedule(cron="30 8 * * *",datetime_kwarg="scheduled_on")@register_task()defmy_task(scheduled_on):returnf"This was scheduled for{scheduled_on.isoformat()}."Management commentBesides standard django management commands arguments, the management command supports following arguments.usage: manage.py worker [--queue QUEUE | --exclude_queue EXCLUDE_QUEUE]
                        [--tick TICK]
                        [--once | --until_done]
                        [--no_recreate | --recreate_only]

optional arguments:
  --queue QUEUE         which queue to run (can be used several times, all
                        queues are run if not provided)
  --exclude_queue EXCLUDE_QUEUE
                        which queue not to run (can be used several times, all
                        queues are run if not provided)
  --tick TICK           frequency in seconds at which the database is checked
                        for new tasks/schedules
  --once                run once then exit (useful for debugging)
  --until_done          run until no tasks are available then exit (useful for
                        debugging)
  --no_recreate         do not (re)populate the schedule table (useful for
                        debugging)
  --recreate_only       populates the schedule table then exit (useful for
                        debugging)Contrib appsdjango_toosimple_q.contrib.mailA queued email backend to send emails asynchronously, preventing your website from failing completely in case the upstream backend is down.InstallationEnable and configure the app insettings.py:INSTALLED_APPS=[...'django_toosimple_q.contrib.mail',...]EMAIL_BACKEND='django_toosimple_q.contrib.mail.backends.QueueBackend'# Actual Django email backend used, defaults to django.core.mail.backends.smtp.EmailBackend, see https://docs.djangoproject.com/en/3.2/ref/settings/#email-backendTOOSIMPLEQ_EMAIL_BACKEND='django.core.mail.backends.smtp.EmailBackend'DevTestsTo run tests locally (by default, tests runs against an in-memory sqlite database):$pipinstall-rrequirements-dev.txt
$pythonmanage.pytestTo run tests against postgres, run the following commands before :# Start a local postgres database$dockerrun-p5432:5432-ePOSTGRES_PASSWORD=postgres-dpostgres# Set and env var$exportTOOSIMPLEQ_TEST_DB=postgres# on Windows: `$Env:TOOSIMPLEQ_TEST_DB = "postgres"`Tests are run automatically on github.ContributeCode style is done with pre-commit :$ pip install -r requirements-dev.txt
$ pre-commit installChangelog2022-03-24 : v0.4.0madelast_checkandlast_runoptional in the admindefinedidfields2021-07-15 : v0.3.0addedcontrib.mailtask replacement now tracked with a FK instead of a statealso run tests on postgresaddeddatetime_kwargargument to schedules2021-06-11 : v0.2.0addedretries,retry_delayoptions for tasksimprove logging2020-11-12 : v0.1.0fixed bug where updating schedule failedfixed worker not doing all available tasks for each tickadded --tick argumentenforce uniqueness of schedule