Package: django-q
A multiprocessing distributed task queue for DjangoFeaturesMultiprocessing worker poolAsynchronous tasksScheduled, cron and repeated tasksSigned and compressed packagesFailure and success database or cacheResult hooks, groups and chainsDjango Admin integrationPaaS compatible with multiple instancesMulti cluster monitorRedis, Disque, IronMQ, SQS, MongoDB or ORMRollbar and Sentry supportRequirementsDjango> = 2.2Django-picklefieldArrowBlessedTested with: Python 3.7, 3.8, 3.9 Django 2.2.X and 3.2.XWarningSince Python 3.7asyncbecame a reserved keyword and was refactored toasync_taskBrokersRedisDisqueIronMQAmazon SQSMongoDBDjango ORMInstallationInstall the latest version with pip:$ pip install django-qAdddjango_qto yourINSTALLED_APPSin your projectssettings.py:INSTALLED_APPS = (
    # other apps
    'django_q',
)Run Django migrations to create the database tables:$ python manage.py migrateChoose a messagebroker, configure and install the appropriate client library.Read the full documentation athttps://django-q.readthedocs.orgConfigurationAll configuration settings are optional. e.g:# settings.py exampleQ_CLUSTER={'name':'myproject','workers':8,'recycle':500,'timeout':60,'compress':True,'cpu_affinity':1,'save_limit':250,'queue_limit':500,'label':'Django Q','redis':{'host':'127.0.0.1','port':6379,'db':0,}}For full configuration options, see theconfiguration documentation.Management CommandsStart a cluster with:$ python manage.py qclusterMonitor your clusters with:$ python manage.py qmonitorMonitor your clusters’ memory usage with:$ python manage.py qmemoryCheck overall statistics with:$ python manage.py qinfoCreating TasksUseasync_taskfrom your code to quickly offload tasks:fromdjango_q.tasksimportasync_task,result# create the taskasync_task('math.copysign',2,-2)# or with a referenceimportmath.copysigntask_id=async_task(copysign,2,-2)# get the resulttask_result=result(task_id)# result returns None if the task has not been executed yet# you can wait for ittask_result=result(task_id,200)# but in most cases you will want to use a hook:async_task('math.modf',2.5,hook='hooks.print_result')# hooks.pydefprint_result(task):print(task.result)For more info seeTasksScheduleSchedules are regular Django models. You can manage them through the
Admin page or directly from your code:# Use the schedule functionfromdjango_q.tasksimportscheduleschedule('math.copysign',2,-2,hook='hooks.print_result',schedule_type=Schedule.DAILY)# Or create the object directlyfromdjango_q.modelsimportScheduleSchedule.objects.create(func='math.copysign',hook='hooks.print_result',args='2,-2',schedule_type=Schedule.DAILY)# Run a task every 5 minutes, starting at 6 today# for 2 hoursimportarrowschedule('math.hypot',3,4,schedule_type=Schedule.MINUTES,minutes=5,repeats=24,next_run=arrow.utcnow().replace(hour=18,minute=0))# Use a cron expressionschedule('math.hypot',3,4,schedule_type=Schedule.CRON,cron='0 22 * * 1-5')For more info check theSchedulesdocumentation.TestingTo run the tests you will need the following in addition to install requirements:py.testpytest-djangoDisque fromhttps://github.com/antirez/disque.gitRedisMongoDBOr you can use the included Docker Compose file.The following commands can be used to run the tests:# Create virtual environmentpython-mvenvvenv# Install requirementsvenv/bin/pipinstall-rrequirements.txt# Install test dependenciesvenv/bin/pipinstallpytestpytest-django# Install django-qvenv/bin/pythonsetup.pydevelop# Run required services (you need to have docker-compose installed)docker-compose-ftest-services-docker-compose.yamlup-d# Run testsvenv/bin/pytest# Stop the services required by tests (when you no longer plan to run tests)docker-compose-ftest-services-docker-compose.yamldownLocaleCurrently available in English, German and French.
Translation pull requests are always welcome.TodoBetter tests and coverageLess dependencies?AcknowledgementsDjango Q was inspired by working withDjango-RQandRQHuman readable hashes byHumanHashRedditors feedback atr/djangoJetBrains for theirOpen Source Support Program