Package: hueyx
hueyxA django extension to run huey with multiple queues.
Multiple queues allow tasks to not block each other and to scale independently.
Only the redis storage is supported.ImportantIf you usehuey 1.xthen installhueyx 0.1.2. Checkout the git taghuey1.x.If you usehuey 2.xthen installhueyx  >= 1.0.UsageInstall it withpipinstallhueyxAdd hueyx in your installed apps.INSTALLED_APPS=['django.contrib.admin','django.contrib.auth','django.contrib.contenttypes','django.contrib.sessions','django.contrib.messages','django.contrib.staticfiles','hueyx',]settings.pyCompared to djhuey, hueyx allows several queues to be defined in the settings.py.HUEYX={'queue_name1':{'connection':{'host':'localhost','port':6379,'db':0,},'consumer':{'workers':1,'worker_type':'process',}},'queue_name2':{'connection':{'connection_pool':ConnectionPool(host='localhost',port=6379,db=1)},'consumer':{'multiple_scheduler_locking':True,'prometheus_metrics':True,'workers':2,'worker_type':'thread',}},'priority_queue_name3':{'huey_class':'huey.PriorityRedisHuey','connection':{'connection_pool':ConnectionPool(host='localhost',port=6379,db=1)},'consumer':{'multiple_scheduler_locking':True,'prometheus_metrics':True,'workers':2,'worker_type':'thread',}},}The settings are almost the same as in djhuey.
Have a look at thehuey documentationto see the exact parameter usage.Exceptions:You can only configure redis as storage engine by configurehuey_classtohuey.RedisHuey,huey.PriorityRedisHuey,huey.RedisExpireHueyorhuey.PriorityRedisExpireHuey.Thenameandbackend_classparameters are not supported.The optionsmultiple_scheduler_lockingandprometheus_metrics_enabledhave been added. See below.The parametersheartbeat_timeoutfordb_taskhas been added. See below.tasks.pyfromhueyx.queuesimporthueyx"""Define which queue you want to use.They are predefined in settings.py."""HUEY_Q1=hueyx('queue_name1')HUEY_Q2=hueyx('queue_name2')@HUEY_Q1.task()defmy_task1():print('my_task1 called')@HUEY_Q1.db_task()defmy_db_task1():print('my_db_task1 called')@HUEY_Q2.task()defmy_task2():print('my_task2 called')@HUEY_Q2.periodic_task(crontab(minute='0',hour='3'))defmy_periodic_task2():print('my_periodic_task2 called')return1@HUEY_Q2.db_task(heartbeat_timeout=120)defmy_heartbeat_task(heartbeat:Heartbeat):withheartbeat.long_running_operation():print('This operation can take a while -> don\'t check for heartbeats')print('Now we check for heartbeats -> call heartbeat() periodically')heartbeat()Push task to queuefromexample.tasksimportmy_task1,my_db_task1,my_task2my_task1()# Task for queue_name1my_db_task1()# Task for queue_name1my_task2()# Task for queue_name2Run consumerConsumers are started with the queue_name../manage.pyrun_hueyxqueue_name1Heartbeat tasksHeartbeat tasks are tasks with the parameterheartbeat_timeout. It defines the timeout in seconds.
They get a Heartbeat object which needs to be called in order to send a heartbeat to redis.
If no heartbeat occurs in set timeout the task is presumed to be dead and will automatically get restarted.heartbeat_timeoutneeds to be at least 120 seconds. It does not work together with the parameterinclude_task.Additional settingsmultiple_scheduler_lockingmultiple_scheduler_lockinghas been added to support multiple huey schedulers.
If you run huey in a cloud environment, you will end up running multiple huey instances which each will
schedule the periodic task.multiple_scheduler_lockingprevents periodic tasks to be scheduled multiple times. It is false by default.Huey signalsOptionally hueyx pushes all huey signals to the redis pubsubhueyx.huey2.signalingif enabled.HUEYX_SIGNALS={'enabled':True,'environment':'your environment'}The format of the message is{'environment':settings.HUEYX_SIGNALS['environment'],'queue':queue,'pid':pid,'signal':signal_name,'task':task_name}The environment parameter is a optional variable.PrometheusThehuey-exporterproject takes the signals und reports it to prometheus.CollaboratorsUpdate hueyx on PyPi