Package: django-huey-monitor
django-huey-monitorDjango based tool for monitoringhuey task queueCurrent implementation will just store all Huey task signals into the database
and display them in the Django admin.Quickstartpipinstalldjango-huey-monitorINSTALLED_APPS=[#...'bx_django_utils',# https://github.com/boxine/bx_django_utils'huey_monitor',#...]Note: You must normally not change your Huey tasks.Collect main-/sub-tasksHuey monitor model can store information about task hierarchy. But this information can't be collected automatically.
You have to store these information in your Task code.e.g.:@task(context=True)defsub_task(task,parent_task_id,chunk_data):# Save the task hierarchy by:TaskModel.objects.set_parent_task(main_task_id=parent_task_id,sub_task_id=task.id,)# ... do something with e.g.: chunk_data ...@task(context=True)defmain_task(task):forchunk_datainsomething:sub_task(parent_task_id=task.id,chunk_data=chunk_data)Working example can be found in the test app here:huey_monitor_tests/test_app/tasks.pyCollect progress informationRunning task can store progress information in a similar way astqdm.
So it's possible to see the progress in admin.Minimal example:@task(context=True)deffoobar_task(task,list_to_process):process_info=ProcessInfo(task,desc='A description of this Job',total=len(list_to_process))foriteminlist_to_process:# ...to something with the {item}...process_info.update(n=1)# add the information that one item was processedIt is also possible to divide the work to several tasks and collect information about the processing of main-/sub-tasks.Working example can be found in the test app here:huey_monitor_tests/test_app/tasks.pysettingsoverride list filter (optional)It is possible to overridelist_filterofSignalInfoModelAdminandTaskModelAdminvia settings.
e.g.:HUEY_MONITOR_SIGNAL_INFO_MODEL_LIST_FILTER=('task__name','signal_name')HUEY_MONITOR_TASK_MODEL_LIST_FILTER=('name','state__signal_name')Note: This both settings are optional.
In this example is the "hostname" filter not present ;)run test projectNote: You can quickly test Huey Monitor with the test project, e.g:~/django-huey-monitor$./manage.shrun_testserveror in an isolated Docker container:~/django-huey-monitor$makeupMore info see below.Screenshots(More Screenshots here:boxine.github.io/django-huey-monitor/)2021-02-22-v030-task-details.png2021-02-22-v030-progress-info1.pngdevelopinginstall dockerclone the projectstart the containerTo start developing e.g.:~$gitclonehttps://github.com/boxine/django-huey-monitor.git
~$cddjango-huey-monitor
~/django-huey-monitor$./manage.py
~/django-huey-monitor$makehelp~/django-huey-monitor$makeupPoint our browser to:http://localhost:8000/Our Makefile contains the following targets:help                           List all commands
install                        install huey monitor package
update                         Update the dependencies as according to the pyproject.toml file
run-dev-server                 Run Django's developer server
test                           Run unittests
tox                            Run unittests via tox
makemessages                   Make and compile locales message files
clean                          Remove created files from the test project (e.g.: SQlite, static files)
build                          Update/Build docker services
up                             Start docker containers
down                           Stop all containers
shell-django                   go into a interactive bash shell in Django container
run-shell-django               Build and start the Django container and go into shell
shell-huey1                    go into a interactive bash shell in Huey worker container 1
shell-huey2                    go into a interactive bash shell in Huey worker container 2
shell-huey3                    go into a interactive bash shell in Huey worker container 3
shell-redis                    go into a interactive bash shell in Redis container
logs                           Display and follow docker logs
logs-django                    Display and follow docker logs only from "django" container
reload-django                  Reload the Django dev server
reload-huey                    Reload the Huey worker
restart                        Restart the containers
fire-test-tasks                Call "fire-test-tasks" manage command to create some Huey Tasks
fire-many-test-tasks           Call "fire-test-tasks" with --count 10000 to create many task entries ;)
fire-parallel-processing-task  Just fire "parallel processing" Huey Task
delete-all-tasks-data          Delete all Task/Signal database entiesIt's also possible to run the test setup with SQLite and Huey immediate setup
without docker:~$gitclonehttps://github.com/boxine/django-huey-monitor.git
~$cddjango-huey-monitor
~/django-huey-monitor$./manage.pyrun_dev_serverBackwards-incompatible changesVersion compatibilityHuey MonitorDjangoPython>v0.7.0v3.2, v4.1, v4.2v3.9, v3.10, v3.11>v0.6.0v3.2, v4.0, v4.1v3.9, v3.10, v3.11>v0.5.0v2.2, v3.1, v3.2v3.7, v3.8, v3.9<=v0.4.0v2.2, v3.0, v3.1v3.7, v3.8, v3.9v0.6.0We refactor the project setup: Developer must reinit the repository.v0.5.0Change CI and remove tests against Django 3.0, but add test run with Django v3.2v0.3.0 -> v0.4.0 - Outsourcing Django stuffbx_py_utilswas split and Django related stuff moved into:bx_django_utilsYou must change your Django settings and replace the app name:INSTALLED_APPS = [#...-     'bx_py_utils',+     'bx_django_utils','huey_monitor',#...]Historydevtbcv0.9.0 - 22.12.2023Fix #135 DisallowedModelAdminLookupAdd "thread" name as change list filter.Enhance test project setupApply manageprojects updatesRemove Python v3.9 supportAdd Django v5.0 to test matrix and remove Django 4.1Enable local AUTOLOGIN as defaultUse unittest "load_tests Protocol" and deny any requests in testsAddhttps://github.com/PyCQA/flake8-bugbearUpdate requirementsv0.8.1 - 20.11.2023BugfixZeroDivisionErrorin adminv0.8.0 - 20.11.2023Make is possible to overridelist_filterofSignalInfoModelAdminandTaskModelAdminvia settingsUpdate local docker dev setupv0.7.1 - 18.08.2023Fix #127: Catch error getting HUEY countsv0.7.0 - 09.08.2023New: Display Huey pending/scheduled/result task counts in adminSwitch to gitmainbranchSwitch frompytestto normalunittestsSwitch frompoetrytopip-toolsUsehttps://github.com/jedie/manage_django_projectfor developingExpand test matrix by remove Django 4.0 and add 4.2Enhance tox configv0.6.0 - 09.01.2023Test against Django v3.2, v4.0, v4.1 and Python v3.9 - v3.11Optimize Admin change list (contributed by henribru)Order sub-tasks chronologically in admin (contributed by Skrattoune)displaying progression for parent_tasks (contributed by Skrattoune)Delegating set_parent_task toProcessInfo.__init__(contributed by Skrattoune)Fix #27 auto crop overlong process info descriptionNEW #102 Unlock all tasks via change list object tool linkFix #81 ExpandTaskModel.descmax. length and makeSignalInfoModel.exception_lineoptionalUpdate docker test setupv0.5.0 - 10.02.2022Refactor models: RemoveTaskProgressModeland store progress information intoTaskModelv0.4.6 - 03.02.2022Display task desc or name in TaskAdmin(contributed bySkrattoune)update requirementsuseDarkeras code formatterCheck linting via PyTest pluginsv0.4.5 - 28.01.2022Fix#46by increment existing TaskProgressModel instances (based oncontributionbySkrattoune)v0.4.4 - 07.01.2022Add missinghuey_monitor.cssfilev0.4.3 - 07.01.2022Add temporarycumulate2parentsflag toProcessInfo(contributed by formacube)v0.4.2 - 25.08.2020suppress the default_app_config attribute in Django 3.2+ (contributed byJonas Svarvaa)v0.4.1 - 02.06.2020BugfixProcessInfo.__str__()#27Check 'desc' length inProcessInfoRemove test against Django 3.0 and add tests with Django 3.2Bugfix localtoxruns and use different Python versionsv0.4.0 - 21.05.2020bx_py_utils was split and Django related stuff moved into:https://github.com/boxine/bx_django_utilsv0.3.0 - 22.02.2020Store and display running task progress information a little bit astqdm#17v0.2.0 - 17.02.2020Store "parent_task" information for main-/sub-tasksv0.1.0 - 21.12.2020Work-a-round if a Huey worker diedFix missing translations from bx_py_utils in test projectSimulate a cluster of Huey worker via docker and the test projectv0.0.1 - 15.12.2021initial releaseLicenseGPL. Patches welcome!Linkshttps://pypi.org/project/django-huey-monitor/