Package: dramatiq-taskstate
dramatiq-taskstateA middleware for Dramatiq (for Django) that keeps track of task state only
when you need it to.Note:When using the term "task" in the documentation: that would generally refer
to the task model in this package. It has nothing to do with Dramatiq or thedjango_dramatiqpackage except that theTaskmodel is an abstraction of
a Dramatiq task. Therefore, this package only operates on theTaskmodel
and not Dramatiq tasks.QuickstartInstall dramatiq-taskstate via pip:pip install dramatiq-taskstateAddtaskstateanddjango.contrib.postgresto yourINSTALLED_APPSin
your project settings.py file:INSTALLED_APPS=['django_dramatiq','...','django.contrib.postgres','...','taskstate',]Run migrate:python manage.py migrateInclude the middleware in yourdjango_dramatiqconfiguration:DRAMATIQ_BROKER={'BROKER':'...','OPTIONS':{# ...},'MIDDLEWARE':[# ...'taskstate.middleware.StateMiddleware',]}Add afor_stateparameter to Dramatiq actors that need task state
to be tracked. The middleware will ignore any tasks that don't have
this argument. Also remember that all values in thefor_statedictionary must be JSON serializable.@dramatiq.actordefmy_actor(arg,for_state={}):passThen, when sending a new task to Dramatiq: thefor_statedictionary can
contain any of the following keys:'for_state':{'user_pk':user.pk,'model_name':'model','app_name':'app','description':'description',}Each time a task's status is updated atask_changedsignal is dispatched
which can be handled like this:fromdjango.dispatchimportreceiverfromtaskstate.middlewareimportStateMiddlewarefromtaskstate.signalsimporttask_changed@receiver(task_changed,sender=StateMiddleware)defhandle_task_changed(sender,task,**kwargs):passKeep in mind that this is not apost_savesignal -- it only fires for
status updates.Reporting task state to the UIOf course, a common case with background tasks is that the progress/state of a
task needs to be displayed to a user somehow. This package includes aWebsocketConsumerthat can be used withdjango-channelsto check the
status of a task. Check the flowchart in the root of the repo for more
information on how this works.Check theget_task_status.jsfile in thetaskstate/staticdirectory for
an example of how to send a request via websockets to get/monitor the task
status/progress. Also, as shown in the flowchart in the root of the repo,
bothtask_changed-- which only handles when the task object's status is
updated, i.e, enqueued, done, etc. -- andpost_savesignals are handled.Routing is included for django-channels. Make sure to use theURLRouterfor your django-channels configuration. You can send data for the
websocket to the following route:/ws/get-task-status/Or, create your own route:fromdjango.urlsimportre_pathfromtaskstate.consumersimportCheckTaskStatuswebsocket_urlpatterns=[re_path(r'^ws/custom-route-task-status/$',CheckTaskStatus.as_asgi()),]Also remember to add the routes to your django-channels router, for example:application=ProtocolTypeRouter({'http':django_asgi_app,'websocket':AllowedHostsOriginValidator(AuthMiddlewareStack(URLRouter(taskstate.routing.websocket_urlpatterns,)),),})A default template is included to render tasks in the UI -- use the following
in your templates (check the template to see which context variables to use):{% include 'taskstate/task_list.html' %}The above template will merely render a list of tasks, however, to check/monitor
the statuses of those tasks include the default script in your HTML before
the closing body tag:<scriptsrc="{% static 'taskstate/get_task_status.js' %}"charset="utf-8"></script>Updating progress percentage of aTaskSee thetask_progress_example.pyfile in the examples directory in the root of
the repo for an example of how to update the progress of a task. Note that
some of the details there are specific to Dramatiq itself.Seen status of aTaskA task can only be marked as seen when it is complete. The seen status of a
set of tasks can be set through another django-channels route:/ws/set-task-seen/Sending a list of task ID's to this route will automatically mark all
completed tasks from the list of ID's as seen. This is handled in the default
JS script -- therefore, check theget_task_status.jsfile for an example.There is also an APS (Advanced Python Scheduler) periodic task that will
delete tasks older than 120 seconds for tasks that have been seen and
have a "final/completed" status like "skipped", "failed" or "done". To add
thecleanup_tasksperiodic job to APS:fromdjango.confimportsettingsfromapscheduler.schedulers.blockingimportBlockingSchedulerfromapscheduler.triggers.cronimportCronTriggerfromtaskstate.tasksimportcleanup_tasksscheduler=BlockingScheduler(timezone=settings.TIME_ZONE)scheduler.add_job(cleanup_tasks.send,trigger=CronTrigger(second='*/240'),# Every 240 secondsmax_instances=1,replace_existing=True,)Task statusesenqueueddelayedrunningfaileddoneskippedGet all completed taskscompleted_tasks=Task.objects.completed()Get tasks for displayTo get all the tasks that have been recently seenandthat have not been
seen (including currently active tasks), use the following:tasks=Task.objects.for_display()This will show tasks that have been seen in the last 30 seconds. To only show
tasks seen in the last 15 seconds use the following:tasks=Task.objects.for_display(seconds_since_seen=15)Management commandsTheclear_tasksmanagement command will delete allTaskobjects currently
in the database irrespective of status.python manage.py clear_tasksCompatibilityPython 3.6+Django 3.2+Only supports PostgreSQL becausedjango.contrib.postgres.fields.ArrayFieldis used. This could be looked at in future.VersioningThis project followssemantic versioning(SemVer).License and code of conductCheck the root of the repo for these files.