Package: darq
[ ~ Dependencies scanned by PyUp.io ~ ]Async task manager with Celery-like features. Fork ofarq.FeaturesCelery-like@taskdecorator, adds.delay()to enqueue jobPropermypytype checking: all arguments passed to.delay()will be checked against the original function signatureGraceful shutdown: waits until running tasks are finishedInstallationDarq usesaioredis1.x as Redis client. Unfortunately, this library has been abandoned, and does not support Python 3.11. I made a fork with compatability fixes:evo-aioredis(https://github.com/evo-company/aioredis-py).Because of this,aioredisis not currently added as Darq dependency, and you must install it yourself:For Python<3.11 you can use:pipinstallaioredis<2.0.0For Python 3.11 (and older versions too) you can use fork:pipinstallevo-aioredis<2.0.0Quick start# some_project/darq_app.pyimportasyncioimportdarqdarq=darq.Darq(redis_settings=darq.RedisSettings(host='redis'))@darq.taskasyncdefadd_to_42(a:int)->int:return42+aasyncdefmain():# Before adding tasks to queue we should connect darq instance to redisawaitdarq.connect()# Direct call job as function:result=awaitadd_to_42(5)# result == 47# Celery-like add task to queue:awaitadd_to_42.delay(a=5)awaitdarq.disconnect()if__name__=='__main__':asyncio.run(main())And start worker:python3-mdarq.cli-Asome_project.darq_app.darqworkerWorker output:15:24:42:Startingworkerfor1functions:some_project.darq_app.add_to_4215:24:42:redis_version=5.0.7mem_usage=834.87Kclients_connected=1db_keys=215:25:08:0.22s→1315f27608e9408392bf5d3310bca38c:darq_app.add_to_42(a=5)15:25:08:0.00s←1315f27608e9408392bf5d3310bca38c:darq_app.add_to_42●47Changelog0.11.2 (2024-01-18)Fix reconnecting after disconnect (thanks to@heimtathurs)0.11.1 (2022-11-30)Add Python 3.11 support (withevo-aioredisdependency instead ofaioredis)RemovepydanticdependencyRemoveaioredisfrom dependencies to allow choose betweenaioredisandevo-aioredis- fork with Python 3.11 compatability0.11.0 (2022-08-03)Added ability to optionally passctxto the task, like this:@task(with_ctx=True)deffoobar(ctx):log.info('Foobar try%s',ctx['job_try'])ctxcontains:job_id,job_try,enqueue_time,score,metadata+ all worker’sctx(including custom context which can be passed viaon_startup). Thanks to@kindermax(https://github.com/seedofjoy/darq/pull/426) !0.10.2 (2022-02-03)Add proper typing for functions wrapped with the @task decorator. Mypy will now check that parameters are passed correctly when callingfunc()andfunc.delay()0.10.1 (2021-07-29)Addsentinel_timeout(defaults to 0.2) param toRedisSettings0.10.0 (2021-07-09)Breaking change: Renamedarq.worker.Functiontodarq.worker.TaskMadejobtotasknaming migrationAdd max_jobs parameter to CLI (thanks to@antonmyronyuk)Fixed bug withexpiresargument:default_job_expirescould not be replaced withNonein@taskor.apply_async0.9.0 (2020-06-24)Breaking change: Addscheduler_ctxparam toon_scheduler_startupandon_scheduler_shutdownto share data between this callbacks. It already hasctx['redis']- instance ofArqRedis0.8.0 (2020-06-22)Breaking change: Changed CLI command format. Before:darq some_project.darq_app.darq. Now:darq-Asome_project.darq_app.darq workerBreaking change: Scheduler (cron jobs) now run’s seperate from worker (seedarq schedulercommand)Breaking change: Changed some function signatures (rename arguments)Breaking change: Removeredis_poolparam fromDarqappAddon_scheduler_startupandon_scheduler_shutdowncallbacks0.7.2 (2020-06-18)Fix some types (cron, OnJobPrepublishType)on_job_prerunnow runs before “task started” log andon_job_postrunnow runs after “task finished” log0.7.1 (2020-05-25).apply_async: Makeargsandkwargsarguments optional0.7.0 (2020-05-25)Forkarqto project and merge it withdarq(It was easier to rewritearqthan to write a wrapper)Breaking change: Remove “magic” params from.delay. For enqueue job with special params added.apply_async.Addwatch-mode to CLI.Fix: Now worker will not run cronjob if it’s functions queue not match with worker’s0.6.0 (2020-03-08)Breaking change: ChangedDarqconstructor from singleconfigparam to separate params.arq_function.coroutinenow has.delaymethod.0.5.0 (2020-03-03)Addon_job_prepublish(metadata, arq_function, args, kwargs)callback.metadatais mutable dict, which will be available atctx['metadata'].0.4.0 (2020-03-03)Adddefault_job_expiresparam to Darq (if the job still hasn’t started after this duration, do not run it). Default - 1 dayAddexpiresparam to@task(if set - overwritesdefault_job_expires)0.3.1 (2020-03-02)Rewrite warm shutdown: now during warm shutdown cron is disabled, on second signal the warm shutdown will be canceled0.3.0 (2020-02-27)Breaking change:on_job_prerunandon_job_postrunnow acceptsarq.worker.Functioninstead of the original function (it can still be accessed atarq_function.coroutine)0.2.1 (2020-02-26)Fixadd_cron_jobsmethod. Tests added.0.2.0 (2020-02-26)Addon_job_prerun(ctx, function, args, kwargs)andon_job_postrun(ctx, function, args, kwargs, result)callbacks.0.1.0 (2020-02-26)Breaking change: Jobs no longer explicitly getJobCtxas the first argument, as in 99.9% cases it doesn’t need it. In future release will be possible to optionally passJobCtxin some way.Breaking change: All cron jobs should be wrapped in@taskdecoratorDirectly passfunctionstoarq.Worker, not names.0.0.3 (2020-02-25).delay()now returnsarq_redis.enqueue_jobresult (Optional[Job])Addpy.typedfileFixedadd_cron_jobstyping0.0.2 (2020-02-24)Addadd_cron_jobsmethod0.0.1 (2020-02-21)First release