Saravanan666/pypi-org huggingface dataset:
Package: celery-signal-receivers
Celery Signal ReceiversMake Django signals asynchronous with Celery tasks. This package allows you to convert and write signal receivers to run in the background as Celery tasks.InstallationThe package isavailable on PyPI:pipinstallcelery-signal-receiversConfigurationIn order to use this package you need to set the path to the celery app object in Django settings:EVENT_SIGNALS_CELERY_APP='myproject.celery.app'UsageThe package is usingDjango Unified Signalsfor passing the message object from sender to receiver. The message object is always expected to be passed when sending the signal. That way receiver knows what type of message will be received. This package automates the process of checking if the send message is following the contract.Let's start by defining the message structure. It can be any class you want.importdataclasses@dataclasses.dataclassclassProfileMessage:id:intname:strNow that we have the message structure defined, we can create the signal. We will useUnifiedSignalclass for that:fromunified_signalsimportUnifiedSignalprofile_updated_signal=UnifiedSignal(ProfileMessage)See thedocumentation of Django Unified Signalsto learn more about sending the signal
with standardized message.Let's now write receiver for the signal. We will use thecelery_signal_receivers.receiverdecorator to convert the receiver to Celery task.fromcelery_signalsimportreceiver_task@receiver_task(profile_updated_signal)defhandle_profile_updated(sender,message:ProfileMessage,**kwargs):print(message.id)print(message.name)...The above task will be executed by celery worker for thehandle_profile_updatedtask. This function works as any other celery task, so you can route it to specific queue, set the priority, etc.app.conf.task_routes={'handle_profile_updated':{'queue':'profile-updated-queue'},}OptionsYou can also pass the celery options to the task using the param in the decorator:@receiver_task(profile_updated_signal,celery_task_options={'queue':'profile-updated-queue'})deffoo(sender,message:ProfileMessage,**kwargs):...The decorator also accepts all other keyword arguments as regulardjango.dispatch.receiverdecorator (ie. same asSignal.connect. For example you can set thedispatch_uidto avoid registering the same receiver multiple times.@receiver_task(profile_updated_signal,dispatch_uid='profile_updated')deffoo(sender,message:ProfileMessage,**kwargs):...LimitationsFor now this package does not support multiple signals passed to the@receiver_taskdecorator.
You should create separate receivers for each signal.
This may be added in the future.
Package: celery-skinos
Celery SkinosCustom consumer for celery integration.Usagefromskinos.custom_consumerimportCustomConsumerDefine a new exchangedefined a new exchange with a name and a binding key (always a topic).
The exchange name must be unique.# add_exchange(str, str) -> ExchangeCustomConsumer.add_exchange('test',"test.*.*")Define a new taskDefine a new message handlerdecoration take 3 arguments:exchange name (must be defined)queue name (must be defined)queue binding keyFunction but have this prototype:(str, Message) -> Anybodyis the payloadmsgis the message object (kombu.transport.myamqp.Message)# consumer(str, str, str) -> Callable[[str, Message], Any]@CustomConsumer.consumer('test','test.test','test.test.*')defcoucou(body,msg):print('payload content :{}'.format(body))print('message object content :{}'.format(msg))Build consumers for Celery integrationBuild consumers itself. all previous methods are just a pre-configuration for this build.
It take one argument, which is the Celery app.# build(Celery) -> NoneCustomConsumer.build(app)Add Sentry handlerYou must init Sentry normally for a Celery project.
Then Skinos is able to catch exception and send it sentry.set sentry to True and set raise to False (i.e: if error occur, error is not re-raise, but ignored)
if you don't use it, default values are False and False# with_sentry(bool, bool) -> Tuple(bool, bool)
CustomConsumer.with_sentry(False, False)Run celeryRun celery normally
Package: celery-smartbase
Celery SmartBasecelery-smartbaseis an extension and improvement of the popularceleryPyPI package. It provides features to avoid duplicates for tasks that are pending or running and to see the "pending" tasks as well in the "jobs" Django model when you integrate Celery with Django. Although this package is improved to work with Django, users can modify the features slightly to make it work with any Python framework, not just Django.Table of ContentsCelery SmartBaseFeaturesSmartBase ClassRequirementsInstallationUsageContributingLicenseAcknowledgmentsFeaturesAvoid Duplicate Tasks: Prevents the creation of duplicate tasks that are either pending or running.View Pending Tasks: Allows you to see the "pending" tasks in the "jobs" Django model when you integrate Celery with Django.These features are developed inside an extension of theBaseTaskclass of the existing Celery Python package. The extension class is calledSmartBase.SmartBase ClassTheSmartBaseclass is an extension of theBaseTaskclass from the Celery package. Here is the code for theSmartBaseclass:classSmartBase(BaseTask):...[Include the full code of the SmartBase class here]RequirementsDjangodjangorestframeworkdjango-model-utilsdjango-celery-resultsdjango-celery-beatkombuceleryInstallationTo install thecelery-smartbasepackage, run the following command:pip install celery-smartbaseUsageTo use thecelery-smartbasepackage, you need to follow these steps:Install thecelery-smartbasepackage using pip.In your Django project, create atasks.pyfile inside your app directory.In thetasks.pyfile, import theSmartBaseclass from thecelery_smartbasepackage and define your tasks using theSmartBaseclass.Here is an example of atasks.pyfile:fromceleryimportshared_taskfromcelery_smartbaseimportSmartBase@shared_task(base=SmartBase,name='add')defadd(x,y):returnx+yIn this example, theaddtask is defined using the@shared_taskdecorator withSmartBaseas the base class.Start the Celery worker to process the tasks.celery -A myproject worker --loglevel=infoReplacemyprojectwith the name of your Django project.ContributingIf you would like to contribute to thecelery-smartbasepackage, please...[Include information on how others can contribute to your package]LicenseThecelery-smartbasepackage is licensed under the...[Include information about the license of your package]Acknowledgments[Include any acknowledgments or credits you would like to include]
Package: celery-sqlalchemy
celery-sqlalchemySQLAlchemy model serialization for celery. Supports SQLAlchemy 1.4 & 2.0.Installpipinstallcelery-sqlalchemyHow does it work?Any model or list of models passed to a celery task as a direct argument will be
serialized/deserialized bycelery-sqlalchemy.Behind the scenesTheorjsonlibrary is used behind the scenes to handle
serialization of commonly used Python types.UsageBy defaultcelery-sqlalchemywill configure Celery to use thejson+sqlalchemycontent type for all tasks and no further configuration is needed.fromcelery_sqlalchemy.celeryimportinitializeasinitialize_celeryfromcelery_sqlalchemy.jsonimportJsonSerializer# setup celerycelery=...# initialize celery-sqlalchemyinitialize_celery(celery,JsonSerializer())# dispatch a taskauthor="Alan Watts"title="Become What You Are"task.delay(Model(author=author,title=title))Using the json+sqlalchemy content type in combination with other content typesThe first way of doing this is by specifying thejson+sqlalchemycontent type as
the default task serializer.fromcelery_sqlalchemy.celeryimportinitializeasinitialize_celeryfromcelery_sqlalchemy.jsonimportJsonSerializer# setup celerycelery=...# initialize celery-sqlalchemy without the `apply_serializer` settinginitialize_celery(celery,JsonSerializer(),apply_serializer=False)# combine the json+sqlalchemy content type with your other content typescelery.conf.accept_content=["json+sqlalchemy","your content type"]celery.conf.result_accept_content=["json+sqlalchemy","your content type"]# set the default task serializercelery.conf.task_serializer="json+sqlalchemy"# dispatch a model taskauthor="Alan Watts"title="Become What You Are"task.delay(Model(author=author,title=title))# dispatch a task using another content typetask.apply_async((author,title),serializer="your content type")The other way of doing this is by not specifying the default task serializer, and
instead usingapply_async()to dispatch all tasks.fromcelery_sqlalchemy.celeryimportinitializeasinitialize_celeryfromcelery_sqlalchemy.jsonimportJsonSerializer# setup celerycelery=...# initialize celery-sqlalchemy without the `apply_serializer` settinginitialize_celery(celery,JsonSerializer(),apply_serializer=False)# combine the json+sqlalchemy content type with your other content typescelery.conf.accept_content=["json+sqlalchemy","your content type"]celery.conf.result_accept_content=["json+sqlalchemy","your content type"]# dispatch a model taskauthor="Alan Watts"title="Become What You Are"task.apply_async((Model(author=author,title=title),),serializer="json+sqlalchemy")# dispatch a task using another content typetask.apply_async((author,title),serializer="your content type")Changelog0.1.6Extract celery support into its own moduleRemovecelery_sqlalchemy.initialize()0.1.5Bump sqlalchemy dependency0.1.4Bug fix: schema model may already be converted to model instance when chaining0.1.3Check list type when converting json back to argument0.1.2Add py.typed for mypy0.1.1Add support for SQLAlchemy 1.40.1.0Initial version with support for common Python types
Package: celery-sql-beat-reloader
IN DEVELOPMENTcelery-sql-beat-reloaderA celery schedule that will sync with a db for the latest schedule of tasks.FeaturesPulls schedule from a database instead of code.Every schedule is timezone aware.Getting StartedPrerequisitesPython 3celery >= 5.2sqlalchemyInstallingInstall from PyPi:$ pip install celery-sql-beat-reloaderInstall from source by cloning this repository:$ git clone git@github.com:sir-wiggles/celery-sql-beat-reloader.git
$ cd celery-sql-beat-reloader
$ python setup.py installUsagecelery_sql_beat_reloaderuses sqlalchemy to manage database connections so you should in theory be able to use this on any flavor of SQL database.SchemaCREATETABLEcelery_beat_schedule(idSERIALPRIMARYKEY,created_dateTIMESTAMPTZDEFAULTNOW(),updated_dateTIMESTAMPTZDEFAULTNOW(),last_scheduled_dateTIMESTAMPTZ,activeBOOLEANDEFAULTTRUE,nameTEXTNOTNULLUNIQUE,taskTEXTNOTNULL,argsJSONBDEFAULT'[]',kwargsJSONBDEFAULT'{}',typeTEXTNOTNULLDEFAULT'cron',secondsINTEGERDEFAULT60,minuteTEXTDEFAULT'*',hourTEXTDEFAULT'*',day_of_weekTEXTDEFAULT'*',day_of_monthTEXTDEFAULT'*',month_of_yearTEXTDEFAULT'*',timezoneTEXTDEFAULT'America/Los_Angeles');NOTE: Table is not auto generated.Column Descriptionsactive- indicates if the schedule is active or not.active=false, the schedule will be ignored at startup or removed at the next sync interval.active=true, the schedule will be include at startup or added at the next sync interval.name- is a unique name for the schedule.task- is the import path of the task e.g.test_app.beat1,test_app.beat2.type- is the type of the schedule and supports two types:cronandperiodic.cron- is a crontab with timezone awareness.periodic- is a schedule with a frequency of seconds from thesecondscolumn.last_scheduled_date- is when the beat was last sent to be processed.NOTE:last_scheduled_datedoes not mean the task was successful or even if the task was processed.  It just means the scheduler sent a request at that time.ConfigIn your celeryapp.confyou'll need to setbeat_reloader_dburlto the url of your database.# test_app.pyimportceleryapp=celery.Celery("test",broker_url="redis://localhost:6379",timezone="UTC",beat_reloader_dburl="postgresql+psycopg2://user:pass@127.0.0.1:5432/db",)@app.taskdefbeat1():print("beat1")@app.taskdefbeat2():print("beat2")Runningcelery--apptest_appbeat-Scelery_sql_beat_reloader:Reloader--max-interval=300--loglevel=INFOSpecifying the--max-intervalargument will allow you to control how frequently the schedule syncs with the databaseAcknowledgmentscelery-sqlalchemy-schedulerdjango-celery-beatcelerybeatrediscelery
Package: celeryt
CeleryTWhat is CeleryT?CeleryTis a project builder for Celery python project.How to download it?python3-mpipinstallcelerytWhat can I do with celeryT?You can create a project skeleton for Celery.You can create a project skeleton for Celery with skinos extension.It init automatically a git repository and can init git flow.You can run celery command for test purpose (basic command)How to run it?new celery project:celeryt new <project_name>new celery project with git flow:celeryt new <project_name> --flownew celery project with skinos:celeryt new <project_name> --skinosrun celery command for dev or test:celeryt runhelp:celeryt helpWhat's inside the project skeletonProject structure<project_name>/
| <project_name>/
|   | __init__.py 
|   | celery.py 
|   | get_settings.py 
|   \ tasks.py
| requirements/
|   | common.txt
|   \ dev.txt
| settings/
|   | base.py
|   | test.py
|   | dev.py
|   | pprd.py
|   \ prod.py
| .coveragerc
| .gitignore
| README.md
| requirements.txt
\ tox.iniproject technologiesApplicationPythonCelerycelery-skinossentryToolstoxcoveragegitgit flowCreditCreated by:Diggersheep (Alexandre COMBEAU)ForUniversité de Strasbourg (Unistra)
Package: celery_tasks_ctl
Overview========Command line utility to list and revoke `Celery <http://www.celeryproject.org/>`_ tasks* Facilitates listing tasks that have not yet been reserved by a Celery instance and are still on the RabbitMQ queue.* Revoke tasks where their ids match your regexHow it works------------It uses the Celery inspect object to get a list of statistics for the tasks currently running in, or reserved by, the cluster. Queued tasks are tasks that the cluster does not yet know about and are currently in the RabbitMQ queue. To get a list of these, the RabbitMQ API is used.Install=======::python setup.py installor::pip install celery-tasks-ctlRun===::celery_tasks_ctl --helpLicense_______Licensed under `MPL 1.1 <http://www.mozilla.org/MPL/1.1/>`_.
Package: celery-task-tigger
celery-task-tigger====[![release](https://img.shields.io/badge/release-0.3-blue.svg)]()[![license](https://img.shields.io/badge/license-MIT-blue.svg)]()[![celery](https://img.shields.io/badge/celery-3%7C4-brightgreen.svg)]()A controllable timing task widgets with Celery## AboutAs is known to all, Celery have a already provides periodic task and it's very perfit. But, Assume this case: After my task was called, I hope it's task can  frequency of execution, and when celery task was started.The above case, Periodic task is hard to practice, Becacuse it's depend on celery beat. ``celery-task-tigger`` do it over simple packaging or implement for solution to do it.## Installation~~~pythonpip install celery-task-tigger~~~## UseageAssume you have aleady install celery and can do it.### BasesOption `max_times` is must be appoint.~~~pythonfrom celery_tasktigger.decorator import tigger_task@app.task(bind=True)@tigger_task(max_times='forever')    # forever is expressed unlimited timedef add(self, x, y):return x + y~~~### max_timesOption `max_times`: The maximum number of execute the task times.Type： ***int***> Note: The value ***'forever'*** is expressed unlimited time.Example:~~~python@app.task(bind=True)@tigger_task(max_times=3)    # after execute 3 times, raise an exceptiondef add(self, x, y):return x + y~~~### countdownOption `countdown`: You can also provide the countdown argument to execute.Type:  ***int***Default: 1 (seconds)Example:~~~python@app.task(bind=True)@tigger_task(max_times='forever', countdown=3)    # each execute in 3 secondsdef add(self, x, y):return x + y~~~## How To Calling Task~~~~python>> from example import add>> add.apply_async((1,2))~~~~you can also delayed execute task, as follow：~~~~python>> from example import add>> add.apply_async((1,2), countdown=4)   # after 4 seconds, begin start task~~~~> About Celery Task, Please see below for details：> [Celery Calling-Tasks Document](http://docs.jinkan.org/docs/celery/userguide/calling.html)## How To Stopif you appoint `max_times='forever'` or provides the bigger values of max_times, you must stop it in programe.~~~~python>> result = add.apply_async((1,2))>> result.revoke()or>> from mycelery import app>> app.control.revoke('task_id')~~~~> See below for details：> [Celery Document——FAQ](http://docs.jinkan.org/docs/celery/faq.html#can-i-cancel-the-execution-of-a-task)## Some screenshots![](http://i1.piimg.com/536217/1ae9af3a274de4c7.gif)## Features- 100％ full compatible with Celery- the frequency of execution for task- ...and many other stuff (o,0)## Author- Boyle Gu
Package: celerytest
celerytest - Integration testing with CeleryWriting (integration) tests that depend on Celery tasks is problematic.
When you manually run a Celery worker together with your tests, it runs
in a separate process and there’s no clean way to address objects
targeted by Celery from your tests. When you use a separate test
database (as with Django for example), you’ll have to duplicate
configuration code so your Celery worker accesses the same database.celerytest provides the ability to run a Celery worker in the background
from your tests. It also allows your tests to monitor the worker and
pause until Celery tasks are completed.Using celerytestTo start a Celery worker in a separate thread, use:app=Celery()# your Celery appworker=start_celery_worker(app)# configure the app for our celery workerTo wait for the worker to finish executing tasks, use:result=some_celery_task.delay()worker.idle.wait()# optionally specify time-outDjangoTo use this with your django app through django-celery, get your app as
such:fromdjcelery.appimportappworker=start_celery_worker(app)TestCaseIf you want to use this in a unittest TestCase, you can use
CeleryTestCaseMixin. If you’re writing unit tests that depend on a
celery worker, though, you’re doing it wrong. For unit tests, you’ll
want to mock your Celery methods and test them separately. You could use
CeleryTestCaseMixin to write integration tests with Celery tasks,
though.fromunittestimportTestCasefromcelerytest.testcaseimportCeleryTestCaseMixin,setup_celery_workerimporttimeapp=Celery()setup_celery_worker(app)# need to setup worker outsideclassSomeTestCase(CeleryTestCaseMixin,TestCase):celery_app=appcelery_concurrency=4deftest_something(self):result=multiply.delay(2,3)self.worker.idle.wait()self.assertEqual(result.get(),6)LettuceTo automatically launch a worker in the background while running a
Lettuce integration test suite, add toterrain.py:# terrain.pyfromlettuceimport*fromcelerytestimportstart_celery_workerfromappimportapp@before.harvestdefinitial_setup(server):# memory transport may not work hereworld.celery=start_celery_worker(app,config="amqp")@after.harvestdefcleanup(server):world.celery.stop()@after.each_stepdefafter_step(step):# make sure we've received any scheduled tasksworld.celery.active.wait(.05)# allow tasks to completeworld.celery.idle.wait(5)InstallationInstall the latest version ofcelerytestfrom PyPI:$ pip install celerytestOr, clone the latest version ofcelerytestfrom GitHub and run
setup:$ git clone git://github.com/RentMethod/celerytest.git
$ cd celerytest
$ ./setup.py install # as root
Package: celery-testutils
Fork of celerytest with minor bug fixescelerytest - Integration testing with CeleryWriting (integration) tests that depend on Celery tasks is problematic.
When you manually run a Celery worker together with your tests, it runs
in a separate process and there’s no clean way to address objects
targeted by Celery from your tests. When you use a separate test
database (as with Django for example), you’ll have to duplicate
configuration code so your Celery worker accesses the same database.celerytest provides the ability to run a Celery worker in the background
from your tests. It also allows your tests to monitor the worker and
pause until Celery tasks are completed.Using celerytestTo start a Celery worker in a separate thread, use:app=Celery()# your Celery appworker=start_celery_worker(app)# configure the app for our celery workerTo wait for the worker to finish executing tasks, use:result=some_celery_task.delay()worker.idle.wait()# optionally specify time-outDjangoTo use this with your django app through django-celery, get your app as
such:fromdjcelery.appimportappworker=start_celery_worker(app)TestCaseIf you want to use this in a unittest TestCase, you can use
CeleryTestCaseMixin. If you’re writing unit tests that depend on a
celery worker, though, you’re doing it wrong. For unit tests, you’ll
want to mock your Celery methods and test them separately. You could use
CeleryTestCaseMixin to write integration tests with Celery tasks,
though.fromunittestimportTestCasefromcelerytest.testcaseimportCeleryTestCaseMixin,setup_celery_workerimporttimeapp=Celery()setup_celery_worker(app)# need to setup worker outsideclassSomeTestCase(CeleryTestCaseMixin,TestCase):celery_app=appcelery_concurrency=4deftest_something(self):result=multiply.delay(2,3)self.worker.idle.wait()self.assertEqual(result.get(),6)LettuceTo automatically launch a worker in the background while running a
Lettuce integration test suite, add toterrain.py:# my_celery_app.pyapp=Celery('my_celery_app',broker='amqp://')# terrain.pyfromlettuceimport*fromcelerytestimportstart_celery_worker# replace this with an import of your actual appfrommy_celery_appimportapp@before.harvestdefinitial_setup(server):# memory transport may not work hereworld.celery=start_celery_worker(app,config="amqp")@after.harvestdefcleanup(server):world.celery.stop()@after.each_stepdefafter_step(step):# make sure we've received any scheduled tasksworld.celery.active.wait(.05)# allow tasks to completeworld.celery.idle.wait(5)InstallationInstall the latest version ofcelerytestfrom PyPI:$ pip install celerytestOr, clone the latest version ofcelerytestfrom GitHub and run
setup:$ git clone git://github.com/RentMethod/celerytest.git
$ cd celerytest
$ ./setup.py install # as root
Package: celerytimer
基于celery实现的一个定时器工作流程celerytimer隐藏了其中大部分内容，让您专注于手头的任务celerytimer是基于celery实现的定时器，需要配合celery使用，使用celerytimer访问受保护的资源非常简单fromceleryimportshared_taskfromcelerytimer.celerytimerimportTimingTasksobj=TimingTasks("127.0.0.1",3306,"root","123456",'runtest')# 数据库的相关配置@obj.executed_task("push_task","推送测试","flag",["account_id"])@shared_taskdefpush_data(executed_data,a):print("值一：",executed_data)print("值三：",a)print("测试成功")executed_data={1602256560:[{"account_id":83022257,"channel_id":55824},]}push_data(executed_data,"a")功能可动态设置定时任务：限制在未来一小时内(celery中存在定时任务超过一小时未执行会重复执行)过滤掉已执行的任务注意点该定时器可以处理未来一小时内的任务，超出部分会抛弃，建议：脚本每10钟执行一次celerytimer装饰器必须在@shared_task上方安装要安装 celerytimer，你可以使用pip$ pip install celerytimer历史V1.1 （2020年10月09日）基于celery的定时器
Package: celery-tracker
Celery TrackerReceive/Sending event tracking data for theCelery.ZabbixPluginSendReceiveFluentPluginSendReceiveReceivePluginJSON response, e.g. athttp://127.0.0.1:27015CustomizeCustomize Documentation - celery-tracker.rtfd.orgSetupPyPI$ pip install celery-trackerGitHubgithub.com/ikeikeikeike/celery-trackerExamplesFor more information, please see theExampleLicenseMIT License
Package: celery_tryton
Celery integration with Tryton.
It defines an abstract base Task class that runs inside a Tryton transaction.
If the task is called from within a Tryton transaction, the context of this one
will be automatically restored when the worker will execute it.Getting StartedHere is a simple application:from celery import Celery
from celery_tryton import TrytonTask, celery_app
from trytond.pool import Pool

celery_app.conf.TRYTON_DATABASE = 'test'

@celery.task(base=TrytonTask)
def hello(user_id):
    User = Pool().get('res.user')
    user = User(user_id)
    return 'hello world, %s' % user.nameConfigurationAll standard celery configuration options will work but in additionTRYTON_DATABASEmust be set to the default Tryton database name and
optionallyTRYTON_CONFIGto load a specific configuration file.Thecelery_appwill load the configuration from theconfigvalue from the
section[celery]defined in the trytond configuration when used from a Tryton
module.ExampleStart a worker:$ celery worker --app=examples.purchase_requestsCallgenerate_alltask:$ celery call examples.purchase_requests.generate_allTo report issues please visit thecelery_tryton bugtracker.