Package: celery-crossover
Celery Crossover (Alpha)AboutCelery crossover aims to make it easier to execute celery tasks from a diffent source code base in the most simple and yet reliable way.Quick Examples1) Simple Alice task execution triggered by BobLets suppose Bob is a service that needs Alice to execute the taskplusdefined on Alice with the following code:Alice:celery_config.py# -*- coding: utf-8 -*-fromkombuimportExchange,QueueBROKER_URL='redis://localhost:6379/1'CELERY_QUEUES=[Queue('alice_queue',Exchange('alice_queue'),routing_key='alice_queue')]alice.py# -*- encoding: utf-8 -*-importcrossoverfromceleryimportCeleryapp=Celery('tasks')app.config_from_object('celery_config')# The line bellow will make Alice's Tasks usable by other services.crossover.register_router(app)@app.task(name='plus',queue='alice_queue')defplus(x,y):_add=x+yreturn_addBob:All that Bob need to do is:exec_task_on_alice.py# -*- encoding: utf-8 -*-fromcrossoverimportClientalice_broker="redis://localhost:6379/1"alice_client=Client(broker=alice_broker)alice_client.plus(x=340,y=210)2) Alice task execution triggered by Bob with callback (Auto Callback)In the same scenario described above, lets suppose Bob need to be notified (have a task executed) after Alice is done with theplustask.
For this case, all we have to do is decorate the Alice's taskpluswith@crossover.callback(auto_callback=True)to have its returned value sent back to Bob. Also, Bob have to define and send to Alice a task to be called by Alice's callback.
That way, Alice and Bob could would be:Alice:alice_celery_config.py# -*- coding: utf-8 -*-fromkombuimportExchange,QueueBROKER_URL='redis://localhost:6379/1'CELERY_QUEUES=[Queue('alice_queue',Exchange('alice_queue'),routing_key='alice_queue')]alice.py# -*- encoding: utf-8 -*-importcrossoverfromceleryimportCeleryapp=Celery('tasks')app.config_from_object('alice_celery_config')# The line bellow will make Alice's Tasks usable by other services.crossover.register_router(app)@app.task(name='plus',queue='alice_queue')@crossover.callback(auto_callback=True)defplus(x,y):_add=x+yreturn_addBob:bob_celery_conf.py# -*- coding: utf-8 -*-fromkombuimportExchange,QueueBROKER_URL='redis://localhost:6379/1'CELERY_QUEUES=[Queue('bob_queue',Exchange('bob_queue'),routing_key='bob_queue')]bob.py# -*- encoding: utf-8 -*-importcrossoverfromceleryimportCeleryfromcelery.utils.logimportget_task_loggerlogger=get_task_logger(__name__)app=Celery('tasks')app.config_from_object('bob_celery_conf')crossover.register_router(app)@app.task(name='plus_callback',queue='bob_queue')defplus_callback(result):logger.info('Got Addition callback ={0}'.format(result))exec_task_on_alice.py# -*- encoding: utf-8 -*-fromcrossoverimportClientfrombobimportplus_callbackalice_broker="redis://localhost:6379/1"alice_client=Client(broker=alice_broker)alice_client.plus(x=340,y=210,callback=plus_callback)3) Alice task execution triggered by Bob with callback (No Auto Callback)In this case, everything is the same as 2) from Bob's perspective but lets suppose Alice's task cant calculate or
determine a response right away so it needs to pass (or persist) the callback metadata for further execution.
It can be done by using de decorator@crossover.callbackwithbind_callback_meta=Truewhich will give
to the task function the callback metadata as its first parameters. Following an example of its usage:alice.py# -*- encoding: utf-8 -*-...@app.task(name='plus',queue='alice_queue')@crossover.callback(bind_callback_meta=True)defplus(callback_meta,x,y):logger.info('Execution actual addition task.')calculate_addition.delay(callback_meta,x,y):@app.task(name='calculate_addition',queue='project_1')defcalculate_addition(callback_meta,x,y):_add=x+ylogger.info('Addition{0}+{1}={2}'.format(x,y,_add))crossover.CallBack(callback_meta)(result=_add)