Package: celery-abc
celery-abcIt's time for some dynamic meta-magic!What is it?Set of tools to create microservice app withceleryas message brocker.When to use it?Say, you want to write some microservices with python. And you want them to work seamlessly with each other - call methods of one service from another like it is a single app.
You may useceleryjust for that. But you actually need to register tasks in the same app. Isn't that a little strange - to keep the code base of ALL the services in each and every of them?
Wouldn't it be more convinient, to share only interfaces? That way each developer could track what they can use of other services, but without actully storing their code.How to use it?We'll need at least two services and a shared directory. In shared directory we'll create an abstract classInterface. Let's call one serviceCallerand the other -Worker.CallerhasInterfaceimported, but knows nothing about the implementation.WorkerhasInterfaceimplemented.Create common directory, share it with all your services. Chances are - you'll need one anyway.In the shared folder create python module. Define your interfaces there. For now,celery-abcrequires forInterfaceto be abstract.fromabcimportABC,abstractmethodclassInterface(ABC):@abstractmethoddefdo_some_stuff(self,arg):pass@abstractmethoddefdo_more_stuff(self,arg):passIn yourWorkerservice importInterfacefrom shared module and implement it. Metaclass for your implementation should beWorkerMetaBasefromcelery-abc. You don't need to register it's methods, just instantiate your class withcelery. Run yourworkeras a celery worker.fromceleryimportCeleryfromcelery_abcimportWorkerMetaBasefromsharedimportInterfaceclassWorker(Interface,metaclass=WorkerMetaBase):defdo_some_stuff(self,arg):returnself.do_more_stuff(arg)defdo_more_stuff(self,arg):returnf"Doing stuff{arg}"celery_app=Celery(...)Worker(celery)InCallerservice importCallerMetaBase, create a new class, that inherits fromInterfaceand haveCallerMetaBaseas a metaclass. Instantiate it with yourceleryapp. Unlikeworker,callermay run just as regular python app. Now, you can call methods, that are implemented in theWorker=)classCaller(Interface,metaclass=CallerMetaBase):passcelery_app=Celery(...)caller=Caller(celery)result=caller.do_some_stuff('Hello!')Here you go, now you can seamlessly call methods ofWorkerfromCaller. As you can see, you can callWorker's methods from each other. But you can not keep any instnce variables inWorker'sself. In the end, microservices are better when they are stateless.When not to use it?If you want to create statefull microservices - you still can usecelery-abc, if only for comunication. But there may be other solutions.If you want to create complex system of classes. OOP is cool, but multiple and chaind inheritance are not supported at the moment.Method signature can't have*argsand**kwargsin it. It is totally fine to pass argumetns this way, but capturing them is not supported yet.Interesting stuffAbstract methodsMetaclassesObject-methodDescriptorsinspectworker's selfExampleProject includes small example module. To run it, create insideconfig.envfile with redis creds and rundocker-compose -f "tests\docker-compose.yaml" up -d --buildRABBITMQ_DEFAULT_USER=...
RABBITMQ_DEFAULT_PASS=...
RABBITMQ_HOST=...
RABBITMQ_PORT=...In the future:Support inheritanceSupport capturing arguments