Package: ate-dispatcher
ate-dispatcherOverviewA pure-Python, thread-based, asynchronous dispatcher used to gather results
from distributed agents that react on a given event.The dispatcher reacts to an event tagged with a given topic, which is relayed
to a set ofProducerobjects (which registered the kind of
events they react to). Each result is then passed to a set ofResultListenerobjects, which will attend to the response depending if they were registered
to attend the given topic.The exposed API is thread-safe, asynchronous and lock-free, which makes it
suitable for tasks that are lightweight and quick.DependenciesThis package is pure-Python, and therefore it does not depend on any external
library, besidestyping-extensions, which is used to import the typing
classes that are not available in older Python 3 versions.InstallationYou can install this library by using conda or pip package managers,
as it follows:# Using condacondainstallate-dispatcher-cconda-forge# Using pippipinstallate-dispatcherLocal developmentIn order to install a local development version forate-dispatcher, it is
possible to invoke pip:pipinstall-e.Package usageate-dispatcherexposes two abstract interfaces (ProducerandResultListener)
as well as the main dispatcher (ATEDispatcher). The former classes are designed
to be inherited, while the last one is designed to be instantiated.Defining a producerAProducerobject is on charge of producing a response to an incoming input
message from the dispatcher that is tagged with a certaintopicthat the
object can handle. It must implementproduce_dispatcher_output.importtimefromtypingimportAnyfromate_dispatcherimportProducer# Defining a producerclassSpecificTopicProducer(Producer):def__init__(self,_id:int,timeout:int,*topics):super().__init__()self.id=_idself.timeout=timeout/1000self.topics=set(topics)defproduce_dispatcher_output(self,topic:str,*args,**kwargs)->Any:time.sleep(timeout)return{'id':self.id,'some_key':topic,'args':args,'kwargs':kwargs}Defining a result listenerAResultListenerobject will receive the responses emitted by theProducerobjects that reacted to a given topic that the listener also supports. This
is the endpoint for the dispatcher architecture, where all end messages will
arrive. TheResultListenersubclasses must implement theprocess_dispatcher_resultmethod.fromtypingimportAnyfromate_dispatcherimportResultListener# Defining a result listenerclassResultListenerExample(ResultListener):def__init__(self):super().__init__()self.responses={}defclear(self):self.responses={}defprocess_dispatcher_result(self,topic:str,response:Any):topic_responses=self.responses.get(topic,[])topic_responses.append(response)self.responses[topic]=topic_responsesCreating, using and destroying the dispatcherAfter defining theProducerandResultListenersubclasses, it is necessary
to instantiate and register theATEDispatcher instance. Since the interfaces
inherit fromthreading.Thread, it is necessary to keep track of their
lifetime via thestartandendmethods.importtime# Import the producer and result listener classesfromspecific_topic_producerimportSpecificTopicProducerfromresult_listener_exampleimportResultListenerExample# Import the dispatcherfromate_dispatcherimportATEDispatcher# Create the dispatcherdispatcher=ATEDispatcher()# Start the dispatcher, the lifetime is delegated to the end developer.dispatcher.start()# Define the producers and register them against the dispatcherproducer1=SpecificTopicProducer(0,200,'topic1','my_topic')producer1.start()producer2=SpecificTopicProducer(1,500,'topic1','topic2','my_topic')producer2.start()fortopicinproducer1.topics:dispatcher.register_result_producer(producer1,topic)fortopicinproducer2.topics:dispatcher.register_result_producer(producer2,topic)# Define the result listeners and register them against the dispatcherlistener1=ResultListenerExample()listener1.start()listener2=ResultListenerExample()listener2.start()fortopicin['topic1','topic2','my_topic']:# The first listener will receive all responses tagged with all topicsdispatcher.register_result_listener(listener1,topic)fortopicin['topic1','my_topic']:# This listener will attend to certain topics.dispatcher.register_result_listener(listener2,topic)Since the dispatcher architecture is completely asynchronous, a trigger message
may indicate a maximum timeout (in milliseconds) for all registered producers
on a given topic to emit their response. Any response received after the
specified timeout will be discarded. Also, the messages will be delivered to
the result listeners as they arrive.# Trigger a dispatcher request with a 4000ms timeout on the topic1dispatcher.send_request('topic1',3,4,5,ttl=4000,keyword='b')# Wait for responses to arrivetime.sleep(1.0)# Both listeners should have received the responses from both producers.expected_responses={'topic1':[{'id':0'some_key':'topic1','args':(3,4,5),'kwargs':{'keyword':'b'}},{'id':1'some_key':'topic1','args':(3,4,5),'kwargs':{'keyword':'b'}},]}assertlistener1.responses==expected_responsesassertlistener2.responses==expected_responses# Clear the listener responseslistener1.clear()listener2.clear()# Trigger a dispatcher request with a 300ms timeout limit on the topic my_topicdispatcher.send_request('my_topic',3,4,5,ttl=300,keyword='b')# Wait for responses to arrivetime.sleep(0.5)# Both listeners should have only received the response from producer1.expected_responses={'my_topic':[{'id':0'some_key':'my_topic','args':(3,4,5),'kwargs':{'keyword':'b'}}]}assertlistener1.responses==expected_responsesassertlistener2.responses==expected_responses# Clear the listener1 responseslistener1.clear()# Trigger a dispatcher request with a 1000ms timeout limit on the topic topic2dispatcher.send_request('topic2',3,4,5,ttl=1000,keyword='b')# Wait for responses to arrivetime.sleep(1.0)# Only the listener1 should have received the message produced by the producer2expected_responses={'topic2':[{'id':1'some_key':'topic2','args':(3,4,5),'kwargs':{'keyword':'b'}}]}assertlistener1.responses==expected_responsesFinally, each registeredProducerandResultListenerinstance can be
deregistered from an specific topic at any time. However, before stopping eitherProducerorResultListenerinstances, each registered topic must be
deregistered.# Deregister the listener2 and the producer1 from certain topicsdispatcher.deregister_result_listener(listener2,'my_topic')dispatcher.deregister_result_producer(producer1,'topic1')# Stopping the producer and result listener instancesfortopicinproducer1.topics:dispatcher.deregister_result_producer(producer1,topic)fortopicinproducer2.topics:dispatcher.deregister_result_producer(producer2,topic)producer1.stop()producer2.stop()fortopicin['topic1','topic2','my_topic']:dispatcher.deregister_result_listener(listener1,topic)fortopicin['topic1','my_topic']:dispatcher.deregister_result_listener(listener2,topic)# Stop the dispatcherdispatcher.stop()Running testsWe use pytest to run tests as it follows:pytest-x-vate_dispatcher/testsChangelogVisit ourCHANGELOGfile to learn more about our new features and improvements.Contribution guidelinesWe follow PEP8 and PEP257. We use MyPy type annotations for all functions and classes declared on this package. Feel free to send a PR or create an issue if you have any problem/question.