Package: aiopubsub
Simple publish-subscribe pattern for asyncio applications.WhyWhen building big applications, separation of concerns is a great way to keep things manageable.
In messaging systems, the publish-subscribe pattern is often used to decouple data producers and data
consumers. We went a step ahead and designed even the internals of our applications around this pattern.We explain our thinking and the workings ofaiopubsubin detail in our articleDesign your app using the pub-sub pattern with aiopubsub.
We recommend reading it before usingaiopubsubin your project.Installationaiopubsubis only compatible with Python 3.8 and higher. There are no plans to support older versions.aiopubsubisavailable on PyPIand you can install it with:pip install aiopubsuborpoetry add aiopubsubHow to use itThe following comprehensive example is explained step-by-step
in our article“Design your app using the pub-sub pattern with aiopubsub”.importasyncioimportdataclassesimportdecimalimportaiopubsub@dataclasses.dataclassclassTrade:timestamp:floatquantity:intprice:decimal.Decimalasyncdefon_trade(key:aiopubsub.Key,trade:Trade)->None:print(f'Processing trade ={trade}with key ={key}.')asyncdefon_nyse_trade(key:aiopubsub.Key,trade:Trade)->None:print(f'Processing trade ={trade}with key ={key}that happened in NYSE')asyncdefmain():# create an aiopubsub hubhub=aiopubsub.Hub()# create a sample of data to sendtrade=Trade(timestamp=123.5,quantity=56,price=decimal.Decimal('1639.43'))# subscriber listens on every trade and calls the `on_trade` functionsubscriber=aiopubsub.Subscriber(hub,'trades')subscribe_key=aiopubsub.Key('*','trade','*')subscriber.add_async_listener(subscribe_key,on_trade)# publisher has a NASDAQ prefix and sends the trade that happened on Google stockpublisher=aiopubsub.Publisher(hub,prefix=aiopubsub.Key('NASDAQ'))publish_key=aiopubsub.Key('trade','GOOGL')publisher.publish(publish_key,trade)# sleep so the event loop can process the actionawaitasyncio.sleep(0.001)# expected output:# Processing trade = Trade(timestamp=123.5, quantity=56, price=Decimal('1639.43'))  with key = ('NASDAQ', 'trade', 'GOOGL').# sample from another stock exchangetrade_nyse=Trade(timestamp=127.45,quantity=67,price=decimal.Decimal('1639.44'))# subscribe only for the NYSE exchangesubscribe_key_nyse=aiopubsub.Key('NYSE','trade','*')subscriber.add_async_listener(subscribe_key_nyse,on_nyse_trade)# publish NYSE tradepublisher_nyse=aiopubsub.Publisher(hub,prefix=aiopubsub.Key('NYSE'))publisher_nyse.publish(aiopubsub.Key('trade','GOOGL'),trade_nyse)# sleep so the event loop can process the actionawaitasyncio.sleep(0.001)# expected output:# Processing trade = Trade(timestamp=127.45, quantity=67, price=Decimal('1639.44'))  with key = ('NYSE', 'trade', 'GOOGL').# Processing trade = Trade(timestamp=127.45, quantity=67, price=Decimal('1639.44'))  with key = ('NYSE', 'trade', 'GOOGL') that happened in NYSE# clean the subscriber before the end of the programawaitsubscriber.remove_all_listeners()if__name__=='__main__':asyncio.run(main())Aiopubsub will uselogwoodif it is installed, otherwise it will default
to the standard logging module. Note thatlogwoodis required to run tests.ArchitectureHubaccepts messages fromPublishersand routes them toSubscribers. Each message is routed by itsKey- an iterable of strings forming a hierarchic namespace. Subscribers may subscribe to wildcard keys,
where any part of the key may be replaced replaced with a*(star).addedSubscriberandremovedSubscribermessagesWhen a new subscriber is added the Hub sends this message{
        "key": ("key", "of", "added", "subscriber"),
        "currentSubscriberCount": 2
}under the key('Hub', 'addedSubscriber', 'key', 'of', 'added', 'subscriber')(the part afteraddedSubscriberis made of the subscribed key). Note thecurrentSubscriberCountfield indicating how many subscribers are currently
subscribed.When a subscriber is removed a message in the same format is sent, but this time under the key('Hub', 'removedSubscriber', 'key', 'of', 'added', 'subscriber').ContributingPull requests are welcome! In particular, we are aware that the documentation could be improved.
If anything aboutaiopubsubis unclear, please feel free tosimply open an issueand we will do our best
to advise and explain 🙂fastenumwas made byQuantlane, a systematic trading firm.
We design, build and run our own stock trading platform.