Package: topix
TopixA functional microframework to use Redis streams in Python.Quickstart 🚀Install Topix:pipinstalltopixWrite a consumer function:# app.pydefconsumer(data):print(data)With Redis running onlocalhost:6379, listen for new items in a stream:python-mtopixconsumerdemo_streamprinter_groupapp:consumerThen, send items to the stream using the commandtopix emitter:python-mtopixemitterdemo_stream'{"hello": "world"}'...or using Python code:fromtopiximportemitemit("demo_stream",{"hello":"world"})That's it. That's the API.Why Topix?I created Topix with a few primary goals and concepts:Topix has a dead-simple API- the "for humans" approach.Small abstraction over the fantastic Redis Streams.Implementation of Topix should favor simple code over new features.If you need heavy-duty task queues, advanced messaging systems, or want an opinionated framework with a "right way" to do things, Topix isn't for you. However, if you want a simple API you can easily manage and build your own abstractions on top of, Topix might be for you.UsageTopix centers around two concepts:consumers, implemented as functions, andemitters, that calltopix.emitorredis.xadd.Writing a Consumer FunctionWith Topix, creating a consumer for a stream is as easy as creating a function. For example, this is a valid consumer function:defprint_item(item):print(item)Specifically, your function needs to be of the followingtype, otherwise you'll get an error:Callable[[Dict[bytes,bytes]],None]Like Redis, Topix works with bytes, so if you want strings (or any other type) you need to bring your own converter.Using Your Consumer FunctionBack to ourprint_itemfunction, we can use this function in a consumer process using the Topix command line. Assuming we savedprint_iteminapp.py, we can run Topix like this:python-mtopixconsumerstream_namegroup_nameapp:print_itemWherestream_nameis the name of the Redis stream,group_nameis the name of the consumer group, andapp:print_itemis the Python path to our function (thinkgunicorn-style imports). By default, Topix will attempt to connect to the Redis url defined in the environment variableTOPIX_REDIS_URL, falling back toredis://localhost:6379/0.At runtime, Topix will import your function, and map it over the stream you give it. Topix automatically handles acknowledging successful commands, the creation of consumer groups, removing the consumer when it exits, and parallel execution of your function using threads.Sending to StreamsTopix defines anemitfunction that sends a Redis-compatible dictionary to the stream you give it, by name:fromtopiximportemitemit("stream_name",{b"hello":b"world"})# Strings are valid as keys too- they just# become bytes "on the other side"emit("stream_name",{"hello":"world"})Topix'semitis athinwrapper around Redis'XADDcommand, and maps the values given toXADDto your consumer function. This means you can emit messages fromanywhere, even from code in another language or not using Topix, and use your same consumer.Typically, you'll want to emit events to your code usingemit(orxadd) programmatically, but for demos (orbashscripts), Topix includes a subcommand to send messages to a stream:python-mtopixemitstream_name'{"hello": "world"}'Additional ConfigurationBy default, Topix uses a number of threads equivalent to the number of CPUs available. You can change the amount of concurrency (using threads) that Topix uses by providing the--concurrencyoption totopix consumer:python-mtopixconsumerstream_namegroup_nameapp:print_item--concurrency2In addition, Topix supports a few additional options, including configurable logging. For a full list of options, try:python-mtopix--helpDevelopmentTo get started hacking on Topix, be sure to install it in development mode.virtualenv.envsource.env/bin/activate# Linux/OSX.env\scripts\activate# Windowspipinstall-rrequirements.dev.txt
pythonsetup.pydevelopTests are provided by Pytest. Plugins forpylintandcoverageare directly integrated:pytest# or python setup.py test# Pytest OutputCoverageReport:

...======XPassed,XSkippedin.38seconds=====Topix checks types with MyPy. Type check your code like this:mypytopixLicenseCopyright 2019 by Madelyn Eriksen under the terms of the MIT License. Seelicensefor full terms.