Package: turbomq
TurboMQ=======**TurboMQ** is a simple message queue system. I hope it is fast enoughto merit the name. In our test it could provide and consume millions ofmessages in a second. But we delegate the final judgment to thedevelopers who use the library. Consider that , currently, it is tooexperimental and there will be dramatical change in both functionalityand protocols.Why TurboMQ is developed?=========================First, I want to explain why a new message queue system is developed.There are many message queue systems available and some of them arepopular and stable like **RabbitMQ** or **ZMQ**. The most importantreason behind this implementation is that most of message queue systemsare designed to handle backend processing like distributing jobs betweennodes to process huge amount of data or just complete the remained partof a business transaction. Certainly, TurboMQ can be used to distributeworks between nodes. Moreover, it originally designed to supportmillions of providers and consumers working with millions of queues andtopics.The most close (as queue functionality) system is **Redis**. It has aremarkable IO mechanism to handle network connections. However, it canjust utilize one core for one instance. Do we really want to use justone core of for example 8 available cores? Or do we want to configureclustering inside one machine to just use all available cores?**ZMQ** is a good library. It is fast, stable and useful for manypurposes. Nonetheless, there is a serious problem in topic-based PUB-SUBqueues. The consumers (subscribers) has to be connected before providers`(missing message problem solver)`_ otherwise the message is going to belost.Technical information=====================**TurboMQ** is a python module. To avoid GIL problems, it is developedusing pure **C** and **Cython**. It uses its own event loop system. Thebenefit is that it is a real multi-threaded event loop and can exploitall available cores. The drawback is that it does not support windows.Are the bad news finished? No, kqueue has not implemented yet and ituses (slow) posix POLL in BSD families. Is there any other good news?Yes, windows and kqueue support is going to be implemented very soon.Installation============Installation is easy. The package can be installed by pip:$ sudo pip install turbomqYou need to download or clone it and then type the python magic:$ sudo python setup.py installUsage============To use **TurboMQ** just import and run the server. The following code runs a server for 10 minutes... code-block:: pythonfrom turbomq import TurboEngineimport time# You can pass the thread count as a second parameter.# Otherwise, it will automatically selects 4 threads per core.e = TurboEngine('tcp://127.0.0.1:33444')e.run()# "run" method will not block the main thread.# So you need to simply wait or run your own loop as you want.time.sleep(10.0 * 60)# "stop" method just shuts TCP sockets down.e.stop()# After destroy all resources will be freed.# Then you can not use this instance anymore.e.destroy()This code sends a message to server and receives it again... code-block:: pythonfrom turbomq import TurboClient# Connects to the server.c = TurboClient('tcp://127.0.0.1:33444')# Creates a mirror queue in client side.q = c.get_queue('test')# Both topic key and data is mandatory in push.q.push('hello', 'turbo')# In pop you need to determine a timeout.# So this will wait two seconds. If timeout is exceeded, it will return None.print(q.pop('hello', 2))