nikipore/stompest — pull request #25 by rtkrruvinskiy (2014-09-12 16:02:45+00:00)
Title: Avoid unintended inflation of messaging handling time

Previously, `async.Stomp._notify` used `task.cooperate` to dispatch all
notifications. However, `task.cooperate` is actually intended to run tasks in
a less CPU-intensive manner by calling the `next` method of the iterator at
fixed time intervals and thus releasing the CPU to be claimed by other tasks.
This meant that we were artifically yielding the CPU while notifying the listeners.
As a result, under high message load, the message arrival rate could easily
surpass the rate at which Twisted was invoking the cooperative tasks, causing
the process to fall behind on message processing and the memory usage to balloon
as there would be more and more messages retrieved from the broker waiting to be
processed.

The solution is not to use `task.cooperate` to notify the listeners and instead
to simply iterate over the listener list and notifying each one. This removes the
yielding behaviour of `task.cooperate` and allows stompest to usually fully
process each message before reading the next one from the broker, offloading
overload handling to the message broker rather than (not) dealing with it in the
Python process.

Testing: Ran test suite and ran ad-hoc high message arrival rate tests.

Resolves #23