mumrah/kafka-python — pull request #31 by mahendra (2013-06-12 08:32:28+00:00)
Title: Optimize auto-commit process

The auto-commit thread is a bit heavy and there are points of optimization. I tried two attempts to do it.

* Spawn the commit thread only if necessary - If there are no messages being consumed, the timer keeps creating new threads at the specified intervals. This may not be necessary. We can control this behaviour such that the timer thread is started only when a message is consumed

* The previous approach optimized the commit thread such that the timer started only when there were messages to be consumed. This goes a step further and ensures the following:
* Only one timer thread is created
* The main app does not block on exit (waiting for timer thread to finish).

This is ensured by having a single thread blocking on an event and keeps calling a function. We use events instead of time.sleep() so as to prevent the python interpreter from running every 50ms checking if the timer has expired (logic copied from threading.Timer)