bbangert/retools — pull request #22 by heynemann (2014-01-22 21:08:07+00:00)
Title: Limiter functionality.

I had to limit running parallel requests and thought of building my own lib. Thankfully I found retools and thought of contributing to it, instead.

The limiter class allows you to lock a specific number of items for a given operation. For instance:

limiter = Limiter(limit=10, prefix='parallel-requests')

for i in range(100):
url = 'http://my.domain.com/%d.html' % i
if limiter.acquire_limit(url):
async_http_request(url, callback=handle_result) # only 10 of these will get executed in parallel.

The app can then do something else or wait until it can acquire a limit. I hope you do consider this a generic enough construct that it should be in retools.

I tried hard to comply to the code notation and docs already in the project. If I missed something, please let me know and I'll gladly update the pull request.

The test covers 100% of the code and with both unit (mock redis) and functional (real redis) tests.