bitly/pynsq — pull request #39 by jehiah (2013-06-27 02:56:57+00:00)
Title: distribute lookupd requests

when configured to use lookupd, Reader currently polls all configured lookupd instances at the same time. It is expected that the client union the responses, but that actually doesn't require the reader to poll at the same time.

Configuration is specified in terms of that poll interval (and some initial jitter is applied). It would be ideal to spread out the requests evenly instead of all at once so you have an option to distribute requests. This would mean that given 4 lookupd instances, they would be queried one at a time based on the poll interval. This allows additional hosts to be added to distribute requests.

Initial jitter is currently set to 10% of the poll interval. It would also be nice to increase this percentage for more evenly distributed requests when a process has multiple Reader objects, or when a host has multiple process that all start at the same time.