racker/eom — pull request #25 by cabrera (2014-04-28 20:49:02+00:00)
Title: feat(governor): leaky bucket algorithm [WIP]

# Commit Log

This PR replaces the existing governor's double-bucket approach with a
leaky bucket implementation of rate limiting.

This leads to a simple implementation to use and reason
about. Furthermore, the following concepts were removed:

- Cache object: use raw dictionaries instead
- period_sec: no longer relevant, as we don't context-switch
- max_sleep_sec: no longer relevant. Max sleep implicit on target rate
- node_count: removed. Makes configuring individual nodes less intuitive

Drain velocity has been added. It may yet be removed. It might be more
reasonable to define a drain acceleration factor and use a combination
of the delta since the last request and the target rate to determine
how much to drain per request from the bucket.

A simplification of the sleep_for function is provided that determines
how much to sleep based on:

(count / limit) * sleep_offset

This makes it very easy to hand-calculate how much one would sleep if
the soft limit is exceeded and how sleep_offset works with that.

This implementation is fully distributed.

# Missing

* Unit test updates
* Parameter tweaking
* Design review