Ratelimiter patch

governor.py: Modified the rate limiting algorithm so that it guarantees request-limiting to <number> requests per second (if required, we can change this to different units like requests per hour, etc.) and removed a need for the server to force a delay upon itself or the client.

test_governor.py: Added the ability to use a real redis instance from the pool for testing. The new rate limiter currently causes two unit tests to fail while using the fake redis instance, but upon using a real redis instance, all tests pass. (Functional test will be updated and uploaded soon).

Observation: The higher the attempted request rate, the more bursty this algorithm is going to be. A very high attempted request rate will exhaust the quota at the beginning of the epoch, and then the client will get hard-throttled. 

This shouldn't be a problem if there is enough headroom in the deployed system to absorb these spikes. If there is not, then noisy neighbor will become a problem (i.e., other tenant's requests will slow down during these spikes).

To smooth out the spikes, you can do soft throttling; when a tenant approaches their quota, you start slowing them down a bit, targeting the ideal rate. I would only implement this after putting the hard limit in production and gathering statistics on live usage to find out how frequent the spikes are, and how well the system is able to absorb them.