Package: pyrate-limiter
PyrateLimiterThe request rate limiter using Leaky-bucket Algorithm.Full project documentation can be found atpyratelimiter.readthedocs.io.ContentsPyrateLimiterContentsFeaturesInstallationQuickstartBasic usageKey conceptsDefining rate limits & bucketsDefining clock & routing logicWrapping all up with LimiterWeightHandling exceeded limitsBucket analogyRate limit exceptionsRate limit delaysBackendsInMemoryBucketSQLiteBucketRedisBucketDecoratorAdvanced UsageComponent-level DiagramTime sourcesLeakingConcurrencyCustom backendFeaturesTracks any number of rate limits and intervals you want to defineIndependently tracks rate limits for multiple services or resourcesHandles exceeded rate limits by either raising errors or adding delaysSeveral usage options including a normal function call, a decoratorOut-of-the-box workable with both sync & asyncIncludes optional SQLite and Redis backends, which can be used to persist limit tracking across
multiple threads, or application restartsInstallationPyrateLimitersupportspython ^3.8Install using pip:pip install pyrate-limiterOr using conda:conda install --channel conda-forge pyrate-limiterQuickstartLet's say you want to limit 5 requests over 2 seconds, and raise an exception if the limit is exceeded:frompyrate_limiterimportDuration,Rate,Limiter,BucketFullExceptionrate=Rate(5,Duration.SECOND*2)limiter=Limiter(rate)# Or you can pass multiple rates# rates = [Rate(5, Duration.SECOND * 2), Rate(10, Duration.MINUTE)]# limiter = Limiter(rates)forrequestinrange(6):try:limiter.try_acquire(request)exceptBucketFullExceptionaserr:print(err)print(err.meta_info)# Bucket for item=5 with Rate limit=5/2.0s is already full# {'error': 'Bucket for item=5 with Rate limit=5/2.0s is already full', 'name': 5, 'weight': 1, 'rate': 'limit=5/2.0s'}Basic UsageKey conceptsClockTimestamp itemsBucketHold items with timestamps.Behave like a FIFO queueIt canleak- popping items that are no longer relevant out of the queueBucketFactoryBucketFactory keeps references to buckets & clocks: determine the exact time that items arrive then route them to their corresponding bucketsHelp schedule background tasks to run buckets'leakperiodically to make sure buckets will not explode from containing too many itemsWhere user define his own logic: routing, condition-checking, timing etc...LimiterThe Limiter's most important responsibility is to make user's life as easiest as possible:Sums up all the underlying logic to a simple, intuitive API to work withHandles async/sync context seamlessly (everything justworksby adding/removingasync/awaitkeyword to the user's code)Provides different ways of interacting with the underlyingBucketFactory(plain method call, decorator, context-manager (TBA))Provides thread-safety using RLockDefining rate limits and bucketsConsider some public API (like LinkedIn, GitHub, etc.) that has rate limits like the following:- 500 requests per hour
- 1000 requests per day
- 10000 requests per monthYou can define these rates using theRateclass.Rateclass has 2 properties only:limitandintervalfrompyrate_limiterimportDuration,Ratehourly_rate=Rate(500,Duration.HOUR)# 500 requests per hourdaily_rate=Rate(1000,Duration.DAY)# 1000 requests per daymonthly_rate=Rate(10000,Duration.WEEK*4)# 10000 requests per monthrates=[hourly_rate,daily_rate,monthly_rate]Rates must be properly ordered:Rates' intervals & limits must be ordered from least to greatestRates' ratio oflimit/intervalmust be ordered from greatest to leastExisting implementations of Bucket come with rate-validation when init. If you are to use your own implementation, use the validator provided by the libfrompyrate_limiterimportvalidate_rate_listassertvalidate_rate_list(my_rates)Then, add the rates to the bucket of your choicesfrompyrate_limiterimportInMemoryBucket,RedisBucketbasic_bucket=InMemoryBucket(rates)# Or, using redisfromredisimportRedisredis_connection=Redis(host='localhost')redis_bucket=RedisBucket.init(rates,redis_connection,"my-bucket-name")# Async Redis would work too!fromredis.asyncioimportRedisredis_connection=Redis(host='localhost')redis_bucket=awaitRedisBucket.init(rates,redis_connection,"my-bucket-name")If you only need a single Bucket for everything, and python's built-intime()is enough for you, then pass the bucket to Limiter then ready to roll!frompyrate_limiterimportLimiter# Limiter constructor accepts single bucket as the only parameter,# the rest are 3 optional parameters with default values as following# Limiter(bucket, clock=TimeClock(), raise_when_fail=True, max_delay=None)limiter=Limiter(bucket)# Limiter is now ready to work!limiter.try_acquire("hello world")If you want to have finer grain control with routing & clocks etc, then you should useBucketFactory.Defining Clock & routing logic with BucketFactoryIf you need more than one type of Bucket, and be able to route items to different buckets based on some condition, you can use BucketFactory to do that.From the above steps, you already have your buckets. Now it's time to define whatTimeis (funny?!). Most of the time (again!?), you can use the existing Clock backend provided bypyrate_limiter.frompyrate_limiter.clockimportTimeClock,MonotonicClock,SQLiteClockbase_clock=TimeClock()PyrateLimitermakes no assumption about users logic, so to map coming items to their correct buckets, implement your ownBucketFactoryclass! At minimum, there are only 2 methods require implementingfrompyrate_limiterimportBucketFactoryfrompyrate_limiterimportAbstractBucketclassMyBucketFactory(BucketFactory):# You can use constructor here,# nor it requires to make bucket-factory work!defwrap_item(self,name:str,weight:int=1)->RateItem:"""Time-stamping item, return a RateItem"""now=clock.now()returnRateItem(name,now,weight=weight)defget(self,_item:RateItem)->AbstractBucket:"""For simplicity's sake, all items route to the same, single bucket"""returnbucketWrapping all up with LimiterPass your bucket-factory to Limiter, and ready to roll!frompyrate_limiterimportLimiterlimiter=Limiter(bucket_factory,raise_when_fail=False,# Default = Truemax_delay=1000,# Default = None)item="the-earth"limiter.try_acquire(item)heavy_item="the-sun"limiter.try_acquire(heavy_item,weight=10000)If your bucket's backend isasync, well, we got you covered! Passingawaitto the limiter is enought to make it scream!awaitlimiter.try_acquire(item)Alternatively, you can useLimiter.try_acquireas a function decorator. But you have to provide amappingfunction that map the wrapped function's arguments to a properlimiter.try_acquireargument - which is a tuple of(str, int)or juststrmy_beautiful_decorator=limiter.as_decorator()defmapping(some_number:int):returnstr(some_number)@my_beautiful_decorator(mapping)defrequest_function(some_number:int):requests.get('https://example.com')# Async would work too!@my_beautiful_decorator(mapping)asyncdefasync_request_function(some_number:int):requests.get('https://example.com')WeightItem can have weight. By default item's weight = 1, but you can modify the weight before passing tolimiter.try_acquire.Item with weight W > 1 when consumed will be multiplied to (W) items with the same timestamp and weight = 1. Example with a big item with weight W=5, when put to bucket, it will be divided to 5 items with weight=1 + following namesBigItem(weight=5, name="item", timestamp=100) => [
    item(weight=1, name="item", timestamp=100),
    item(weight=1, name="item", timestamp=100),
    item(weight=1, name="item", timestamp=100),
    item(weight=1, name="item", timestamp=100),
    item(weight=1, name="item", timestamp=100),
]Yet, putting this big, heavy item into bucket is expected to be transactional & atomic - meaning either all 5 items will be consumed or none of them will. This is made possible as bucketput(item)always check for available space before ingesting. All of the Bucket's implementations provided byPyrateLimiterfollows this rule.Any additional, custom implementation of Bucket are expected to behave alike - as we have unit tests to cover the case.SeeAdvanced usage optionsbelow for more details.Handling exceeded limitsWhen a rate limit is exceeded, you have two options: raise an exception, or add delays.Bucket analogyAt this point it's useful to introduce the analogy of "buckets" used for rate-limiting. Here is a
quick summary:This library implements theLeaky Bucket algorithm.It is named after the idea of representing some kind of fixed capacity -- like a network or service -- as a bucket.The bucket "leaks" at a constant rate. For web services, this represents theideal or permitted request rate.The bucket is "filled" at an intermittent, unpredicatble rate, representing theactual rate of requests.When the bucket is "full", it will overflow, representingcanceled or delayed requests.Item can have weight. Consuming a single item with weight W > 1 is the same as consuming W smaller, unit items - each with weight=1, with the same timestamp and maybe same name (depending on however user choose to implement it)Rate limit exceptionsBy default, aBucketFullExceptionwill be raised when a rate limit is exceeded.
The error contains ameta_infoattribute with the following information:name: The name of item it receivedweight: The weight of item it receivedrate: The specific rate that has been exceededHere's an example that will raise an exception on the 4th request:rate=Rate(3,Duration.SECOND)bucket=InMemoryBucket([rate])clock=TimeClock()classMyBucketFactory(BucketFactory):defwrap_item(self,name:str,weight:int=1)->RateItem:"""Time-stamping item, return a RateItem"""now=clock.now()returnRateItem(name,now,weight=weight)defget(self,_item:RateItem)->AbstractBucket:"""For simplicity's sake, all items route to the same, single bucket"""returnbucketlimiter=Limiter(MyBucketFactory())for_inrange(4):try:limiter.try_acquire('item',weight=2)exceptBucketFullExceptionaserr:print(err)# Output: Bucket with Rate 3/1.0s is already fullprint(err.meta_info)# Output: {'name': 'item', 'weight': 2, 'rate': '3/1.0s', 'error': 'Bucket with Rate 3/1.0s is already full'}The rate part of the output is constructed as:limit / interval. On the above example, the limit
is 3 and the interval is 1, hence theRate 3/1.Rate limit delaysYou may want to simply slow down your requests to stay within the rate limits instead of canceling
them. In that case you pass themax_delayargument the maximum value of delay (typically inmswhen use human-clock).limiter=Limiter(factory,max_delay=500)# Allow to delay up to 500msAsmax_delayhas been passed as a numeric value, when ingesting item, limiter will:First, try to ingest such item using the routed bucketIf it fails to put item into the bucket, it will callwait(item)on the bucket to see how much time remains until the bucket can consume the item again?Comparing thewaitvalue to themax_delay.ifmax_delay>=wait: delay (wait + 50ms as latency-tolerance) using eitherasyncio.sleeportime.sleepuntil the bucket can consume againifmax_delay<wait: it raisesLimiterDelayExceptionif Limiter'sraise_when_fail=True, otherwise silently fail and return FalseExample:frompyrate_limiterimportLimiterDelayExceptionfor_inrange(4):try:limiter.try_acquire('item',weight=2,max_delay=200)exceptLimiterDelayExceptionaserr:print(err)# Output:# Actual delay exceeded allowance: actual=500, allowed=200# Bucket for 'item' with Rate 3/1.0s is already fullprint(err.meta_info)# Output: {'name': 'item', 'weight': 2, 'rate': '3/1.0s', 'max_delay': 200, 'actual_delay': 500}BackendsA few different bucket backends are available:InMemoryBucket using python built-in list as bucketRedisBucket, using err... redis, with both async/sync supportSQLite, using sqlite3InMemoryBucketThe default bucket is stored in memory, using pythonlistfrompyrate_limiterimportInMemoryBucket,Rate,Durationrates=[Rate(5,Duration.MINUTE*2)]bucket=InMemoryBucket(rates)This bucket only availabe insyncmode. The only constructor argument isList[Rate].RedisBucketRedisBucket usesSorted-Setto store items with key being item's name and score item's timestamp
Because it is intended to work with both async & sync, we provide a classmethodinitfor itfrompyrate_limiterimportRedisBucket,Rate,Duration# Using synchronous redisfromredisimportConnectionPoolfromredisimportRedisrates=[Rate(5,Duration.MINUTE*2)]pool=ConnectionPool.from_url("redis://localhost:6379")redis_db=Redis(connection_pool=pool)bucket_key="bucket-key"bucket=RedisBucket.init(rates,redis_db,bucket_key)# Using asynchronous redisfromredis.asyncioimportConnectionPoolasAsyncConnectionPoolfromredis.asyncioimportRedisasAsyncRedispool=AsyncConnectionPool.from_url("redis://localhost:6379")redis_db=AsyncRedis(connection_pool=pool)bucket_key="bucket-key"bucket=awaitRedisBucket.init(rates,redis_db,bucket_key)The API are the same, regardless of sync/async. If AsyncRedis is being used, callingawait bucket.method_name(args)would just work!SQLiteBucketIf you need to persist the bucket state, a SQLite backend is available.Manully create a connection to Sqlite and pass it along with the table name to the bucket class:frompyrate_limiterimportSQLiteBucket,Rate,Durationimportsqlite3rates=[Rate(5,Duration.MINUTE*2)]conn=sqlite3.connect("/var/mydb.sqlite",isolation_level="EXCLUSIVE",check_same_thread=False,)table="my-bucket-table"bucket=SQLiteBucket(rates,conn,table)DecoratorLimiter can be used as decorator, but you have to provide amappingfunction that maps the wrapped function's arguments tolimiter.try_acquirefunction arguments. The mapping function must return either a tuple of(str, int)or just astrThe decorator can work with both sync & async functiondecorator=limiter.as_decorator()defmapping(*args,**kwargs):return"demo",1@decorator(mapping)defhandle_something(*args,**kwargs):"""function logic"""@decorator(mapping)asyncdefhandle_something_async(*args,**kwargs):"""function logic"""Advanced UsageComponent level diagramTime sourcesTime source can be anything from anywhere: be it python's built-in time, or monotonic clock, sqliteclock, or crawling from world time server(well we don't have that, but you can!).frompyrate_limiterimportTimeClock# use python' time.time()frompyrate_limiterimportMonotonicClock# use python time.monotonic()Clock's abstract interface only requires implementing a methodnow() -> int. And it can be both sync or async.LeakingTypically bucket should not hold items forever. Bucket's abstract interface requires its implementation must be provided withleak(current_timestamp: Optional[int] = None).Theleakmethod when called is expected to remove any items considered outdated  at that moment. During Limiter lifetime, all the buckets'leakshould be called periodically.BucketFactoryprovide a method calledschedule_leakto help deal with this matter. Basically, it will run1(one)background task for a specific bucket, where interval betweenleakcall by default is the bucket's longest rate's interval * 2.# Runnning a background task (whether it is sync/async - doesnt matter)# calling the bucket's leak, using the clock as `bucket.leak(clock.now())`# interval between call = bucket.rates[-1].interval * 2 (ms)factory.schedule_leak(bucket,clock)You can change this calling interval by overriding BucketFactory'sleak_interval. For example, use the first rate's interval or a fixed valueclassMyBucketFactory(BucketFactory):...otherimplementationsdefleak_interval(self,bucket):ifisinstance(bucket,InMemoryBucket):returnbucket.rates[0].intervalifisinstance(bucket,RedisBucket):return10_000returnsuper().leak_interval()When dealing with leak using BucketFactory, the author's suggestion is, we can be pythonic about this by implementing a constructorclassMyBucketFactory(BucketFactory):defconstructor(self,clock,buckets):self.clock=clockself.buckets=bucketsforbucketinbuckets:self.schedule_leak(bucket,clock)ConcurrencyGenerally, Lock is provided at Limiter's level, except SQLiteBucket case.Custom backendsIf these don't suit your needs, you can also create your own bucket backend by implementingpyrate_limiter.AbstractBucketclass.One ofPyrateLimiterdesign goals is powerful extensibility and maximum ease of development.It must be not only be a ready-to-use tool, but also a guide-line, or a framework that help implementing new features/bucket free of the most hassles.Due to the composition nature of the library, it is possbile to write minimum code and validate the result:Fork the repoImplement your bucket withpyrate_limiter.AbstractBucketAdd your owncreate_bucketmethod intests/conftest.pyand pass it to thecreate_bucketfixtureRun the test suite to validate the resultIf the tests pass through, then you are just good to go with your new, fancy bucket!