Package: bucketratelimiter
BucketRateLimiterbucketratelimiteris the collection of rate limiters, which are based onBucketconception.How to install:pip install bucketratelimiterExamples:You can find complete examples on how to useAsyncioBucketTimeRateLimiterandMThreadedBucketTimeRateLimiterin
folderexamplesof this repository. Also you can checktestsfolder and get some tricks from unittests.How to use:Create rate limiter:frombucketratelimiterimportAsyncioBucketTimeRateLimiter# max_size = 4 and recovery_time = 1.0# means that we would like to limit something to 4 attempts per second# max_size - is a size of internal bucket# recovery_time - time to make bucket full again# rest_time - is the time to sleep for workers which can not proceed further due to rate limiter# callback - is a function which should be called after task will be completedASYNC_LIMITER=AsyncioBucketTimeRateLimiter(max_size=4,recovery_time=1.0,rest_time=0.2,callback=None,)frombucketratelimiterimportMThreadedBucketTimeRateLimiterASYNC_LIMITER=MThreadedBucketTimeRateLimiter(max_size=4,recovery_time=1.0,rest_time=0.2,callback=None,)"Wrap" some function in ratelimiter:importasynciofrombucketratelimiterimportAsyncioBucketTimeRateLimiterlimiter=AsyncioBucketTimeRateLimiter()@limiter# we use decorator to limit the function to a certain number of attempts per secondasyncdefsome_func_to_limit(sleep_time:float=1.0)->None:awaitasyncio.sleep(sleep_time)importtimefrombucketratelimiterimportMThreadedBucketTimeRateLimiterlimiter=MThreadedBucketTimeRateLimiter()@limiter# we use decorator to limit the function to a certain number of attempts per seconddefsome_func_to_limit(sleep_time:float=1.0)->None:time.sleep(sleep_time)Activate ratelimiter logic:limiter=AsyncioBucketTimeRateLimiter()asyncdefmain_entry_point()->None:"""Main entry point of our asyncio app."""q=asyncio.Queue()fortaskinTASKS_TO_COMPLETE:awaitq.put(task)# use LIMITER as context manager to ensure its correct activation and end of workasyncwithlimiter:forwin[worker(q)for_inrange(1,WORKER_NUM+1)]:asyncio.create_task(w)awaitq.join()limiter=MThreadedBucketTimeRateLimiter()defmain_entry_point()->None:"""Main entry point of our multithreading app."""q=Queue()fortaskinTASKS_TO_COMPLETE:q.put(task)# use LIMITER as context manager to ensure its correct activation and end of workwithLIMITER:for_inrange(1,WORKER_NUM+1):Thread(target=worker,args=(q,),daemon=True).start()q.join()HOW TO USE LOW LEVEL API:Use without context manager:# Use RateLimiter's method to activate and deactivate it's inner logic# instead of using context managerstry:limiter.activate()finally:limiter.deactivate()Use without functions decoration:importasynciofrombucketratelimiterimportAsyncioBucketTimeRateLimiterlimiter=AsyncioBucketTimeRateLimiter()asyncdefsome_func_to_limit(sleep_time:float=1.0)->None:awaitasyncio.sleep(sleep_time)...asyncwithlimiter:awaitlimiter.wrap_operation(some_func_to_limit,sleep_time=1.0)# ATTENTION !# Do not use wrap_operation to some function more than once# Do not apply decorator to some function if you use wrap_operation# It can lead to unexpected resultsimporttimefrombucketratelimiterimportMThreadedBucketTimeRateLimiterlimiter=MThreadedBucketTimeRateLimiter()defsome_func_to_limit(sleep_time:float=1.0)->None:time.sleep(sleep_time)...withlimiter:limiter.wrap_operation(some_func_to_limit,sleep_time=1.0)FOR CONTRIBUTORS:Clone the project:https://github.com/ArtyomKozyrev8/BucketRateLimiter.git
cd BucketRateLimiterCreate a new virtualenv:python3 -m venv env
source env/bin/activateInstall all requirements:pip install -e '.[develop]'Run Tests:mypy --strict bucketratelimiter

pytest --cov=bucketratelimiter tests/

coverage report

coverage html