Package: token-throttler
Token throttlerToken throttleris an extendable rate-limiting library somewhat based on atoken bucket algorithm.Table of contentsInstallationFeaturesUsageManual usage exampleDecorator usage exampleFastAPI usage exampleStorageConfigurationGlobal configuration usageInstance configuration usage1. InstallationToken throttler is available on PyPI:$python-mpipinstalltoken-throttlerToken throttler officially supports Python >= 3.7.NOTE: Depending on the storage engine you pick, you can install token throttler with extras:$python-mpipinstalltoken-throttler[redis]2. FeaturesBlocking (TokenThrottler) and non-blocking (TokenThrottlerAsync)Global throttler(s) configurationConfigurable token throttler cost and identifierMultiple buckets per throttler per identifierBuckets can be added/removed manually or by adictconfigurationManual usage or usage via decoratorDecorator usage supports async code tooCustom decorator can be writtenExtendable storage engine (eg. Redis)3. UsageToken throttler supports both manual usage and via decorator.Decorator usage supports both async and sync.1) Manual usage example:fromtoken_throttlerimportTokenBucket,TokenThrottlerfromtoken_throttler.storageimportRuntimeStoragethrottler:TokenThrottler=TokenThrottler(cost=1,storage=RuntimeStorage())throttler.add_bucket(identifier="hello_world",bucket=TokenBucket(replenish_time=10,max_tokens=10))throttler.add_bucket(identifier="hello_world",bucket=TokenBucket(replenish_time=30,max_tokens=20))defhello_world()->None:print("Hello World")foriinrange(10):throttler.consume(identifier="hello_world")hello_world()ifthrottler.consume(identifier="hello_world"):hello_world()else:print("bucket_one ran out of tokens")2) Decorator usage example:fromtoken_throttlerimportTokenBucket,TokenThrottler,TokenThrottlerExceptionfromtoken_throttler.storageimportRuntimeStoragethrottler:TokenThrottler=TokenThrottler(1,RuntimeStorage())throttler.add_bucket("hello_world",TokenBucket(10,10))throttler.add_bucket("hello_world",TokenBucket(30,20))@throttler.enable("hello_world")defhello_world()->None:print("Hello World")foriinrange(10):hello_world()try:hello_world()exceptTokenThrottlerException:print("bucket_one ran out of tokens")3) FastAPI usage example:fromfastapiimportDepends,FastAPI,RequestfrompydanticimportBaseModelfromtoken_throttlerimportTokenThrottler,TokenBucketfromtoken_throttler.storageimportRuntimeStoragefromtoken_throttler.ext.fastapiimportFastAPIThrottlerapp:FastAPI=FastAPI()ban_hammer:TokenThrottler=TokenThrottler(cost=1,storage=RuntimeStorage())classUser(BaseModel):id:intname:stru1:User=User(id=1,name="Test")users:list[User]=[u1]defcreate_buckets()->None:foruserinusers:ban_hammer.add_bucket(str(user.id),TokenBucket(replenish_time=10,max_tokens=10))# For example purposes only - feel free to load your users with the bucket(s) anywhere elseapp.add_event_handler("startup",create_buckets)# Add your auth logic heredefget_auth_user()->User:returnu1# Since this is a FastAPI dependency, you can have the `Request` object here too if needed (e.g. storing the JWT token in# a request lifecycle).# You can also use sub-dependencies like shown below - `auth_user`asyncdefget_user_id(request:Request,auth_user:User=Depends(get_auth_user))->str:returnstr(auth_user.id)# You can configure the dependency per route, on a router level or on global application level.# Pass your unique user identifier (e.g. JWT subject, apikey or user's ID) as a callback parameter# of the `enable` method.@app.get("/throttle",dependencies=[Depends(FastAPIThrottler(ban_hammer).enable(get_user_id))])asyncdefread_users():return{"detail":"This is throttled URL"}For other examples seeexamplesdirectory.4. StorageTokenThrottlersupportsRuntimeStorageandRedisStorage.TokenThrottlerAsyncsupportsRedisStorageAsyncIf you want your own storage engine, feel free to extend thetoken_throttler.storage.BucketStorageortoken_throttler.storage.BucketStorageAsyncclasses.For storage examples seeexamplesdirectory.5. ConfigurationToken throttler supports global and per instance configurations.Configuration params:IDENTIFIER_FAIL_SAFE(defaultFalse) - if invalid identifier is given as a param for theconsumemethod andIDENTIFIER_FAIL_SAFEis set toTrue, noKeyErrorexception will be raised andconsumewill act like a limitless bucket is being consumed.ENABLE_THREAD_LOCK(defaultFalse) - if set toTrue, throttler will acquire a thread lock upon callingconsumemethod and release
the lock once theconsumeis finished. This avoids various race conditions at a slight performance cost.Global configuration usageGlobal configuration means that all the throttler instances will use the same configuration from
theThrottlerConfigGlobalclass singleton (!) instance -default_config.fromtoken_throttlerimportTokenBucket,TokenThrottler,default_configfromtoken_throttler.storageimportRuntimeStoragedefault_config.set({"ENABLE_THREAD_LOCK":False,"IDENTIFIER_FAIL_SAFE":True,})throttler:TokenThrottler=TokenThrottler(1,RuntimeStorage())throttler.add_bucket("hello_world",TokenBucket(10,10))throttler.add_bucket("hello_world",TokenBucket(30,20))...Whenever you change the options insidedefault_configobject, all of theTokenThrottlerinstances, that are not
using the instance specific configuration, will get updated with the new settings.NOTE: If any modifications are attempted on thedefault_configobject during runtime, aRuntimeWarningwill be emitted. Thedefault_configobject is designed to be a singleton instance of theThrottlerConfigGlobalclass, responsible for storing and managing configuration settings. To ensure the proper functioning of the throttling mechanism and to maintain the integrity of the configuration, it is recommended to avoid making runtime changes to thedefault_configobject. Any such modifications may result in unexpected behavior or inconsistencies in the throttling behavior. Instead, consider utilizinginstance-specific configuration.Instance configuration usageInstance configuration is to be used whenTokenThrottlerinstance(s) needs a different configuration that the global one.
If no instance configuration is passed,TokenThrottlerobject will use the global configuration.fromtoken_throttlerimportThrottlerConfig,TokenBucket,TokenThrottlerfromtoken_throttler.storageimportRuntimeStoragethrottler:TokenThrottler=TokenThrottler(1,RuntimeStorage(),ThrottlerConfig({"ENABLE_THREAD_LOCK":True,"IDENTIFIER_FAIL_SAFE":True,}),)throttler.add_bucket("hello_world",TokenBucket(10,10))throttler.add_bucket("hello_world",TokenBucket(30,20))...