Package: deckar01-ratelimit
This project is a fork oftomasbasham/ratelimitthat implements asliding logfor correctness and provides persistance via sqlite. See the usage section onPersistencefor more details. Turning on persistence is highly
recommended, especially during development, to ensure rate limits are respected
between application restarts.APIs are a very common way to interact with web services. As the need to
consume data grows, so does the number of API calls necessary to remain up to
date with data sources. However many API providers constrain developers from
making too many API calls. This is know as rate limiting and in a worst case
scenario your application can be banned from making further API calls if it
abuses these limits.This packages introduces a function decorator preventing a function from being
called more often than that allowed by the API provider. This should prevent
API providers from banning your applications by conforming to their rate
limits.InstallationPyPiAdd this line to your application’s requirements.txt:deckar01-ratelimitAnd then execute:$pipinstall-rrequirements.txtOr install it yourself:$pipinstalldeckar01-ratelimitGitHubInstalling the latest version from Github:$gitclonehttps://github.com/deckar01/ratelimit$cdratelimit$pythonsetup.pyinstallUsageTo use this package simply decorate any function that makes an API call:fromratelimitimportlimitsimportrequestsFIFTEEN_MINUTES=900@limits(calls=15,period=FIFTEEN_MINUTES)defcall_api(url):response=requests.get(url)ifresponse.status_code!=200:raiseException('API response:{}'.format(response.status_code))returnresponseThis function will not be able to make more then 15 API call within a 15 minute
time period.The arguments passed into the decorator describe the number of function
invocation allowed over a specified time period (in seconds). If no time period
is specified then it defaults to 15 minutes (the time window imposed by
Twitter).If a decorated function is called more times than that allowed within the
specified time period then aratelimit.RateLimitExceptionis raised. This
may be used to implement a retry strategy such as anexpoential backofffromratelimitimportlimits,RateLimitExceptionfrombackoffimporton_exception,expoimportrequestsFIFTEEN_MINUTES=900@on_exception(expo,RateLimitException,max_tries=8)@limits(calls=15,period=FIFTEEN_MINUTES)defcall_api(url):response=requests.get(url)ifresponse.status_code!=200:raiseException('API response:{}'.format(response.status_code))returnresponseAlternatively to cause the current thread to sleep until the specified time
period has ellapsed and then retry the function use thesleep_and_retrydecorator. This ensures that every function invocation is successful at the
cost of halting the thread.fromratelimitimportlimits,sleep_and_retryimportrequestsFIFTEEN_MINUTES=900@sleep_and_retry@limits(calls=15,period=FIFTEEN_MINUTES)defcall_api(url):response=requests.get(url)ifresponse.status_code!=200:raiseException('API response:{}'.format(response.status_code))returnresponsePersistenceIf a limit needs to be respected between application restarts or shared by
multiple processes, thestorageargument can be used to save the limit
state to disk and load it automatically.fromratelimitimportlimits,sleep_and_retryimportrequestsFIFTEEN_MINUTES=900@sleep_and_retry@limits(calls=15,period=FIFTEEN_MINUTES,storage='ratelimit.db')defcall_api(url):response=requests.get(url)ifresponse.status_code!=200:raiseException('API response:{}'.format(response.status_code))returnresponseIf multiple limits need to be persisted, thenameargument can be used to
store them in the same database using different tables.fromratelimitimportlimits,sleep_and_retryimportrequestsHOUR=3600DAY=24*HOUR@sleep_and_retry@limits(calls=15,period=HOUR,storage='ratelimit.db',name='hourly_limit')@sleep_and_retry@limits(calls=100,period=DAY,storage='ratelimit.db',name='daily_limit')defcall_api(url):response=requests.get(url)ifresponse.status_code!=200:raiseException('API response:{}'.format(response.status_code))returnresponseLicenseThis project is licensed under theMIT License.