Package: lc-api-manager
api-manager-pyOverviewMany APIs with rate limits push responsibility on API users to manage rate limiting, including those with SDKs.api-manager-pyis a Python library that aims to abstract away the complexity of managing rate limits,
allowing developers to focus on retrieving desired data without hacking together buggy, case-by-case solutions.
The API management functionality does not require using a specific library for interacting with an API, and will
automatically cache responses based on input parameters to reduce network IO. Simply implement a small API client
interface and start making requests.InstallationInstall from PyPi (preferred method)pipinstalllc-api-managerInstall from GitHub with Pippipinstallgit+https://github.com/libcommon/api-manager-py.git@vx.x.x#egg=lc_api_managerwherex.x.xis the version you want to download.Install by Manual DownloadTo download the source distribution and/or wheel files, navigate tohttps://github.com/libcommon/api-manager-py/tree/releases/vx.x.x/dist, wherex.x.xis the version you want to install,
and download either via the UI or with a tool like wget. Then to install run:pipinstall<downloadedfile>Donotchange the name of the file after downloading, as Pip requires a specific naming convention for installation files.Dependenciesapi-manager-pydepends on thelc-cachelibrary for caching API responses. Only
Python versions >= 3.6 are officially supported.Getting StartedThe first step is to implement anAPIClientand choose a library to make HTTP requests. One common choice is theRequestslibrary, which we'll use to implement a client for theGitHub REST API
v3. The domain for GitHub's API ishttps://api.github.com, so the client'srequestmethod
only needs the HTTP method (GET,POST, etc.), API endpoint (i.e.,/repos/<username>/<repo_name>), and optionalheaders,params, anddatadictionaries (see:Requests documentation).fromhashlibimportsha256fromtypingimportAny,Dict,OptionalimportrequestsfromrequestsimportResponsefromlc_api_managerimportAPIClientclassGitHubAPIClient(APIClient):"""API client for GitHub Rest API v3."""__slots__=("_headers",)def__init__(self,auth_token:Optional[str]=None)->None:"""Initialize API client with optional GitHub API oauth tokensee: https://developer.github.com/v3/#authentication."""ifauth_token:self._headers={"Authorization":"token{}".format(auth_token)}else:self._headers=dict()defprocess_response_for_cache(self,response:Optional[Response])->Optional[str]:"""Return the SHA-256 hash of the API response if not None."""ifresponse:returnsha256(response.text.encode("utf8")).hexdigest()returnNonedefrequest(http_method:str,api_endpoint:str,headers:Optional[Dict[str,Any]],params:Optional[Dict[str,Any]],data:Optional[Dict[Any,Any]])->Response:"""Make request to GitHub REST API endpoint with providedheaders, URL parameters, and data and return response."""# Merge authorization header with provided headersmerged_headers=self._headersifheaders:merged_headers.update(headers)# Construct full URL and make requesturl="https://api.github.com/{}".format(api_endpoint.lstrip("/"))response=requests.request(http_method.upper(),url,params=params,data=data,headers=merged_headers)# Raise error if status code is 4XX or 5XXresponse.raise_for_status()returnresponseWith a functionalAPIClient, we can start making requests and caching them using the built-inAPIManagerclass:fromlc_api_managerimportAPIManagerfromlc_cacheimportHashmapCachedefmain()->int:"""Make 60 unauthenticated requests to an API endpoint in rapid succession."""api_manager=APIManager(3600,# GitHub API allows 60 unauthenticated requests per hour60,GitHubAPIClient(),HashmapCache())# Make 60 requests to the same API endpointfor_inrange(60):# The API manager will make the request on first iteration,# but will return cached response on the other 59response=api_manager.request("GET","repos/libcommon/api-manager-py",params=dict(per_page=100))# Check rate limit status, should be 59 requests remaining# See: https://developer.github.com/v3/rate_limit/response=api_manager.request("GET","rate_limit")requests_remaining=response.json().get("resources").get("core").get("remaining")assert(requests_remaining==59)return0if__name__=="__main__":main()If you are running multiple Python processes requesting data from the same API, and want to ensure that all of them
respect the rate limit requirements, you could override theAPIManager.update_statemethod. TheAPIManagerconstructor
has a parameter calledupdated_state_before_request, which defaults to False. If you set this toTrue, theupdate_statemethod will be called before every API request, and thus can be used to sync rate limiting state across multiple processes.
For example, you could use the/rate_limitGitHub API endpoint to implement this method:fromlc_api_managerimportAPIManagerdefGitHubAPIManager(APIManager):"""API manager for GitHub REST API v3 that syncs rate limiting state."""__slots__=()def__init__(self,*args,**kwargs)->None:ifnotkwargs.get("update_state_before_request"):kwargs["update_state_before_request"]=Truesuper().__init__(*args,**kwargs)defupdate_state(self)->None:"""Make request to /rate_limit endpoint and update rate limit status."""response=self._client.request("GET","rate_limit")requests_remaining=response.json().get("resources").get("core").get("remaining")self._count=self._threshold-requests_remainingContributing/SuggestionsContributions and suggestions are welcome! To make a feature request, report a bug, or otherwise comment on existing
functionality, please file an issue. For contributions please submit a PR, but make sure to lint, type-check, and test
your code before doing so. Thanks in advance!