Package: api-client
Python API ClientA client for communicating with an api should be a clean abstraction
over the third part api you are communicating with. It should be easy to
understand and have the sole responsibility of calling the endpoints and
returning data.To achieve this,APIClienttakes care of the other (often duplicated)
responsibilities, such as authentication and response handling, moving
that code away from the clean abstraction you have designed.Quick linksInstallationClient in actionAdding retries to requestsWorking with paginated responsesAuthenticating your requestsHandling the formats of your responsesCorrectly encoding your outbound request dataHandling bad requests and responsesEndpoints as codeExtensionsInstallationpipinstallapi-clientUsageSimple ExamplefromapiclientimportAPIClientclassMyClient(APIClient):deflist_customers(self):url="http://example.com/customers"returnself.get(url)defadd_customer(self,customer_info):url="http://example.com/customers"returnself.post(url,data=customer_info)>>>client=MyClient()>>>client.add_customer({"name":"John Smith","age":28})>>>client.list_customers()[...,{"name":"John Smith","age":28},]TheAPIClientexposes a number of predefined methods that you can call
This example usesgetto perform a GET request on an endpoint.
Other methods include:post,put,patchanddelete. More
information on these methods is documented in theInterface.For a more complex use case example, see:Extended exampleRetryingTo add some robustness to your client, the power oftenacityhas been harnessed to add a@retry_requestdecorator to theapiclienttoolkit.This will retry any request which responds with a 5xx status_code (which is normally safe
to do as this indicates something went wrong when trying to make the request), or when anUnexpectedErroroccurs when attempting to establish the connection.@retry_requesthas been configured to retry for a maximum of 5 minutes, with an exponential
backoff strategy.  For more complicated uses, the user can use tenacity themselves to create
their own custom decorator.Usage:fromapiclientimportretry_requestclassMyClient(APIClient):@retry_requestdefretry_enabled_method():...For more complex use cases, you can build your own retry decorator using
tenacity along with the custom retry strategy.For example, you can build a retry decorator that retriesAPIRequestErrorwhich waits for 2 seconds between retries and gives up after 5 attempts.importtenacityfromapiclient.retryingimportretry_if_api_request_errorretry_decorator=tenacity.retry(retry=retry_if_api_request_error(),wait=tenacity.wait_fixed(2),stop=tenacity.stop_after_attempt(5),reraise=True,)Or you can build a decorator that will retry only on specific status
codes (following a failure).retry_decorator=tenacity.retry(retry=retry_if_api_request_error(status_codes=[500,501,503]),wait=tenacity.wait_fixed(2),stop=tenacity.stop_after_attempt(5),reraise=True,)PaginationIn order to support contacting pages that respond with multiple pages of data when making get requests,
add a@paginateddecorator to your client method.@paginatedcan paginate the requests either where
the pages are specified in the query parameters, or by modifying the url.Usage is simple in both cases; paginator decorators take a Callable with two required arguments:by_query_params-> callable takesresponseandprevious_page_params.by_url-> callable takesresponesandprevious_page_url.
The callable will need to return either the params in the case ofby_query_params, or a new url in the
case ofby_url.
If the response is the last page, the function should return None.Usage:fromapiclientimportpaginateddefnext_page_by_params(response,previous_page_params):# Function reads the response data and returns the query param# that tells the next request to go to.return{"next":response["pages"]["next"]}defnext_page_by_url(response,previous_page_url):# Function reads the response and returns the url as string# where the next page of data lives.returnresponse["pages"]["next"]["url"]classMyClient(APIClient):@paginated(by_query_params=next_page_by_params)defpaginated_example_one():...@paginated(by_url=next_page_by_url)defpaginated_example_two():...Authentication MethodsAuthentication methods provide a way in which you can customize the
client with various authentication schemes through dependency injection,
meaning you can change the behaviour of the client without changing the
underlying implementation.The apiclient supports the following authentication methods, by specifying
the initialized class on initialization of the client, as follows:client=ClientImplementation(authentication_method=<AuthenticationMethodClass>(),response_handler=...,request_formatter=...,)NoAuthenticationThis authentication method simply does not add anything to the client,
allowing the api to contact APIs that do not enforce any authentication.Example:client=ClientImplementation(authentication_method=NoAuthentication(),response_handler=...,request_formatter=...,)QueryParameterAuthenticationThis authentication method adds the relevant parameter and token to the
client query parameters.  Usage is as follows:client=ClientImplementation(authentication_method=QueryParameterAuthentication(parameter="apikey",token="secret_token"),response_handler=...,request_formatter=...,)Example. Contacting a url with the following datahttp://api.example.com/users?age=27Will add the authentication parameters to the outgoing request:http://api.example.com/users?age=27&apikey=secret_tokenHeaderAuthenticationThis authentication method adds the relevant authorization header to
the outgoing request.  Usage is as follows:client=ClientImplementation(authentication_method=HeaderAuthentication(token="secret_value"),response_handler=...,request_formatter=...,)# Constructs request header:{"Authorization":"Bearer secret_value"}TheAuthorizationparameter andBearerscheme can be adjusted by
specifying on method initialization.authentication_method=HeaderAuthentication(token="secret_value"parameter="apikey",scheme="Token",)# Constructs request header:{"apikey":"Token secret_value"}Or alternatively, when APIs do not require a scheme to be set, you can
specify it as a value that evaluates to False to remove the scheme from
the header:authentication_method=HeaderAuthentication(token="secret_value"parameter="token",scheme=None,)# Constructs request header:{"token":"secret_value"}Additional header values can be passed in as a dict here when API's require more than one
header to authenticate:authentication_method=HeaderAuthentication(token="secret_value"parameter="token",scheme=None,extra={"more":"another_secret"})# Constructs request header:{"token":"secret_value","more":"another_secret"}BasicAuthenticationThis authentication method enables specifying a username and password to APIs
that require such.client=ClientImplementation(authentication_method=BasicAuthentication(username="foo",password="secret_value"),response_handler=...,request_formatter=...,)CookieAuthenticationThis authentication method allows a user to specify a url which is used
to authenticate an initial request, made at APIClient initialization,
with the authorization tokens then persisted for the duration of the
client instance in cookie storage.These cookies use thehttp.cookiejar.CookieJar()and are set on the
session so that all future requests contain these cookies.As the method of authentication at the endpoint is not standardised
across API's, the authentication method can be customized using one of
the already defined authentication methods;QueryParameterAuthentication,HeaderAuthentication,BasicAuthentication.client=ClientImplementation(authentication_method=(CookieAuthentication(auth_url="https://example.com/authenticate",authentication=HeaderAuthentication("1234-secret-key"),),response_handler=...,request_formatter=...,)Response HandlersResponse handlers provide a standard way of handling the final response
following a successful request to the API.  These must inherit fromBaseResponseHandlerand implement theget_request_data()method which
will take therequests.Responseobject and parse the data accordingly.The apiclient supports the following response handlers, by specifying
the class on initialization of the client as follows:The response handler can be omitted, in which case no formatting is applied to the
outgoing data.client=ClientImplementation(authentication_method=...,response_handler=<ResponseHandlerClass>,request_formatter=...,)RequestsResponseHandlerHandler that simply returns the originalResponseobject with no
alteration.Example:client=ClientImplementation(authentication_method=...,response_handler=RequestsResponseHandler,request_formatter=...,)JsonResponseHandlerHandler that parses the response data tojsonand returns the dictionary.
If an error occurs trying to parse to json then aUnexpectedErrorwill be raised.Example:client=ClientImplementation(authentication_method=...,response_handler=JsonResponseHandler,request_formatter=...,)XmlResponseHandlerHandler that parses the response data to anxml.etree.ElementTree.Element.
If an error occurs trying to parse to xml then aUnexpectedErrorwill be raised.Example:client=ClientImplementation(authentication_method=...,response_handler=XmlResponseHandler,request_formatter=...,)Request FormattersRequest formatters provide a way in which the outgoing request data can
be encoded before being sent, and to set the headers appropriately.These must inherit fromBaseRequestFormatterand implement theformat()method which will take the outgoingdataobject and format accordingly
before making the request.The apiclient supports the following request formatters, by specifying
the class on initialization of the client as follows:client=ClientImplementation(authentication_method=...,response_handler=...,request_formatter=<RequestFormatterClass>,)JsonRequestFormatterFormatter that converts the data into a json format and adds theapplication/jsonContent-type header to the outgoing requests.Example:client=ClientImplementation(authentication_method=...,response_handler=...,request_formatter=JsonRequestFormatter,)ExceptionsThe exception handling forapi-clienthas been designed in a way so that all exceptions inherit from
one base exception type:APIClientError.  From there, the exceptions have been broken down into the
following categories:ResponseParseErrorSomething went wrong when trying to parse the successful response into the defined format.  This could be due
to a misuse of the ResponseHandler, i.e. configuring the client with anXmlResponseHandlerinstead of
aJsonResponseHandlerAPIRequestErrorSomething went wrong when making the request.  These are broken down further into the following categories to provide
greater granularity and control.RedirectionErrorA redirection status code (3xx) was returned as a final code when making the
request. This means that no data can be returned to the client as we could
not find the requested resource as it had moved.ClientErrorA clienterror status code (4xx) was returned when contacting the API. The most common cause of
these errors is misuse of the client, i.e. sending bad data to the API.ServerErrorThe API was unreachable when making the request.  I.e. a 5xx status code.UnexpectedErrorAn unexpected error occurred when using the client.  This will typically happen when attempting
to make the request, for example, the client never receives a response.  It can also occur to
unexpected status codes (>= 600).Custom Error HandlingError handlers allow you to customize the way request errors are handled in the application.Create a new error handler, extendingBaseErrorHandlerand implement theget_exceptionstatic method.Pass the custom error handler into your client upon initialization.Example:fromapiclient.error_handlersimportBaseErrorHandlerfromapiclientimportexceptionsfromapiclient.responseimportResponseclassMyErrorHandler(BaseErrorHandler):@staticmethoddefget_exception(response:Response)->exceptions.APIRequestError:"""Parses client errors to extract bad request reasons."""if400<=response.get_status_code()<500:json=response.get_json()returnexceptions.ClientError(json["error"]["reason"])returnexceptions.APIRequestError("something went wrong")In the above example, you will notice that we are utilising an internalResponseobject. This has been designed to abstract away the underlying response
returned from whatever strategy that you are using. TheResponsecontains the following
methods:get_original: returns the underlying response object. This has been implemented
for convenience and shouldn't be relied on.get_status_code: returns the integer status code.get_raw_data: returns the textual data from the response.get_json: should return the json from the response.get_status_reason: returns the reason for any HTTP error code.get_requested_url: returns the url that the client was requesting.Request StrategyThe design of the client provides a stub of a client, exposing the required methods;get,post, etc. And this then calls the implemented methods of a request strategy.This allows us to swap in/out strategies when needed. I.e. you can write your own
strategy that implements a different library (e.g.urllib). Or you could pass in a
mock strategy for testing purposes.Example strategy for testing:fromunittest.mockimportMockfromapiclientimportAPIClientfromapiclient.request_strategiesimportBaseRequestStrategydeftest_get_method():"""test that the get method is called on the underlying strategy.This does not execute any external HTTP call."""mock_strategy=Mock(spec=BaseRequestStrategy)client=APIClient(request_strategy=mock_strategy)client.get("http://google.com")mock_strategy.get.assert_called_with("http://google.com",params=None)EndpointsThe apiclient also provides a convenient way of defining url endpoints with
use of the@endpointdecorator.  In order to decorate a class with@endpointthe decorated class must define abase_urlattribute along with the required
resources.  The decorator will combine the base_url with the resource.Example:fromapiclientimportendpoint@endpoint(base_url="http://foo.com")classEndpoint:resource="search">>>Endpoint.resource"http://foo.com/search"ExtensionsMarshalling JSONapi-client-jsonmarshal: automatically
marshal to/from JSON into plain python dataclasses. Full usage examples can be found in the extensions home page.Pydanticapi-client-pydantic: validate request data and converting json straight
to pydantic class.Extended Examplefromapiclientimport(APIClient,endpoint,paginated,retry_request,HeaderAuthentication,JsonResponseHandler,JsonRequestFormatter,)fromapiclient.exceptionsimportAPIClientError# Define endpoints, using the provided decorator.@endpoint(base_url="https://jsonplaceholder.typicode.com")classEndpoint:todos="todos"todo="todos/{id}"defget_next_page(response):return{"limit":response["limit"],"offset":response["offset"]+response["limit"],}# Extend the client for your API integration.classJSONPlaceholderClient(APIClient):@paginated(by_query_params=get_next_page)defget_all_todos(self)->dict:returnself.get(Endpoint.todos)@retry_requestdefget_todo(self,todo_id:int)->dict:url=Endpoint.todo.format(id=todo_id)returnself.get(url)# Initialize the client with the correct authentication method,# response handler and request formatter.>>>client=JSONPlaceholderClient(authentication_method=HeaderAuthentication(token="<secret_value>"),response_handler=JsonResponseHandler,request_formatter=JsonRequestFormatter,)# Call the client methods.>>>client.get_all_todos()[{'userId':1,'id':1,'title':'delectus aut autem','completed':False},...,{'userId':10,'id':200,'title':'ipsam aperiam voluptates qui','completed':False}]>>>client.get_todo(45){'userId':3,'id':45,'title':'velit soluta adipisci molestias reiciendis harum','completed':False}# REST APIs correctly adhering to the status codes to provide meaningful# responses will raise the appropriate exeptions.>>>client.get_todo(450)# NotFound: 404 Error: Not Found for url: https://jsonplaceholder.typicode.com/todos/450>>>try:...client.get_todo(450)...exceptAPIClientError:...print("All client exceptions inherit from APIClientError")"All client exceptions inherit from APIClientError"APIClient InterfaceTheAPIClientprovides the following public interface:post(self, endpoint: str, data: dict, params: OptionalDict = None)Delegate to POST method to send data and return response from endpoint.get(endpoint: str, params: OptionalDict = None)Delegate to GET method to get response from endpoint.put(endpoint: str, data: dict, params: OptionalDict = None)Delegate to PUT method to send and overwrite data and return response from endpoint.patch(endpoint: str, data: dict, params: OptionalDict = None)Delegate to PATCH method to send and update data and return response from endpointdelete(endpoint: str, params: OptionalDict = None)Delegate to DELETE method to remove resource located at endpoint.get_request_timeout() -> floatBy default, all requests have been set to have a default timeout of 10.0 s.  This
is to avoid the request waiting forever for a response, and is recommended
to always be set to a value in production applications.  It is however possible to
override this method to return the timeout required by your application.MentionsMany thanks toJetBrainsfor supplying me with a license to use their product in the development
of this tool.