Package: keboola.http-client
Python HTTP ClientIntroductionThis library serves as tool to work effectively when sending requests to external services. The library wraps on top of therequestslibrary and implements a couple useful method, such as in-built retry, exception raising, etc.It is being developed by the Keboola Data Services team and officially supported by Keboola. It aims to simplify the Keboola Component creation process, by removing the necessity to write complicated code to work with the APIs effectively.LinksAPI Documentation:API DocsSource code:https://github.com/keboola/python-http-clientPYPI project code:https://test.pypi.org/project/keboola.http-clientDocumentation:https://developers.keboola.com/extend/component/python-component-libraryQuick StartInstallationThe package may be installed via PIP:pip install keboola.http-clientStructure and FunctionalityThe package contains a single core module:keboola.http_client- Contains theHttpClientclass for easy manipulation with APIs and external servicesHttpClientThe core class that serves as a tool to communicate with external services. The class is a wrapper around therequestslibrary with implemented retry mechanism, and automatic error handling in case of HTTP error returned.For each HTTP method, following methods are implemented in theHttpClient:GET -get(),get_raw()POST -post(),post_raw()PATCH -patch(),patch_raw()UPDATE -update(),update_raw()PUT -put(),put_raw()DELETE -delete(),delete_raw()The difference between_raw()methods and their non-_raw()counterparts is, that raw methods will returnrequests.Responseobject, while non-raw methods will return a json body if the request is successful and raise an error if an HTTP error is encountered.All abovementioned methods support all parameters supported byrequests.request()functions - as described in thedocumentation.InitializationThe core class iskeboola.http_client.HttpClient, which can be initialized by specifying thebase_urlparameter:fromkeboola.http_clientimportHttpClientBASE_URL='https://connection.keboola.com/v2/storage/'cl=HttpClient(BASE_URL)Default argumentsForHttpClient, it is possible to define default arguments, which will be sent with every request. It's possible to definedefault_http_header,auth_headeranddefault_params- a default header, a default authentication header and default parameters, respectively.fromkeboola.http_clientimportHttpClientBASE_URL='https://connection.keboola.com/v2/storage/'AUTH_HEADER={'x-storageapi-token':'1234-STORAGETOKENSTRING'}DEFAULT_PARAMS={'include':'columns'}DEFAULT_HEADER={'Content-Type':'application/json'}cl=HttpClient(BASE_URL,default_http_header=DEFAULT_HEADER,auth_header=AUTH_HEADER,default_params=DEFAULT_PARAMS)Basic authenticationBy specifying theauthargument, theHttpClientwill utilize the basic authentication.fromkeboola.http_clientimportHttpClientBASE_URL='https://connection.keboola.com/v2/storage/'USERNAME='TestUser'PASSWORD='@bcd1234'cl=HttpClient(BASE_URL,auth=(USERNAME,PASSWORD))Simple POST requestMaking a simple POST request usingpost_raw()method.fromkeboola.http_clientimportHttpClientBASE_URL='https://www.example.com/change'cl=HttpClient(BASE_URL)data={'attr_1':'value_1','attr_2':'value_2'}header={'content-type':'application/json'}response=cl.post_raw(data=data,headers=header)ifresponse.okisnotTrue:raiseValueError(response.json())else:print(response.json())Making a simple POST request usingpost()method.fromkeboola.http_clientimportHttpClientBASE_URL='https://www.example.com/change'cl=HttpClient(BASE_URL)data={'attr_1':'value_1','attr_2':'value_2'}header={'content-type':'application/json'}response=cl.post(data=data,headers=header)Working with URL pathsEach of the methods takes an optional positional argumentendpoint_path. If specified, the value of theendpoint_pathwill be appended to the URL specified in thebase_urlparameter, when initializing the class. When appending theendpoint_path, theurllib.parse.urljoin()function is used.The below code will send a POST request to the URLhttps://example.com/api/v1/events:fromkeboola.http_clientimportHttpClientBASE_URL='https://example.com/api/v1'cl=HttpClient(BASE_URL)header={'token':'token_value'}cl.post_raw('events',headers=header)It is also possible to override this behavior by using parameteris_absolute_path=True. If specified, the value ofendpoint_pathwill not be appended to thebase_urlparameter, but will rather be used as an absolute URL to which the HTTP request will be made.In the below code, thebase_urlparameter is set tohttps://example.com/api/v1, but the base URL will be overriden by specifyingis_absolute_path=Trueand the HTTP request will be made to the URL specified in thepost()request -https://anothersite.com/v2.fromkeboola.http_clientimportHttpClientBASE_URL='https://example.com/api/v1'cl=HttpClient(BASE_URL)header={'token':'token_value'}cl.post_raw('https://anothersite.com/v2',headers=header,is_absolute_path=True)Raw request ExampleA simple request made with default authentication header and parameters.importosfromkeboola.http_clientimportHttpClientBASE_URL='https://connection.keboola.com/v2/'TOKEN=os.environ['TOKEN']cl=HttpClient(BASE_URL,auth_header={'x-storageapi-token':TOKEN})request_params={'exclude':'components'}response=cl.get_raw('storage',params=request_params)ifresponse.okisTrue:print(response.json())Building HTTP client based on HTTPClient ExampleThis example demonstrates the default use of the HTTPClient as a base for REST API clients.fromkeboola.http_clientimportHttpClientBASE_URL='https://connection.eu-central-1.keboola.com/v2/storage'MAX_RETRIES=10classKBCStorageClient(HttpClient):def__init__(self,storage_token):HttpClient.__init__(self,base_url=BASE_URL,max_retries=MAX_RETRIES,backoff_factor=0.3,status_forcelist=(429,500,502,504),auth_header={"X-StorageApi-Token":storage_token})defget_files(self,show_expired=False):params={"showExpired":show_expired}returnself.get('files',params=params)cl=KBCStorageClient("my_token")print(cl.get_files())