Package: popget
A simple no-bells-and-whistles REST-API client, optionally supporting async requests.We use this for service-to-service requests in our heterogenous
microservices environment.UsagepipinstallpopgetConfigurationSettings are intended to be configured primarily via a python file, such
as your existing Djangosettings.py.
To bootstrap this, there are a couple of env vars to control how config
is loaded:POPGET_APP_CONFIGshould be an import path to a python module, for example:POPGET_APP_CONFIG=django.conf.settingsPOPGET_CONFIG_NAMESPACESets the prefix used for loading further config values from env and
config file. Defaults toPOPGET.See source ofpopget/conf/defaults.pyfor more details.Some useful config keys (all of which are prefixed withPOPGET_by default):<namespace>_CLIENT_DEFAULT_USER_AGENTwhen making requests, popget will use this
string as the user agent.<namespace>_CLIENT_TIMEOUTifNonethen no timeout, otherwise this timeout
(in seconds) will be applied to all requests. Requests which timeout will
return a 504 response, which will be raised as anHTTPError.APIClientYou will sub-classAPIClientto make your API. You do not need to
instantiate the client, all methods are class-methods.egfrompopgetimportAPIClient,Arg,GetEndpointclassThingServiceClient(APIClient):classConfig:base_url='http://things.depop.com'get_things=GetEndpoint('/things/{user_id}/',# url format stringquerystring_args=(Arg('type',required=True),# required querystring param (validated on call)))Results in a client method you can call like:data=ThingServiceClient.get_things(user_id=2345,type='cat')Which will perform a request like:GEThttp://things.depop.com/things/2345/?type=catIf response was"Content-Type:application/json"thendatais
already deserialized.Under Python 3 there is a further distinction betweenstrandbytes.
If the Content-Type header containstext/then the returned value
will be encoded tostr(by underlyingpython-requestslib).
Other content types will returnbytes.We useraise_for_statusso anything >= 400 will raise arequests.HTTPError.APIEndpointAPIEndpointis the base class for endpoint methods.GetEndpoint,PostEndpoint,PutEndpoint,PatchEndpointandDeleteEndpointare provided for convenience, allowing to omit the method arg.Params from url path (format string), querystring and request headers
(format string of value portion) will be extracted and made available
as kwargs on the resulting callable method on your client class.This means arg names must be unique across all three sources of args.
This is feasible because path and header args can be freely chosen when
implementing the client (they are just format string identifiers rather
than part of the REST API itself like querystring args are).e.g.frompopgetimportAPIClient,Arg,GetEndpointclassThingServiceClient(APIClient):classConfig:base_url='http://things.depop.com'get_things=GetEndpoint('/things/{user_id}/',# url (format string)querystring_args=(Arg('type',required=True),Arg('offset_id'),Arg('limit',default=25),),request_headers={# added to all requests'Authorization':'Bearer{access_token}'# (format string)})This will give you a client with aget_thingsmethod you can call like:response_data=ThingServiceClient.get_things(user_id=123,type='cat',offset_id='65345ff34e344ab53c',limit=20,access_token='87a64c98b62d39e8625f',)Querystring args can have a callable as the default value, e.g.:fromdatetimeimportdatetimefrompopgetimportAPIClient,Arg,GetEndpointdefnow():returndatetime.now().isoformat()classThingServiceClient(APIClient):classConfig:base_url='http://things.depop.com'get_things=GetEndpoint('/things/{user_id}/',# url (format string)querystring_args=(Arg('since',default=now),),request_headers={# added to all requests'Authorization':'Bearer{access_token}'# (format string)})response_data=ThingServiceClient.get_things(user_id=123)# GET http://things.depop.com/things/123/?since=2018-02-09T13:31:10.569481You can still pass extra args down into therequestslib on a per-call
basis by using_request_kwargs:response_data=ThingServiceClient.get_things(user_id=123,type='cat',offset_id='65345ff34e344ab53c',limit=20,access_token='87a64c98b62d39e8625f',_request_kwargs={'headers':{'X-Depop-WTF':'something something'}},)And for calls with a request body:frompopgetimportAPIClient,PostEndpoint,BodyTypeclassThingServiceClient(APIClient):classConfig:base_url='http://things.depop.com'new_thing=PostEndpoint('/things/',body_required=True,body_type=BodyType.FORM_ENCODED,# or BodyType.JSON - sets content-type headerrequest_headers={'Authorization':'Bearer{access_token}',})response_data=ThingServiceClient.new_thing(access_token='87a64c98b62d39e8625f',body={'type':'dog','name':'fido',})You can also pass a customrequests.Sessioninstance on a per-request basis using the_sessionkwarg:fromdjango.confimportsettingsfromrequests_oauthlibimportOAuth1Sessionfrommyproject.twitter.clientimportTwitterClientdeftweet_from(user,message,**extra):oauth_session=OAuth1Session(settings.TWITTER_CONSUMER_KEY,client_secret=settings.TWITTER_CONSUMER_SECRET,resource_owner_key=user.tw_access_token,resource_owner_secret=user.tw_access_token_secret,)body={'status':message,}body.update(extra)returnTwitterClient.update_status(body=body,_session=oauth_session)AsynchronousOptional support for asynchronous requests is provided, via arequests-futuresbackend.pip install popget[threadpool]An async variant of theAPIClientis provided which will have both async and blocking
versions of all endpoint methods.frompopgetimportArg,GetEndpointfrompopget.async.threadpoolimportAPIClientimportrequestsclassThingServiceClient(APIClient):classConfig:base_url='http://things.depop.com'get_things=GetEndpoint('/things/{user_id}/',# url format stringquerystring_args=(Arg('type',required=True),# required querystring param (validated on call)),)# blocking:data=ThingServiceClient.get_things(user_id=2345,type='cat')# async:future=ThingServiceClient.async_get_things(user_id=2345,type='cat')# response is parsed and may raise, as for blocking requeststry:data=future.result()exceptrequests.exceptions.HTTPErrorase:print(repr(e))The async endpoint methods will return a standardconcurrent.futures.Futureobject.SeePython docs.You can customise the name of the generated async endpoint methods:classThingServiceClient(APIClient):classConfig:base_url='http://things.depop.com'async_method_template='{}__async'get_things=GetEndpoint('/things/{user_id}/',# url format stringquerystring_args=(Arg('type',required=True),# required querystring param (validated on call)),)future=ThingServiceClient.get_things__async(user_id=2345,type='cat')CompatibilityThis project is tested against:Python 2.7Python 3.7Running the testsCircleCIThe easiest way to test the full version matrix is to install the
CircleCI command line app:https://circleci.com/docs/2.0/local-jobs/(requires Docker)The cli does not support ‘workflows’ at the moment so you have to run
the two Python version jobs separately:circlecibuild--jobpython-2.7circlecibuild--jobpython-3.7py.test (single python version)It’s also possible to run the tests locally, allowing for debugging of
errors that occur.Decide which Python version you want to test and create a virtualenv:pyenvvirtualenv3.7.4popgetpipinstall-rrequirements-test.txtpy.test-v-s--ipdbtests/