Package: sequoia-client-sdk
Python Sequoia Client SDKA Python Client SDK for interacting with client services.The central idea is that Client SDK allows python application code to communicate with thePiksel PaletteRESTful RESTful services.
Users can also search, filter and select their response collections.Installationpipinstallsequoia-client-sdkUsageCreating a SequoiaClientTo create the client it is necessary to provide the url for the serviceregistryand named arguments specifying the
credentials for the auth_type being used. If no auth_type is specified, then the default CLIENT_GRANT is used:client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",grant_client_id="clientId",grant_client_secret="clientSecret")Authentication typesWhen creating the client, authentication type can be specified using the parameterauth_type:client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",auth_type=AuthType.CLIENT_GRANT,grant_client_id="clientId",grant_client_secret="clientSecret")The Sequoia RESTful services have an OAuth token-based authorisation model, meaning that the Client SDK must first
acquire a time-limited access token before making further requests. CLIENT_GRANT or BYO_TOKEN types should be used.It is also possible to connect to the client via a proxy using two-way TLS authentication. In this case, MUTUAL
auth_type should be used.There are four authentication types:CLIENT_GRANT typeThis is the default type. With CLIENT_GRANT modegrant_client_idandgrant_client_secretparameters are
used to get an access token. The access token is refreshed automatically when expired. Optionally,byo_tokenparameter can be provided when instantiating the client, and will be used until it is expired.
Then the access token is refreshed automatically.client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",auth_type=AuthType.CLIENT_GRANT,grant_client_id="clientId",grant_client_secret="clientSecret")BYO_TOKEN typeWith this methodbyo_tokenis required. That access token will be used to authenticate requests. The access token will
be used along the client life and won’t be refreshed.NO_AUTH typeMode used when no authentication is required.MUTUAL typeMode used when mutual TLS authentication is required. Paths to local client certificate, client key and a server
certificate files must be provided in the client_cert, client_key and server_cert arguments respectively.client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",auth_type=AuthType.MUTUAL,client_cert="/certs/client_cert.pem",client_key="/certs/client_key.pem",server_cert="/certs/server_cert.pem",...Content TypeBy default the client sets “Content-Type” and “Accept’ header values of http requests to  “application/vnd.piksel+json”.
A different content type for these headers can be specified in the content_type parameter when creating a client.client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",auth_type=AuthType.MUTUAL,client_cert="/certs/client_cert.pem",client_key="/certs/client_key.pem",server_cert="/certs/server_cert.pem",content_type="application/json")Creating an endpointAn endpoint defines the resource on which to perform the operations.profile_endpoint=client.workflow.profilescontent_endpoint=client.metadata.contentsAPI methodsReadRetrieves one resource given its reference and owner and returns the response retrieved.endpoint.read(owner,ref)BrowseRetrieves the list of resources that matches with the criteria and returns the response.endpoint.browse(owner,criteria)StoreCreates one or more resources and returns the response retrieved.endpoint.store(owner,json)Criteria API for Requesting DataThe SDK supports a fluent criteria API to abstract client code from
the details of the Sequoia query syntax.
This API allows to provide filters to retrieve the queried data and a way to request for related resources and its fields:CriterionThe way to provide the filter to get specific data is by using the criterion this way.endpoint.browse("testmock",Criteria().add_criterion(StringExpressionFactory.field("contentRef").equal_to("testmock:sampleContent")))This alternative way is also supported:endpoint.browse("testmock",Criteria().add(criterion=StringExpressionFactory.field("contentRef").equal_to("testmock:sampleContent")))The following filtering criteria are supported:equalToStringExpressionFactory.field("engine").equal_to("diesel")Will generate the criteria expression equivalent to: field=diesel (withEngine=diesel)Inclusion of related documentsThe SDK support inclusion of related documents up to 1 level (direct relationships).Both, direct and indirect relationships, are allowed. In each case resource’sreferenceare needed to perform the mapping.Criteria().add_inclusion(Inclusion.resource('assets'))This alternative way is also supported:Criteria().add(inclusion=Inclusion.resource('assets'))Selecting fieldsThe SDK allows to specify which fields will be present in the response, discarding the rest of them.For now it can be used only for InclusionsCriteria().add(inclusion=Inclusion.resource('assets').fields('name','ref'))Paginating resultsIteratorBrowse responses can be paginated. To paginate results, browse response has to be used as an iterator.forresponseinendpoint.browse('testmock'):resources=response.resourcesNot iteratorIf browse function is not used as an iterator, only first page is retrieved. i.e:response=endpoint.browse('testmock')resources_in_page_1=response.resourcesWith continueSequoia services allow to paginate using the parametercontinue, which will return the link to get the following page in themetaof the response.
Thebrowsecan be call repeatedly while there are pages to be read.
Optionally, you can set the number of items per page.forresponseinendpoint.browse('testmock',query_string='continue=true&perPage=2'):resources=response.resourcesPaginating linked resourcesInclusionWhen doing an inclusion, service returns a list of linked resources. Those resources can be paginated. Let’s assume a browse of contents is performed with assets resource as an inclusion. To perform pagination:forlinked_assetsinendpoint.browse('testmock').linked('assets'):forlinked_assetinlinked_assets:asset_name=linked_asset['name']If linked response is not used as an iterator, only first page of linked resources is retrieved:linked_assets=endpoint.browse('testmock').linked('assets')forlinked_assetinlinked_assets.resources:asset_name=linked_asset['name']Retrying requestsWhen a request is returning a retrievable status code, a retry strategy can be configured withbackoff_strategy. By defaultbackoff_strategyis{'wait_gen':backoff.constant,'interval':0,'max_tries':10}We can set a different backoff strategy.client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",grant_client_id="clientId",grant_client_secret="clientSecret",backoff_strategy={'wait_gen':backoff.expo,'max_tries':5,'max_time':300})Here an exponential strategy will be used, set in the constructor.For a more fine grain customization the retry policy can also be set for specific requests by passing the parameter
at method level. This is valid forread,browse,getandrequestmethods.
The backoff strategy configured in the constructor will be used for all the other queries in which another backoff
strategy is not given.
Here you can see an example:client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",grant_client_id="clientId",grant_client_secret="clientSecret",backoff_strategy={'wait_gen':backoff.expo,'max_tries':5,'max_time':300})assets_endpoint=client.metadata.contentsresponse1=assets_endpoint.browse(self.owner,criteria.Criteria().add_criterion(criteria.StringExpressionFactory.field('ref').equal_to('test:c0007')).add_inclusion(criteria.Inclusion.resource('assets')),backoff_strategy={'wait_gen':backoff.constant,'interval':0,'max_tries':20})response2=assets_endpoint.browse(self.owner,criteria.Criteria().add_criterion(criteria.StringExpressionFactory.field('ref').equal_to('test:c02')).add_inclusion(criteria.Inclusion.resource('categories')))For that example, the first browse will retry the query up to 20 times while the second browse will do up to 5
or until the the max time is reached.Retry when status codeYou can also provide a number of HTTP status codes to perform the retry of the query, this is, when the query you are
performing returns one of the status codes you’ve specified, the query is automatically retried.
The key word you have to use for this isretry_http_status_codeswithin the backoff_strategy dictionary.For instance:client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",grant_client_id="clientId",grant_client_secret="clientSecret",backoff_strategy={'wait_gen':backoff.expo,'max_tries':5,'max_time':300,'retry_http_status_codes':[404,409]})Whenmax_timeis set to None or not passed, a default value is automatically set to avoid possible undesired behaviour
such as infinite loops. The default value is set to 120 seconds.For more info about backoff strategieshttps://github.com/litl/backoffRetry when empty resultYou can also set up the retries policy for the case in that the resources you are querying for are missing in the
response. This is useful when you are quite sure the data you are querying will eventually exist in the service even
though it doesn’t exist yet.The way to configure this is by using the parameterretry_when_empty_resultin the method you use to query the
service, this is valid forread,browse,getandrequestmethods.The parameterretry_when_empty_resultaccepts either aboolean valueto specify all resources are expected,
either they are the main resources or the linked ones, or adictionaryin which you can explicitly specify the
type of resources you are expecting to have in the response.
In cases these resources are missing the query will be retried.Let’s see an example:client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",grant_client_id="clientId",grant_client_secret="clientSecret",backoff_strategy={'retry_when_empty_result':True})assets_endpoint=client.metadata.contentsresponse=assets_endpoint.browse(self.owner,criteria.Criteria().add_criterion(criteria.StringExpressionFactory.field('ref').equal_to('test:c0007')).add_inclusion(criteria.Inclusion.resource('categories')).add_inclusion(criteria.Inclusion.resource('assets')),retry_when_empty_result=True)This way you are asking to retry the query when the response has no data for the main resource and for the inclusions
you are querying for.This is, if your query look likehttps://metadata-sandbox.sequoia.piksel.com/data/contents?include=assets,categories&owner=test&withRef=test:c0007the query will be retried until the content test:c0007 is returned and it has at least one asset and
one category in the response too. Or the retries reach the limit.A finer configuration using a dictionary is allowed so you can specify which resources have to be checked this way:client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",grant_client_id="clientId",grant_client_secret="clientSecret")assets_endpoint=client.metadata.contentsresponse=assets_endpoint.browse(self.owner,criteria.Criteria().add_criterion(criteria.StringExpressionFactory.field('ref').equal_to('test:c0007')).add_inclusion(criteria.Inclusion.resource('categories')).add_inclusion(criteria.Inclusion.resource('assets')),retry_when_empty_result={'contents':True,'assets':False,'categories':True})In that example both resources contents and categories are checked to be returned, but not assets.In case the limit of retries is reached and that condition is not fulfilled the latest response is returned.
Bear in mind that the response can very likely have a status code of 200 and a body with data.Remember, as specified above amax_timeis automatically set even though it is not given.Correlation IDEvery request to Sequoia RESTful services is added with a unique correlation id in the headers.--requestheaders--...x-correlation-id:f0fca55f3da85..6336cb20fda36...The SDK allows to set a correlation id at the client to be added to all the subsequent requests.client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",...correlation_id="custom_correlation_id_1234",...)endpoint.browse(owner,criteria)--requestheaders--...x-correlation-id:custom_correlation_id_1234...It also allows to provide both an user and an application ids so each operation request will be set with
an unique generated correlation id having these values as prefix.
This correlation id will be shared by all related requests derived by that operation: browse, store, etc
(e.g. the subsequents paging requests in a browse operation).Both parametersuser_idandapplication_idhas to be provided, providing just one
you won’t have a prefix in the correlation id.client=Client("https://registry-sandbox.sequoia.piksel.com/services/testmock",...user_id="user123",application_id="app101",...)endpoint.browse(owner,criteria)--requestheaders--...x-correlation-id:user123/app101/cbd05bd7-3099-4dcb-aeff-806ccec3292a...endpoint.browse(owner,criteria)--requestheaders--...x-correlation-id:user123/app101/9becd6c7-8ef0-44c4-a240-6c02c583957f...The parametercorrelation_idhas precedence overuser_idandapplication_id.DevelopmentIt has been tested for Python 3.5 and 3.6You can use the included command line toolmaketo work with this projectPreparing environmentCreate new virtualenvIt’s encouraging to create a new virtual environment and install all the dependencies in it.
You can use these commands:mkdir-p~/.virtualenvsvirtualenv-ppython3.6~/.virtualenvs/sequoia-python-client-sdkworkonsequoia-python-client-sdkpipinstall-rrequirements.txtpipinstall-rrequirements_test.txtTestingThere are two different ways of running the tests.Run tests on the current environmentUsingpytestoption will run all the unit tests over your environment.maketestRun tests on every compatible python versionWhile using the optiontestwill set up a virtual environment for the supported version of Python, i.e. 3.5 and 3.6 and will run all the tests on each of them.maketest-allIf you are usingpyenvand found issues running this command because tox isn’t able to create the virtualenvs, just add the python versions you have installed to the file.python-versionlike this:echo"3.6.9">>.python-versionecho"3.7.7">>.python-versionecho"3.8.3">>.python-versionLintTo make sure the code fulfills the format runmakelintHistory1.0.0First release.1.1.0 (2017-10-25)Upgrade to Python 3.61.2.0 (2019-03-06)Librariesurllib3andrequestsupgraded to solve security issues:CVE-2018-20060CVE-2018-180741.2.1 (2019-03-26)Load yaml config file for testing in a safer way as specified inPyYAML.2.0.0 (2019-06-06)Removing python 2.7 compatibility.Adding backoff to http requests. Configurable backoff from client creation.Librariesurllib3andrequestsupgraded to solve security issues.2.1.0 (2019-09-30)Modifying setup.cfg to allow different version formats (i.e development versions).Paging withcontinueparameter.When token is expired, it is updated automatically with CLIENT_GRANT auth type.2.1.1 (2019-10-02)Token fetching not restarting backoff. Retries continuing its count instead of restarting it when there is a invalid token.2.2.0 (2020-08-13)Allowing to providecorrelation_idvalue when the client is created.Caching tokens bygrant_client_idandtoken_urlto avoid calling identity in case credentials are cached.PageBrowser keeping a response cache to avoid duplicated requests.Bug fixed when paging main content. Query params should to be added to next url.NewAuthType.MUTUAL.3.0.0 (2020-10-06)Removingtransaction_idvalue when the client is created.Allowing to provideuser_idandapplication_idvalues as correlation id prefix.4.0.0 (2020-10-21)Python 3.5 support removed.Python 3.7 supported.Python 3.8 supported.Pagination withcontinueparameter over linked resources supported.Requirements upgraded.4.0.1 (2020-12-22)When token is expired, it is updated automatically with CLIENT_GRANT auth type,the 401 response wasn’t managed to do so, only the exception was.
Now the 401 response is treated like that.4.0.2 (2021-03-04)Two new methods added to Criterion object so the fluent API is easier to use:add_inclusionandadd_criterion.4.1.0 (2021-04-13)New keywordretry_http_status_codesfor thebackoff_strategyto retry specific http status codes.Prospector version upgraded to 1.3.1 so it works with python versions 3.9.4, 3.8.9, 3.7.10, 3.6.13.Lint issues solved (OAuth2SessionTokenManagementWrapper request method signature).GitHub Actions configured to run lint and unit tests.4.2.0 (2021-04-13)Python 3.9 supported.Requirements upgraded.Drop the use of some libraries: jsonpickle, twine.Tox is installed in the Makefile when used.4.3.0 (2021-04-27)New parameterretry_when_empty_resultfor theread,browse,getandrequestmethods to retry the query when resources are missing in the response.4.4.0 (2021-05-03)Thebackoff_strategycan be specified in theread,browse,getandrequestmethods so it can be different from the one passed in the constructor.Set up the logger name to allow a better logging configuration