Package: osdupy
osdupyA simple python client for theOSDUdata platform.ContentsClientsSimpleOsduClientAwsServicePrincipalOsduClientAwsOsduClientCurrently supported methodsInstallationTestsUsageInstantiating the SimpleOsduClientInstantiating the AwsServicePrincipalOsduClientInstantiating the AwsOsduClientUsing the clientSearch for records by querySearch with pagingGet a recordUpsert recordsList groupmembership for the current userList membership of a particular groupAdd a user to a particular groupRelease NotesClientsChoose the client that best meets your needs. The same methods are all supported for each.SimpleOsduClientBYOT: Bring your own token. Great for backend service or business logic that supplements a
front-end application.This client assumes you are obtaining a token yourself (e.g. via your application's
login form or otheer mechanism. With this SimpleOsduClient, you simply provide that token.
With this simplicity, you are also then respnsible for reefreeshing the token as needed and
re-instantiating the client with the new token.AwsOsduServicePrincipalClientRequires:boto3==1.15.*Good for batch tasks that don't have an interactive front-end. Token management is handled
with the boto3 library directly through the Cognito service. You have to supply additional arguments for this.For OSDU on AWS, this client is usually simpler than the AwsOsduClient as long as you have IAM credentials to access the necessary resources. You only need to provide the OSDU resource_prefix, region, and profile.AwsOsduClientRequires:boto3==1.15.*Good for batch tasks that don't have an interactive front-end. Token management is handled
with the boto3 library directly through the Cognito service. You have to supply additional arguments for this.For OSDU on AWS, this client is useful in the case where you may want to perform actions as a specific OSDU user rather than as the ServicePrinicpal.Currently supported methodssearchqueryquery_with_pagingstoragequery_all_kindsget_recordget_recordsget_all_record_versionsget_record_versionstore_recordsdelete_recordpurge_recorddatasetget_dataset_registryget_dataset_registriesget_storage_instructionsregister_datasetget_retrieval_instructionsentitlementsget_groupsget_group_membersadd_group_memberdelete_group_membercreate_groupInstallationpipinstallosdupyTestsRun unit testspython-munittest-vtests.unitRun integration testspython-munittest-vtests.integrationUsageInstantiating the SimpleOsduClientIf environment variableOSDU_API_URLis set, then it does not need to be passed as an argument. Otherwise it must be passed as keyword argument.fromosdu.clientimportSimpleOsduClientdata_partition='osdu'token='token-received-from-front-end-app'# With env var `OSDU_API_URL` set in current environment.osdu_client=SimpleOsduClient(data_partition,token)# Without env var set.api_url='https://your.api.base_url.com'osdu_client=SimpleOsduClient(data_partition,token,api_url=api_url)Instantiating the AwsServicePrincipalOsduClientfromosdu.clientimportAwsOsduClientdata_partition='osdu'resource_prefix='osdur3mX'osdu_client=AwsServicePrincipalOsduClient(data_partition,resource_prefix,profile=os.environ['AWS_PROFILE'],region=os.environ['AWS_DEFAULT_REGION'])Instantiating the AwsOsduClientThe only required argument isdata_partition. If your environment variables (below) have been set, then client can be instantiated with onlydata_partitionas an argument.
Environment variables:OSDU_API_URLOSDU_CLIENT_IDOSDU_USEROSDU_PASSWORDAWS_PROFILEAWS_SECRETHASHfromosdu.clientimportAwsOsduClientdata_partition='osdu'osdu_client=AwsOsduClient(data_partition)If you have not set the above environment variales—or you have only set some—then you will need to pass any undefined as args when instantiating the client.fromgetpassimportgetpassfromosdu.clientimportAwsOsduClientapi_url='https://your.api.url.com'# Must be base URL onlyclient_id='YOURCLIENTID'user='username@testing.com'password=getpass()data_partition='osdu'profile='osdu-dev'message=user+client_iddig=hmac.new(client_secret.encode('UTF-8'),msg=message.encode('UTF-8'),digestmod=hashlib.sha256).digest()secretHash=base64.b64encode(dig).decode()osdu_client=AwsOsduClient(data_partition,api_url=api_url,client_id=client_id,user=user,password=password,secret_hash=secretHash,profile=profile)Using the clientBelow are just a few usage examples. Seeintegration testsfor more comprehensive usage examples.Search for records by queryquery={"kind":f"osdu:wks:*:*"}result=osdu_client.search.query(query)# { results: [ {...}, .... ], totalCount: ##### }Search with pagingFor result sets larger than 1,000 records, use the query with paging method.page_size=100# Number of records per page (1-1000)query={"kind":f"osdu:wks:*:*","limit":page_size}result=osdu_client.search.query_with_paging(query)# Iterate over the pages to do something with the results.forpage,total_countinresult:forrecordinpage:# Do stuff with record...Get a recordrecord_id='opendes:doc:123456789'result=osdu_client.storage.get_record(record_id)# { 'id': 'opendes:doc:123456789', 'kind': ..., 'data': {...}, 'acl': {...}, .... }Upsert recordsnew_or_updated_record='./record-123.json'withopen(new_or_updated_record,'r')as_file:record=json.load(_file)result=osdu_client.storage.store_records([record])List groupmembership for the current userresult=osdu_client.entitlements.get_groups()# {#  "desId": "user@example.org",#  "groups": [#    {#      "description": "Datalake Plugin-Manager users",#      "email": "service.plugin.user@osdu.example.com",#      "name": "service.plugin.user"#    },#    {#      "description": "Datalake csv-parser admins",#      "email": "service.csv-parser.admin@osdu.example.com",#      "name": "service.csv-parser.admin"#    },#    #...#       {#     "description": "The viewer of the datalake csv-parser service",#     "email": "service.csv-parser.viewer@osdu.example.com",#     "name": "service.csv-parser.viewer"#   }# ],# "memberEmail": "user@example.com"# }List membership of a particular groupresult=osdu_client.entitlements.get_group_members('users@osdu.example.com')#{#  "members": [#    {#      "email": "serviceprincipal@testing.com",#      "role": "OWNER"#    },#    {#      "email": "user@example.com",#      "role": "OWNER"#    },#    {#      "email": "noaccess@testing.com",#      "role": "OWNER"#    }#  ]#}Add a user to a particular groupAdd a user (user@example.com) to groups to give entitlement to search for and retrieve data.query={"email":"user@example.com",#OWNER or MEMBER"role":"MEMBER",}result=osdu_client.entitlements.add_group_member('users.datalake.viewers@osdu.example.com',query)query={"email":"user@example.com",#OWNER or MEMBER"role":"OWNER",}result=osdu_client.entitlements.add_group_member('service.search.admin@osdu.example.com',query)Delete user from a particular groupRemove a user (user@example.com) from a group.query={"email":"user@example.com",#OWNER or MEMBER"role":"MEMBER",}result=osdu_client.entitlements.delete_group_member('users.datalake.viewers@osdu.example.com',query)query={"email":"user@example.com",#OWNER or MEMBER"role":"OWNER",}result=osdu_client.entitlements.delete_group_member('service.search.admin@osdu.example.com',query)