Package: origo-sdk-python
Deprecation warningThis module is deprecated in favor oforigo-sdk.The module contents has moved from theorigonamespace toorigo.sdk. Ie.from origo.data.upload import Uploadis changed tofrom origo.sdk.data.upload import Upload.ConfigurationWhen calling any classes interacting with the Origo API and there are no Config params passed to the constructor, a config object will be
automaticly created for you based on environment variablesEnvironment variablesDefault, will pick up configuration from current environment.
The credentials is resolved automatically if you do not set a specific Auth config, in the following order:Client Credentials: If you have added client_id / client_secret to the config. Or if you use the
environment variable equivalent:ORIGO_CLIENT_ID/ORIGO_CLIENT_SECRET.Username And Password:  If you have added username / password to the config. Or if you use the
environment variable equivalent:ORIGO_USERNAME/ORIGO_PASSWORD.# keycloak user
export ORIGO_USERNAME=my-user

# keycloak password for ORIGO_USERNAME
export ORIGO_PASSWORD=my-password

# keycloak client
export ORIGO_CLIENT_ID=my-machine-client

# keycloak secret for ORIGO_CLIENT_ID
export ORIGO_CLIENT_SECRET=some-generated-secure-string


# overrides default environment (dev), but will be trumped by --env=<environment> on the commandline
export ORIGO_ENVIRONMENT=dev|prod

# If you are sending events and have been assigned a API key
export ORIGO_API_KEY=your-api-keyGetting Credentials:username/passwordare synced with Oslo municipalities Active Directory so any user with an association can
use their personal account to access the SDK.Forclient credentialsplease contact the data platform team.dataplattform[at]oslo.kommune.noTODO: Named profilesIf environment variables are not available, the system will try to load from a default profile: Located in ~/.origo/configurationUsageTable of contents:Upload dataSending eventsCreate and manage event streamsCreating datasets with versions and editionsUpload dataWhen uploading data you need to refer to an existing dataset that you own, a version and an edition.
If these are non existent then you can create them yourself. This can be achievedusing the sdk,
or you can use ourcommand line interface.fromorigo.data.uploadimportUploadfromorigo.configimportConfigorigo_config=Config()# If necessary you can override default valuesorigo_config.config["cacheCredentials"]=Falsedata_uploader=Upload(config=origo_config)# Upload file 'data.json' to dataset-id/version/editiondataset_id="my-dataset-id"version="my-version"# example value: 1edition="my-edition"# example value: 20200618T114038filename="/path-to-file/data.json"# Note! filename must be pointing to an existing file on your diskupload_response=data_uploader.upload(filename,dataset_id,version,edition)print(upload_response)# {#     "result": True,#     "trace_id": "my-dataset-id-54a3c78e-86a3-4631-8f28-0252fe1c7c13"# }Thetrace_idreturned by the upload method can be used to "trace" the steps involved in the upload process:fromorigo.statusimportStatus...status=Status(config=origo_config)trace_events=status.get_status(trace_id)print(trace_events)# [#     {#         "trace_id": "my-dataset-1a2bc345-6789-1234-567d-8912ef34a567",#         "trace_status": "STARTED",#         "trace_event_id": "1a2b3cd4-eef5-6aa7-bccd-e889912334f5",#         "trace_event_status": "OK",#         "component": "data-uploader",#         ...#     },#     {#         "trace_id": "my-dataset-1a2bc345-6789-1234-567d-8912ef34a567",#         "trace_status": "CONTINUE",#         ...#     },#     {#         "trace_id": "my-dataset-1a2bc345-6789-1234-567d-8912ef34a567",#         "trace_event_id": "1aa2b345-678c-9de1-f2a3-4566bcd78912",#         "trace_status": "FINISHED",#         "trace_event_status": "OK",#         ...#     }# ]Download dataWhen downloading data you need to refer to an existing dataset that you own, a version and an edition.
If these are non existent then you can create them yourself. This can be achievedusing the sdk,
or you can use ourcommand line interface.fromorigo.data.downloadimportDownloadfromorigo.configimportConfigorigo_config=Config(env="dev")# If necessary you can override default config valuesorigo_config.config["cacheCredentials"]=Falsedata_downloader=Download(config=origo_config)dataset_id="your-dataset-id"version="1"edition="latest"# Downloading a fileres1=data_downloader.download(dataset_id,version,edition,"my/preferred/output/path")print(res1)# {#     "downloaded_files": ["my/preferred/output/path/file_name.csv"]# }Sending eventsIn order to start sending events you will need access to an event stream. If such an event stream is already
in place you are good to go. If not, you can create one either byusing the sdk,
or byusing our command line interface.fromorigo.event.post_eventimportPostEventfromorigo.configimportConfigorigo_config=Config()# If necessary you can override default config valuesorigo_config.config["cacheCredentials"]=Trueevent_poster=PostEvent(config=origo_config)dataset_id="some-dataset-id"version="1"event={"foo":"bar"}res=event_poster.post_event(event,dataset_id,version)# res:# {'message': 'Ok'}# Method also supports list of dictionariesevent_list=[{"foo":"bar"},{"foo":"bar"}]res2=event_poster.post_event(event_list,dataset_id,version)# res2:# {'message': 'Ok'}Create and manage event streamsIn order to create an event stream you need to have defined a dataset and a version,
unless these already exist. Defining a dataset and a version can be
achievedusing the sdk,
or you can use ourcommand line interface.
You do not need to define an edition in order to create an event stream.fromorigo.event.event_stream_clientimportEventStreamClient# Using default configuration for dev-environmentevent_stream_client=EventStreamClient(env="dev")dataset_id="some-dataset-id"version="1"# Creating a new event stream:create_response=event_stream_client.create_event_stream(dataset_id,version)# create_response:# {'message': 'Accepted'}# Getting info about the event streamevent_stream_info=event_stream_client.get_event_stream_info(dataset_id,version)# event_stream_info:# { 'createdAt': '2020-01-29T07:02:32.598520+00:00',#   'createdBy': 'jd',#   'id': 'test-stream-manager/1',#   'status': 'CREATE_IN_PROGRESS'#   }# Note! You must wait until the event stream has status=ACTIVE#       before you can successfully send events to the stream# Deleting the event streamdelete_response=event_stream_client.delete_event_stream(dataset_id,version)# delete_response:# {'message': 'Delete initiated'}Creating datasets with versions and editionsfromorigo.data.datasetimportDatasetfromorigo.configimportConfigorigo_config=Config()# If necessary you can override default valuesorigo_config.config["cacheCredentials"]=False# Create a new datasetdataset=Dataset(config=origo_config)dataset_metadata={"title":"Precise Descriptive Title","description":"Describe your dataset here","keywords":["some-keyword"],"accessRights":"public","confidentiality":"green","objective":"Exemplify how to create a new dataset","contactPoint":{"name":"Your name","email":"your_email@domain.com","phone":"999555111"},"publisher":"name of organization or person responsible for publishing the data"}new_dataset=dataset.create_dataset(data=dataset_metadata)# new_dataset:# { 'Id': 'precise-descriptive-title',#   'Type': 'Dataset',#   '_links': {'self': {'href': '/datasets/precise-descriptive-title'}},#   'accessRights': 'public',#   'confidentiality': 'green',#   'contactPoint': { 'email': 'your_email@domain.com',#                     'name': 'Your name',#                     'phone': '999555111'},#   'description': 'Describe your dataset here',#   'keywords': ['some-keyword'],#   'objective': 'Exemplify how to create a new dataset',#   'publisher': 'name of organization or person responsible for publishing the '#                'data',#   'title': 'Precise Descriptive Title'}# create version for new dataset:version_data={"version":"1"}new_version=dataset.create_version(new_dataset["Id"],data=version_data)# new_version:# { 'Id': 'precise-descriptive-title/1',#   'Type': 'Version',#   '_links': { 'self': { 'href': '/datasets/precise-descriptive-title/versions/1'}},#   'version': '1'}# create edition for new_dataset/new_version:importdatetime# Note! edition-field must be ISO 8601 with utc offsetedition_data={"edition":str(datetime.datetime.utcnow().replace(tzinfo=datetime.timezone.utc).isoformat()),"description":"My edition description","startTime":"2019-01-01","endTime":"2019-12-31"}new_edition=dataset.create_edition(new_dataset["Id"],new_version["version"],data=edition_data)# new_edition# { 'Id': 'precise-descriptive-title/1/20200115T130439',#   'Type': 'Edition',#   '_links': { 'self': { 'href': '/datasets/precise-descriptive-title/versions/1/editions/20200115T130439'}},#   'description': 'My edition description',#   'edition': '2020-01-15T13:04:39.041778+00:00',#   'endTime': '2019-12-31',#   'startTime': '2019-01-01'}