Package: bodosdk
Bodo Platform SDKA simple SDK for Bodo Cloud Platform.List of contents:Getting StartedJob resourceCluster resourceWorkspace resourceCloud configInstance Role ManagerBilling moduleGetting startedThe first step is to create anAPI Tokenin the Bodo Platform for
Bodo SDK authentication.
Navigate toAPI Tokensin the Admin Console to generate a token.
Copy and save the token'sClient IDandSecret Keyand use them for BodoClient
definition:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)Alternatively, setBODO_CLIENT_IDandBODO_SECRET_KEYenvironment variables
to avoid requiring keys:frombodosdk.clientimportget_bodo_clientclient=get_bodo_client()Other bodo client optionsprint_logs - default False, if enabled all API calls will be printedfrombodosdk.clientimportget_bodo_clientfrombodosdk.modelsimportWorkspaceKeyskeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys,print_logs=True)Job resourceModule responsible for managing jobs in a workspaceBodo Platform Batch JobsCreate batch job definitionList batch job definitionsGet batch job definition by idGet batch job definition by nameRemove batch job definitionSubmit a batch job runList batch job runsGet batch job runCancel batch job runCancel all job runs on a cluster UUIDsCheck batch job run statusSubmit SQL job runJob Run waiterget job logsBodo Platform Batch JobsCreate batch job definitionBodoClient.job.create_batch_job_definition(job_definition: CreateBatchJobDefinition)Creates batch job definition in the given workspace.Example 1: Create batch job definition for a workspace source scriptfrombodosdk.modelsimportWorkspaceKeys,BatchJobDefinitionfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportCreateBatchJobDefinition,JobConfig,JobSource,JobSourceType,SourceCodeType,\WorkspaceDef,RetryStrategykeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)workspace_source_def=JobSource(type=JobSourceType.WORKSPACE,definition=WorkspaceDef(path="Example-path/batch-job-defs",),)retry_strategy=RetryStrategy(num_retries=1,retry_on_timeout=False,delay_between_retries=2,)jobConfig=JobConfig(source=workspace_source_def,source_code_type=SourceCodeType.PYTHON,sourceLocation="test.py",args=None,retry_strategy=retry_strategy,timeout=10000,env_vars=None,)createBatchJobDef=CreateBatchJobDefinition(name="test-job",config=jobConfig,description="test-batch-job-description-attempt",cluster_config={"bodoVersion":"2023.1.3","instance_type":"c5.2xlarge","workers_quantity":2,"accelerated_networking":False,},)jobdef=client.job.create_batch_job_definition(createBatchJobDef)Example 2: Create batch job definition for a git source scriptfrombodosdk.modelsimportWorkspaceKeys,CreateBatchJobDefinition,BatchJobDefinitionfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportCreateBatchJobDefinition,JobConfig,JobSource,JobSourceType,SourceCodeType,\WorkspaceDef,RetryStrategykeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)git_source_def=JobSource(type=JobSourceType.GIT,definition=GitDef(repo_url='https://github.com/Bodo-inc/Bodo-examples.git',username='XYZ',token='XYZ'),)retry_strategy=RetryStrategy(num_retries=1,retry_on_timeout=False,delay_between_retries=2,)jobConfig=JobConfig(source=git_source_def,source_code_type=SourceCodeType.PYTHON,sourceLocation="test.py",args=None,retry_strategy=retry_strategy,timeout=10000,env_vars=None,)createBatchJobDef=CreateBatchJobDefinition(name="test-job",config=jobConfig,description="test-batch-job-description-attempt",cluster_config={"bodoVersion":"2023.1.3","instance_type":"c5.2xlarge","workers_quantity":2,"accelerated_networking":False,},)jobdef=client.job.create_batch_job_definition(createBatchJobDef)**Example 3. Create batch job definition for a S3 source scriptTo run a script file located on an S3 bucket, the cluster must have the required permissions to read the files from S3.
This can be provided by creating an Instance Role with access to the required S3 bucket.
Please make sure to specify an Instance Role that should be attached to the Job Cluster.
The policy attached to the roles should provide access to both the bucket and its contents.frombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportRetryStrategyfrombodosdk.modelsimport(CreateRoleDefinition,InstanceRole,JobConfig,JobSource,JobSourceType,S3Source,SourceCodeType,WorkspaceKeys,CreateBatchJobDefinition)keys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)git_source_def=JobSourcekeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')role_definition=CreateRoleDefinition(name="test-sdk-role-creation",description="testing",data=InstanceRole(role_arn="arn:aws:iam::accountID:role/name"))list_of_instance_roles=client.instance_role.list()role_to_use=Noneforroleinlist_of_instance_roles:ifrole.name=='role_i_want_to_use':role_to_use=rolebreaks3_job_source=JobSource(type=JobSourceType.S3,definition=S3Source(bucket_path='s3://path-to-my-bucket/my_job_script_folder/',type=JobSourceType.S3,bucket_region='region',),)createBatchJobDef=CreateBatchJobDefinition(name="test-job",config=JobConfig(source=s3_job_source,source_code_type=SourceCodeType.PYTHON,sourceLocation="to_sql.py",args=None,timeout=10000,env_vars=None),description="test-batch-job-description-attempt",cluster_config={"bodo_version":"2023.1.3","instance_type":"c5.2xlarge","workers_quantity":2,"accelerated_networking":False,"instance_role_uuid":role_to_use.uuid,})jobdef=client.job.create_batch_job_definition(createBatchJobDef)List batch job definitionsBodoClient.job.list_batch_job_definitions(page: int, size: int, order: PaginationOrder)Parameters:ParameterTypeDescriptionRequiredpageintThe page number has a lower limit of 1, default value 1NosizeintThe size has a maximum allowed value of 100 and default value of 5NoorderPaginationOrderThe order in which the elements would be displayed. Default is created_at ASCNoReturns:Return lists all batch job runs in the given workspace filtered by given parameters - List[JobRunResponse]Lists all batch job definitions in the given workspace.Example:fromtypingimportListfrombodosdk.modelsimportPersonalKeys,WorkspaceKeys,JobConfig,SourceCodeType,RetryStrategy,JobSourceType,\WorkspaceDef,CreateBatchJobDefinitionfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportCreateJobRun,JobSource,JobRunStatus,BatchJobDefinitionResponsekeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)jobdefs:List[BatchJobDefinitionResponse]=client.job.list_batch_job_definitions()Get batch job definition by idBodoClient.job.get_batch_job_definition(job_definition_id: str)Gets specific batch job definition by id.Example:fromtypingimportListfrombodosdk.modelsimportPersonalKeys,WorkspaceKeys,JobConfig,SourceCodeType,RetryStrategy,JobSourceType,\WorkspaceDef,CreateBatchJobDefinitionfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportCreateJobRun,JobSource,JobRunStatus,BatchJobDefinitionResponsekeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)jobdef:BatchJobDefinitionResponse=client.job.get_batch_job_definition('04412S5b-300e-42db-84d4-5f22f7506594')Get batch job definition by nameBodoClient.job.get_batch_job_definition_by_name(name: str)Return the batch job definition based on the name providedExample:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportBatchJobDefinitionResponsekeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)jobdef:BatchJobDefinitionResponse=client.job.get_batch_job_definition('batch-job-1')Remove batch job definitionBodoClient.job.remove_batch_job_definition(job_definition_id: str)Removes specific batch job definition by id.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)client.job.remove_batch_job_definition('04412S5b-300e-42db-84d4-5f22f7506594')Submit a batch job runBodoClient.job.submit_batch_job_run(job_run: CreateJobRun)Submits a job run for a given batch job definition.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportCreateJobRunkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)client.job.submit_batch_job_run(CreateJobRun(batchJobDefinitionUUID='04412S5b-300e-42db-84d4-5f22f7506594',clusterUUID='12936Q5z-109d-89yi-23c4-3d91u1219823'))List batch job runsBodoClient.job.list_batch_job_runs()Parameters:ParameterTypeDescriptionRequiredbatch_job_idsList[UUID]List of Ids of the batch job definitionsNostatusesList[JobRunStatus]List of Job Run status as filtersNocluster_idsList[UUID]List of cluster ids as filtersNostarted_atList[UUID]started at time stamp filterNofinished_atList[UUID]finished at timestamp filterNopageintThe page number as integer. Default page 1Nopage_sizeintThe number of elements in a page. Default page_size = 10NoorderPaginationOrderThe order of listing for the job run ordered by created_atNoReturns:Return lists all batch job runs in the given workspace filtered by given parameters - List[JobRunResponse]Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportJobRunStatuskeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)jobruns=client.job.list_batch_job_runs(statuses=[JobRunStatus.FAILED],cluster_ids=['ba62e653-312a-490e-9457-71d7bc096959'])List batch job runs by batch job nameBodoClient.job.list_job_runs_by_batch_job_name()Parameters:ParameterTypeDescriptionRequiredbatch_job_namesList[str]List of Ids of the batch job definitionsNostatusesList[JobRunStatus]List of Job Run StatusesNocluster_idsList[str]Cluster IDs filter for the batch job runNostarted_atList[str]Started at time filterNofinished_atList[str]Finished at time filterNopageList[str]Page no to fetch. Default value 1Nopage_sizeintNo of elements in a page. Default value 10NoorderPaginationOrderJob run sorting order by created_at. Default value ASCNoLists all batch job runs in the given workspace filtered by given parameters.Returns:Returns a list of JobRunResponse - List[JobRunResponse]Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.jobimportJobRunStatuskeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)jobruns=client.job.list_job_runs_by_batch_job_name(batch_job_names=['production-job-1'],statuses=[JobRunStatus.FAILED],cluster_ids=['ba62e653-312a-490e-9457-71d7bc096959'])Get batch job runBodoClient.job.get_batch_job_run(uuid: str)Returns batch job run based on the job_run_id provided.Parameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Returns batch job run for the idYesReturns:ParameterTypeDescriptionnamestrThe name of the job runclusterUUIDOptional[Union[UUID, None]]The cluster UUID on which the job run was executedclusterOptional[JobCluster]The cluster on which the job run was executedtypeJobRunTypeThe type of Job RunconfigJobConfigThe JobConfig of the clustersubmittedAtdatetimeThe time at which the job was submittedfinishedAtOptional[Union[datetime, None]]The time at which the job finishedstartedAtOptional[Union[datetime, None]]The time at which job startedstatusJobRunStatusJob run statusbatchJobDefinitionConfigOverridesOptional[JobConfigOverride]The job run config overridebatch_job_definitionOptional[BatchJobDefinitionUUID]UUID of batch job definitionreasonOptional[str]The reason for job statue changesubmitterOptional[str]Email of the person who submitted the jobGets specific batch job run by job_run_idExample:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)jobrun=client.job.get_batch_job_run('04412S5b-300e-42db-84d4-5f22f7506594')Cancel batch job runBodoClient.job.cancel_batch_job_run(uuid: Union[str, UUID])Cancels specific batch job run by job_run_idParameters:ParameterTypeDescriptionRequireduuiduuid: Union[str, UUID]UUID of the job runYesReturns:None returned on successExample:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)client.job.cancel_batch_job_run('04412S5b-300e-42db-84d4-5f22f7506594')Cancel all job runs on a cluster UUIDsBodoClient.job.cancel_all_job_runs(cluster_uuids: Union[List[str], List[UUID]])Cancels all the job runs for a set of cluster UUIDs provided as a function parameterParameters:ParameterTypeDescriptionRequiredcluster_uuidsUnion[List[str], List[UUID]]Cancels all job runs on a list of cluster UUIDsYesReturns:None returned on successExample:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)client.job.cancel_all_job_runs(['04412S5b-300e-42db-84d4-5f22f7506594'])Check batch job run statusBodoClient.job.check_job_run_status(batch_job_run_uuid: Union[str, UUID])->JobRunStatusChecks status of specific batch job run by id.Parameters:ParameterTypeDescriptionRequiredbatch_job_run_uuidUnion[str, UUID]Fetches theYesReturns:JobRunStatus returned on successExample:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)status=client.job.check_job_run_status('04412S5b-300e-42db-84d4-5f22f7506594')Submit SQL job runBodoClient.job.submit_sql_job_run(create_sql_job_run: CreateSQLJobRun)Submits a SQL query as a job run. The SQL job run contains the SQL query text and the cluster UUID on which the query
will be executed along with catalog and the query tags. The query tags field accepts a JSON which returns the query
tags associated with a query.!!! note
This needs a database [catalog][catalog] to be configured in the workspace.Parameters(CreateSQLJobRun):ParameterTypeDescriptionRequiredDefaultJob Run TypeJobRunTypeType of Job RunYesNoneclusterUUIDstrThe cluster UUID on which the job run will be executedNoNonecatalogstrThe catalog which the query useYesNonesql_query_textstrThe SQL query textYesNonequery_tagsDict[str, str]The query tags associated with the queryNo{}timeoutintThe timeout for the query in minutesNo60env_varsDict[str, str]The environment variables for the queryNo{}retry_strategyRetryStrategyThe retry strategy for the queryNoNonecluster_configJobClusterDefinitionJob Cluster ConfigurationNoNoneargsUnion[str, Dict]The arguments for the queryNo{}Returns:ParameterTypeDescriptionnamestrThe name of the job runclusterUUIDOptional[Union[UUID, None]]The cluster UUID on which the job run was executedclusterOptional[JobCluster]The cluster on which the job run was executedtypeJobRunTypeThe type of Job RunconfigJobConfigThe JobConfig of the clustersubmittedAtdatetimeThe time at which the job was submittedfinishedAtOptional[Union[datetime, None]]The time at which the job finishedstartedAtOptional[Union[datetime, None]]The time at which job startedstatusJobRunStatusJob run statusbatchJobDefinitionConfigOverridesOptional[JobConfigOverride]The job run config overridebatch_job_definitionOptional[BatchJobDefinitionUUID]UUID of batch job definitionreasonOptional[str]The reason for job statue changesubmitterOptional[str]Email of the person who submitted the jobExample:frombodosdk.modelsimportWorkspaceKeys,CreateSQLJobRunfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_client(keys)job_run=client.job.submit_sql_job_run(CreateSQLJobRun(clusterUUID='04412S5b-300e-42db-84d4-5f22f7506594',catalog="SNOWFLAKE_CATALOG",sql_query_text="SELECT * FROM PUBLIC.TABLE LIMIT 10",query_tags={"DAG_ID":"398482","MACHINE_ID":"1934"}))Job Run waiterBodoClient.job.get_job_run_waiter()Returns a waiter object that waits until the job run uuid specified finishes.waiter.wait()To wait for job run to be finished, invoke the waiter.wait() function,
which can take the following parameters.Parameters:ParameterTypeDescriptionRequireduuidstrCreate a SQL Job Run based on the parameters that are passedYeson_successCallableCallable executed on success with job object passed as argumentNoon_failureCallableCallable executed on failure with job object passed as argumentNoon_timeoutCallableCallable executed on failure with job_uuid passed as argumentNocheck_periodintTime in seconds between status checks for the wait function. Default is 10 secondsNotimeoutintTime in seconds after which timeout error will be raised. Default is noneNoReturns:ParameterTypeDescriptionnamestrThe name of the job runclusterUUIDOptional[Union[UUID, None]]The cluster UUID on which the job run was executedclusterOptional[JobCluster]The cluster on which the job run was executedtypeJobRunTypeThe type of Job RunconfigJobConfigThe JobConfig of the clustersubmittedAtdatetimeThe time at which the job was submittedfinishedAtOptional[Union[datetime, None]]The time at which the job finishedstartedAtOptional[Union[datetime, None]]The time at which job startedstatusJobRunStatusJob run statusbatchJobDefinitionConfigOverridesOptional[JobConfigOverride]The job run config overridebatch_job_definitionOptional[BatchJobDefinitionUUID]UUID of batch job definitionreasonOptional[str]The reason for job statue changesubmitterOptional[str]Email of the person who submitted the jobfromtypingimportCallabledefwait(self,uuid,on_success:Callable=None,on_failure:Callable=None,on_timeout:Callable=None,check_period=10,timeout=None):passBy default, returns job model if no callbacks is provided. There is option to pass callable objects as following
parameters:Example 1. Success/Failure callbacks:frombodosdk.modelsimportWorkspaceKeys,CreateJobRunfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)input_job=CreateJobRun(clusterUUID='<cluster-uuid>',batchJobDefinitionUUID='<batch-job-definition-uuid>')job_run=client.job.submit_batch_job_run(input_job)waiter=client.job.get_job_run_waiter()defsuccess_callback(job):print("in success callback",job.status)deffailure_callback(job):print('in failure callback',job.status)result=waiter.wait(job_run.uuid,on_success=success_callback,on_failure=failure_callback)Example 2. Timeout callback:frombodosdk.modelsimportWorkspaceKeys,CreateJobRunfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)input_job=CreateJobRun(clusterUUID='<cluster-uuid>',batchJobDefinitionUUID='<batch-job-definition-uuid>')job_run=client.job.submit_batch_job_run(input_job)waiter=client.job.get_job_run_waiter()deftimeout_callback(job_uuid):print(f'Waiter timeout for{job_uuid}')returnjob_uuidresult=waiter.wait(job_run.status,on_timeout=timeout_callback,timeout=1)Get job logsBodoClient.job.get_job_logs(job_uuid)Returns specific stdout and stderr urls along with expiration timestamp in workspace.
Also, downloads specific stdout and stderr logs in workspace and additionally provides links as wellstdout: Standard output of the program executionstderr: Standard error messages of the program executionDownloads files as below and overrides if they already existstdout_{uuid}.txtstderr_{uuid}.txtExample:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.models.jobimportJobRunLogsResponsefrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)logs:JobRunLogsResponse=client.job.get_job_logs('8c32aec5-7181-45cc-9e17-8aff35fd269e')Cluster resourceModule responsible for managing clusters in workspace.get available instance typesget available imagescreate clusterlist clustersget clusterremove clusterscale clusterstop clusterrestart clusterpause clusterresume clustermodify clusterget tasks for clusterAvailability Zone SelectionWhen creating a cluster, you can specify the availability zone in which the cluster will be created. However, cluster creation might fail if the availability zone does not have sufficient capacity to create the cluster.
Even after the cluster is created, resuming or scaling it might fail if the availability zone does not have sufficient capacity to resume or scale the cluster.Bodo supports anauto_azflag in cluster creation which is by default set toTrue. When enabled
create, scale and resume tasks attempt to automatically select an availability zone with sufficient capacity for said cluster. If you want to disable this behavior, setauto_aztoFalsein theClusterDefinitionobject.Available instance typesBodoClient.cluster.get_available_instance_types(region:str) -> Dict[str, InstanceCategory]Parameters:ParameterTypeDescriptionRequiredregionstrAzure / AWS regionYesReturns:Return dictionary of instance types available for given region grouped by instance category.InstanceCategory:FieldTypeDescriptionnamestrName of instance instance categoryinstance_typesDict[str, InstanceType]Dict with all instances in the categoryfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)instance_types=client.cluster.get_available_instance_types('us-west-2')Available imagesBodoClient.cluster.get_available_images(region:str) -> Dict[str, BodoImage]Parameters:ParameterTypeDescriptionRequiredregionstrAzure / AWS regionYesReturns:Return dictionary of images available for given region where key is bodo version and value isBodoImage:BodoImage:FieldTypeDescriptionimage_idstrImage idbodo_versionstrBodo versionfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)images=client.cluster.get_available_images('us-west-2')Create clusterBodoClient.cluster.create(cluster_definition: ClusterDefinition) -> ClusterResponseCreates a cluster in the workspace based on the instance type, no of workers and whether the instance is a spot instance. The cluster can be configured to have an auto-pause and auto-stop time in minutes to pause and stop the cluster when there is no activity.If you choose to create a cluster with spot instances, please note:Spot instance clusters are only supported on AWS at this moment.Spot instance has lower cost at the expense of reliability. We recommend to use instance types with lower reclaim rate according toAWS spot instance advisor.Spot instance clusters cannot be paused/resumed. Please use stop/restart insteadAuto pause on spot instance clusters is not allowed. Please use auto stop insteadSpot instance clusters will have a 60-minute auto stop by default to avoid accidental long-running clustersParameter:cluster_definition: ClusterDefinitionFieldTypeDescriptionRequirednamestrCluster nameYesinstance_typestrInstance type of cluster nodesYesworkers_quantityintNumber of nodes in the clusterYesbodo_versionstrBodo version available on the clusterYesdescriptionOptional[str]Cluster descriptionNoimage_idOptional[str]Image id used on cluster nodesNoauto_stopOptional[int]Cluster auto stop value [min]Noauto_pauseOptional[int]Cluster auto pause value [min]Noavailability_zoneOptional[str]Availability zone for clusterNoaws_deployment_subnet_idOptional[str][AWS] Subnet id for clusterNoinstance_role_uuidOptional[str][AWS] Instance roleNoauto_azOptional[bool][AWS] Whether the cluster has Auto AZ enabledNouse_spot_instanceOptional[bool][AWS] Whether the cluster uses spot instanceNois_job_dedicatedOptional[bool]whether the cluster is a job dedicated clusterNoaccelerated_networkingOptional[bool]Whether the cluster uses accelerated networking (e.g. EFA on AWS)NoReturns:ObjectClusterResponse:FieldTypeDescriptionnamestrCluster nameuuidUnion[str, UUID]Cluster UUIDstatusClusterStatusCluster statusdescriptionOptional[str]Cluster descriptioninstance_typestrInstance type of cluster nodesworkers_quantityintNumber workers in clustersauto_stopOptional[int]Cluster auto stop value [min]auto_pauseOptional[int]Cluster auto pause value [min]bodo_versionOptional[str]Bodo version used in cluster VMsimage_idOptional[str]Image id used cluster VMscores_per_workerOptional[int]Number of cores per workeraccelerated_networkingOptional[bool]Whether the cluster uses accelerated networkingcreated_atstrDate when cluster was createdlast_known_activityOptional[str]Date of last known cluster activityis_job_dedicatedOptional[bool]whether the cluster is a job dedicated clusternode_metadataOptional[object]Metada data about nodes (IP, instance id)asg_metadataOptional[object][AWS] Auto scaling group metadataaws_deployment_subnet_idOptional[str][AWS] Subnet id for clusterauto_azOptional[bool][AWS] Whether the cluster has Auto AZ enabledExample:Create a regular clusterfrombodosdk.modelsimportWorkspaceKeys,ClusterDefinitionfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)cluster_definition=ClusterDefinition(name="test",instance_type="c5.large",workers_quantity=2,use_spot_instance=False,auto_pause=100,image_id="ami-038d89f8d9470c862",bodo_version="2022.4",description="my desc here",auto_az=False,)result_create=client.cluster.create(cluster_definition)Example:Create a spot instance cluster:frombodosdk.modelsimportWorkspaceKeys,ClusterDefinitionfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)cluster_definition=ClusterDefinition(name="test-spot",instance_type="c5.large",workers_quantity=2,use_spot_instance=True,auto_stop=100,image_id="ami-038d89f8d9470c862",bodo_version="2022.4",description="my desc here",auto_az=False,)result_create=client.cluster.create(cluster_definition)List clustersBodoClient.cluster.list(page: int, page_size: int) -> ClusterListParameters:ParameterTypeDescriptionRequiredpageintThe page number has a lower limit of 1, default value 1FalsesizeintThe size has a maximum allowed value of 100 and default value of 10FalseReturns:ClusterListFieldsTypeDescriptiondataList[ClusterResponse]Contains a list of Cluster ResponsemetadataPageMetadataContains fieldspage,size,total_pages,total_items, andorderObjectClusterResponse:FieldTypeDescriptionnamestrCluster nameuuidUnion[str, UUID]Cluster UUIDstatusClusterStatusCluster statusdescriptionOptional[str]Cluster descriptioninstance_typestrInstance type of cluster nodesworkers_quantityintNumber workers in clustersauto_stopOptional[int]Cluster auto stop value [min]auto_pauseOptional[int]Cluster auto pause value [min]bodo_versionOptional[str]Bodo version used in cluster VMsimage_idOptional[str]Image id used cluster VMscores_per_workerOptional[int]Number of cores per workeraccelerated_networkingOptional[bool]Whether the cluster uses accelerated networkingcreated_atstrDate when cluster was createdlast_known_activityOptional[str]Date of last known cluster activityis_job_dedicatedOptional[bool]whether the cluster is a job dedicated clusternode_metadataOptional[object]Metada data about nodes (IP, instance id)asg_metadataOptional[object][AWS] Auto scaling group metadataaws_deployment_subnet_idOptional[str][AWS] Subnet id for clusterauto_azOptional[bool][AWS] Whether the cluster has Auto AZ enabledExample: Get the first 10 clusters in the workspace using the default parameter values.frombodosdk.modelsimportWorkspaceKeys,ClusterResponsefrombodosdk.clientimportget_bodo_clientfromtypingimportListkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)clusters:List[ClusterResponse]=client.cluster.list().dataExample: Get all clusters with chunck of 10 clusters.frombodosdk.clientimportget_bodo_clientfrombodosdk.modelsimport(WorkspaceKeys,)keys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)cluster_list=[]page=1size=10clusters=client.cluster.list(page=page,page_size=size)print('Total Clusters',clusters.metadata.total_items)total_items=clusters.metadata.total_itemscluster_list=clusters.datawhilelen(cluster_list)<total_items:page+=1clusters=client.cluster.list(page=page,page_size=size)cluster_list+=clusters.dataGet clusterBodoClient.cluster.get(uuid : str) -> ClusterResponseParameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDTrueReturns:ObjectClusterResponse:FieldTypeDescriptionnamestrCluster nameuuidUnion[str, UUID]Cluster UUIDstatusClusterStatusCluster statusdescriptionOptional[str]Cluster descriptioninstance_typestrInstance type of cluster nodesworkers_quantityintNumber workers in clustersauto_stopOptional[int]Cluster auto stop value [min]auto_pauseOptional[int]Cluster auto pause value [min]bodo_versionOptional[str]Bodo version used in cluster VMsimage_idOptional[str]Image id used cluster VMscores_per_workerOptional[int]Number of cores per workeraccelerated_networkingOptional[bool]Whether the cluster uses accelerated networkingcreated_atstrDate when cluster was createdlast_known_activityOptional[str]Date of last known cluster activityis_job_dedicatedOptional[bool]whether the cluster is a job dedicated clusternode_metadataOptional[object]Metada data about nodes (IP, instance id)asg_metadataOptional[object][AWS] Auto scaling group metadataaws_deployment_subnet_idOptional[str][AWS] Subnet id for clusterauto_azOptional[bool][AWS] Whether the cluster has Auto AZ enabledExample:frombodosdk.modelsimportWorkspaceKeys,ClusterResponsefrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)clusters:ClusterResponse=client.cluster.get('<CLUSTER-UUID>')Remove clusterBodoClient.client.remove(uuid: Union[str, UUID], force_remove: bool = False, mark_as_terminated: bool = False)Method removing cluster from platformParameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDTrueforce_removeUnion[str, UUID]Try to remove cluster even if there is cluster activityFalsemark_as_terminatedUnion[str, UUID]Mark cluster as removed without removing resources, may be useful if cluster creation failed and normal removal is failingFalseReturns:ReturnsNoneif successful. Otherwise, raises exception.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfromtypingimportListkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.cluster.remove('<CLUSTER-UUID>')Scale clusterBodoClient.cluster.scale(scale_cluster: ScaleCluster) -> ClusterResponseChanges number of nodes in cluster (AWS only)Parameter:scale_cluster: ScaleClusterFieldTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesworkers_quantityintNumber of nodes in the clusterYesReturns:ObjectClusterResponse:FieldTypeDescriptionnamestrCluster nameuuidUnion[str, UUID]Cluster UUIDstatusClusterStatusCluster statusdescriptionOptional[str]Cluster descriptioninstance_typestrInstance type of cluster nodesworkers_quantityintNumber workers in clustersauto_stopOptional[int]Cluster auto stop value [min]auto_pauseOptional[int]Cluster auto pause value [min]bodo_versionOptional[str]Bodo version used in cluster VMsimage_idOptional[str]Image id used cluster VMscores_per_workerOptional[int]Number of cores per workeraccelerated_networkingOptional[bool]Whether the cluster uses accelerated networkingcreated_atstrDate when cluster was createdlast_known_activityOptional[str]Date of last known cluster activityis_job_dedicatedOptional[bool]whether the cluster is a job dedicated clusternode_metadataOptional[object]Metada data about nodes (IP, instance id)asg_metadataOptional[object][AWS] Auto scaling group metadataaws_deployment_subnet_idOptional[str][AWS] Subnet id for clusterauto_azOptional[bool][AWS] Whether the cluster has Auto AZ enabledExample:frombodosdk.modelsimportWorkspaceKeys,ScaleCluster,ClusterResponsefrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)NEW_WORKERS_QUANTITY=3scale_cluster=ScaleCluster(uuid='<CLUSTER-UUID>',workers_quantity=NEW_WORKERS_QUANTITY)cluster:ClusterResponse=client.cluster.scale(scale_cluster)Stop clusterBodoClient.cluster.stop(uuid: Union[str, UUID])Stops any cluster activity. You will not incur any charges for stopped cluster. You can restart it again at any time.Parameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesReturns:ReturnsNoneif successful. Otherwise, raises exception.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.cluster.stop('<CLUSTER-UUID>')Restart clusterBodoClient.cluster.restart(uuid: Union[str, UUID])Restarts cluster. You can restart cluster only if it is stopped.Parameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesReturns:ReturnsNoneif successful. Otherwise, raises exception.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.cluster.stop('<CLUSTER-UUID>')Pause clusterBodoClient.cluster.pause(uuid: Union[str, UUID])Pause cluster. You can pause cluster only if it is running.Parameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesReturns:ReturnsNoneif successful. Otherwise, raises exception.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.cluster.pause('<CLUSTER-UUID>')Resume clusterBodoClient.cluster.resume(uuid: Union[str, UUID])Resume cluster. You can resume cluster only if it is paused.Parameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesReturns:ReturnsNoneif successful. Otherwise, raises exception.Example:frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.cluster.resume('<CLUSTER-UUID>')Modify Cluster metadataBodoClient.cluster.modify(modify_cluster: ModifyCluster) -> ClusterResponseThis function can be used to edit cluster metadata for a given cluster. The properties that we can edit are
description, autopause time, autostop time, bodo-version, instance type, instance role, flag for auto availability zone selection and the number of workers.
Changing the number of workers will kick off a scaling event on the cluster, which will resume the cluster if it is in paused state.
The modify function also supports modifying a subset of property part if the ModifyCluster object like listed in the example below.
The cluster modification can only happen when the cluster is in stopped state. The fields that aren't required to
be modified are optional and don't necessarily have to be passed during the call to the API.
Note: Disabling theauto_azflag without specifying anavailability_zonein the same request might result in the cluster failing.
So make sure to provide a fallback zone to avoid failures.Parameter:modify_cluster: ModifyClusterFieldTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesauto_stopOptional[int]Cluster auto stop value [min]Noauto_pauseOptional[int]Cluster auto pause value [min]NodescriptionOptional[str]Cluster descriptionNoworkers_quantityOptional[int]Number of nodes in the clusterNoinstance_role_uuidOptional[str][AWS] Instance roleNoinstance_typeOptional[str]Instance type used in clusterNobodo_versionOptional[str]Bodo version available on the clusterNoauto_azOptional[bool][AWS] Whether the cluster is Auto AZ enabledNoavailability_zoneOptional[str]Availability zone for the clusterNoReturns:ObjectClusterResponse:FieldTypeDescriptionnamestrCluster nameuuidUnion[str, UUID]Cluster UUIDstatusClusterStatusCluster statusdescriptionOptional[str]Cluster descriptioninstance_typestrInstance type of cluster nodesworkers_quantityintNumber workers in clustersauto_stopOptional[int]Cluster auto stop value [min]auto_pauseOptional[int]Cluster auto pause value [min]bodo_versionOptional[str]Bodo version used in cluster VMsimage_idOptional[str]Image id used cluster VMscores_per_workerOptional[int]Number of cores per workeraccelerated_networkingOptional[bool]Whether the cluster uses accelerated networkingcreated_atstrDate when cluster was createdlast_known_activityOptional[str]Date of last known cluster activityis_job_dedicatedOptional[bool]whether the cluster is a job dedicated clusternode_metadataOptional[object]Metada data about nodes (IP, instance id)asg_metadataOptional[object][AWS] Auto scaling group metadataaws_deployment_subnet_idOptional[str][AWS] Subnet id for clusterauto_azOptional[bool][AWS] Whether the cluster has Auto AZ enabledExample:frombodosdk.modelsimportWorkspaceKeys,ModifyCluster,ClusterResponsefrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')role_definition=CreateRoleDefinition(name="test-sdk-role-creation",description="testing-instance-role-creation",data=InstanceRole(role_arn="arn:aws:iam::427443013497:role/testing_bucket_with_my_script"))result_create_role:CreateRoleResponse=client.instance_role.create(role_definition)client=get_bodo_client(keys)modify_cluster=ModifyCluster(uuid="<cluster-uuid>",auto_pause=60,auto_stop=0,workers_quantity=4,description="using the SDK",instance_type="c5.large",instance_role_uuid=result_create_role.uuid,bodo_version="2022.4",auto_az=True,)partial_modify_cluster=ModifyCluster(uuid="<cluster-uuid>",autopause=120,)new_cluster:List[ClusterResponse]=client.cluster.modify(modify_cluster)new_cluster_partial:List[ClusterResponse]=client.cluster.modify(partial_modify_cluster)Example:Detach Custom Instance Role . Replace the custom instance role with default role which is automatically created for a clusterdetach_custom_instance_role=ModifyCluster(uuid="<cluster-uuid>",instance_role_uuid="default",)new_cluster_partial:ClusterResponse=client.cluster.modify(detach_custom_instance_role)List tasks for a clusterBodoClient.cluster.list_tasks(uuid: Union[str, UUID]) -> List[TaskInfo]Gets all taks for cluster.Parameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Cluster UUIDYesReturns:List[TaskInfo]
ObjectTaskInfo:FieldTypeDescriptionuuidstrCluster task UUIDstatusTaskStatusCluster task statustask_typestrTask typelogsstrCluster descriptionExample:frombodosdk.modelsimportWorkspaceKeys,TaskInfofrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)tasks:List[TaskInfo]=client.cluster.list_tasks(uuid)Workspace resourceModule responsible for managing workspaces in an organization.workspace getting startedcreate workspacelist workspacesget workspaceremove workspaceassign userlist workspace tasksWorkspace getting startedIn order to work with Workspace, users need to generate Personal Tokens, under Admin Console, from the Bodo Platform Dashboard.
Then instantiate a PersonalKeys object with the generatedclient_idandsecret_id. Then pass in this personal key while
instantiating a client objectfrombodosdk.modelsimportPersonalKeyspersonal_keys=PersonalKeys(client_id='<CLIENT-ID>',secret_id='<SECRET-ID>',)client=get_bodo_organization_client(personal_keys)Create WorkspaceBodoClient.workspace.create(workspace_definition: WorkspaceDefinition) -> WorkspaceCreatedResponseCreates a workspace with the specifications passed in through a WorkspaceDefinition object under the
user's organization`BodoClient.workspace.create(workspace_definition: WorkspaceDefinition) -> WorkspaceCreatedResponse`

Creates a workspace with the specifications passed in through a `WorkspaceDefinition` object under the
user's organization.

**Parameter:** `workspace_definition: WorkspaceDefinition`

| Field               | Type                       | Description                                                                                                                                                                                                                                                                                                  | Required |
| ------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- |
| `name`              | `str`                      | Workspace name                                                                                                                                                                                                                                                                                               | Yes      |
| `region`            | `str`                      | Region where workspace should be placed                                                                                                                                                                                                                                                                      | Yes      |
| `cloud_config_uuid` | `Union[UUID, str]`         | Existing AWS or Azure cloud config uuid                                                                                                                                                                                                                                                                      | Yes      |
| `storage_endpoint`  | `Optional[bool]`           | Enable additonal endpoints [Microsoft.Storage](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-service-endpoints-overview) or [S3 Gateway](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html). <br/>There's no additional charge for using service endpoints. | No       |
| `aws_network_data`  | `Optional[AWSNetworkData]` | Specific for network data for AWS                                                                                                                                                                                                                                                                            | No       |

**Returns:**
Object `WorkspaceCreatedResponse:`

| Field               | Type            | Description                               |
| ------------------- | --------------- | ----------------------------------------- |
| `name`              | `str`           | Workspace name                            |
| `uuid`              | `str`           | Workspace UUID                            |
| `status`            | `str`           | Workspace status                          |
| `region`            | `str`           | Region where workspace should be placed.  |
| `organization_uuid` | `str`           | Organization UUID                         |
| `data`              | `Optional[Any]` | Specific configuration data for workspace |
| `created_by`        | `Optional[str]` | Email address of the workspace creator    |
| `cloud_config`      | `Optional[Any]` | Cloud config data                         |

**Example: Create new Workspace**

```python
from bodosdk.models import PersonalKeys
from bodosdk.models import WorkspaceDefinition
personal_keys = PersonalKeys(
    client_id='<CLIENT-ID>',
    secret_id='<SECRET-ID>',
)
client = get_bodo_organization_client(personal_keys)
wd = WorkspaceDefinition(
    name="<WORSPACE-NAME>",
    cloud_config_uuid="<CONFIG-UUID>",
    region="<WORKSPACE-REGION>"
)
resp = client.workspace.create(wd)List WorkspacesBodoClient.workspace.list()Returns:Returns a list of all workspaces defined under this organization asList[WorkspaceListItem]FieldTypeDescriptionnamestrWorkspace nameuuidstrWorkspace UUIDstatusWorkspaceStatusWorkspace statusproviderstrWorkspace provider (AWS or AZURE)regionstrRegion where workspace should be placed.organization_uuidstrOrganization UUIDdataOptional[Any]Specific configuration data for workspaceserver_timeOptional[str]Current server timecloud_configOptional[Any]Cloud config datacreated_byOptional[str]Email address of the workspace creatorExample:`BodoClient.workspace.list()`Returnsalistofallworkspacesdefinedunderthisorganization.Thewith_taskbooleancontrolsprintingouttasksrunningintheworkspaces.ThereturnedlistisalistofGetWorkspaceResponseobjectfrom bodosdk.models import PersonalKeys
personal_keys = PersonalKeys(
client_id='',
secret_id='',
)
client = get_bodo_organization_client(personal_keys)
resp = client.workspace.list()### Get Workspace<a id="get-workspace"></a>

`BodoClient.workspace.get(uuid: str)`
Returns information about the workspace with the given uuid. Returns a GetWorkspaceResponse object with details about the workspace uuid mentioned.BodoClient.workspace.get(uuid: Union[str, UUID]) -> WorkspaceResponseParameters:ParameterTypeDescriptionRequireduuidUnion[str, UUID]Workspace UUIDYesReturns:Returns the workspace details provided the workspaceuuid-WorkspaceResponseFieldTypeDescriptionnamestrWorkspace nameuuidstrWorkspace UUIDstatusWorkspaceStatusWorkspace statusregionstrRegion where workspace is placed.organization_uuidstrOrganization UUIDdataOptional[Any]Specific configuration data for workspacecloud_configOptional[Any]Cloud config datacreated_byOptional[str]Email address of the workspace creatorExample:frombodosdk.modelsimportPersonalKeyspersonal_keys=PersonalKeys(client_id='<CLIENT-ID>',secret_id='<SECRET-ID>',)client=get_bodo_organization_client(personal_keys)resp=client.workspace.get("<WORKSPACE-UUID>")Remove WorkspaceBodoClient.workspace.remove(uuid: Union[str, UUID], mark_as_terminated: bool = False)Removes the workspace with the provided uuid. The operation is only successful if all resources within the workspaces(jobs, clusters, notebooks) are already terminated.Parameters:FieldTypeDescriptionRequireduuidUnion[str, UUID]Workspace UUIDYesmark_as_terminatedboolMark role as terminated without removing resources,may be useful if role creation failed and deletion is failingNoReturns:Returns None if successful. Otherwise, raises exception.Example:frombodosdk.modelsimportPersonalKeyspersonal_keys=PersonalKeys(client_id='<CLIENT-ID>',secret_id='<SECRET-ID>',)client=get_bodo_organization_client(personal_keys)resp=client.workspace.remove("<WORKSPACE-UUID>")Assign userBodoClient.workspace.assign_users(workspace_uuid: Union[str, UUID], users: List[UserAssignment])Assign user to workspace.Parameters:ParameterTypeDescriptionRequiredworkspace_uuidUnion[str, UUID]Workspace UUIDYesusersList[UserAssignment]List of users that will be assigned to a given workspaceYesReturns:Returns None if successful. Otherwise, raises exception.Example:frombodosdk.modelsimportPersonalKeyspersonal_keys=PersonalKeys(client_id='<CLIENT-ID>',secret_id='<SECRET-ID>',)client=get_bodo_organization_client(personal_keys)workspace_uuid="<some uuid>"users:List[UserAssignment]=[UserAssignment(email="example@example.com",skip_email=True,bodo_role=BodoRole.ADMIN)]client.workspace.assign_users(workspace_uuid,users)List Workspace tasksBodoClient.workspace.get_tasks(workspace_uuid: Union[str, UUID]) -> List[TaskInfo]Returns:Return a list of workspace tasks in the workspace -List[TaskInfo]FieldTypeDescriptionuuidstrWorkspace task uuidstatusTaskStatusStatus of workspace tasktask_typeWorkspaceStatusType of workspace tasklogsstrLogs from specific taskExamplefrombodosdk.modelsimportPersonalKeyspersonal_keys=PersonalKeys(client_id='<CLIENT-ID>',secret_id='<SECRET-ID>',)client=get_bodo_organization_client(personal_keys)resp=client.workspace.get_tasks("<WORKSPACE-UUID>")Cloud ConfigModule responsible for creating cloud configurations for organization.create cloud configurationlist cloud configurationsget cloud configCreate configBodoClient.cloud_config.create(config: Union[CreateAwsCloudConfig, CreateAzureCloudConfig])Create cloud configuration for cloudAWS examplefrombodosdk.modelsimportOrganizationKeys,CreateAwsProviderData,CreateAwsCloudConfig,AwsCloudConfigfrombodosdk.clientimportget_bodo_clientkeys=OrganizationKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)config=CreateAwsCloudConfig(name='test',aws_provider_data=CreateAwsProviderData(tf_backend_region='us-west-1',access_key_id='xyz',secret_access_key='xyz'))config:AwsCloudConfig=client.cloud_config.create(config)Azure examplefrombodosdk.modelsimportOrganizationKeys,CreateAzureProviderData,CreateAzureCloudConfig,AzureCloudConfigfrombodosdk.clientimportget_bodo_clientkeys=OrganizationKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)config=CreateAzureCloudConfig(name='test',azure_provider_data=CreateAzureProviderData(tf_backend_region='eastus',tenant_id='xyz',subscription_id='xyz',resource_group='MyResourceGroup'))config:AzureCloudConfig=client.cloud_config.create(config)Get configBodoClient.cloud_config.list()Get list of cloud configs.frombodosdk.modelsimportOrganizationKeys,AzureCloudConfig,AwsCloudConfigfrombodosdk.clientimportget_bodo_clientfromtypingimportUnion,Listkeys=OrganizationKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)configs:List[Union[AwsCloudConfig,AzureCloudConfig]]=client.cloud_config.list()Get configBodoClient.cloud_config.get(uuid: Union[str, UUID])Get cloud config by uuid.frombodosdk.modelsimportOrganizationKeys,AzureCloudConfig,AwsCloudConfigfrombodosdk.clientimportget_bodo_clientfromtypingimportUnionkeys=OrganizationKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)config:Union[AwsCloudConfig,AzureCloudConfig]=client.cloud_config.get('8c32aec5-7181-45cc-9e17-8aff35fd269e')Instance Role ManagerModule responsible for managing AWS roles in workspace.create rolelist rolesget roleremove roleCreate roleBodoClient.instance_role.create()Creates an AWS role with the specified role definition with a given AWS role arn.frombodosdk.modelsimportWorkspaceKeys,CreateRoleDefinition,CreateRoleResponsefrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)role_definition=CreateRoleDefinition(name="test-sdk-role-creation",description="testing",data=InstanceRole(role_arn="arn:aws:iam::1234567890:role/testing"))result_create:CreateRoleResponse=client.instance_role.create(role_definition)List rolesBodoClient.instance_role.list()Returns list of all roles in workspacefrombodosdk.modelsimportWorkspaceKeys,InstanceRoleItemfrombodosdk.clientimportget_bodo_clientfromtypingimportListkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)result_list:List[InstanceRoleItem]=client.instance_role.list()Get roleBodoClient.instance_role.get(cluster_uuid)Returns role by uuidfrombodosdk.modelsimportWorkspaceKeys,InstanceRoleItemfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)clusters:InstanceRoleItem=client.instance_role.get('<CLUSTER-UUID>')Remove roleBodoClient.instance_role.remove(cluster_uuid, mark_as_terminated=False)ParametersTypeDescriptionRequiredmark_as_terminatedBooleanMark role as terminated without removing resources,may be useful if role creation failed and deletion is failingNofrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.instance_role.remove('<ROLE-UUID>')CatalogModule responsible for storing database catalogscreate catalogget catalog by uuidget catalog by namelist catalogsupdate catalogremove catalogremove all catalogsCreate CatalogBodoClient.catalog.create()Stores the Database Catalogfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.catalogimportCatalogDefinition,SnowflakeConnectionDefinitionkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)# Type Support for Snowflakesnowflake_definition=SnowflakeConnectionDefinition(host="test.snowflake.com",port=443,username="test-username",password="password",database="test-db",warehouse="test-wh",role="test-role")# For other databases, need to defined as JSONconnection_data={"host":"test.db.com","username":"test-username","password":"*****","database":"test-db",}catalog_definition=CatalogDefinition(name="catalog-1",description="catalog description",catalogType="SNOWFLAKE",# Currently Support Snowflakedata=snowflake_definition)client.catalog.create(catalog_definition)Get Catalog by UUIDBodoClient.catalog.get_catalog()Retrieves the Catalog details by UUIDfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.catalogimportCatalogInfokeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)catalog_info:CatalogInfo=client.catalog.get("<CATALOG-UUID>")Get Catalog by NameBodoClient.catalog.get_by_name()Retrieves the Catalog details by UUIDfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.catalogimportCatalogInfokeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)catalog_info:CatalogInfo=client.catalog.get_by_name("test-catalog")List CatalogsBodoClient.catalog.list()Retrieves all catalogs in a workspace.frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.catalogimportCatalogInfokeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)catalog_info:CatalogInfo=client.catalog.list()Update CatalogBodoClient.catalog.update()Updates the Database Catalogfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.catalogimportCatalogDefinition,SnowflakeConnectionDefinitionkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)# Type Support for Snowflakesnowflake_definition=SnowflakeConnectionDefinition(host="update.snowflake.com",port=443,username="test-username",password="password",database="test-db",warehouse="test-wh",role="test-role")new_catalog_def=CatalogDefinition(name="catalog-1",description="catalog description",catalogType="SNOWFLAKE",# Currently Support Snowflakedata=snowflake_definition)client.catalog.update("<CATALOG-UUID>",new_catalog_def)Remove Catalog by UUIDBodoClient.catalog.remove()Deletes a Database Catalog by UUIDfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.catalog.remove("<CATALOG-UUID>")Remove all CatalogsBodoClient.catalog.remove()Deletes a Database Catalog by UUIDfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.catalog.remove_all()Secret GroupsModule responsible for separating secrets into multiple groups.A default secret group will be created at the time of workspace creation.
Users can define custom secret groups using the following functions.create secret grouplist secret groupsupdate secret groupdelete secret groupCreate Secret GroupBodoClient.secret_group.create()Create a secret groupfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secret_groupimportSecretGroupDefinitionkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)secret_group_definition=SecretGroupDefinition(name="sg-1",# Name should be unique to that workspacedescription="secret group description",)client.secret_group.create(secret_group_definition)List Secret GroupsBodoClient.secret_group.list()List all the secret groups in a workspace.frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secret_groupimportSecretGroupInfofromtypingimportListkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)groups_list:List[SecretGroupInfo]=client.secret_group.list()Update Secret GroupBodoClient.secret_group.update()Updates the secret group descriptionfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secret_groupimportSecretGroupInfo,SecretGroupDefinitionkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)update_secret_group_def=SecretGroupDefinition(name="sg-1",# Cannot modify the name in the groupdescription="secret group description",)groups_data:SecretGroupInfo=client.secret_group.update(update_secret_group_def)Delete Secret GroupBodoClient.secret_group.remove()Removes the secret group.Note: Can only remove if all the secrets in the group are deletedfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)client.secret_group.remove("<secret-group-uuid>")SecretsModule responsible for creating secrets.create secretget secretlist secretslist secrets by secret groupupdate secretdelete secretCreate SecretBodoClient.secrets.create()Create the secret in a secret group.frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secretsimportSecretDefinitionkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)secret_definition=SecretDefinition(name="secret-1",data={"key":"value"},secret_group="<secret-group-name>"#If not defined, defaults to default to secret group)client.secrets.create(secret_definition)Get Secrets by UUIDBodoClient.secrets.get()Retrieves the Secrets by UUIDfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secretsimportSecretInfokeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)secret_info:SecretInfo=client.secrets.get("<secret-uuid>")List Secrets by WorkspaceBodoClient.secrets.list()List the secrets in a workspacefrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secretsimportSecretInfofromtypingimportListkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)secrets_info:List[SecretInfo]=client.secrets.list()List Secrets by Secret GroupBodoClient.secrets.list_by_group()List the Secrets by Secret Groupfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secretsimportSecretInfofromtypingimportListkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)secrets_info:List[SecretInfo]=client.secrets.list_by_group("<secret-group-name>")Update SecretBodoClient.secrets.update()Updates the secret.frombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secretsimportSecretDefinitionkeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)update_secret_def=SecretDefinition(data={"key":"value"})client.secrets.update("<secret-uuid>",update_secret_def)Delete Secrets by UUIDBodoClient.secrets.remove()Delete the Secret by UUIDfrombodosdk.modelsimportWorkspaceKeysfrombodosdk.clientimportget_bodo_clientfrombodosdk.models.secretsimportSecretInfokeys=WorkspaceKeys(client_id='XYZ',secret_key='XYZ')client=get_bodo_client(keys)secret_info:SecretInfo=client.secrets.remove("<secret-uuid>")Billing moduleBilling module provides access to billing information related to a particular workspace.Get job run billing report CSVGet cluster level billing report CSVGet job run billing report CSVBodoClient.billing.get_job_run_price_export(started_at: str, finished_at: str, workspace_uuid: Union[str, UUID])Provides a CSV download link for the billing report, specifically on the job run level, displaying EC2 costs per job run
within the defined startTime and endTime range for all workspaces.For a particular workspace, provide the workspaceUUID which can be obtained fromlist-workspacesThe billing report includes essential fields such as start time, end time, duration, worker count, instance type, and
associated costs. This link remains active for a duration of 7 days and is exclusively available to AWS customers.
It's important to note that reports can only be generated for a 30-day time period.Parameters:ParametersTypeDescriptionRequiredstarted_atUnion[str, date]Start date of the reportYesfinished_atUnion[str, date]End date of the reportYesworkspace_uuidUnion[str, UUID]Workspace UUID, returns all workspaces in organization when nullNoReturns:FieldsTypeDescriptionurlstringS3 Pre-signed URL which contains the reportCSV report for date rangeThe following Python code generates a CSV report for a specified date range, starting from September 5th to
September 6th for a given workspace UUID. .
You can also provide the same date range with a timestamp in the ISO8601 format, such as2023-09-05T11:15:00Z. After running the code, it will display a link to the CSV report that can be clicked to
initiate the download:frombodosdk.modelsimportOrganizationKeysfrombodosdk.clientimportget_bodo_organization_clientkeys=OrganizationKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_organization_client(keys)print(client.billing.get_job_run_price_export('2023-09-05','2023-09-06','WORKSPACE-UUID'))Get cluster level billing report CSVBodoClient.billing.get_cluster_price_export(started_at: str, finished_at: str, workspace_uuid: Union[str, UUID])Provides a CSV download link for the billing report, specifically at the cluster level, displaying EC2 costs per cluster run
within the defined startTime and endTime range among all workspaces.For a particular workspace, provide the workspaceUUID which can be obtained fromlist-workspacesThe billing report includes essential fields such as start time, end time, duration, worker count, instance type, and associated costs.
This link remains active for a duration of 7 days and is exclusively available to AWS customers.
It's important to note that reports can only be generated for a 30-day time period.Parameters:ParametersTypeDescriptionRequiredstarted_atUnion[str, date]Start date of the reportYesfinished_atUnion[str, date]End date of the reportYesworkspace_uuidUnion[str, UUID]Workspace UUID,returns all workspaces in organization when nullNoReturns:FieldsTypeDescriptionurlstringS3 Pre-signed URL which contains the reportCSV report for date rangeThe following Python code generates a CSV report for a specified date range, starting from September 5th to
September 6th for a given workspace UUID.
You can also provide the same date range with a timestamp in the ISO8601 format, such as2023-09-05T11:15:00Z.
After running the code, it will display a link to the CSV report that can be clicked to initiate the download:frombodosdk.modelsimportOrganizationKeysfrombodosdk.clientimportget_bodo_organization_clientkeys=OrganizationKeys(client_id="XYZ",secret_key="XYZ")client=get_bodo_organization_client(keys)print(client.billing.get_cluster_price_export('2023-09-05','2023-09-06','WORKSPACE-UUID'))