Package: databricks-client
databricks-clientAboutA REST client for theDatabricks REST API.This module is a thin layer allowing to build HTTPRequests.
It does not expose API operations as distinct methods, but rather exposes generic methods allowing
to build API calls.The Databricks API sometimes returns 200 error codes and HTML content when the request is not
properly authenticated. The client intercepts such occurrences (detecting non-JSON returned content)
and wraps them into an exception.This open-source project is not developed by nor affiliated with Databricks.Installingpip install databricks-clientUsageimportdatabricks_clientclient=databricks_client.create("https://northeurope.azuredatabricks.net/api/2.0")client.auth_pat_token(pat_token)client.ensure_available()clusters_list=client.get('clusters/list')forclusterinclusters_list["clusters"]:print(cluster)Usage with a newly provisioned workspaceIf using this module as part of a provisioning job, you need to callclient.ensure_available().When the first user logs it to a new Databricks workspace, workspace provisioning is triggered,
and the API is not available until that job has completed (that usually takes under a minute,
but could take longer depending on the network configuration). In that case you would get an
error such as the following when calling the API:"Succeeded{"error_code":"INVALID_PARAMETER_VALUE","message":"Unknown worker environment WorkerEnvId(workerenv-4312344789891641)"}The methodclient.ensure_available(url="instance-pools/list", retries=100, delay_seconds=6)prevents this error by attempting to
connect to the provided URL and retries as long as the workspace is in provisioning
state, or until the given number of retries has elapsed.Usage with Azure Active DirectoryNote: Azure AD authentication for Databricks is currently in preview.The client generates short-lived Azure AD tokens. If you need to use your client for longer
than the lifetime (typically 30 minutes), rerunclient.auth_azureadperiodically.Azure AD authentication with Azure CLIInstall the Azure CLI.pip install databricks-client[azurecli]
az loginimportdatabricks_clientclient=databricks_client.create("https://northeurope.azuredatabricks.net/api/2.0")client.auth_azuread("/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-workspace")# or client.auth_azuread(resource_group="my-rg", workspace_name="my-workspace")client.ensure_available()clusters_list=client.get('clusters/list')forclusterinclusters_list["clusters"]:print(cluster)This is recommended with Azure DevOps Pipelines using theAzure CLI task.Azure AD authentication with ADALpip install databricks-client
pip install adalimportdatabricks_clientimportadalauthority_host_uri='https://login.microsoftonline.com'authority_uri=authority_host_uri+'/'+tenant_idcontext=adal.AuthenticationContext(authority_uri)deftoken_callback(resource):returncontext.acquire_token_with_client_credentials(resource,client_id,client_secret)["accessToken"]client=databricks_client.create("https://northeurope.azuredatabricks.net/api/2.0")client.auth_azuread("/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/my-rg/providers/Microsoft.Databricks/workspaces/my-workspace",token_callback)# or client.auth_azuread(resource_group="my-rg", workspace_name="my-workspace", token_callback=token_callback)client.ensure_available()clusters_list=client.get('clusters/list')forclusterinclusters_list["clusters"]:print(cluster)Example usagesGenerating a PAT tokenresponse=client.post('token/create',json={"lifetime_seconds":60,"comment":"Unit Test Token"})pat_token=response['token_value']Uploading a notebookimportbase64withopen(notebook_file,"rb")asf:file_content=f.read()client.post('workspace/import',json={"content":base64.b64encode(file_content).decode('ascii'),"path":notebook_path,"overwrite":False,"language":"PYTHON","format":"SOURCE"})