Package: databricks-labs-blueprint
Databricks Labs BlueprintBaseline for Databricks Labs projects written in Python. Sources are validated withmypyandpylint. SeeContributing instructionsif you would like to improve this project.InstallationBatteries IncludedBasic Terminal User Interface (TUI) PrimitivesSimple Text QuestionsConfirming ActionsSingle Choice from ListSingle Choice from DictionaryMultiple Choices from DictionaryUnit Testing PromptsNicer Logging FormatterRendering on Dark BackgroundRendering in Databricks NotebooksIntegration With Your AppIntegration withconsole_scriptEntrypointsParallel Task ExecutionCollecting ResultsCollecting Errors from Background TasksStrict Failures from Background TasksApplication and Installation StateInstall FolderDetecting Current InstallationDetecting Installations From All UsersSaving@dataclassconfigurationSaving CSV filesLoading@dataclassconfigurationConfiguration Format EvolutionUploading Untyped FilesListing All Files in the Install FolderUnit Testing Installation StateAssert Rewriting with PyTestBuilding WheelsReleased Version DetectionUnreleased Version DetectionApplication Name DetectionPublishing Wheels to Databricks WorkspaceDatabricks CLI'sdatabricks labs ...RouterAccount-level CommandsIntegration with Databricks ConnectStarting New ProjectsNotable Downstream ProjectsProject SupportInstallationYou can install this project viapip:pip install databricks-labs-blueprintBatteries IncludedThis library contains a proven set of building blocks, tested in production throughUCXand projects.Basic Terminal User Interface (TUI) PrimitivesYour command-line apps do need testable interactivity, which is provided byfrom databricks.labs.blueprint.tui import Prompts. Here are some examples of it:[back to top]Simple Text QuestionsUseprompts.question()as a bit more involved thaninput()builtin:fromdatabricks.labs.blueprint.tuiimportPromptsprompts=Prompts()answer=prompts.question('Enter a year',default='2024',valid_number=True)print(answer)Optional arguments are:default(str) - use given value if user didn't input anythingmax_attempts(int, default 10) - number of attempts to throw exception after invalid or empty inputvalid_number(bool) - input has to be a valid numbervalid_regex(bool) - input has to be a valid regular expressionvalidate- function that takes a string and returns boolean, likelambda x: 'awesome' in x, that could be used to further validate input.[back to top]Confirming ActionsUseprompts.confirm()to guard any optional or destructive actions of your app:ifprompts.confirm('Destroy database?'):print('DESTROYING DATABASE')[back to top]Single Choice from ListUse to select a value from a list:answer=prompts.choice('Select a language',['Python','Rust','Go','Java'])print(answer)[back to top]Single Choice from DictionaryUse to select a value from the dictionary by showing users sorted dictionary keys:answer=prompts.choice_from_dict('Select a locale',{'Українська':'ua','English':'en'})print(f'Locale is:{answer}')[back to top]Multiple Choices from DictionaryUse to select multiple items from dictionaryanswer=prompts.multiple_choice_from_dict('What projects are written in Python? Select [DONE] when ready.',{'Databricks Labs UCX':'ucx','Databricks SDK for Python':'sdk-py','Databricks SDK for Go':'sdk-go','Databricks CLI':'cli',})print(f'Answer is:{answer}')[back to top]Unit Testing PromptsUseMockPromptswith regular expressions as keys and values as answers. The longest key takes precedence.fromdatabricks.labs.blueprint.tuiimportMockPromptsdeftest_ask_for_int():prompts=MockPrompts({r".*":""})res=prompts.question("Number of threads",default="8",valid_number=True)assert"8"==res[back to top]Nicer Logging FormatterThere's a basic logging configuration available forPython SDK, but the default output is not pretty and is relatively inconvenient to read. Here's how make output from Python's standard logging facility more enjoyable to read:fromdatabricks.labs.blueprint.loggerimportinstall_loggerinstall_logger()importlogginglogging.root.setLevel("DEBUG")# use only for development or demo purposeslogger=logging.getLogger("name.of.your.module")logger.debug("This is a debug message")logger.info("This is an table message")logger.warning("This is a warning message")logger.error("This is an error message",exc_info=KeyError(123))logger.critical("This is a critical message")Here are the assumptions made by this formatter:Most likely you're forwarding your logs to a file already, this log formatter is mainly for visual consumption.The average app or Databricks Job most likely finishes running within a day or two, so we display only hours, minutes, and seconds from the timestamp.We gray out debug messages, and highlight all other messages. Errors and fatas are additionally painted with red.We shorten the name of the logger to a readable chunk only, not to clutter the space. Real-world apps have deeply nested folder structures and filenames likesrc/databricks/labs/ucx/migration/something.py, which translate intodatabricks.labs.ucx.migration.somethingfully-qualified Python module names, that get reflected into__name__top-level code environmentspecial variable, that you idiomatically use with logging aslogger.getLogger(__name__). This log formatter shortens the full module path to a more readabled.l.u.migration.something, which is easier to consume from a terminal screen or a notebook.We only show the name of the thread if it's other thanMainThread, because the overwhelming majority of Python applications are single-threaded.[back to top]Rendering on Dark BackgroundHere's how the output would look like on dark terminal backgrounds, including those from GitHub Actions:[back to top]Rendering in Databricks NotebooksAnd here's how things will appear when executed from Databricks Runtime as part of notebook or a workflow:[back to top]Integration With Your AppJust place the following code in your wheel's top-most__init__.pyfile:fromdatabricks.labs.blueprint.loggerimportinstall_loggerinstall_logger(level="INFO")And place this idiomatic# ... insert this into the top of your filefromdatabricks.labs.blueprint.entrypointimportget_loggerlogger=get_logger(__file__)# ... top of the file insert end... and you'll be able to benefit from the readable console stderr formatting everywhereEach time you'd need to turn on debug logging, just invokelogging.root.setLevel("DEBUG")(even in notebook).[back to top]Integration withconsole_scriptEntrypointsWhen you invoke Python as an entry point to your wheel (also known asconsole_scripts),__name__top-level code environmentwould always be equal to__main__. But you really want to get the logger to be named after your Python module and not just__main__(seerendering in Databricks notebooks).If you create adist/logger.pyfile with the following contents:fromdatabricks.labs.blueprint.entrypointimportget_logger,run_mainlogger=get_logger(__file__)defmain(first_arg,second_arg,*other):logger.info(f'First arg is:{first_arg}')logger.info(f'Second arg is:{second_arg}')logger.info(f'Everything else is:{other}')logger.debug('... and this message is only shown when you are debugging from PyCharm IDE')if__name__=='__main__':run_main(main)... and invoke it withpython dist/logger.py Hello world, my name is Serge, you should get back the following output.13:46:42  INFO [dist.logger] First arg is: Hello
13:46:42  INFO [dist.logger] Second arg is: world,
13:46:42  INFO [dist.logger] Everything else is: ('my', 'name', 'is', 'Serge')Everything is made easy thanks torun_main(fn)helper.[back to top]Parallel Task ExecutionPython applies global interpreter lock (GIL) for compute-intensive tasks, though IO-intensive tasks, like calling Databricks APIs through Databricks SDK for Python, are not subject to GIL. It's quite a common task to perform multiple different API calls in parallel, though it is overwhelmingly difficult to do multi-threading right.concurrent.futures import ThreadPoolExecutoris great, but sometimes we want something even more high level. This library helps you navigate the most common road bumps.[back to top]Collecting ResultsThis library helps you filtering out empty results from background tasks, so that the downstream code is generally simpler. We're also handling the thread pool namind, so that the name of the list of tasks properly gets into log messages. After all background tasks completed their execution, we log something likeFinished 'task group name' tasks: 50% results available (2/4). Took 0:00:00.000604.fromdatabricks.labs.blueprint.parallelimportThreadsdefnot_really_but_fine():logger.info("did something, but returned None")defdoing_something():logger.info("doing something important")returnf'result from{doing_something.__name__}'logger.root.setLevel('DEBUG')tasks=[not_really_but_fine,not_really_but_fine,doing_something,doing_something]results,errors=Threads.gather("task group name",tasks)assert['result from doing_something','result from doing_something']==resultsassert[]==errorsThis will log the following messages:14:20:15 DEBUG [d.l.blueprint.parallel] Starting 4 tasks in 20 threads
14:20:15  INFO [dist.logger][task_group_name_0] did something, but returned None
14:20:15  INFO [dist.logger][task_group_name_1] did something, but returned None
14:20:15  INFO [dist.logger][task_group_name_1] doing something important
14:20:15  INFO [dist.logger][task_group_name_1] doing something important
14:20:15  INFO [d.l.blueprint.parallel][task_group_name_1] task group name 4/4, rps: 7905.138/sec
14:20:15  INFO [d.l.blueprint.parallel] Finished 'task group name' tasks: 50% results available (2/4). Took 0:00:00.000604[back to top]Collecting Errors from Background TasksInspired by Go Language's idiomatic error handling approach, this library allows for collecting errors from all of the background tasks and handle them separately. For all other cases, we recommend usingstrict failuresfromdatabricks.sdk.errorsimportNotFoundfromdatabricks.labs.blueprint.parallelimportThreadsdefworks():returnTruedeffails():raiseNotFound("something is not right")tasks=[works,fails,works,fails,works,fails,works,fails]results,errors=Threads.gather("doing some work",tasks)assert[True,True,True,True]==resultsassert4==len(errors)This will log the following messages:14:08:31 ERROR [d.l.blueprint.parallel][doing_some_work_0] doing some work task failed: something is not right: ...
...
14:08:31 ERROR [d.l.blueprint.parallel][doing_some_work_3] doing some work task failed: something is not right: ...
14:08:31 ERROR [d.l.blueprint.parallel] More than half 'doing some work' tasks failed: 50% results available (4/8). Took 0:00:00.001011[back to top]Strict Failures from Background TasksUseThreads.strict(...)to raiseManyErrorwith the summary of all failed tasks:fromdatabricks.sdk.errorsimportNotFoundfromdatabricks.labs.blueprint.parallelimportThreadsdefworks():returnTruedeffails():raiseNotFound("something is not right")tasks=[works,fails,works,fails,works,fails,works,fails]results=Threads.strict("doing some work",tasks)# this line won't get executedassert[True,True,True,True]==resultsThis will log the following messages:...
14:11:46 ERROR [d.l.blueprint.parallel] More than half 'doing some work' tasks failed: 50% results available (4/8). Took 0:00:00.001098
...
databricks.labs.blueprint.parallel.ManyError: Detected 4 failures: NotFound: something is not right[back to top]Application and Installation StateThere always needs to be a location, where you put application code, artifacts, and configuration.
TheInstallationclass is used to manage the~/.{product}folder on WorkspaceFS to tracktyped files.
It provides methods for serializing and deserializing objects of a specific type, as well as managing thestorage locationfor those objects. The class includes methods for loading and saving objects, uploading and downloading
files, and managing the installation folder.TheInstallationclass can be helpful for unit testing by allowing you to mock the file system and control
the behavior of theloadandsavemethods.
Seeunit testingfor more details.[back to top]Install FolderTheinstall_foldermethod returns the path to the installation folder on WorkspaceFS. The installation folder
is used to store typed files that are managed by theInstallationclass.Publishing wheelsupdate theversion.jsonfile in the install folder.If aninstall_folderargument is provided to the constructor of theInstallationclass, it will be used
as the installation folder. Otherwise, the installation folder will be determined based on the current user's
username. Specifically, the installation folder will be/Users/{user_name}/.{product}, where{user_name}is the username of the current user and{product}is thename of the productassociated with the installation. Here is an example of how you can use theinstall_foldermethod:fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallation# Create an Installation object for the "blueprint" productinstall=Installation(WorkspaceClient(),"blueprint")# Print the path to the installation folderprint(install.install_folder())# Output: /Users/{user_name}/.blueprintIn this example, theInstallationobject is created for the "blueprint" product. Theinstall_foldermethod
is then called to print the path to the installation folder. The output will be/Users/{user_name}/.blueprint,
where{user_name}is the username of the current user.You can also provide aninstall_folderargument to the constructor to specify a custom installation folder.
Here is an example of how you can do this:fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallation# Create an Installation object for the "blueprint" product with a custom installation folderinstall=Installation(WorkspaceClient(),"blueprint",install_folder="/my/custom/folder")# Print the path to the installation folderprint(install.install_folder())# Output: /my/custom/folderIn this example, theInstallationobject is created for the "blueprint" product with a custom installation
folder of/my/custom/folder. Theinstall_foldermethod is then called to print the path to the installation
folder. The output will be/my/custom/folder.[back to top]Detecting Current InstallationInstallation.current(ws, product)returns theInstallationobject for the given product in the current workspace.If the installation is not found, aNotFounderror is raised. Ifassume_userargument is True, the method
will assume that the installation is in the user's home directory and return it if found. If False, the method
will only return an installation that is in the/Applicationsdirectory.fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallationws=WorkspaceClient()# current user installationinstallation=Installation.current(ws,"blueprint",assume_user=True)assert"/Users/foo/.blueprint"==installation.install_folder()# workspace global installationinstallation=Installation.current(ws,"blueprint")assert"/Applications/blueprint"==installation.install_folder()[back to top]Detecting Installations From All UsersInstallation.existing(ws, product)Returns a collection of all existing installations for the given product in the current workspace.This method searches for installations in the root /Applications directory and home directories of all users in the workspace.
Let's say, usersfoo@example.comandbar@example.cominstalledblueprintproduct in their home folders. The following
code will print/Workspace/bar@example.com/.blueprintand/Workspace/foo@example.com/.blueprint:forblueprintinInstallation.existing(ws,"blueprint"):print(blueprint.install_folder())[back to top]Saving@dataclassconfigurationThesave(obj)method saves a dataclass instance of typeTto a file on WorkspaceFS. If nofilenameis provided,
the name of thetype_refclass will be used as the filename. Any missing parent directories are created automatically.
If the object has a__version__attribute, the method will add aversionfield to the serialized object
with the value of the__version__attribute. Seeconfiguration format evolutionfor more details.save(obj)works with JSON and YAML configurations without the need to supplyfilenamekeyword
attribute. When you need to saveCSV files, thefilenameattribute is required. If you need to
upload arbitrary and untyped files, use theupload()method.Here is an example of how you can use thesavemethod:fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallationinstall=Installation(WorkspaceClient(),"blueprint")@dataclassclassMyClass:field1:strfield2:strobj=MyClass('value1','value2')install.save(obj)# Verify that the object was saved correctlyloaded_obj=install.load(MyClass)assertloaded_obj==objIn this example, theInstallationobject is created for the "blueprint" product. A dataclass object of typeMyClassis then created and saved to a file using thesavemethod. The object is then loaded from the file
using theloadmethodand compared to the original object to verify that
it was saved correctly.[back to top]Saving CSV filesYou may need to upload a CSV file to Databricks Workspace, so that it's easier editable from a Databricks Workspace UI
or tools like Google Sheets or Microsoft Excel. If non-technical humands don't need to edit application state,
usedataclassesfor configuration. CSV files currently don't supportformat evolution.The following example will saveworkspaces.csvfile with two records and a header:fromdatabricks.sdkimportWorkspaceClientfromdatabricks.sdk.service.provisioningimportWorkspacefromdatabricks.labs.blueprint.installationimportInstallationinstallation=Installation(WorkspaceClient(),"blueprint")installation.save([Workspace(workspace_id=1234,workspace_name="first"),Workspace(workspace_id=1235,workspace_name="second"),],filename="workspaces.csv")# ~ $ databricks workspace export /Users/foo@example.com/.blueprint/workspaces.csv# ... workspace_id,workspace_name# ... 1234,first# ... 1235,second[back to top]Loading@dataclassconfigurationTheload(type_ref[, filename])method loads an object of typetype_reffrom a file on WorkspaceFS. If nofilenameis
provided, the__file__attribute oftype_refwill be used as the filename, otherwise the library will figure out the name
based on a class name.fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallation@dataclassclassSomeConfig:# <-- auto-detected filename is `some-config.json`version:strws=WorkspaceClient()installation=Installation.current(ws,"blueprint")cfg=installation.load(SomeConfig)installation.save(SomeConfig("0.1.2"))installation.assert_file_written("some-config.json",{"version":"0.1.2"})[back to top]Configuration Format EvolutionAs time progresses, your application evolves. So does the configuration file format with it. This library provides
a common utility to seamlessly evolve configuration file format across versions, providing callbacks to convert
from older versions to newer.If the type has a__version__attribute, the method will check that the version of the object in the file
matches the expected version. If the versions do not match, the method will attempt to migrate the object to
the expected version using a method namedv{actual_version}_migrateon thetype_refclass. If the migration
is successful, the method will return the migrated object. If the migration is not successful, the method will
raise anIllegalStateexception. Let's say, we have/Users/foo@example.com/.blueprint/config.ymlfile with
only theinitial: 999as content, which is from older installations of theblueprintproduct:fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallation@dataclassclassEvolvedConfig:__file__="config.yml"__version__=3initial:intadded_in_v1:intadded_in_v2:int@staticmethoddefv1_migrate(raw:dict)->dict:raw["added_in_v1"]=111raw["version"]=2returnraw@staticmethoddefv2_migrate(raw:dict)->dict:raw["added_in_v2"]=222raw["version"]=3returnrawinstallation=Installation.current(WorkspaceClient(),"blueprint")cfg=installation.load(EvolvedConfig)assert999==cfg.initialassert111==cfg.added_in_v1# <-- added by v1_migrate()assert222==cfg.added_in_v2# <-- added by v2_migrate()[back to top]Uploading Untyped FilesTheupload(filename, raw_bytes)andupload_dbfs(filename, raw_bytes)methods upload raw bytes to a file on
WorkspaceFS (or DBFS) with the givenfilename, creating any missing directories where required. This method
is used to upload files that are not typed, i.e., they do not use the@dataclassdecorator.installation=Installation(ws,"blueprint")target=installation.upload("wheels/foo.whl",b"abc")assert"/Users/foo/.blueprint/wheels/foo.whl"==targetThe most common example is awheel, which we already integrate withInstallationframework.[back to top]Listing All Files in the Install FolderYou can usefiles()method to recursively list all files in theinstall folder.[back to top]Unit Testing Installation StateYou can create aMockInstallationobject and use it to override the default installation folder and the contents
of the files in that folder. This allows you to test the of your code in different scenarios, such as when a file
is not found or when the contents of a file do not match the expected format.For example, you have the followingWorkspaceConfigclass that is serialized intoconfig.ymlon your workspace:@dataclassclassWorkspaceConfig:__file__="config.yml"__version__=2inventory_database:strconnect:Config|None=Noneworkspace_group_regex:str|None=Noneinclude_group_names:list[str]|None=Nonenum_threads:int|None=10database_to_catalog_mapping:dict[str,str]|None=Nonelog_level:str|None="INFO"workspace_start_path:str="/"Here's the only code necessary to verify that specific content got written:fromdatabricks.labs.blueprint.installationimportMockInstallationinstallation=MockInstallation()installation.save(WorkspaceConfig(inventory_database="some_blueprint"))installation.assert_file_written("config.yml",{"version":2,"inventory_database":"some_blueprint","log_level":"INFO","num_threads":10,"workspace_start_path":"/",})This method is far superior than directly comparing raw bytes content via mock:ws.workspace.upload.assert_called_with("/Users/foo/.blueprint/config.yml",yaml.dump({"version":2,"num_threads":10,"inventory_database":"some_blueprint","include_group_names":["foo","bar"],"workspace_start_path":"/","log_level":"INFO",}).encode("utf8"),format=ImportFormat.AUTO,overwrite=True,)And it's even better if you use PyTest, where we have evendeeper integration.[back to top]Assert Rewriting with PyTestIf you are usingPyTest, then add this to yourconftest.py, so that
the assertions are more readable:importpytestpytest.register_assert_rewrite('databricks.labs.blueprint.installation')[back to top]Building WheelsWe recommend deploying applications as wheels, which are part of theapplication installation. But versioning, testing, and deploying those is often a tedious process.Released Version DetectionWhen you deploy your Python app as a wheel, every time it has to have a different version. This library detects__about__.pyfile automatically anywhere in the project root and reads__version__variable from it. We supportSemVerversioning scheme.Publishing wheelsupdateversion.jsonfile in theinstall folder.fromdatabricks.labs.blueprint.wheelsimportProductInfoproduct_info=ProductInfo(__file__)version=product_info.released_version()logger.info(f'Version is:{version}')[back to top]Unreleased Version DetectionWhen you develop your wheel and iterate on testing it, it's often required to upload a file with different name each time you build it. We usegit describe --tagscommand to fetch the latest SemVer-compatible tag (e.g.v0.0.2) and append the number of commits with timestamp to it. For example, if the released version isv0.0.1, then the unreleased version would be something like0.0.2+120240105144650. We verify that this version is compatible with both SemVer andPEP 440.Publishing wheelsupdateversion.jsonfile in theinstall folder.product_info=ProductInfo(__file__)version=product_info.unreleased_version()is_git=product_info.is_git_checkout()is_unreleased=product_info.is_unreleased_version()logger.info(f'Version is:{version}')logger.info(f'Git checkout:{is_git}')logger.info(f'Is unreleased:{is_unreleased}')[back to top]Application Name DetectionLibrary can infer the name of application by taking the directory name when__about__.pyfile is located within the current project. Seereleased version detectionfor more details.fromdatabricks.labs.blueprint.wheelsimportProductInfoproduct_info=ProductInfo(__file__)logger.info(f'Product name is:{product_info.product_name()}')[back to top]Publishing Wheels to Databricks WorkspaceBefore you execute a wheel on Databricks, you have to build it and upload it. This library provides detectsreleasedorunreleasedversion of the wheel, copies it over to a temporary folder, changes the__about__.pyfile with the right version, and builds the wheel in the temporary location, so that it's not polluted with build artifacts.Wheelsis a context manager, so it removes all temporary files and folders atherwithblock finishes. This library is successfully used to concurrently test wheels on Shared Databricks Clusters through notebook-scoped libraries.Every callwheels.upload_to_wsfs()updatesversion.jsonfile in theinstall folder, which holdsversionfield with the current wheel version. There's alsowheelfield, that contains the path to the current wheel file on WorkspaceFS.fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.installationimportInstallationfromdatabricks.labs.blueprint.wheelsimportProductInfo,WheelsV2w=WorkspaceClient()product_info=ProductInfo(__file__)installation=Installation(w,product_info.product_name())withWheelsV2(installation,product_info)aswheels:remote_wheel=wheels.upload_to_wsfs()logger.info(f'Uploaded to{remote_wheel}')This will print something like:15:08:44  INFO [dist.logger] Uploaded to /Users/serge.smertin@databricks.com/.blueprint/wheels/databricks_labs_blueprint-0.0.2+120240105150840-py3-none-any.whlYou can also dowheels.upload_to_dbfs(), though you're not able to set any access control over it.[back to top]Databricks CLI'sdatabricks labs ...RouterThis library contains common utilities for Databricks CLI entrypoints defined inlabs.ymlfile. Here's the example metadata for a tool namedblueprintwith a singlemecommand and flag named--greeting, that hasHelloas default value:---name:blueprintdescription:Common libraries for Databricks Labsinstall:script:src/databricks/labs/blueprint/__init__.pyentrypoint:src/databricks/labs/blueprint/__main__.pymin_python:3.10commands:-name:medescription:shows current usernameflags:-name:greetingdefault:Hellodescription:Greeting prefixAnd here's the content forsrc/databricks/labs/blueprint/__main__.pyfile, that executesdatabricks labs blueprint mecommand withdatabricks.sdk.WorkspaceClientautomatically injected into an argument with magical namew:fromdatabricks.sdkimportWorkspaceClientfromdatabricks.labs.blueprint.entrypointimportget_loggerfromdatabricks.labs.blueprint.cliimportAppapp=App(__file__)logger=get_logger(__file__)@app.commanddefme(w:WorkspaceClient,greeting:str):"""Shows current username"""logger.info(f"{greeting},{w.current_user.me().user_name}!")if"__main__"==__name__:app()[back to top]Account-level CommandsAs you may have noticed, there were only workspace-level commands, but you can also nave native account-level command support. You need to specify theis_accountproperty when declaring it inlabs.ymlfile:commands:# ...-name:workspacesis_account:truedescription:shows current workspacesand@app.command(is_account=True)will get youdatabricks.sdk.AccountClientinjected intoaargument:fromdatabricks.sdkimportAccountClient@app.command(is_account=True)defworkspaces(a:AccountClient):"""Shows workspaces"""forwsina.workspaces.list():logger.info(f"Workspace:{ws.workspace_name}({ws.workspace_id})")[back to top]Integration with Databricks ConnectInvoking Sparksession using Databricks Connectfromdatabricks.sdkimportWorkspaceClientfromdatabricks.connectimportDatabricksSession@app.commanddefexample(w:WorkspaceClient):"""Building Spark Session using Databricks Connect"""spark=DatabricksSession.builder().sdk_config(w.config).getOrCreate()spark.sql("SHOW TABLES")[back to top]Starting New ProjectsThis tooling makes it easier to start new projects. First, install the CLI:databricks labs install blueprintAfter, create new project in a designated directory:databricks labs blueprint init-project --target /path/to/folder[back to top]Notable Downstream ProjectsThis library is used in the following projects:UCX - Automated upgrade to Unity Catalog[back to top]Project SupportPlease note that this project is provided for your exploration only and is not
formally supported by Databricks with Service Level Agreements (SLAs). They are
provided AS-IS, and we do not make any guarantees of any kind. Please do not
submit a support ticket relating to any issues arising from the use of this project.Any issues discovered through the use of this project should be filed as GitHubIssues on this repository.
They will be reviewed as time permits, but no formal SLAs for support exist.