Package: databricks-dbapi
A thin wrapper aroundpyhiveandpyodbcfor creating aDBAPIconnection to Databricks Workspace and SQL Analytics clusters. SQL Analytics clusters require theSimba ODBC driver.Also provides SQLAlchemy Dialects usingpyhiveandpyodbcfor Databricks clusters. Databricks SQL Analytics clusters only support thepyodbc-driven dialect.InstallationInstall using pip. Youmustspecify at least one of the extras {hiveorodbc}. ForodbctheSimba driveris required:pipinstalldatabricks-dbapi[hive,odbc]For SQLAlchemy support install with:pipinstalldatabricks-dbapi[hive,odbc,sqlalchemy]UsagePyHiveTheconnect()function returns apyhiveHive connection object, which internally wraps athriftconnection.Connecting withhttp_path,host, and atoken:importosfromdatabricks_dbapiimporthivetoken=os.environ["DATABRICKS_TOKEN"]host=os.environ["DATABRICKS_HOST"]http_path=os.environ["DATABRICKS_HTTP_PATH"]connection=hive.connect(host=host,http_path=http_path,token=token,)cursor=connection.cursor()cursor.execute("SELECT * FROM some_table LIMIT 100")print(cursor.fetchone())print(cursor.fetchall())Thepyhiveconnection also provides async functionality:importosfromdatabricks_dbapiimporthivefromTCLIService.ttypesimportTOperationStatetoken=os.environ["DATABRICKS_TOKEN"]host=os.environ["DATABRICKS_HOST"]cluster=os.environ["DATABRICKS_CLUSTER"]connection=hive.connect(host=host,cluster=cluster,token=token,)cursor=connection.cursor()cursor.execute("SELECT * FROM some_table LIMIT 100",async_=True)status=cursor.poll().operationStatewhilestatusin(TOperationState.INITIALIZED_STATE,TOperationState.RUNNING_STATE):logs=cursor.fetch_logs()formessageinlogs:print(message)# If needed, an asynchronous query can be cancelled at any time with:# cursor.cancel()status=cursor.poll().operationStateprint(cursor.fetchall())ODBCThe ODBC DBAPI requires the Simba ODBC driver.Connecting withhttp_path,host, and atoken:importosfromdatabricks_dbapiimportodbctoken=os.environ["DATABRICKS_TOKEN"]host=os.environ["DATABRICKS_HOST"]http_path=os.environ["DATABRICKS_HTTP_PATH"]connection=odbc.connect(host=host,http_path=http_path,token=token,driver_path="/path/to/simba/driver",)cursor=connection.cursor()cursor.execute("SELECT * FROM some_table LIMIT 100")print(cursor.fetchone())print(cursor.fetchall())SQLAlchemy Dialectsdatabricks+pyhiveInstalling registers thedatabricks+pyhivedialect/driver with SQLAlchemy. Fill in the required information when passing the engine URL.fromsqlalchemyimport*fromsqlalchemy.engineimportcreate_enginefromsqlalchemy.schemaimport*engine=create_engine("databricks+pyhive://token:<databricks_token>@<host>:<port>/<database>",connect_args={"http_path":"<cluster_http_path>"})logs=Table("my_table",MetaData(bind=engine),autoload=True)print(select([func.count("*")],from_obj=logs).scalar())databricks+pyodbcInstalling registers thedatabricks+pyodbcdialect/driver with SQLAlchemy. Fill in the required information when passing the engine URL.fromsqlalchemyimport*fromsqlalchemy.engineimportcreate_enginefromsqlalchemy.schemaimport*engine=create_engine("databricks+pyodbc://token:<databricks_token>@<host>:<port>/<database>",connect_args={"http_path":"<cluster_http_path>","driver_path":"/path/to/simba/driver"})logs=Table("my_table",MetaData(bind=engine),autoload=True)print(select([func.count("*")],from_obj=logs).scalar())Refer to the following documentation for more details on hostname, cluster name, and http path:DatabricksAzure DatabricksRelatedpyhivethriftpyodbc