Package: harlequin-adbc
harlequin-adbcThis repo provides the Harlequin adapter forADBC(arrow database connectivity).WarningADBC is a very new database connectivity method compared to ODBC, JDBC that aims at providing a more efficient way to transfer columnar data over the wire.Due to the recency of its development, there is various level of functionality across drivers which may cause issues.Installationharlequin-adbcdepends onadbc_driver_manager,harlequinandpyarrow, so installing this package with also install these dependencies.I also recommend installing the extra adbc drivers for the databases you plan on connecting to otherwise you will have to download the driver yourself from another place and provide thedriver_pathas a cli argument. The driver path connection method hasn't been thoroughly test.If you don't install the driver but provide a--drive-typecli argument you will get anImportErrorUsing pipTo install the adapter using pip into an activated virtual environment:pip install harlequin-adbcwith one of the driver extras:pip install harlequin-adbc[snowflake]Using poetrypoetry add harlequin-adbcwith one of the driver extras:poetry add harlequin-adbc[snowflake]Using pipxIf you do not already have Harlequin installed:pipx install harlequin-adbcpipx install harlequin-adbc[snowflake]If you would like to add ADBC adapter to an existing Harlequin installation:pipx inject harlequin harlequin-adbcMake sure to also inject the driver package you want as well. The following options are:adbc-driver-flightsqladbc-driver-postgresqladbc-driver-snowflakeadbc-driver-sqliteExample usage:pipx inject harlequin adbc-driver-snowflakeUsage and ConfigurationYou can open Harlequin with the ADBC adapter by selecting it with the-aoption and passing in a connection string. The format of the connection string will depend on the driver you are using. You will also need to provide either the--driver-typeor the--driver-path.Driver Type (preferred)When you use the --driver-type option it will try to dynamically use the driver package of the type selectedadbc-driver-{driver type}. That is why it's crucial to also have that package also install in the virtual environment where Harlequin is installed.--driver-typewith one of the following optionsflightsql, postgresql, snowflake, sqlite, duckdbDriver PathThe other option is to pass the file path location of the adbc driver that you are using. Note this method is not well tested.--driver-pathDB Kwargs String (Optional)Since the drivers implement so many different options to pass through when you connect to the database this is a way to pass through these options. The format of the string is key=value separated by ;--db-kwargs-strExample:--db-kwargs-str "username=flight_username;password=flight_password;adbc.flight.sql.client_option.tls_skip_verify=true"This will parse the string and pass through these values into thedb_kwargsvalue in thedbapi.connect()method. Note the parser relies on the ; and = so if any of of the parameters in the string contain either of these characters it's not going to work. This is a known limitation and something being worked on.Snowflake DriverThe Snowflake URI should be of one of the following formats:user[:password]@account/database/schema[?param1=value1&paramN=valueN]user[:password]@account/database[?param1=value1&paramN=valueN]user[:password]@host:port/database/schema?account=user_account[&param1=value1&paramN=valueN]host:port/database/schema?account=user_account[&param1=value1&paramN=valueN]Check theSnowflake ADBC Driver Docsfor more details.Example usage:harlequin -a adbc "$SNOWFLAKE_URI" --driver-type snowflakeFlightSQL DriverExample usage:harlequin -P None -a adbc "grpc+tls://localhost:31337" --driver-type flightsql --db-kwargs-str "username=flight_username;password=flight_password;adbc.flight.sql.client_option.tls_skip_verify=true"Check theFlightSQL ADBC Driver Docsfor more details.Postgres DriverThe Postgres URI should be in the format of aPostgres DSN:harlequin -a postgres "postgres://my-user:my-pass@localhost:5432/my-database"DuckDB and SQLiteWhile DuckDB and SQLite both have ADBC drivers it's not recommend to use them. Harlequin natively supports both of the databases without having to install any other dependencies.Know IssuesSnowflake adbc driver seems to be the only driver that returns thexdbc_data_typefromadbc_get_objects()Snowflake adbc driver has a bug withadbc_get_table_schema()that returnsadbc_driver_manager.OperationalError: IO: sql: expected 12 destination arguments in Scan, not 11The PostgreSQL adbc driver is overall buggy and when executing queries you might get the errorIO: [libpq] Fetch header failed: no COPY in progressSQLite and DuckDB don't have the same level ofdepthforadbc_get_objects()compared to other dbs which causes weird issues.DuckDB driver uses different names for theadbc_get_objects()which causes things to break.