connection-helper 0.13.3
A collection of helper for sql connections
Requires Python: >=3.11
Keywords: sql, duckdb, sqlite

# connection-helper

![PyPI - Version](https://img.shields.io/pypi/v/connection-helper) ![GitHub last commit](https://img.shields.io/github/last-commit/smeisegeier/connection-helper?logo=github) ![GitHub License](https://img.shields.io/github/license/smeisegeier/connection-helper?logo=github) ![py](https://img.shields.io/badge/python->=3.11-blue.svg?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAxMDAgMTAwIj4KICA8ZGVmcz4KICAgIDxsaW5lYXJHcmFkaWVudCBpZD0icHlZZWxsb3ciIGdyYWRpZW50VHJhbnNmb3JtPSJyb3RhdGUoNDUpIj4KICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iI2ZlNSIgb2Zmc2V0PSIwLjYiLz4KICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iI2RhMSIgb2Zmc2V0PSIxIi8+CiAgICA8L2xpbmVhckdyYWRpZW50PgogICAgPGxpbmVhckdyYWRpZW50IGlkPSJweUJsdWUiIGdyYWRpZW50VHJhbnNmb3JtPSJyb3RhdGUoNDUpIj4KICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzY5ZiIgb2Zmc2V0PSIwLjQiLz4KICAgICAgPHN0b3Agc3RvcC1jb2xvcj0iIzQ2OCIgb2Zmc2V0PSIxIi8+CiAgICA8L2xpbmVhckdyYWRpZW50PgogIDwvZGVmcz4KCiAgPHBhdGggZD0iTTI3LDE2YzAtNyw5LTEzLDI0LTEzYzE1LDAsMjMsNiwyMywxM2wwLDIyYzAsNy01LDEyLTExLDEybC0yNCwwYy04LDAtMTQsNi0xNCwxNWwwLDEwbC05LDBjLTgsMC0xMy05LTEzLTI0YzAtMTQsNS0yMywxMy0yM2wzNSwwbDAtM2wtMjQsMGwwLTlsMCwweiBNODgsNTB2MSIgZmlsbD0idXJsKCNweUJsdWUpIi8+CiAgPHBhdGggZD0iTTc0LDg3YzAsNy04LDEzLTIzLDEzYy0xNSwwLTI0LTYtMjQtMTNsMC0yMmMwLTcsNi0xMiwxMi0xMmwyNCwwYzgsMCwxNC03LDE0LTE1bDAtMTBsOSwwYzcsMCwxMyw5LDEzLDIzYzAsMTUtNiwyNC0xMywyNGwtMzUsMGwwLDNsMjMsMGwwLDlsMCwweiBNMTQwLDUwdjEiIGZpbGw9InVybCgjcHlZZWxsb3cpIi8+CgogIDxjaXJjbGUgcj0iNCIgY3g9IjY0IiBjeT0iODgiIGZpbGw9IiNGRkYiLz4KICA8Y2lyY2xlIHI9IjQiIGN4PSIzNyIgY3k9IjE1IiBmaWxsPSIjRkZGIi8+Cjwvc3ZnPgo=)

## usage

install / update package

```bash
uv add connection-helper -U
```

include in python

```python
from connection_helper import sql, pgp
```

## why use connection-helper

`connection-helper` bundles some packages for connecting to sql databases

- `sql` is added as convenience wrapper for retrieving data from sql databases
  - `connect_sql()` to get get data from `['mssql', 'sqlite','postgres']`
  - `load_sql_to_sqlite` connect to a sql db and transfer a list of tables to `sqlite`
  - `load_sqlite_to_parquet()` to get all tables from a sqlite file as parquets
  - `unpack_files_to_duckdb()` return a tuple of all files of a dir (csv or parquet) into high performance duckdb objects
  - `print_meta()` print meta information of a certain database (niche case)
  - `load_from_mssql()` load data from an MSSQL database into a Pandas DataFrame
  - `save_to_mssql()` save data from a Pandas DataFrame to an MSSQL database
  - `load_file_to_duckdb()` load data from various sources into a duckdb database, using pandas read functions (_experimental_)
  - `load_sqlite_to_duckdb()` converts a SQLite database to a DuckDB database
  - `load_mssql_to_duckdb()` converts a MSSQL database to a DuckDB database
  - `apply_duckdb_type_overrides()` applies column type overrides to tables in an existing DuckDB database
  - `add_duckdb_meta_table()` adds a meta table to an existing DuckDB database  
  - `optimize_duckdb()` copies a DuckDB database into a new one and applies optimizations to reduce size

</br>

- `pgp` cryptographic tools adapted from [python-gnupg](https://github.com/vsajip/python-gnupg). this is a wrapper around `gnupg`, but only offers some convenience or tailored options. feel free to use the original library or the [GNU Privacy Guard](https://gnupg.org/).
  - `encrypt()` a message for one or more recipient(s) with a public key
  - `decrypt()` a message with a private key. Passphrase must be provided via env variable
  - `find_key()` in keyring
  - ...

> 💡 The pgp methods require a pgp public/private key in the keyring

</br>

- `sec` secrets module. ⚠️ this package is optional and must be extra installed with `pip install 'connection-helper[sec]'` (enclose in '')
  - `get_infisical_secrets()` retrieve secrets from [Infisical](https://infisical.com/) project (requires existing account and machine identities)
  - `get_bitwarden_secrets()` retrieve secrets from [Bitwarden](https://bitwarden.com/) vault (requires existing account and machine identities)