airbook-duckdb-utils-vb 1.0.0
Utility functions for DuckDB to BigQuery data type mapping and other database operations
License: MIT
Requires Python: >=3.8
Keywords: duckdb bigquery database mapping airbook

# Airbook DuckDB Utils

A Python package providing utility functions for DuckDB to BigQuery data type mapping and other database operations.

## Installation

### From Private PyPI (Recommended)

```bash
pip install airbook-duckdb-utils --extra-index-url https://your-private-pypi-server.com/simple/ --trusted-host your-private-pypi-server.com
```

### From Source

```bash
git clone https://github.com/airbook/airbook-duckdb-utils.git
cd airbook-duckdb-utils
pip install -e .
```

## Usage

### Basic Type Mapping

```python
from airbook_duckdb_utils import map_duckdb_data_type

# Map individual types
bigquery_type = map_duckdb_data_type('INTEGER')
print(bigquery_type)  # Output: NUMERIC

bigquery_type = map_duckdb_data_type('VARCHAR')
print(bigquery_type)  # Output: STRING

bigquery_type = map_duckdb_data_type('DECIMAL(18,3)')
print(bigquery_type)  # Output: NUMERIC
```

### Advanced Usage

```python
from airbook_duckdb_utils.type_mapping import (
    map_duckdb_data_type,
    get_type_mapping_dict,
    is_supported_type
)

# Check if a type is supported
if is_supported_type('TIMESTAMP'):
    mapped_type = map_duckdb_data_type('TIMESTAMP')
    print(f"TIMESTAMP maps to {mapped_type}")

# Get all supported mappings
all_mappings = get_type_mapping_dict()
print(f"Total supported types: {len(all_mappings)}")

# Example: Process a schema
schema = [
    ('id', 'BIGINT'),
    ('name', 'VARCHAR'),
    ('created_at', 'TIMESTAMP'),
    ('metadata', 'JSON'),
    ('score', 'DECIMAL(10,2)')
]

bigquery_schema = []
for column_name, duckdb_type in schema:
    bq_type = map_duckdb_data_type(duckdb_type)
    bigquery_schema.append((column_name, bq_type))
    
print("BigQuery Schema:")
for col_name, col_type in bigquery_schema:
    print(f"  {col_name}: {col_type}")
```

## Supported Type Mappings

| DuckDB Type | BigQuery Type | Notes |
|-------------|---------------|-------|
| INTEGER, BIGINT, etc. | NUMERIC | All integer variants |
| FLOAT, DOUBLE | FLOAT64 | Floating point numbers |
| VARCHAR, TEXT | STRING | Text data |
| BOOLEAN | BOOLEAN | Boolean values |
| DATE | DATE | Date values |
| TIMESTAMP | DATETIME | Timestamp without timezone |
| TIMESTAMPTZ | TIMESTAMP | Timestamp with timezone |
| JSON | JSON | JSON data |
| ARRAY, LIST | JSON | Arrays stored as JSON |
| STRUCT | JSON | Structured data as JSON |
| BLOB | BYTES | Binary data |

## Development

### Setup Development Environment

```bash
git clone https://github.com/airbook/airbook-duckdb-utils.git
cd airbook-duckdb-utils
pip install -e .[dev]
```

### Running Tests

```bash
pytest tests/
```

### Code Formatting

```bash
black airbook_duckdb_utils/
flake8 airbook_duckdb_utils/
mypy airbook_duckdb_utils/
```

## Publishing to Private PyPI

### Build the Package

```bash
python setup.py sdist bdist_wheel
```

### Upload to Private PyPI

```bash
twine upload --repository-url https://your-private-pypi-server.com/legacy/ dist/*
```

## Authentication

For private PyPI access, configure your `~/.pypirc`:

```ini
[distutils]
index-servers = 
    private-pypi

[private-pypi]
repository: https://your-private-pypi-server.com/legacy/
username: your-username
password: your-password
```

Or use environment variables:
```bash
export TWINE_USERNAME=your-username
export TWINE_PASSWORD=your-password
export TWINE_REPOSITORY_URL=https://your-private-pypi-server.com/legacy/
```

## License

MIT License - see LICENSE file for details.

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Run the test suite
6. Submit a pull request

## Changelog

### v1.0.0
- Initial release
- DuckDB to BigQuery type mapping functionality
- Support for all major DuckDB data types
- Comprehensive test coverage