quixlake-sdk 0.2.1
Python SDK for QuixLake - Easy data querying and management
License: MIT
Requires Python: >=3.8
Keywords: database, analytics, sql, parquet, duckdb, data-lake

# QuixLake Python SDK

A Python client library for interacting with QuixLake API. Provides easy-to-use methods for querying, inserting, and managing data without having to handle HTTP requests and response parsing manually.

## Features

- 🔍 **Simple SQL Queries**: Execute SQL queries and get pandas DataFrames
- 📊 **Data Management**: Insert, compact, and repartition tables
- 🗂️ **Partition Support**: Work with Hive-partitioned data
- 🔧 **Easy Setup**: Simple installation and configuration
- 📓 **Jupyter Ready**: Perfect for data analysis notebooks

## Installation

### From wheel (recommended)
```bash
pip install dist/quixlake_sdk-*.whl
```

### Development mode
```bash
pip install -e .
```

### With Jupyter support
```bash
pip install 'dist/quixlake_sdk-*.whl[jupyter]'
```

## Quick Start

```python
from quixlake import QuixLakeClient
import pandas as pd

# Initialize client
client = QuixLakeClient(base_url="http://localhost")

# Query data
df = client.query("SELECT * FROM my_table LIMIT 10")
print(df.head())

# Get available tables
tables = client.get_tables()
print("Available tables:", tables)

# Insert data
new_data = pd.DataFrame({
    'id': [1, 2, 3],
    'name': ['Alice', 'Bob', 'Charlie'],
    'machine': ['3D_PRINTER_0', '3D_PRINTER_1', '3D_PRINTER_0']
})

result = client.insert(
    table_name="users",
    data=new_data,
    hive_columns=["machine"]
)
print("Inserted:", result)
```

## API Reference

### QuixLakeClient

#### `__init__(base_url="http://localhost", timeout=30)`
Initialize the client with QuixLake API base URL.

#### `query(sql, explain_analyze=False)`
Execute SQL query and return pandas DataFrame.
- `sql`: SQL SELECT statement
- `explain_analyze`: Enable query execution plan analysis

#### `get_tables()`
Get list of available tables.

#### `get_partitions(table_name)`
Get partition tree structure for a table.

#### `get_partition_info(table_name)`
Get partition structure information for a table.

#### `insert(table_name, data, hive_columns=None, timestamp_column=None, timestamp_format="day")`
Insert pandas DataFrame into table with optional partitioning.
- `table_name`: Target table name
- `data`: pandas DataFrame to insert
- `hive_columns`: List of columns for Hive partitioning
- `timestamp_column`: Column for timestamp-based partitioning
- `timestamp_format`: 'day', 'hour', or 'month'

#### `compact(table_name, target_file_size_mb=128)`
Compact table files to optimize performance.

#### `repartition(table_name, hive_columns=None, timestamp_column=None, timestamp_format="day", target_file_size_mb=128)`
Repartition table with new partition scheme.

#### `delete(table_name, where_clause=None, delete_table=False, partitions=None)`
Delete data from table with various deletion modes.

## Examples

### Basic Querying
```python
client = QuixLakeClient()

# Simple query
df = client.query("SELECT COUNT(*) as total FROM events_hive")

# Query with explain analyze
df = client.query("SELECT * FROM events_hive WHERE machine = '3D_PRINTER_0'", explain_analyze=True)
```

### Data Management
```python
# Get table information
tables = client.get_tables()
partition_info = client.get_partition_info("events_hive")

# Insert partitioned data
df = pd.read_csv("new_events.csv")
client.insert(
    table_name="events_hive",
    data=df,
    hive_columns=["machine", "experiment_name"],
    timestamp_column="ts_ms",
    timestamp_format="day"
)

# Compact table
result = client.compact("events_hive", target_file_size_mb=256)

# Repartition table
result = client.repartition(
    table_name="events_hive",
    hive_columns=["machine", "year", "month"],
    timestamp_column="ts_ms",
    timestamp_format="month"
)
```

### Using Context Manager
```python
with QuixLakeClient() as client:
    df = client.query("SELECT * FROM my_table")
    print(df.describe())
```

## Building the SDK

Run the build script to create a wheel package:

```bash
./build.sh
```

This will:
1. Clean previous builds
2. Install build dependencies
3. Create wheel package in `dist/` directory
4. Show installation instructions

## Development

### Install in development mode
```bash
pip install -e .[dev]
```

### Run tests
```bash
pytest
```

### Code formatting
```bash
black src/
```

## Error Handling

The SDK provides clear error messages for common issues:

- **Partition Mismatch**: When trying to insert data with incompatible partition structure
- **Query Errors**: SQL syntax errors or execution failures
- **Network Issues**: Connection timeouts or API unavailability

```python
try:
    df = client.query("SELECT * FROM non_existent_table")
except ValueError as e:
    print(f"Query error: {e}")
```

## Requirements

- Python 3.8+
- pandas >= 1.3.0
- requests >= 2.25.0

## License

MIT License - see LICENSE file for details.