semvec/pypi-packages huggingface dataset:
scd2-bq-engine 0.1.0
BigQuery SCD Type 2 dimension builder with SQLX generation
License: MIT
Requires Python: >=3.9
Keywords: bigquery, scd2, dataform, dbt, data-warehouse, dimension

# SCD2 BigQuery Engine

**Version**: 0.1.0  
**License**: MIT  
**Status**: ✅ Production Ready

## Overview

BigQuery-first SCD Type 2 dimension builder with SQLX template generation for Dataform/dbt.

Generates production-ready SQLX files that implement full SCD Type 2 logic including:
- Hash-based change detection
- Effective date tracking
- Soft deletes
- Late arrival handling
- BigQuery optimizations (partitioning, clustering)

## Why This Module?

- dbt snapshots are dbt-locked and Snowflake-optimized
- Existing SCD2 tutorials use stored procedures (hard to version control)
- No declarative, config-driven SCD2 solution for BigQuery

## Features (v0.1.0 MVP)

- ✅ SQLX template generator for standard SCD2 pattern
- ✅ Hash-based change detection (MD5/SHA256)
- ✅ Effective dating with late arrival handling
- ✅ Soft delete support
- ✅ YAML config → SQLX output
- ✅ CLI + Python API

## Quick Start

### Installation
```bash
pip install scd2-bq-engine
```

### Initialize Configuration
```bash
# Create a new dimension configuration
scd2-bq init dim_employee \
    --source-table project.dataset.stg_employees \
    --business-keys employee_number \
    --tracked-columns first_name,last_name,job_code,department
```

### Generate SQLX
```bash
# Generate SCD2 dimension SQLX file
scd2-bq generate \
    --config dim_employee_config.yaml \
    --output-file dim_employee.sqlx
```

### Python API
```python
from scd2_bq_engine import SCD2Generator, SCD2Config

# Create configuration
config = SCD2Config(
    dimension_name="dim_employee",
    source_table="project.dataset.stg_employees",
    business_keys=["employee_number"],
    tracked_columns=["first_name", "last_name", "job_code"]
)

# Generate SQLX
generator = SCD2Generator(config)
generator.write_sqlx("dim_employee.sqlx")
```

## Directory Structure

```
scd2-bq-engine/
├── src/scd2_bq_engine/
│   ├── generator.py          # Core SQLX generator
│   ├── templates/            # Jinja2 SQLX templates
│   └── validators.py         # Config validation
├── tests/
├── examples/
└── docs/
```

## License

MIT - see [LICENSE](LICENSE)
featheros 0.1.21
A Python framework for building full-stack web applications with minimal code
License: MIT
Requires Python: >=3.8

# FeatherOS

A Python framework for building full-stack web applications with minimal code.

## Features

- **Minimal Code**: Build full-stack apps with just Python
- **No JavaScript Required**: Frontend and backend in one language
- **Plugin System**: Extend functionality with plugins managed through owner admin panel
- **Database Abstraction**: Simple ft.db.* API for all database operations
- **CLI Tools**: Easy project scaffolding and development

## Quick Start

### For Developers Using FeatherOS

```bash
# Install the framework
pip install featheros

# Create a new app
feather init my-app
cd my-app

# Run the development server
feather serve
```

### Example App

```python
import feather as ft

app = ft.App(title="My App")

@app.page("/")
def home():
    ft.title("Welcome to FeatherOS")
    
    with ft.form() as form:
        name = ft.input("Name", required=True)
        email = ft.input("Email", type="email")
        submit = ft.button("Submit")
    
    if form.submitted:
        ft.db.save("users", {"name": name.value, "email": email.value})
        ft.success(f"Welcome {name.value}!")
```

## For Framework Owners

### Running the Admin Panel & Server

```bash
# Install dependencies
pip install -r requirements.txt

# Run both plugin server and admin panel
python main.py
```

- Admin Panel: http://localhost:5000/admin
- Plugin Server API: http://localhost:5000/api

## Architecture

- **feather/**: Core framework package
- **server/**: Plugin distribution server
- **admin/**: Owner admin panel for managing plugins
- **plugins/**: Example starter plugins
- **docs/**: Documentation
- **examples/**: Example applications

## License

MIT
synthetic-payroll-lab 0.1.0
Generate realistic enterprise payroll test data with chaos patterns
License: MIT
Requires Python: >=3.9
Keywords: payroll, synthetic-data, testing, data-engineering

# Synthetic Payroll Lab

**Version**: 0.1.0  
**License**: MIT  
**Status**: MVP Development

## Overview

Generate realistic, enterprise-messy payroll and timekeeping test data with configurable "chaos" knobs to stress-test data pipelines.

## Why This Module?

Existing synthetic data tools (Faker, Mockaroo) don't understand payroll domain semantics or simulate common enterprise chaos patterns like:
- Late arriving facts
- Schema drift
- Foreign key orphans
- Timezone errors
- Duplicate records

## Features (v0.1.0 MVP)

- ✅ Generate 6 core payroll domains (employees, jobs, schedules, timecards, payroll runs, cost centers)
- ✅ CSV/JSON output with Hive-style partitioning
- ✅ Configurable chaos patterns (duplicates, nulls, late arrivals, schema drift)
- ✅ Deterministic mode (seed for reproducibility)
- ✅ CLI + Python API

## Quick Start

```bash
# Install
pip install synthetic-payroll-lab

# Generate test data
synthetic-payroll generate \
    --config payroll_config.yaml \
    --output-dir ./landing \
    --start-date 2024-01-01 \
    --end-date 2024-12-31 \
    --employees 50000
```

## Python API

```python
from synthetic_payroll_lab import PayrollGenerator, ChaosConfig

gen = PayrollGenerator(
    employees=50000,
    start_date="2024-01-01",
    chaos=ChaosConfig(
        duplicate_rate=0.02,      # 2% duplicate rows
        null_spike_rate=0.01,     # 1% random null injection
        late_arrival_pct=0.15,    # 15% timecards arrive T+2 days
        schema_drift_days=90,     # Column added every 90 days
        timezone_error_rate=0.03  # 3% wrong timezone
    )
)

gen.generate_all_domains(output_path="./landing", format="csv")
```

## Configuration

See [config_reference.md](docs/config_reference.md) for full YAML schema.

## Roadmap

- **v0.1.0** (MVP): Core domains + basic chaos
- **v0.2.0**: SCD2 dimension changes, retro adjustments
- **v1.0.0**: Multi-region support, PII variants, Parquet output

## Contributing

Issues and PRs welcome! See [CONTRIBUTING.md](CONTRIBUTING.md).

## License

MIT - see [LICENSE](LICENSE)