semvec/pypi-packages huggingface dataset:
vd-dlt 0.1.3
Core DLT ingestion framework for VibeData pipelines
Requires Python: >=3.9

# vd-dlt

Core DLT ingestion framework for VibeData pipelines. Provides config resolution, credential management (Azure Key Vault), pipeline execution, and observability.

## Installation

```bash
# Core only
pip install vd-dlt

# With pipeline dependencies (dlt, pyarrow)
pip install vd-dlt[pipeline]

# With Notion connector
pip install vd-dlt[notion]

# With Notion schema (defaults, docs)
pip install vd-dlt[notion-schema]

# Everything for Notion
pip install vd-dlt[pipeline,notion,notion-schema]
```

## Quick Start

```python
from vd_dlt import PipelineRunner

runner = PipelineRunner(
    vault_url="https://my-vault.vault.azure.net/",
)
result = runner.run("my_source_name")
print(f"Loaded {result.total_rows_loaded} rows")
```

## Architecture

The framework uses a 4-level config hierarchy (most specific wins):

1. **Connector Defaults** - from connector schema package
2. **Source Config** - from source YAML (including `default_sync`)
3. **Group Config** - optional grouping within source
4. **Resource Config** - per-table overrides