microsynth 0.1.0
Conditional microdata synthesis using normalizing flows
Requires Python: >=3.10
Keywords: imputation, microdata, normalizing-flows, privacy, survey-data, synthesis

# micro

Conditional microdata synthesis using normalizing flows.

[![PyPI](https://img.shields.io/pypi/v/micro.svg)](https://pypi.org/project/micro/)
[![Tests](https://github.com/CosilicoAI/micro/actions/workflows/test.yml/badge.svg)](https://github.com/CosilicoAI/micro/actions/workflows/test.yml)
[![Docs](https://github.com/CosilicoAI/micro/actions/workflows/docs.yml/badge.svg)](https://cosilicoai.github.io/micro)

## Overview

`micro` synthesizes survey microdata while preserving:

- **Conditional relationships**: Generate target variables given demographics
- **Zero-inflated distributions**: Handle variables that are 0 for many observations
- **Joint correlations**: Preserve relationships between target variables
- **Hierarchical structures**: Keep household/firm compositions intact

## Installation

```bash
pip install micro
```

## Quick Start

```python
from micro import Synthesizer
import pandas as pd

# Load training data with known target variables
training_data = pd.read_csv("survey_with_income.csv")

# Initialize synthesizer
synth = Synthesizer(
    target_vars=["income", "expenditure", "savings"],
    condition_vars=["age", "education", "region"],
)

# Fit on training data
synth.fit(training_data, weight_col="weight", epochs=100)

# Generate synthetic targets for new demographics
new_demographics = pd.read_csv("demographics_only.csv")
synthetic = synth.generate(new_demographics)
```

## Why `micro`?

| Feature | micro | CT-GAN | TVAE | synthpop |
|---------|-------|--------|------|----------|
| Conditional generation | ✅ | ❌ | ❌ | ❌ |
| Zero-inflation handling | ✅ | ❌ | ❌ | ⚠️ |
| Exact likelihood | ✅ | ❌ | ❌ | N/A |
| Stable training | ✅ | ⚠️ | ✅ | ✅ |
| Preserves source structure | ✅ | ❌ | ❌ | ⚠️ |

### Use Cases

- **Survey enhancement**: Impute income variables from tax data onto census demographics
- **Privacy-preserving synthesis**: Generate synthetic data that preserves statistical properties without copying real records
- **Data fusion**: Combine variables from multiple surveys with different sample designs
- **Missing data imputation**: Fill in missing values conditioned on observed variables

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                      Synthesizer                         │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  Training:                                               │
│  ┌──────────┐    ┌──────────────┐    ┌──────────────┐  │
│  │ Training │───▶│ Transformer  │───▶│ Normalizing  │  │
│  │   Data   │    │ (log, std)   │    │    Flow      │  │
│  └──────────┘    └──────────────┘    └──────────────┘  │
│                                                          │
│  Generation:                                             │
│  ┌──────────┐    ┌──────────────┐    ┌──────────────┐  │
│  │ Context  │───▶│ Zero + Flow  │───▶│  Inverse     │  │
│  │  Vars    │    │   Sampling   │    │  Transform   │  │
│  └──────────┘    └──────────────┘    └──────────────┘  │
│                                                          │
└─────────────────────────────────────────────────────────┘
```

## Documentation

Full documentation at [cosilicoai.github.io/micro](https://cosilicoai.github.io/micro)

- [Tutorial](https://cosilicoai.github.io/micro/tutorial.html)
- [API Reference](https://cosilicoai.github.io/micro/api.html)
- [Benchmarks](https://cosilicoai.github.io/micro/benchmarks.html)

## Benchmarks

See [benchmarks/](benchmarks/) for comparisons against:

- **CT-GAN**: Conditional Tabular GAN (from SDV)
- **TVAE**: Tabular VAE (from SDV)
- **Copulas**: Gaussian copula synthesis (from SDV)
- **synthpop**: CART-based synthesis (R package, via rpy2)

## Citation

```bibtex
@software{micro2024,
  author = {Cosilico},
  title = {micro: Conditional microdata synthesis using normalizing flows},
  year = {2024},
  url = {https://github.com/CosilicoAI/micro}
}
```

## License

MIT License - see [LICENSE](LICENSE) for details.