ppcef 0.1.0
CEL: Counterfactual Explanations Library - A framework for generating probabilistically plausible counterfactual explanations using normalizing flows
Requires Python: >=3.10
Keywords: counterfactual-explanations, explainable-ai, xai, machine-learning, normalizing-flows, interpretability, ppcef

# PPCEF: Probabilistically Plausible Counterfactual Explanations using Normalizing Flows

A Python framework for generating and evaluating counterfactual explanations in machine learning models. The main contribution is **PPCEF** (Probabilistically Plausible Counterfactual Explanations using Normalizing Flows), a novel method that uses normalizing flows as density estimators within an optimization framework to generate high-quality, plausible counterfactual explanations.

<p align="center">
<img src="graphic.svg" alt="PPCEF Framework Overview" width="800"/>
</p>

## Abstract

We present **PPCEF**, a novel method specifically tailored for generating probabilistically plausible counterfactual explanations. This approach utilizes normalizing flows as density estimators within an unconstrained optimization framework, effectively balancing distance, validity, and probabilistic plausibility in the produced counterfactuals. Our method is notable for its computational efficiency and ability to process large and high-dimensional datasets, making it particularly applicable in real-world scenarios. A key aspect of **PPCEF** is its focus on the plausibility of counterfactuals, ensuring that the generated explanations are coherent and realistic within the context of the original data. Through comprehensive experiments across various datasets and models, we demonstrate that **PPCEF** can successfully generate high-quality counterfactual explanations, highlighting its potential as a valuable tool in enhancing the interpretability and transparency of machine learning systems.

## Table of Contents

- [Key Features](#key-features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Library Structure](#library-structure)
- [Counterfactual Methods](#counterfactual-methods)
- [Datasets](#datasets)
- [Models](#models)
- [Metrics](#metrics)
- [Running Experiments](#running-experiments)
- [Documentation](#documentation)
- [Contributing](#contributing)
- [Citation](#citation)
- [Contact](#contact)

## Key Features

- **Multiple CF Method Families**: Local, global, and group counterfactual methods
- **Normalizing Flow Integration**: State-of-the-art density estimation for plausibility
- **Comprehensive Metrics**: 17+ evaluation metrics for counterfactual quality
- **Hydra Configuration**: Flexible experiment management with YAML configs
- **21 Built-in Datasets**: Classification and regression tasks
- **Extensible Architecture**: Easy to add new methods, models, and metrics
- **PyTorch-based**: Modern deep learning framework
- **Cross-validation Support**: Robust evaluation with k-fold CV
- **Preprocessing Pipeline**: Composable feature transformations

## Installation

Clone the repository and set up the environment:

```shell
git clone git@github.com:ofurman/counterfactuals.git
cd counterfactuals
./setup_env.sh
```

Or install dependencies manually with [uv](https://github.com/astral-sh/uv):

```shell
uv sync
```

**Requirements**: Python >= 3.10

## Quick Start

```python
import torch
from counterfactuals.datasets import MethodDataset
from counterfactuals.cf_methods import PPCEF
from counterfactuals.models import MaskedAutoregressiveFlow, MLPClassifier
from counterfactuals.losses import BinaryDiscLoss
from counterfactuals.metrics import evaluate_cf

# Load dataset with preprocessing
dataset = MethodDataset.from_config("config/datasets/moons.yaml")
train_loader = dataset.train_dataloader(batch_size=128, shuffle=True)
test_loader = dataset.test_dataloader(batch_size=128, shuffle=False)

# Train discriminative model (classifier)
disc_model = MLPClassifier(
    input_size=dataset.input_size,
    hidden_layer_sizes=[256, 256],
    target_size=1,
    dropout=0.2,
)
disc_model.fit(train_loader, test_loader, epochs=5000, patience=300, lr=1e-3)

# Train generative model (normalizing flow)
gen_model = MaskedAutoregressiveFlow(
    features=dataset.input_size,
    hidden_features=8,
    context_features=1,
)
gen_model.fit(train_loader, test_loader, num_epochs=1000)

# Generate counterfactuals
cf_method = PPCEF(
    gen_model=gen_model,
    disc_model=disc_model,
    disc_model_criterion=BinaryDiscLoss(),
)
log_prob_threshold = torch.quantile(gen_model.predict_log_prob(test_loader), 0.25)
result = cf_method.explain_dataloader(
    test_loader,
    alpha=100,
    log_prob_threshold=log_prob_threshold,
    epochs=4000,
)

# Evaluate results
X_cf = result.x_origs + result.x_cfs
metrics = evaluate_cf(
    disc_model=disc_model,
    gen_model=gen_model,
    X_cf=X_cf,
    X_train=dataset.X_train,
    y_train=dataset.y_train,
    X_test=result.x_origs,
    y_test=result.y_origs,
    y_target=result.y_cf_targets,
    continuous_features=dataset.numerical_features,
    categorical_features=dataset.categorical_features,
    median_log_prob=log_prob_threshold,
)
```

### Jupyter Notebooks

Example notebooks are available in `notebooks/`:

| Notebook | Description |
|----------|-------------|
| [`ppcef.ipynb`](notebooks/ppcef.ipynb) | Basic PPCEF usage |
| [`rppcef.ipynb`](notebooks/rppcef.ipynb) | Regional PPCEF for group explanations |
| [`categorical_ppcef.ipynb`](notebooks/categorical_ppcef.ipynb) | Handling categorical features |
| [`toy_example.ipynb`](notebooks/toy_example.ipynb) | Simple visualization examples |

## Library Structure

```
counterfactuals/
├── cf_methods/           # Counterfactual explanation methods
│   ├── local/            # Instance-level methods (PPCEF, DiCE, WACH, etc.)
│   ├── global_/          # Model-level methods (GLOBE-CE, AReS)
│   └── group/            # Cohort-level methods (RPPCEF, GLANCE)
├── models/               # ML models
│   ├── discriminative/   # Classifiers (MLP, LogisticRegression, NODE)
│   ├── generative/       # Density estimators (MAF, RealNVP, NICE, KDE)
│   └── regression/       # Regressors (MLP, LinearRegression)
├── datasets/             # Dataset loading and configuration
├── preprocessing/        # Feature transformation pipeline
├── dequantization/       # Categorical feature handling for flows
├── losses/               # Loss functions for CF optimization
├── metrics/              # Evaluation metrics
├── pipelines/            # Experiment orchestration
│   ├── nodes/            # Pipeline components
│   └── conf/             # Hydra configuration files
├── plotting/             # Visualization utilities
└── utils.py              # Helper functions

config/
└── datasets/             # Dataset YAML configurations (21 datasets)

docs/
├── library_overview.md   # Comprehensive package documentation
└── ppcef_pipeline.md     # Pipeline guide
```

## Counterfactual Methods

### Local Methods (Instance-level)

| Method | Class | Description |
|--------|-------|-------------|
| **PPCEF** | `PPCEF` | Probabilistically Plausible CF with normalizing flows (main contribution) |
| **PPCEFR** | `PPCEFR` | PPCEF for regression tasks |
| **DiCE** | `DICE` | Diverse Counterfactual Explanations |
| **CEM** | `CEM_CF` | Contrastive Explanation Method |
| **CET** | `CET` | Counterfactual Explanation Tree |
| **WACH** | `WACH` | Wachter-style gradient-based CF |
| **Artelt** | `Artelt` | Artelt's CF method |
| **SACE** | `SACE`, `CaseBasedSACE` | (Case-based) SACE methods |
| **CEGP** | `CEGP` | CF with Gaussian Processes |
| **C-CHVAE** | `CCHVAE` | Conditional Heterogeneous VAE |
| **DiCoFlex** | `DiCoFlex` | Diverse Counterfactual Flex |
| **LiCE** | `LiCE` | LIME-style CF (requires pyomo/onnx/omlt) |

### Global Methods (Model-level)

| Method | Class | Description |
|--------|-------|-------------|
| **GLOBE-CE** | `GLOBE_CE` | Global Counterfactual Explanations |
| **AReS** | `AReS` | Actionable Recourse Summaries |

### Group Methods (Cohort-level)

| Method | Class | Description |
|--------|-------|-------------|
| **RPPCEF** | `RPPCEF` | Regional PPCEF with shared interventions |
| **GLANCE** | `GLANCE` | Group-level CF method |

## Datasets

The library includes 21 pre-configured datasets:

**Classification:**
`adult`, `adult_census`, `audit`, `bank_marketing`, `compas`, `credit_default`, `diabetes`, `digits`, `german_credit`, `give_me_some_credit`, `heloc`, `law`, `lending_club`, `mnist`, `moons`, `wine`, `blobs`

**Regression:**
`concrete`, `toy_regression`, `wine_quality_regression`, `yacht`

Dataset configurations are in `config/datasets/*.yaml` and support:
- Automatic feature type detection (continuous/categorical)
- Actionability flags for features
- Cross-validation splits
- Train/test split configuration

## Models

### Discriminative Models

| Model | Class | Use Case |
|-------|-------|----------|
| MLP Classifier | `MLPClassifier` | General classification |
| Logistic Regression | `LogisticRegression` | Binary classification |
| Multinomial LR | `MultinomialLogisticRegression` | Multiclass |
| NODE | `NODE` | Neural Oblivious Decision Ensembles |

### Generative Models

| Model | Class | Description |
|-------|-------|-------------|
| MAF | `MaskedAutoregressiveFlow` | Primary normalizing flow |
| RealNVP | `RealNVP` | Real-valued Non-Volume Preserving |
| NICE | `NICE` | Non-linear Independent Components |
| KDE | `KDE` | Kernel Density Estimation baseline |

### Regression Models

| Model | Class |
|-------|-------|
| MLP Regressor | `MLPRegressor` |
| Linear Regression | `LinearRegression` |

## Metrics

The library provides comprehensive evaluation metrics:

| Category | Metrics |
|----------|---------|
| **Validity** | `coverage`, `validity`, `actionability` |
| **Sparsity** | `sparsity` |
| **Distance** | `proximity_continuous_euclidean`, `proximity_continuous_manhattan`, `proximity_continuous_mad`, `proximity_categorical_hamming`, `proximity_categorical_jaccard`, `proximity_l2_jaccard`, `proximity_mad_hamming` |
| **Plausibility** | `prob_plausibility`, `log_density_cf`, `log_density_test` |
| **Outlier Detection** | `lof_scores_cf`, `lof_scores_test`, `isolation_forest_scores_cf`, `isolation_forest_scores_test` |

## Running Experiments

### Using Hydra Pipelines

```shell
# Run PPCEF pipeline
uv run python counterfactuals/pipelines/run_ppcef_pipeline.py

# With custom configuration
uv run python counterfactuals/pipelines/run_ppcef_pipeline.py \
  dataset.config_path=config/datasets/heloc.yaml \
  disc_model.model=disc_model/mlp_large \
  counterfactuals_params.target_class=1
```

### Available Pipelines

| Pipeline | Method |
|----------|--------|
| `run_ppcef_pipeline.py` | PPCEF (main) |
| `run_ppcefr_pipeline.py` | PPCEF for regression |
| `run_rppcef_pipeline.py` | Regional PPCEF |
| `run_dice_pipeline.py` | DiCE |
| `run_cem_pipeline.py` | CEM |
| `run_cet_pipeline.py` | CET |
| `run_cchvae_pipeline.py` | C-CHVAE |
| `run_wach_pipeline.py` | WACH |
| `run_artelt_pipeline.py` | Artelt |
| `run_cegp_pipeline.py` | CEGP |
| `run_globe_ce_pipeline.py` | GLOBE-CE |
| `run_ares_pipeline.py` | AReS |
| `run_glance_pipeline.py` | GLANCE |

## Documentation

- [`docs/library_overview.md`](docs/library_overview.md) - Comprehensive package documentation
- [`docs/ppcef_pipeline.md`](docs/ppcef_pipeline.md) - Detailed PPCEF pipeline guide
- [`AGENTS.md`](AGENTS.md) - Development guidelines and coding standards

## Contributing

Contributions are welcome! Before opening a PR:

1. Read [`AGENTS.md`](AGENTS.md) and [`docs/ppcef_pipeline.md`](docs/ppcef_pipeline.md) to understand the workflow
2. Use `uv` for all operations:
   ```shell
   uv sync                     # Install dependencies
   uv run ruff check --fix     # Lint and fix
   uv run pytest               # Run tests
   ```
3. Follow the coding standards:
   - Python 3.10+, PEP 8 compliant
   - Full type hints everywhere
   - Google-style docstrings
   - Line length: 100 characters
4. Keep patches small and well-documented
5. Update or add tests when behavior changes

To add new dependencies:
```shell
uv add <package>
```

## Citation

```bibtex
@inbook{inbook,
  author = {Wielopolski, Patryk and Furman, Oleksii and Stefanowski, Jerzy and Zieba, Maciej},
  year = {2024},
  month = {10},
  pages = {},
  title = {Probabilistically Plausible Counterfactual Explanations with Normalizing Flows},
  isbn = {9781643685489},
  doi = {10.3233/FAIA240584}
}
```

## Contact

For questions or comments, please contact via LinkedIn: TBA