omnibin 0.1.7
A package for generating comprehensive binary classification reports with visualizations and confidence intervals
Requires Python: >=3.11

[![Tests](https://github.com/kitamura-felipe/omnibin/actions/workflows/test.yml/badge.svg)](https://github.com/kitamura-felipe/omnibin/actions/workflows/test.yml)
[![Deploy](https://github.com/kitamura-felipe/omnibin/actions/workflows/deploy.yml/badge.svg)](https://github.com/kitamura-felipe/omnibin/actions/workflows/deploy.yml)


# Omnibin

A Python package for generating comprehensive binary classification reports with visualizations and confidence intervals.

## Try it Online

You can try Omnibin directly in your browser through our [Hugging Face Space](https://felipekitamura-omnibin.hf.space).

## Installation

```bash
pip install omnibin
```

## Usage

```python
import pandas as pd
from omnibin import generate_binary_classification_report, ColorScheme

# Load your data
data = pd.read_csv("data/scores.csv")
y_true = data['y_true'].values
y_scores = data['y_pred'].values

# Generate comprehensive classification report
report_path = generate_binary_classification_report(
    y_true=y_true,                    # Array of true binary labels (0 or 1)
    y_scores=y_scores,                # Array of predicted probabilities or scores
    output_path="classification_report.pdf",  # Path to save the PDF report
    n_bootstrap=1000,                 # Number of bootstrap iterations for confidence intervals
    random_seed=42,                   # Random seed for reproducibility
    dpi=300,                          # DPI for plot resolution
    color_scheme=ColorScheme.DEFAULT  # Color scheme for plots (DEFAULT, MONOCHROME, or VIBRANT)
)
```

## Input Format

The input data should be provided as:
- `y_true`: Array of true binary labels (0 or 1)
- `y_pred`: Array of predicted probabilities or scores

## Features

- Generates a comprehensive PDF report with:
  - ROC curve with confidence bands
  - Precision-Recall curve with confidence bands
  - Metrics vs. threshold plots
  - Confusion matrix at optimal threshold
  - Calibration plot
  - Summary table with confidence intervals
- Calculates optimal threshold using Youden's J statistic
- Provides confidence intervals using bootstrapping
- Supports both probability and score-based predictions

## Metrics Included

- Accuracy
- Sensitivity (Recall)
- Specificity
- Positive Predictive Value (Precision)
- Matthews Correlation Coefficient
- F1 Score
- AUC-ROC
- AUC-PR

All metrics include 95% confidence intervals calculated through bootstrapping.

## Output

The package generates a PDF report containing:
1. ROC and Precision-Recall curves with confidence bands
2. Metrics plotted across different thresholds
3. Confusion matrix at the optimal threshold
4. Calibration plot
5. Summary table with all metrics and their confidence intervals 

## Example

Here are examples of the visualizations generated by Omnibin:

### ROC and Precision-Recall Curves
![ROC and PR Curves](results/plots/roc_pr.png)

### Metrics vs Threshold
<img src="results/plots/metrics_threshold.png">

### Confusion Matrix
<img src="results/plots/confusion_matrix.png">

### Calibration Plot
<img src="results/plots/calibration.png">

### Prediction Distribution
<img src="results/plots/prediction_distribution.png">

### Metrics Summary
![Metrics Summary](results/plots/metrics_summary.png)