Benchmarking classical unsupervised outlier detectors under controlled anomaly injection
Domain: ml (anomaly-detection, machine-learning)
Topic: Benchmarking unsupervised outlier detection methods (IsolationForest, LocalOutlierFactor, OneClassSVM, EllipticEnvelope) on UCI datasets with injected anomalies
Venue: ARC-Bench 2026
Headline metric: roc_auc (maximize)

Research briefing:
Classical unsupervised anomaly-detection methods remain widely used in
production tabular pipelines because they are interpretable enough,
CPU-efficient, and available in standard libraries. Yet their relative
behavior can change sharply as anomaly prevalence and feature scaling vary.
IsolationForest is often robust in heterogeneous feature spaces, while
distance/density methods like LocalOutlierFactor can be strong when local
neighborhoods are informative but may degrade in higher dimensions.

A practical benchmark should avoid external downloads and still provide
realistic control over anomaly rate. One reliable strategy is to start from
clean sklearn classification datasets and inject synthetic anomalies into the
test split by sampling uniformly beyond feature-wise quantile ranges of the
training data. This yields known binary ground truth for evaluation while
preserving a realistic inlier distribution.

A credible CPU-scale study here should compare IsolationForest,
LocalOutlierFactor (novelty mode), OneClassSVM, and EllipticEnvelope under a
shared preprocessing pipeline (e.g., StandardScaler), evaluate ROC-AUC and
PR-AUC across at least two datasets and multiple seeds, and include a simple
baseline such as random scoring. Since contamination is rarely known exactly
in practice, the benchmark should also probe at least two injected anomaly
rates.

The main decision-relevant question is not whether any method can detect easy
anomalies, but which method is consistently best across datasets and
contamination levels under strict CPU constraints. A concise, quantitative
hypothesis-driven comparison is therefore the goal.

*Which classical unsupervised outlier detector is most robust across tabular datasets and injected anomaly rates when measured by ROC-AUC and PR-AUC under a single-core CPU budget?*

Hypotheses:
statement: IsolationForest achieves higher mean ROC-AUC than OneClassSVM on at least 2 of 3 datasets when averaged over anomaly rates {0.05, 0.10} and ≥3 seeds., statement: At anomaly rate 0.10, LocalOutlierFactor attains the best PR-AUC (rank 1) on at least 1 of 3 datasets., statement: Each implemented detector (IsolationForest, LocalOutlierFactor, OneClassSVM, EllipticEnvelope) outperforms a random-score baseline in ROC-AUC by at least 0.10 on the pooled mean across all evaluated datasets/rates.

Experiment design:
research_question: Which among IsolationForest, LocalOutlierFactor, OneClassSVM, and EllipticEnvelope is most robust across datasets and injected anomaly rates for unsupervised outlier detection under CPU constraints?
conditions: name: isolation_forest
description: sklearn IsolationForest with contamination set to injected rate, 200 trees, default feature subsampling., name: local_outlier_factor_novelty
description: sklearn LocalOutlierFactor in novelty=True mode, n_neighbors in [20, 35], contamination set to injected rate., name: one_class_svm_rbf
description: sklearn OneClassSVM with RBF kernel, nu matched to injected rate, gamma='scale'., name: elliptic_envelope
description: sklearn EllipticEnvelope with contamination set to injected rate and robust covariance estimation., name: random_score_baseline
description: Baseline assigning i.i.d. uniform random anomaly scores to test points.
baselines: random_score_baseline, one_class_svm_rbf as a classical margin-based reference
metrics: name: roc_auc
direction: maximize
description: ROC-AUC of anomaly scores against injected ground-truth labels on the test set, averaged over seeds., name: pr_auc
direction: maximize
description: Average precision / PR-AUC for anomaly class on the test set, averaged over seeds., name: fit_predict_time_sec
direction: minimize
description: Wall-clock seconds for model fit plus test scoring per (dataset, rate, seed), reported as mean.
datasets: name: breast_cancer
source: sklearn.datasets.load_breast_cancer, name: wine
source: sklearn.datasets.load_wine, name: digits
source: sklearn.datasets.load_digits
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 480

Requirements:

Scoring rubric:
requirements: A credible experiment benchmarking unsupervised outlier detectors (IsolationForest, LocalOutlierFactor, OneClassSVM, EllipticEnvelope, or equivalents) with injected anomalies: methods are implemented as distinct code paths, runs cover multiple datasets across anomaly rates, and results address H1/H2/H3 directionally.
judging_note: Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Well-motivated substitutes (e.g., HBOS, KNN-based outlier) that test the same scientific question should be credited.
sub_tasks: requirements: The outlier-detection conditions and anomaly-injection pipeline are implemented correctly.
sub_tasks: requirements: The submission implements multiple distinct detector code paths — typically including IsolationForest, LOF, OneClassSVM, and an elliptic/Gaussian-envelope method — rather than aliases to one shared estimator.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: A reproducible anomaly-injection routine is implemented for test data at one or more explicit contamination rates with binary ground-truth anomaly labels.
task_category: Code Development
finegrained_task_category: Experimental Setup, requirements: The code uses multiple datasets (sklearn built-ins or comparable) and applies a consistent preprocessing pipeline (including feature scaling) before model fitting.
task_category: Code Development
finegrained_task_category: Dataset and Model Acquisition, requirements: Execution records anomaly-detection metrics for each condition.
sub_tasks: requirements: Execution produces a machine-readable metrics artifact containing numeric ROC-AUC and PR-AUC (or equivalents) for each implemented method on at least one (dataset, rate) cell.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Reported results aggregate over multiple random seeds and include at least two anomaly rates, with a dispersion measure. Honest small-seed runs with variance reported are preferable to a single run.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: The run logs a wall-clock timing measure per method and demonstrates completion within a CPU-only workflow.
task_category: Code Execution
finegrained_task_category: Logging, Analysis & Presentation, requirements: Quantitative analysis addresses H1/H2/H3 directionally.
sub_tasks: requirements: The submission compares mean ROC-AUC of IsolationForest vs OneClassSVM per dataset and conveys whether IsolationForest is meaningfully better on most datasets — judge directionally against H1.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission conveys whether LOF is competitive on PR-AUC for at least one dataset at a higher anomaly rate (H2) and whether detectors meaningfully outperform a random baseline on pooled ROC-AUC (H3).
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: The README or writeup describes setup and anomaly injection, reports ROC-AUC/PR-AUC results, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and discusses limitations (synthetic anomalies, dataset scope, hyperparameter sensitivity, seed count). No strict word-count requirement.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation