Comparing graph- and pseudo-label-based semi-supervised learning on small tabular datasets
Domain: ml (machine-learning, semi-supervised-learning)
Topic: Comparing semi-supervised learning strategies (LabelPropagation, LabelSpreading, SelfTrainingClassifier) on small UCI datasets with 10-20% labeled data
Venue: ARC-Bench 2026
Headline metric: test_accuracy (maximize)

Research briefing:
Semi-supervised learning is attractive when labels are scarce but unlabeled
examples are cheap. In sklearn, LabelPropagation and LabelSpreading provide
graph-based transductive approaches, while SelfTrainingClassifier offers a
pseudo-labeling wrapper around a standard supervised base learner. These
methods rely on different assumptions: graph smoothness over neighborhoods
versus confidence-thresholded iterative self-labeling. On small tabular
datasets, their relative behavior can change quickly as the labeled fraction
drops from moderate (20%) to very low (10%).

A useful CPU-scale study should compare these methods under the same splits,
feature preprocessing, and random seeds, with explicit control of labeled
fraction. Because the key claim of semi-supervision is label efficiency, the
experiment should include a supervised-only baseline trained on the same
labeled subset and evaluated on a fixed held-out test set. This makes gains
attributable to unlabeled-data usage rather than favorable splits.

The study should report not only test accuracy but also balanced accuracy and
macro-F1, since small tabular datasets can have class imbalance and accuracy
alone may hide minority-class degradation. Repeating runs across several seeds
is necessary because which points are labeled strongly affects outcomes at low
label rates.

Practically, the benchmark should stay lightweight: sklearn datasets only,
no downloads, and model choices that complete within minutes on a single CPU
core. The focus is not SOTA performance but robust relative comparisons across
label regimes and datasets.

*How do LabelPropagation, LabelSpreading, and SelfTrainingClassifier compare in label-efficiency versus a supervised-only baseline when only 10–20% of training labels are available on small sklearn tabular datasets?*

Hypotheses:
statement: At 10% labeled data, at least one semi-supervised method (LabelPropagation, LabelSpreading, or SelfTrainingClassifier) achieves test accuracy at least 3 percentage points higher than the supervised-only baseline on at least 2 of 3 datasets, averaged over ≥5 seeds., statement: Across datasets at 10% labeled data, LabelSpreading with RBF kernel attains mean test accuracy greater than or equal to LabelPropagation with RBF kernel in at least 2 of 3 datasets (seed-averaged)., statement: For SelfTrainingClassifier, increasing labeled fraction from 10% to 20% improves mean test accuracy by at least 1 absolute percentage point on at least 2 of 3 datasets.

Experiment design:
research_question: How do graph-based (LabelPropagation/LabelSpreading) and pseudo-labeling (SelfTrainingClassifier) methods compare to supervised-only training under 10% and 20% labeled-data regimes on small tabular benchmarks?
conditions: name: supervised_logreg_labeled_only
description: LogisticRegression trained only on the labeled subset of the training split; unlabeled points ignored., name: label_propagation_rbf
description: LabelPropagation with RBF kernel; unlabeled training labels set to -1 and transductive predictions used for test inference., name: label_spreading_rbf
description: LabelSpreading with RBF kernel under the same masked-label protocol., name: self_training_logreg
description: SelfTrainingClassifier wrapping LogisticRegression with confidence threshold (e.g., 0.8), fit on mixed labeled/unlabeled training data.
baselines: supervised_logreg_labeled_only is the primary baseline, 10% labeled regime serves as a lower-label baseline versus 20% within each method
metrics: name: test_accuracy
direction: maximize
description: Classification accuracy on held-out test split, averaged over seeds., name: balanced_accuracy
direction: maximize
description: Balanced accuracy on held-out test split, averaged over seeds., name: macro_f1
direction: maximize
description: Macro-averaged F1 score on held-out test split, averaged over seeds.
datasets: name: breast_cancer
source: sklearn.datasets.load_breast_cancer, name: wine
source: sklearn.datasets.load_wine, name: digits
source: sklearn.datasets.load_digits
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 420

Requirements:

Scoring rubric:
requirements: A credible experiment comparing semi-supervised methods (LabelPropagation, LabelSpreading, SelfTrainingClassifier, or equivalents) against a supervised-only baseline under limited-label regimes: methods are implemented correctly, runs cover multiple datasets with repeated seeds, and results address H1/H2/H3 directionally.
judging_note: Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Well-motivated alternative semi-supervised methods that test the same scientific question should be credited.
sub_tasks: requirements: Semi-supervised and baseline conditions are implemented correctly with explicit label masking.
sub_tasks: requirements: The submission implements multiple distinct conditions — typically including at least one graph-based method (LabelPropagation or LabelSpreading), a self-training method, and a supervised-only baseline — as separate code paths.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: Label masking for semi-supervised runs is explicit and correct: only a defined labeled fraction retains class labels and the remaining training labels are set to the unlabeled marker.
task_category: Code Development
finegrained_task_category: Experimental Setup, requirements: The submission uses multiple datasets (sklearn built-ins or comparable), with a held-out test split and consistent preprocessing across conditions.
task_category: Code Development
finegrained_task_category: Dataset and Model Acquisition, requirements: Execution covers multiple label fractions and produces benchmark metrics.
sub_tasks: requirements: Execution outputs numeric test accuracy and at least one additional metric (e.g., balanced accuracy or macro-F1) for each implemented (condition, dataset, label-fraction) cell in a machine-readable artifact.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Each reported cell is aggregated over multiple random seeds with a dispersion measure. Honest small-seed runs with variance reported are preferable to a single run.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Results address H1/H2/H3 directionally with quantitative comparisons.
sub_tasks: requirements: The submission compares each semi-supervised method versus the supervised-only baseline at a low labeled fraction and conveys whether any method provides a meaningful gain on most datasets — judge directionally against H1.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission reports a direct LabelSpreading vs LabelPropagation comparison per dataset and conveys whether LabelSpreading is at least competitive on most datasets (H2).
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission reports self-training accuracy at both low and higher labeled fractions and conveys whether the gain with more labels is meaningful (H3).
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: The README or writeup describes setup and methods, reports per-dataset metrics, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and discusses limitations (dataset scope, seed count variance, sensitivity to masking/hyperparameters). No strict word-count requirement.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation