Filter-based vs embedded L1 feature selection with injected noise features
Domain: ml (machine-learning, feature-selection)
Topic: Investigating filter-based feature selection (mutual_info_classif, chi2, f_classif) vs embedded L1-logistic selection on UCI classification datasets with injected irrelevant features
Venue: ARC-Bench 2026
Headline metric: test_accuracy (maximize)

Research briefing:
Feature selection on tabular classification problems is often presented as a
choice between simple univariate filters and embedded sparse models.
Univariate filters such as chi-squared, ANOVA F-score, and mutual
information are computationally cheap and easy to apply in a pipeline, but
they evaluate each feature independently and can miss interactions. Embedded
L1-regularized logistic regression performs selection during model fitting,
potentially yielding feature subsets that better align with predictive
structure when many irrelevant variables are present.

A practical stress test is to inject synthetic irrelevant features into
otherwise standard benchmark datasets. This setup creates controlled feature
dilution while preserving original labels and base difficulty. Under such
dilution, methods that can ignore noise should maintain accuracy and avoid
selecting many synthetic features. Conversely, methods sensitive to spurious
univariate associations may degrade as noise dimension increases.

A credible CPU-only study should compare at least three filter selectors
(mutual_info_classif, chi2, f_classif) and an embedded L1 selector in a
shared downstream classifier pipeline, evaluate on multiple sklearn
classification datasets, and report both predictive performance and selection
quality. Including multiple random seeds is important because both noise
injection and train/test splits can change apparent selector rankings.

The key aim is not to reproduce a single published table, but to determine
whether embedded sparsity offers robustness advantages over filters as
irrelevant dimensions grow, and whether that robustness generalizes across
datasets with different feature scales and class structures.

*Does embedded L1-logistic feature selection retain predictive performance and reject injected irrelevant features more reliably than univariate filter methods under controlled feature-noise injection?*

Hypotheses:
statement: With 5x injected irrelevant features (relative to original feature count), L1-logistic embedded selection achieves higher mean test_accuracy than each filter method (mutual_info_classif, chi2, f_classif) on at least 2 of 3 datasets, averaged over >=5 seeds., statement: At fixed selected-feature budget k=20 (or all features if p<20), L1-logistic selects a lower fraction of injected irrelevant features than the average of the three filter methods on all evaluated datasets., statement: From no-injection to 5x-injection settings, mean test_accuracy drop for L1-logistic is <= 3 percentage points on at least 2 of 3 datasets.

Experiment design:
research_question: Does embedded L1-logistic feature selection outperform univariate filter methods in robustness to injected irrelevant features on sklearn tabular classification datasets?
conditions: name: filter_mutual_info_k20
description: SelectKBest(mutual_info_classif, k=20 or p if p<20) followed by LogisticRegression classifier., name: filter_chi2_k20
description: MinMax scaling to nonnegative domain, then SelectKBest(chi2, k=20 or p if p<20) followed by LogisticRegression classifier., name: filter_f_classif_k20
description: SelectKBest(f_classif, k=20 or p if p<20) followed by LogisticRegression classifier., name: embedded_l1_logistic
description: L1-penalized LogisticRegression (saga/liblinear) used as embedded selector; keep top-20 by absolute coefficient magnitude (or nonzero if <=20), retrain LogisticRegression on selected subset.
baselines: filter_f_classif_k20 as a standard univariate linear-statistic baseline, filter_mutual_info_k20 as a nonlinear dependency baseline
metrics: name: test_accuracy
direction: maximize
description: Held-out test accuracy averaged over >=5 seeds for each dataset and injection level., name: noise_feature_selection_rate
direction: minimize
description: Fraction of selected features that come from injected irrelevant columns., name: accuracy_drop_0x_to_5x_pp
direction: minimize
description: Absolute percentage-point drop in test_accuracy from 0x to 5x injection.
datasets: name: breast_cancer
source: sklearn.datasets.load_breast_cancer, name: wine
source: sklearn.datasets.load_wine, name: digits
source: sklearn.datasets.load_digits
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 420

Requirements:

Scoring rubric:
requirements: A credible experiment studying filter-based feature selection (mutual information, chi-square, ANOVA F) versus embedded L1-logistic selection under injected irrelevant features: conditions are implemented, runs cover multiple datasets with multiple seeds and injection levels, and results address H1/H2/H3 directionally.
judging_note: Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Alternative filter methods or embedded selectors (e.g., tree-based importance) that test the same scientific question should be credited.
sub_tasks: requirements: Feature-selection conditions and the noise-injection pipeline are implemented correctly.
sub_tasks: requirements: The submission implements multiple distinct selection conditions — typically including mutual information, chi-square, ANOVA F, and L1-logistic (or a comparable embedded method) — rather than reusing one selector for all conditions.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: The pipeline injects synthetic irrelevant features at one or more defined levels (including a no-injection baseline and a higher-injection case) and tracks which columns are injected so a noise-selection rate can be computed.
task_category: Code Development
finegrained_task_category: Experimental Setup, requirements: The submission uses multiple datasets (sklearn built-ins or comparable) and a reasonable train/test split.
task_category: Code Development
finegrained_task_category: Dataset and Model Acquisition, requirements: Execution produces metrics across selectors and injection levels.
sub_tasks: requirements: Execution outputs a structured metrics artifact containing numeric test accuracy and a noise-selection-rate style measure for each implemented condition on at least one dataset with at least two injection levels.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Reported metrics are aggregated over multiple random seeds per (dataset, condition, injection level) cell with a dispersion measure. Honest small-seed runs with variance reported are preferable to a single run.
task_category: Code Execution
finegrained_task_category: Hyperparameter Tuning, requirements: Results address H1/H2/H3 directionally with a clear narrative.
sub_tasks: requirements: The submission compares mean test accuracy at the higher injection level between the embedded L1-logistic method and each filter method per dataset and conveys whether L1 tends to be better — judge directionally against H1.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission reports the noise-selection rate for each method and conveys whether the embedded method selects fewer irrelevant features than the filter-method average on most datasets (H2).
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: The submission reports the accuracy drop from low-injection to high-injection for the embedded method and conveys whether the drop is small on most datasets (H3).
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The README or writeup describes the experimental setup, reports key metric tables, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as synthetic-noise realism, k-choice, classifier dependence, and seed/dataset scope. No strict word-count requirement.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation