Dimensionality reduction methods for preserving cluster structure in synthetic high-dimensional data
Domain: ml (machine-learning, unsupervised-learning)
Topic: Investigating dimensionality reduction techniques (PCA, t-SNE, UMAP) for preserving cluster structure in high-dimensional synthetic datasets
Venue: ARC-Bench 2026
Headline metric: silhouette_score (maximize)

Research briefing:
Dimensionality reduction is frequently used before visualization and
clustering, but different methods optimize different objectives and can
distort neighborhood and global geometry in distinct ways. PCA is linear and
emphasizes variance preservation, while t-SNE and UMAP are nonlinear manifold
methods designed to preserve local structure. In practice, users often infer
cluster quality from 2D embeddings without checking whether separability in
the embedding reflects true high-dimensional structure.

A CPU-scale investigation can probe this mismatch by generating synthetic
datasets where ground-truth cluster labels are known and controllable. By
varying cluster overlap, anisotropy, and manifold geometry (e.g., Gaussian
blobs, anisotropic transforms, and noisy moons embedded in high dimensions),
one can evaluate whether each reducer preserves cluster structure under
different regimes instead of relying on a single toy example.

A credible experiment should compare PCA, t-SNE, and UMAP under a common
pipeline: reduce to 2 dimensions, then score structure using metrics such as
silhouette score and adjusted Rand index after a fixed clustering backend
(e.g., KMeans with true k). Runtime should also be tracked because nonlinear
methods may yield better structure at substantially higher computational cost,
which matters in constrained environments.

The goal is not to claim a universally best reducer, but to quantify tradeoffs
between structure preservation and efficiency in settings where intrinsic
geometry differs. This produces actionable guidance for method choice based on
data regime rather than defaults.

*Which dimensionality reduction method (PCA, t-SNE, or UMAP) best preserves cluster structure across distinct synthetic high-dimensional regimes when evaluated by silhouette score, ARI, and runtime?*

Hypotheses:
statement: On nonlinear manifold data (noisy two-moons embedded into 50D), at least one nonlinear reducer (t-SNE or UMAP) achieves silhouette_score at least 0.10 higher than PCA after 2D reduction and KMeans clustering, averaged over >=5 seeds., statement: On linearly separable Gaussian blobs in 50D, PCA achieves silhouette_score within 0.05 of the best nonlinear method (t-SNE or UMAP), averaged over >=5 seeds., statement: PCA has the lowest mean wall-clock reduction time and is at least 3x faster than both t-SNE and UMAP on at least 2 of 3 datasets.

Experiment design:
research_question: Which reducer best balances cluster-structure preservation and compute cost across linear and nonlinear synthetic high-dimensional datasets?
conditions: name: pca_2d
description: PCA with n_components=2 using sklearn.decomposition.PCA., name: tsne_2d
description: t-SNE with n_components=2, perplexity in [20, 40], learning_rate='auto', init='pca'., name: umap_2d
description: UMAP-like reducer: use umap.UMAP(n_components=2, n_neighbors in [15, 30], min_dist in [0.0, 0.3]) if available; if unavailable, use sklearn.manifold.SpectralEmbedding(n_components=2) as a documented fallback proxy., name: identity_no_reduction
description: Baseline that applies KMeans directly in the original high-dimensional space (no dimensionality reduction).
baselines: identity_no_reduction as no-DR clustering baseline, pca_2d as linear DR baseline
metrics: name: silhouette_score
direction: maximize
description: Silhouette score computed on the 2D embedding (or original space for identity baseline) using predicted KMeans labels; primary metric., name: ari
direction: maximize
description: Adjusted Rand Index between KMeans labels and known synthetic ground-truth labels., name: fit_transform_time_sec
direction: minimize
description: Wall-clock time for dimensionality reduction fit_transform only (seconds).
datasets: name: gaussian_blobs_50d
source: sklearn.datasets.make_blobs with 4 centers, n_features=50, cluster_std=1.2, name: anisotropic_blobs_50d
source: make_blobs in 10D expanded/projection to 50D then linear anisotropic transform, name: embedded_moons_50d
source: sklearn.datasets.make_moons(noise=0.08) embedded into 50D by random linear projection plus Gaussian noise
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 600

Requirements:

Scoring rubric:
requirements: A credible experiment studying how linear (PCA) vs non-linear (t-SNE / UMAP-like) dimensionality reduction preserves cluster structure on synthetic high-dimensional datasets: methods share a consistent evaluation pipeline, execution covers multiple datasets with repeated seeds, and conclusions address H1/H2/H3 directionally.
judging_note: Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. UMAP may be unavailable in the environment — a documented fallback (e.g., Isomap, LLE) that tests the same scientific question should be credited rather than penalized.
sub_tasks: requirements: The dimensionality-reduction and clustering conditions are implemented in a way that supports a fair comparison.
sub_tasks: requirements: The submission implements the main reduction conditions relevant to the hypotheses — a linear method (PCA), at least one nonlinear method (t-SNE, UMAP, or a documented substitute such as Isomap), and preferably a no-reduction identity baseline — as distinct code paths.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: The submission generates multiple synthetic high-dimensional datasets that stress different geometries (e.g., isotropic blobs, anisotropic blobs, embedded non-linear manifolds such as moons) with reproducible seeds and retained ground-truth labels.
task_category: Code Development
finegrained_task_category: Dataset and Model Acquisition, requirements: A consistent post-reduction clustering backend (e.g., KMeans with k matching the true cluster count) and a shared metric-computation pipeline are applied uniformly across conditions.
task_category: Code Development
finegrained_task_category: Experimental Setup, requirements: Execution outputs quantitative cluster-quality and runtime metrics.
sub_tasks: requirements: Execution produces a machine-readable metrics artifact with numeric cluster-quality scores (silhouette, ARI, or equivalents) and a wall-clock reduction-time measure per (dataset, condition) cell.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Reported metrics are aggregated over multiple random seeds per (dataset, condition) with some dispersion measure. More seeds are better, but an honest small-seed run with variance reported is preferable to a single run.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Results are analyzed against the hypotheses with interpretable evidence.
sub_tasks: requirements: The submission compares nonlinear methods vs PCA on the nonlinear-manifold dataset using a cluster-quality metric (silhouette or ARI) and conveys whether the nonlinear methods produce meaningfully better cluster separation — judge directionally against H1.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission discusses PCA vs the best nonlinear method on the linearly-separable blobs dataset and conveys whether PCA is competitive (comparable quality) in that regime.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission reports runtime rankings across methods and conveys whether PCA is markedly faster than the nonlinear methods on most datasets.
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: The README or writeup describes setup, reports the key metric numbers, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as synthetic-only scope, hyperparameter sensitivity, or substitutions for unavailable methods. No strict word-count requirement.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation