Comparing clustering algorithms on synthetic non-convex and anisotropic shapes
Domain: ml (machine-learning, clustering)
Topic: Comparing clustering algorithms (k-means, agglomerative, DBSCAN, HDBSCAN, spectral) on synthetic shapes (moons, circles, blobs, anisotropic) with ground-truth labels
Venue: ARC-Bench 2026
Headline metric: adjusted_rand_score (maximize)

Research briefing:
Clustering algorithms encode very different geometric assumptions. K-means
prefers spherical, equal-variance groups; agglomerative clustering can adapt
to hierarchical structure depending on linkage; DBSCAN finds dense regions
and can recover non-convex shapes while labeling outliers as noise; spectral
clustering can separate manifolds when graph affinity is appropriate.
Because these assumptions interact strongly with data geometry, a single
benchmark score often hides systematic failure modes.

Synthetic datasets are ideal for a fast but meaningful comparison because we
can generate known structures with ground-truth labels: intertwined moons,
concentric circles, and anisotropic Gaussian blobs. These patterns expose
strengths and weaknesses that are difficult to isolate in real-world data.
With labeled synthetic data, external clustering metrics like adjusted rand
index (ARI) and normalized mutual information (NMI) directly quantify
partition recovery quality.

A credible CPU-scale study should compare several algorithms under a shared
protocol: standardized inputs, fixed train-size generation per seed, and
explicit handling of methods that output noise labels. It should include at
least one centroid baseline (k-means) and one density method (DBSCAN), then
test whether non-linear methods systematically outperform centroid methods on
non-convex shapes while avoiding large regressions on anisotropic blobs.

The practical goal is not to crown one universally best algorithm, but to
establish data-shape-dependent guidance. If shape complexity changes which
method wins, practitioners should choose clustering tools by geometric prior
rather than habit.

*How strongly does dataset geometry (non-convex vs anisotropic) determine which clustering algorithm achieves the best ground-truth agreement under a fixed CPU-budget protocol?*

Hypotheses:
statement: On the two non-convex datasets (moons and circles), at least one of {DBSCAN, SpectralClustering} achieves mean ARI that is at least 0.15 higher than k-means, averaged over ≥5 seeds., statement: On the anisotropic-blobs dataset, agglomerative clustering (ward linkage) achieves ARI greater than or equal to k-means in the seed-averaged result (difference >= 0.00)., statement: No single algorithm ranks first in ARI on all evaluated datasets; i.e., the best-ARI method differs on at least one dataset from the global winner.

Experiment design:
research_question: How strongly does dataset geometry (non-convex vs anisotropic) determine which clustering algorithm achieves the best agreement with ground-truth labels under a fixed CPU-budget protocol?
conditions: name: kmeans_k2_or_k3
description: KMeans baseline with n_clusters set to dataset ground-truth class count; n_init>=10., name: agglomerative_ward
description: AgglomerativeClustering with ward linkage and n_clusters equal to ground-truth class count., name: dbscan_tuned_eps
description: DBSCAN with eps selected from a small grid per dataset (e.g., 0.1 to 0.5) and fixed min_samples (e.g., 5)., name: spectral_rbf
description: SpectralClustering with RBF affinity and n_clusters equal to ground-truth class count.
baselines: kmeans_k2_or_k3 as centroid-based baseline, agglomerative_ward as hierarchical baseline
metrics: name: adjusted_rand_score
direction: maximize
description: Adjusted Rand Index against ground-truth labels, averaged over seeds., name: normalized_mutual_info
direction: maximize
description: Normalized Mutual Information against ground-truth labels., name: silhouette_score
direction: maximize
description: Internal cohesion/separation score computed from predicted labels (excluding noise-only failures).
datasets: name: moons
source: sklearn.datasets.make_moons (n_samples~800, noise~0.08), name: circles
source: sklearn.datasets.make_circles (n_samples~800, noise~0.06, factor~0.5), name: anisotropic_blobs
source: sklearn.datasets.make_blobs followed by linear transformation to create anisotropy
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 420

Requirements:

Scoring rubric:
requirements: A credible experiment comparing clustering algorithms across synthetic geometric datasets: clustering conditions are implemented, execution covers multiple datasets with repeated seeds and ARI-centric reporting, and the analysis addresses H1/H2/H3 directionally.
judging_note: Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Well-motivated algorithm substitutes (e.g., HDBSCAN for DBSCAN) that test the same question should be credited.
sub_tasks: requirements: The clustering conditions and dataset generators are implemented correctly.
sub_tasks: requirements: The submission implements multiple distinct clustering conditions — typically k-means, agglomerative, DBSCAN, and/or spectral — via sklearn APIs with separate code paths and method-specific hyperparameters.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: The submission generates multiple synthetic datasets with varied geometry (e.g., moons, circles, anisotropic blobs), preserves ground-truth labels, and standardizes features before clustering where appropriate.
task_category: Code Development
finegrained_task_category: Dataset and Model Acquisition, requirements: The experiment framework includes seed control and a consistent per-dataset protocol for cluster-count-dependent methods.
task_category: Code Development
finegrained_task_category: Experimental Setup, requirements: Execution produces clustering metrics for each condition.
sub_tasks: requirements: Execution outputs a metrics artifact containing numeric adjusted_rand_score (or equivalent) for each implemented condition on at least one dataset.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Reported results are aggregated over multiple random seeds per (dataset, condition) with a dispersion estimate. Honest small-seed runs with variance reported are preferable to a single run.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: For DBSCAN (or equivalent density-based method), the run documents a reasonable choice of eps/min_samples (either via a small search or a justified fixed value), logged in the artifacts.
task_category: Code Execution
finegrained_task_category: Hyperparameter Tuning, requirements: Results address H1/H2/H3 directionally with interpretable evidence.
sub_tasks: requirements: The submission compares ARI of density-based / spectral methods against k-means on nonlinear-geometry datasets and conveys whether they produce meaningfully better cluster assignments — judge directionally against H1.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The submission conveys dataset-wise ARI outcomes — whether agglomerative-ward is at least competitive with k-means on anisotropic blobs (H2) and whether any single method wins across all datasets (H3).
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: The README or writeup describes setup, key ARI/NMI findings per dataset, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations (synthetic-only scope, hyperparameter sensitivity, seed count, metric dependence). No strict word-count requirement.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation