HyperCore: Coreset Selection under Noise via Hypersphere Models

## Abstract

Abstract The goal of coreset selection methods is to identify representative subsets of datasets for efficient model training.
Yet, existing methods often ignore the possibility of annotation errors and require fixed pruning ratios, making them impractical in real-world settings.
We present HyperCore, a robust and adaptive coreset selection framework designed explicitly for noisy environments.
HyperCore leverages lightweight hypersphere models learned per class, embedding in-class samples close to a hypersphere center while naturally segregating out-of-class samples based on their distance.
By using Youden’s J statistic, HyperCore can adaptively select pruning thresholds, enabling automatic, noise-aware data pruning without hyperparameter tuning.
Our experiments reveal that HyperCore consistently surpasses state-of-the-art coreset selection methods, especially under noisy and low-data regimes.
HyperCore effectively discards mislabeled and ambiguous points, yielding compact yet highly informative subsets suitable for scalable and noise-free learning.

## 1 Introduction

Modern deep learning excels with scale, but scale comes at a cost.
Training on massive datasets drains resources and introduces noise, creating a growing need for efficient and robust data selection [34, 4, 41, 13].
In practice, acquiring or maintaining such large datasets is often infeasible due to storage limits, privacy constraints, or annotation costs [8, 37].
Coreset selection seeks to address this challenge by identifying a small, informative subset that preserves the performance of training on the full dataset [moser2025coreset, 31, 9, 3].
Beyond efficiency, coresets can improve robustness by excluding noisy, redundant, or overly difficult examples, reducing overfitting and sharpening generalization [2, 13].
As highlighted by Sorscher et al. [31], with the right pruning ratio, a well-selected coreset can even outperform full-data training, a surprising and powerful result, which has been verified in some applications [23, 21, 38, 22, 6].

Despite these benefits, selecting an optimal coreset remains nontrivial [41, 29].
Most methods rely on gradient heuristics [25, 20, 14], influence estimation [33, 25], or decision boundary estimates [7, 19].
Yet, current methods struggle with noise, computational overhead, and lack of class-awareness - key challenges in real-world applications [40].
Crucially, these approaches often prune via fixed sampling budgets rather than adapting to the natural density or ambiguity within each class [1, 31, 9, 41].

Our method, HyperCore, offers a new perspective.
We train lightweight hypersphere models [32, 28, 18] that learn to separate in-class from out-of-class samples in a class-conditional embedding space.
Here, “out-of-class” refers not only to samples from other classes but also to mislabeled, ambiguous, or corrupted inputs that appear atypical when measured against a given class distribution.
Treating such points as outliers is natural in a per-class setting, since they provide conflicting training signals for that class.
By measuring the distance of each point to its class-specific hypersphere center, we obtain an interpretable conformity score.

To determine the hypersphere decision boundary, we adaptively select pruning thresholds without tuning hyperparameters.
More explicitly, we exploit the *Youden’s J statistic* [39], a well-known criterion from signal detection theory, to filter uncertain or atypical examples in a per-class, data-driven way.
This thresholding adapts automatically to class imbalance, ambiguity, and noise, in contrast to fixed global ratios.

HyperCore is computationally lightweight: each class-specific model is trained independently, enabling parallelization across classes; thresholding reduces to a one-pass scan over sorted distances, i.e., $O(n_{c}\log n_{c})$ per class.
In practice, this cost scales linearly with the number of classes and can be amortized by parallel workers or shared backbones.

Our contributions can be summarized as follows:

- •
We introduce a simple *class-wise hypersphere formulation* with fixed centers and pseudo-Huber loss, yielding interpretable conformity scores and avoiding costly center estimation.
- •
We propose *adaptive pruning via Youden’s J statistic*, eliminating the need for global ratio tuning and naturally adjusting to per-class density and noise.
- •
We demonstrate *robustness under label noise and high pruning ratios*, where HyperCore outperforms state-of-the-art methods across ImageNet-1K and CIFAR-10.
- •
We provide *scalability analysis*, showing that HyperCore remains efficient due to embarrassingly parallel training and near-linear complexity.

## 2 Preliminaries

### 2.1 Coreset Selection

Consider a supervised learning setup, where the training set
$\mathcal{T}=\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{N}$
contains $N$ i.i.d. samples drawn from an unknown distribution $P$. Each input $\mathbf{x}_{i}\in\mathcal{X}$ is paired with a label $y_{i}\in\mathcal{Y}$.

###### Definition 1 (Coreset Selection)

The goal is to extract a subset $\mathcal{S}\subset\mathcal{T}$ with $\lvert\mathcal{S}\rvert\ll\lvert\mathcal{T}\rvert$, such that training a model $\theta^{\mathcal{S}}$ on $\mathcal{S}$ achieves comparable generalization to training $\theta^{\mathcal{T}}$ on the full dataset $\mathcal{T}$:

$$ $\mathcal{S}^{*}=\mathop{\arg\min}_{\begin{subarray}{c}\mathcal{S}\subset\mathcal{T}:\;\frac{\lvert\mathcal{S}\rvert}{\lvert\mathcal{T}\rvert}\approx 1-\alpha\end{subarray}}\;\;\mathbb{E}_{(\mathbf{x},y)\sim P}\left[\mathcal{L}(\mathbf{x},y;\,\theta^{\mathcal{S}})-\mathcal{L}(\mathbf{x},y;\,\theta^{\mathcal{T}})\right],$ (1) $$

where $\alpha\in(0,1)$ denotes the pruning ratio and $(1-\alpha)$ is the retained fraction. $\mathcal{L}$ is the task-specific loss.

Notation. Throughout the paper we use $\alpha$ for the pruning ratio and $(1-\alpha)$ for the retained fraction.

While simple in form, this objective is difficult to achieve in practice.
Effective coreset selection depends on identifying training samples that best support generalization.
Popular approaches estimate sample importance via gradients, influence scores, or diversity heuristics [24, 30, 36, 41].
Yet, these methods are often sensitive to noise, expensive to compute, or agnostic to class structure.

###### Definition 1 (Coreset Selection)

### 2.2 Hypersphere Classifier

Hypersphere classifiers, such as Deep SVDD [28] and FCDD [18], represent a class of anomaly detection methods that embed nominal data into a compact region of the feature space while mapping anomalies away.

###### Definition 2 (Hypersphere Classifier)

Given a set of samples $\mathbf{x}_{1},\dots,\mathbf{x}_{n}$ with labels $y_{i}\in\{0,1\}$ (where $y_{i}=0$ denotes a nominal sample and $y_{i}=1$ denotes an anomaly), a hypersphere classifier seeks to learn a neural network mapping $\phi(\mathbf{x};W)$ with parameters $W$ and a randomly (non-trivial) center $\mathbf{c}\in\mathbb{R}^{d}$ by optimizing the objective:

$$ $\min_{W}\frac{1}{n}\sum_{i=1}^{n}\Biggl[(1-y_{i})\,h\left(\|\phi(\mathbf{x}_{i};W)-\mathbf{c}\|\right)-y_{i}\log\ \left(1-\exp\ \left(-h\left(\|\phi(\mathbf{x}_{i};W)-\mathbf{c}\|\right)\ \right)\ \right)\Biggr],$ (2) $$

where $h:\mathbb{R}\to\mathbb{R}$ is the pseudo-Huber loss [11] defined as
$h(a)=\sqrt{a^{2}+1}-1$.
This loss function robustly penalizes deviations, interpolating from a quadratic penalty for small distances to a linear penalty for larger deviations.

Coreset Selection Context. This geometric intuition naturally facilitates threshold-based selection of representative data points, making hypersphere classifiers particularly suitable for robust coreset selection tasks.
In a one-vs-rest view, we ask if a sample is *typical of class $c$*. Four sources tend to fall farther from the $c$-center in a class-conditional embedding: (i) true out-of-class samples, (ii) mislabeled instances whose content supports another class, (iii) borderline examples mixing evidence from multiple classes, and (iv) low-quality or corrupted inputs. All four inject conflicting gradients for class $c$, so excluding them when curating $c$’s coreset improves representativeness.
In our experiments section, we show empirically that our derived coresets preferentially retain clean, central exemplars.

###### Definition 2 (Hypersphere Classifier)

## 3 Methodology

Our method is built upon class-wise hypersphere models that assess the representativeness of each sample by measuring the distance to a fixed hypersphere center, as illustrated in [Figure 1](https://arxiv.org/html/2509.21746v1#S3.F1) (left).
Specifically, we classify samples from the target class as normal, and all others as anomalies.

Figure: Figure 1: Left: Visualization of HyperCore. In-class samples are pulled toward the center, while out-of-class samples are pushed away, creating a clear separation. Right: Illustration of adaptive pruning ratio selection via Youden’s J statistic. Two candidate thresholds are compared, with the purple threshold yielding a higher $J$ value and thus being considered more optimal for pruning.
Refer to caption: x1.png

### 3.1 HyperCore Loss with Zero Center

A key simplification of HyperCore is anchoring the hypersphere center at the origin, i.e. $\mathbf{c}=\mathbf{0}$, which avoids explicit center estimation and enables lightweight optimization.
Let $\phi(\mathbf{x};W)\in\mathbb{R}^{d}$ denote the learned embedding of input $\mathbf{x}$ under parameters $W$.
The embedding norm $\|\phi(\mathbf{x};W)\|$ is then used as a measure of conformity.

###### Definition 3 (HyperCore Loss)

Given a sample $\mathbf{x}$ with label $y\in\{0,1\}$ (where $y=0$ denotes an in-class sample and $y=1$ an out-of-class sample), the HyperCore loss is defined as

$$ $L_{\text{HyperCore}}(\mathbf{x};W)=(1-y)\,h\!\left(\|\phi(\mathbf{x};W)\|\right)-y\,\log\!\Biggl(1-\exp\!\Bigl(-h\!\left(\|\phi(\mathbf{x};W)\|\right)\Bigr)\Biggr),$ (3) $$

where $h(a)=\sqrt{a^{2}+1}-1$ is the pseudo-Huber loss [11].
This loss penalizes small embedding norms for anomalies and encourages in-class samples to lie close to the origin.

We now show that the trivial solution $W=0$ (mapping all inputs to $\mathbf{0}$) is not optimal.

###### Lemma 1 (Balanced Sampling Prevents Trivial Collapse)

Assume that each training batch is balanced, i.e. the number of in-class ($y=0$) samples equals the number of out-of-class ($y=1$) samples.
Let $W_{0}$ be the all-zero weight configuration such that $\phi(\mathbf{x};W_{0})=\mathbf{0}$ for all $\mathbf{x}$.
Then the HyperCore loss at $W_{0}$ is unbounded:

$$ $L_{\text{HyperCore}}(\mathbf{x};W_{0})\to\infty,$ (4) $$

which rules out the trivial solution as optimal.

###### Proof

At $W\to W_{0}$, we have $\phi(\mathbf{x};W)\to\mathbf{0}$ and hence $\|\phi(\mathbf{x};W)\|\to 0$.
Since $h(0)=0$, the loss for in-class samples ($y=0$) vanishes.
For out-of-class samples ($y=1$), the term becomes

$$ $-\log\!\left(1-\exp\!\bigl(-h(0)\bigr)\right)=-\log(1-e^{0})=-\log(0)\to\infty.$ (5) $$

Thus, even a single out-of-class sample in a batch makes $L_{\text{HyperCore}}(\mathbf{x};W_{0})$ diverge.

###### Definition 3 (HyperCore Loss)

###### Lemma 1 (Balanced Sampling Prevents Trivial Collapse)

###### Proof

### 3.2 Static Pruning: A Baseline for Comparison

As a baseline, we adopt fixed pruning ratios.
Let $\mathcal{T}_{c}^{\mathrm{in}}$ denote the in-class dataset for class $c$.
Each sample $\mathbf{x}_{i}\in\mathcal{T}_{c}^{\mathrm{in}}$ is mapped to an embedding with norm

$$ $d_{i}=\|\phi_{c}(\mathbf{x}_{i})\|.$ (6) $$

Given a pruning fraction $\alpha$, we retain the $(1-\alpha)\cdot|\mathcal{T}_{c}^{\mathrm{in}}|$ samples with the smallest $d_{i}$.
Formally,

$$ $\mathcal{S}_{c}^{\mathrm{fixed}}=\{(\mathbf{x}_{i},c)\in\mathcal{T}_{c}^{\mathrm{in}}\mid d_{i}\leq\tau_{c}^{\mathrm{fixed}}\},$ (7) $$

where $\tau_{c}^{\mathrm{fixed}}$ is chosen such that exactly $(1-\alpha)\cdot|\mathcal{T}_{c}^{\mathrm{in}}|$ samples are kept.
The global coreset is $\mathcal{S}^{\mathrm{fixed}}=\bigcup_{c=0}^{C-1}\mathcal{S}_{c}^{\mathrm{fixed}}$.

Although simple, this approach does not adapt to class-specific density or noise levels.
Furthermore, finding an effective $\alpha$ typically requires testing multiple candidates, increasing overhead.

### 3.3 HyperCore with Adaptive Pruning Ratio

Instead of fixing $\alpha$, we determine class-specific thresholds via Youden’s $J$ statistic [39].
For each class $c$, let

$$ $\displaystyle D_{c}^{\mathrm{in}}$ $\displaystyle=\{d_{i}=\|\phi(\mathbf{x}_{i};W)\|:(\mathbf{x}_{i},c)\in\mathcal{T}_{c}^{\mathrm{in}}\},\text{ and }$ $\displaystyle D_{c}^{\mathrm{out}}$ $\displaystyle=\{d_{j}=\|\phi(\mathbf{x}_{j};W)\|:(\mathbf{x}_{j},c)\in\mathcal{T}\setminus\mathcal{T}_{c}^{\mathrm{in}}\}.$ (8) $$

For any candidate threshold $\tau$, we define

$$ $\text{TPR}_{c}(\tau)=\frac{|\{d\in D_{c}^{\mathrm{in}}:d\leq\tau\}|}{|D_{c}^{\mathrm{in}}|},\qquad\text{FPR}_{c}(\tau)=\frac{|\{d\in D_{c}^{\mathrm{out}}:d\leq\tau\}|}{|D_{c}^{\mathrm{out}}|}.$ (9) $$

Youden’s $J$ statistic, as illustrated in [Figure 1](https://arxiv.org/html/2509.21746v1#S3.F1) (right), is then

$$ $J_{c}(\tau)=\text{TPR}_{c}(\tau)-\text{FPR}_{c}(\tau).$ (10) $$

The optimal threshold is

$$ $\tau_{c}^{*}=\operatorname*{arg\,max}_{\tau\in D_{c}^{\mathrm{in}}}J_{c}(\tau).$ (11) $$

The class-specific coreset is

$$ $\mathcal{S}_{c}=\{(\mathbf{x}_{i},c)\in\mathcal{T}_{c}^{\mathrm{in}}\mid d_{i}\leq\tau_{c}^{*}\},$ (12) $$

and the global coreset is again the union $\mathcal{S}=\bigcup_{c=0}^{C-1}\mathcal{S}_{c}$.

###### Lemma 2 (Threshold search complexity)

For class $c$, computing $\tau_{c}^{*}=\arg\max_{\tau\in D_{c}^{\mathrm{in}}}J_{c}(\tau)$ requires sorting $D_{c}^{\mathrm{in}}$ and a single linear scan, i.e., $O(n_{c}\log n_{c})$ time and $O(n_{c})$ memory. Summed over classes, the total time is $\sum_{c}O(n_{c}\log n_{c})$ and memory $\sum_{c}O(n_{c})$, with $\sum_{c}n_{c}=N$.

###### Lemma 2 (Threshold search complexity)

### 3.4 Practical Remarks and Complexity

Training overhead. Each $\phi_{c}$ is a small network trained on $\mathcal{T}_{c}^{\mathrm{in}}\cup\mathcal{T}_{c}^{\mathrm{out}}$, typically much cheaper than full-dataset gradient-based selection.

Thresholding overhead. For each class, sorting $D_{c}^{\mathrm{in}}$ costs $O(n_{c}\log n_{c})$, where $n_{c}=|\mathcal{T}_{c}^{\mathrm{in}}|$. Across classes, the complexity is near-linear in $N$.

No fraction tuning. HyperCore automatically derives class-specific pruning ratios without requiring $\alpha$. A global budget can be enforced if needed, but allowing classes to self-threshold often yields greater robustness.

## 4 Experiments

In this section, we present experiments on ImageNet-1K [5] and CIFAR-10 [16] that assess HyperCore across several dimensions, including overall coreset quality, runtime efficiency, and robustness.

Backbone and training.
For all experiments, we use ResNet-18 [10], following training protocols from DeepCore [9].
Models are trained with SGD for 200 epochs using a cosine-annealed learning rate (initial 0.1), momentum 0.9, weight decay $5\times 10^{-4}$, and standard data augmentation (random crop + flip).
On CIFAR-10, we use a batch size of 128; on ImageNet-1K, a batch size of 256.
To establish upper-bound references, we also train ResNet-18 on random subsets of varying fractions of the full dataset.

HyperCore training.
Each class-specific HyperCore model is trained on balanced batches (half in-class, half out-of-class).
We use Adam with learning rate $10^{-4}$, batch size 128 for CIFAR-10 and 512 for ImageNet, and train for 100 epochs per class.
Since classes are independent, training runs fully in parallel, making the overall wall-clock cost scale with available compute rather than the number of classes.

Robustness protocol.
To evaluate label noise tolerance, we adopt the poisoning setup of Zhang et al. [40], injecting label noise and malicious relabeling into the training data.

**Table 1: Coreset selection performance on ImageNet-1K. We evaluate various pruning methods by training randomly initialized ResNet-18 models on their selected subsets and testing on the full ImageNet validation set. DeepFool and GraNd are excluded due to their substantial memory and computational demands.**
| Fraction $(1-\alpha)$ | 10% | 20% | 30% | 40% | 50% | 100% |
| --- | --- | --- | --- | --- | --- | --- |
| Herding [35] | 29.17$\pm$0.23 | 41.26$\pm$0.43 | 48.71$\pm$0.23 | 54.65 $\pm$ 0.07 | 58.92 $\pm$ 0.19 | 69.52$\pm$0.45 |
| k-Center Greedy [29] | 48.11$\pm$0.29 | 59.06$\pm$0.22 | 62.91$\pm$0.22 | 64.93 $\pm$ 0.22 | 66.04 $\pm$ 0.05 | 69.52$\pm$0.45 |
| Forgetting [33] | 55.31$\pm$0.07 | 60.36$\pm$0.12 | 62.45$\pm$0.11 | 63.97 $\pm$ 0.01 | 65.06 $\pm$ 0.02 | 69.52$\pm$0.45 |
| CAL[19] | 46.08$\pm$0.10 | 53.71$\pm$0.19 | 58.11$\pm$0.13 | 61.17$\pm$0.06 | 63.67 $\pm$0.28 | 69.52$\pm$0.45 |
| Craig [20] | 51.39$\pm$0.13 | 59.33$\pm$0.22 | 62.72$\pm$0.13 | 64.96$\pm$0.00 | 66.29 $\pm$0.00 | 69.52$\pm$0.45 |
| GradMatch [14] | 47.57$\pm$0.32 | 56.29$\pm$0.31 | 60.62$\pm$0.28 | 64.40$\pm$0.33 | 65.02 $\pm$ 0.50 | 69.52$\pm$0.45 |
| Glister [15] | 47.02$\pm$0.29 | 55.93$\pm$0.17 | 60.38$\pm$0.17 | 62.86$\pm$0.07 | 65.07$\pm$0.08 | 69.52$\pm$0.45 |
| HyperCore (ours) | 49.94$\pm$0.02 | 58.12$\pm$0.11 | 62.96$\pm$0.01 | 64.96$\pm$0.04 | 65.32$\pm$0.13 | 69.52$\pm$0.45 |

### 4.1 ImageNet-1K Results

Our evaluation on ImageNet-1K in [Table 1](https://arxiv.org/html/2509.21746v1#S4.T1) demonstrates that HyperCore achieves consistently strong performance, positioning itself among the top-performing coreset selection methods despite being explicitly designed with robustness as its primary objective.
Specifically, at moderate to high retained fractions (30–50%), HyperCore matches or slightly surpasses established state-of-the-art methods such as Craig and GradMatch.
Even at more aggressive pruning (e.g., $(1-\alpha)=10\%$–$20\%$ retained), HyperCore closely follows the best-performing methods and achieves highly competitive results.

**Table 2: Fixed coreset selection accuracy on CIFAR-10 using randomly initialized ResNet-18 models [10]. Bold entries indicate the highest performance at each data fraction.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding [35] | 19.8$\pm$2.7 | 29.2$\pm$2.4 | 31.1$\pm$2.9 | 50.7$\pm$1.6 | 63.1$\pm$3.4 | 75.2$\pm$1.0 | 80.8$\pm$1.5 | 85.4$\pm$1.2 | 88.4$\pm$0.6 | 90.9$\pm$0.4 | 94.4$\pm$0.1 | 95.6$\pm$0.1 |
| k-Center Greedy [29] | 19.9$\pm$0.9 | 25.3$\pm$0.9 | 32.6$\pm$1.6 | 55.6$\pm$2.8 | 74.6$\pm$0.9 | 87.3$\pm$0.2 | 91.0$\pm$0.3 | 92.6$\pm$0.2 | 93.5$\pm$0.5 | 94.3$\pm$0.2 | 95.5$\pm$0.2 | 95.6$\pm$0.1 |
| Forgetting [33] | 21.3$\pm$1.2 | 29.7$\pm$0.3 | 35.6$\pm$1.0 | 51.1$\pm$2.0 | 66.9$\pm$2.0 | 86.6$\pm$1.0 | 91.7$\pm$0.3 | 93.0$\pm$0.2 | 94.1$\pm$0.2 | 94.6$\pm$0.2 | 95.4$\pm$0.1 | 95.6$\pm$0.1 |
| GraNd [25] | 14.6$\pm$0.8 | 17.2$\pm$0.8 | 18.6$\pm$0.8 | 28.9$\pm$0.5 | 41.3$\pm$1.3 | 71.1$\pm$1.3 | 88.3$\pm$1.0 | 93.0$\pm$0.4 | 94.8$\pm$0.1 | 95.2$\pm$0.1 | 95.5$\pm$0.1 | 95.6$\pm$0.1 |
| CAL [19] | 23.1$\pm$1.8 | 31.7$\pm$0.9 | 39.7$\pm$3.8 | 60.8$\pm$1.4 | 69.7$\pm$0.8 | 79.4$\pm$0.9 | 85.1$\pm$0.7 | 87.6$\pm$0.3 | 89.6$\pm$0.4 | 90.9$\pm$0.4 | 94.7$\pm$0.2 | 95.6$\pm$0.1 |
| DeepFool [7] | 18.7$\pm$0.9 | 26.4$\pm$1.1 | 28.3$\pm$0.6 | 47.7$\pm$3.5 | 61.2$\pm$2.8 | 82.7$\pm$0.5 | 90.8$\pm$0.5 | 92.9$\pm$0.2 | 94.4$\pm$0.1 | 94.8$\pm$0.1 | 95.6$\pm$0.1 | 95.6$\pm$0.1 |
| Craig [20] | 19.3$\pm$0.3 | 29.1$\pm$1.6 | 32.8$\pm$1.8 | 42.5$\pm$1.7 | 59.9$\pm$2.1 | 78.1$\pm$2.5 | 90.0$\pm$0.5 | 92.8$\pm$0.2 | 94.3$\pm$0.2 | 94.8$\pm$0.1 | 95.5$\pm$0.1 | 95.6$\pm$0.1 |
| GradMatch [14] | 17.4$\pm$1.6 | 27.1$\pm$1.1 | 27.7$\pm$2.0 | 41.8$\pm$2.4 | 55.5$\pm$2.3 | 78.1$\pm$2.0 | 89.6$\pm$0.7 | 92.7$\pm$0.5 | 94.1$\pm$0.2 | 94.7$\pm$0.3 | 95.4$\pm$0.1 | 95.6$\pm$0.1 |
| Glister [15] | 18.4$\pm$1.3 | 26.5$\pm$0.7 | 29.4$\pm$1.9 | 42.1$\pm$1.0 | 56.8$\pm$1.8 | 77.2$\pm$2.4 | 88.8$\pm$0.6 | 92.7$\pm$0.4 | 94.2$\pm$0.1 | 94.8$\pm$0.2 | 95.5$\pm$0.1 | 95.6$\pm$0.1 |
| HyperCore (ours) | 24.5$\pm$1.3 | 35.4$\pm$1.0 | 40.7$\pm$1.0 | 60.3$\pm$1.3 | 71.1$\pm$0.9 | 83.5$\pm$0.5 | 88.6$\pm$0.4 | 91.1$\pm$0.3 | 92.3$\pm$0.1 | 93.1$\pm$0.1 | 95.0$\pm$0.2 | 95.6$\pm$0.1 |

Figure: Figure 2: Time-Measurement on CIFAR-10. HyperCore ranks among the fastest techniques, including training, averaging only 4 minutes per class and benefiting from a parallelizable design.
Refer to caption: x2.png

### 4.2 CIFAR-10 Results

Comparisons to other coreset baselines are shown in [Table 2](https://arxiv.org/html/2509.21746v1#S4.T2).
HyperCore achieves *up to 5.6% higher accuracy than the best baseline at aggressive pruning levels*.
More specifically, HyperCore is on par for larger retained fractions while reaching the overall best results between $(1-\alpha)=0.1\%$ and $10\%$.
Furthermore, the parallelizable design of HyperCore enables us to obtain these outcomes with significantly reduced execution times, as shown in [Figure 2](https://arxiv.org/html/2509.21746v1#S4.F2).

**Table 3: Fixed coreset selection performance under label noise on CIFAR-10, where 10% of the training labels are randomly corrupted by assigning them to incorrect classes. Bold entries indicate the highest performance at each data fraction.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding [35] | 11.4$\pm$0.9 | 10.8$\pm$0.5 | 11.1$\pm$0.9 | 10.6$\pm$1.1 | 11.7$\pm$0.9 | 26.0$\pm$3.4 | 50.1$\pm$1.3 | 71.0$\pm$0.6 | 79.1$\pm$1.8 | 84.6$\pm$0.6 | 90.4$\pm$0.1 | 90.8$\pm$0.1 |
|  | -8.4 | -18.4 | -20.0 | -40.1 | -51.4 | -49.2 | -30.7 | -14.4 | -9.3 | -6.3 | -4.0 | -4.8 |
| k-Center Greedy [29] | 12.6$\pm$1.3 | 14.3$\pm$0.8 | 16.1$\pm$1.0 | 29.6$\pm$1.6 | 41.7$\pm$3.0 | 62.0$\pm$2.0 | 73.8$\pm$1.8 | 80.2$\pm$0.7 | 83.9$\pm$0.7 | 86.6$\pm$0.5 | 90.4$\pm$0.3 | 90.8$\pm$0.1 |
|  | -7.3 | -11.0 | -16.5 | -26.0 | -32.9 | -25.3 | -17.2 | -12.4 | -9.6 | -7.7 | -5.1 | -4.8 |
| Forgetting [33] | 21.8$\pm$1.6 | 31.1$\pm$1.0 | 35.0$\pm$1.3 | 52.8$\pm$1.2 | 66.4$\pm$1.3 | 83.2$\pm$1.0 | 88.9$\pm$0.2 | 90.7$\pm$0.2 | 91.0$\pm$0.4 | 91.5$\pm$0.4 | 90.9$\pm$0.1 | 90.8$\pm$0.1 |
|  | +0.5 | +1.4 | -0.6 | +1.7 | -0.5 | -3.4 | -2.8 | -2.3 | -3.1 | -3.1 | -4.5 | -4.8 |
| GraNd [25] | 11.5$\pm$0.9 | 11.9$\pm$0.8 | 11.1$\pm$0.6 | 10.8$\pm$1.1 | 10.6$\pm$1.2 | 25.4$\pm$0.9 | 44.8$\pm$2.0 | 67.2$\pm$2.6 | 79.4$\pm$1.3 | 86.3$\pm$0.3 | 90.2$\pm$0.2 | 90.8$\pm$0.1 |
|  | -3.1 | -5.3 | -7.5 | -18.1 | -30.7 | -45.7 | -43.5 | -25.8 | -15.4 | -8.9 | -5.3 | -4.8 |
| CAL [19] | 21.3$\pm$1.7 | 30.8$\pm$1.0 | 36.8$\pm$1.3 | 59.9$\pm$0.8 | 71.3$\pm$1.0 | 80.0$\pm$0.2 | 83.9$\pm$0.6 | 87.1$\pm$0.3 | 89.1$\pm$0.2 | 90.6$\pm$0.2 | 91.9$\pm$0.1 | 90.8$\pm$0.1 |
|  | -1.8 | -0.9 | -2.9 | -0.9 | -1.6 | +0.6 | -1.2 | -0.5 | -0.5 | -0.3 | -2.8 | -4.8 |
| DeepFool [7] | 17.4$\pm$0.9 | 21.6$\pm$1.4 | 25.3$\pm$1.3 | 33.6$\pm$0.4 | 43.9$\pm$3.2 | 65.5$\pm$1.6 | 77.0$\pm$1.3 | 84.5$\pm$0.5 | 86.6$\pm$0.9 | 88.7$\pm$0.5 | 90.8$\pm$0.2 | 90.8$\pm$0.1 |
|  | -1.3 | -4.8 | -3.0 | -14.1 | -17.3 | -17.2 | -13.8 | -8.4 | -7.8 | -6.1 | -4.8 | -4.8 |
| Craig [20] | 19.5$\pm$1.4 | 20.2$\pm$1.4 | 24.8$\pm$1.1 | 30.4$\pm$0.9 | 31.7$\pm$1.6 | 39.2$\pm$1.5 | 58.4$\pm$2.9 | 73.1$\pm$1.4 | 81.4$\pm$0.6 | 85.3$\pm$0.4 | 90.5$\pm$0.3 | 90.8$\pm$0.1 |
|  | +0.2 | -8.9 | -8.0 | -12.1 | -28.2 | -38.9 | -31.6 | -19.7 | -12.9 | -9.5 | -5.0 | -4.8 |
| GradMatch [14] | 15.7$\pm$2.0 | 21.4$\pm$0.7 | 23.0$\pm$1.6 | 27.6$\pm$2.4 | 31.4$\pm$2.5 | 37.7$\pm$2.1 | 55.6$\pm$2.5 | 72.0$\pm$1.4 | 80.3$\pm$0.4 | 85.3$\pm$0.5 | 90.2$\pm$0.2 | 90.8$\pm$0.1 |
|  | -1.7 | -5.7 | -4.7 | -14.2 | -24.1 | -40.4 | -34.0 | -20.7 | -13.8 | -9.4 | -5.2 | -4.8 |
| Glister [15] | 14.9$\pm$2.0 | 20.9$\pm$1.5 | 24.5$\pm$1.3 | 29.0$\pm$1.9 | 31.7$\pm$2.1 | 40.2$\pm$2.3 | 57.2$\pm$1.3 | 72.2$\pm$1.3 | 80.6$\pm$0.4 | 85.4$\pm$0.5 | 90.2$\pm$0.2 | 90.8$\pm$0.1 |
|  | -3.5 | -5.6 | -4.9 | -13.1 | -25.1 | -37.0 | -31.6 | -20.5 | -13.6 | -9.4 | -5.3 | -4.8 |
| HyperCore (ours) | 20.9$\pm$1.5 | 32.6$\pm$1.1 | 41.4$\pm$1.0 | 60.5$\pm$1.4 | 70.0$\pm$0.9 | 84.2$\pm$0.9 | 89.1$\pm$0.4 | 91.0$\pm$0.5 | 92.6$\pm$0.1 | 93.5$\pm$0.2 | 93.7$\pm$0.3 | 90.8$\pm$0.1 |
|  | -3.6 | -2.8 | +0.7 | +0.2 | -1.1 | +0.7 | +0.5 | -0.1 | +0.3 | +0.4 | -1.3 | -4.8 |

In [Table 3](https://arxiv.org/html/2509.21746v1#S4.T3), we investigate how each coreset selection strategy holds up when 10% of the CIFAR-10 training labels are corrupted with random misassignments [40].
As one might expect, most approaches struggle at very small retained fractions (e.g., $(1-\alpha)=0.1\%$ or $0.5\%$).
In fact, if we look at the leftmost columns, methods such as GraNd or DeepFool perform particularly poorly, sometimes falling below 15% accuracy.
Yet even at these high pruning levels, it is notable that Forgetting stands out with a slightly higher result at $(1-\alpha)=0.1\%$.

For larger retained fractions (0.5% and beyond), one sees the advantages of HyperCore sharpen.
For instance, at 1% of the data, HyperCore achieves 41.4% accuracy, surpassing the second-best method (CAL) by a margin of about 4.6%. This gap widens in the mid-range fractions (10%, 20%, 30%), underscoring the resilience of our method to label noise: whereas other approaches tend to plateau or fade noticeably, HyperCore keeps the performance or advances.
As expected, accuracies converge at 90% dataset usage since almost all data is included.
Notably, from 40% onward, HyperCore empirically validates the theory of Sorscher et al. [31] by surpassing even full dataset performance.
Further experiments (see appendix) with VGG-16, InceptionNet, ResNet-50, and WRN-16-8 confirm these observations: HyperCore is exceptionally efficient under noisy conditions.

**Table 4: CIFAR-10 performance for training on full dataset vs on adaptively derived coresets (HyperCore).**
| Poisoning | Accuracy [%] | $\alpha_{\text{HyperCore}}$ [%] |  |
| --- | --- | --- | --- |
| Full Dataset | HyperCore |  |  |
| 0% | 95.6$\pm$0.1 | 95.6$\pm$0.2 | 00.6$\pm$0.4 |
| 10% | 90.8$\pm$0.1 | 94.8$\pm$0.2 | 16.4$\pm$1.3 |
| 20% | 87.5$\pm$0.4 | 93.5$\pm$0.3 | 29.4$\pm$2.4 |
| 30% | 85.9$\pm$0.3 | 91.1$\pm$0.3 | 41.2$\pm$2.7 |
| 40% | 83.7$\pm$0.5 | 86.9$\pm$1.3 | 50.5$\pm$4.0 |

### 4.3 Adaptive Coreset Selection

[Table 4](https://arxiv.org/html/2509.21746v1#S4.T4) demonstrates the adaptive coreset selection across various levels of label poisoning. When no label noise is present, HyperCore matches the accuracy obtained by training on the full dataset, illustrating that adaptive selection does not compromise model performance. More notably, as the level of label noise increases, HyperCore significantly outperforms training on the full dataset.

Figure: Figure 3: Average hypersphere radii (adaptive thresholds) and their standard deviations as a function of the relabeling percentage. The plot reveals that both the mean radius and its variability increase with higher levels of label poisoning, reflecting a broader dispersion in the embedding space and an adaptive expansion of the decision boundary to accommodate noise.
Refer to caption: x3.png

The [Figure 3](https://arxiv.org/html/2509.21746v1#S4.F3) shows that as the percentage of relabeling increases (i.e., as the level of label poisoning grows), the adaptive thresholds - the hypersphere radii - tend to increase. This behavior suggests that with more noise, the embeddings become more dispersed; in order to retain as many true inlier samples as possible, the model adapts by enlarging the decision boundary. Moreover, the rising standard deviation across classes indicates that the impact of label noise is not uniform: some classes experience a greater shift in their radii than others. In short, the model compensates for increased uncertainty by raising the threshold, which, though it might seem counterintuitive at first, is necessary to maintain robust discrimination between inliers and outliers under noisy conditions.

### 4.4 Analysis of Youden’s J Statistics

Regarding adaptiveness, [Figure 4](https://arxiv.org/html/2509.21746v1#S4.F4) compares the key performance curves of HyperCore under increasing levels of artificially introduced label noise.
In the left panel, we plot the confusion-based rates, namely True Positive Rate (TPR), False Positive Rate (FPR), True Negative Rate (TNR), and False Negative Rate (FNR), as a function of the poisoning percentage.
Broadly speaking, positive rates denote included samples, while negative rates mean they are excluded.
Despite the growing noise, we observe that TPR and TNR remain consistently higher than FPR and FNR, indicating that our hypersphere models selectively exclude corrupted or ambiguous samples.
Meanwhile, FPR and FNR only moderately increase, suggesting that HyperCore successfully mitigates the risk of discarding genuine samples or retaining mislabeled ones.

In the right panel, we plot Youden’s $J$ in orange, alongside the fraction of removed data in blue.
Even as the fraction removed escalates for severe noise, Youden’s $J$ remains relatively stable.
This interplay demonstrates that the pruning decisions of HyperCore are not overly conservative.
Although HyperCore discards an increasingly large portion of the data under extreme mislabeling, it still identifies informative inliers with sufficient reliability to maintain a viable Youden’s $J$.
Overall, the figure underscores strong resilience to label noise.

Figure: Figure 4: Left: Confusion-based metrics (TPR, FPR, TNR, FNR) under increasing label poisoning in CIFAR-10. Right: Youden’s $J$ (orange) and fraction of removed samples (blue). Both plots highlight HyperCore’s robust selection behavior across varying degrees of poisoned labels (error-bands highlight the variance between classes).
Refer to caption: x4.png

## 5 Related Work

#### Distance-Based Pruning and Anomaly Detection.

In parallel, a rich literature on anomaly or outlier detection capitalizes on distance metrics.
One-class methods such as *Support Vector Data Description* (SVDD) [32] enclose normal samples in a minimal-radius hypersphere, labeling points outside as outliers.
Deep SVDD extends this idea to a learned representation, forcing inliers near a randomly-sampled center in feature space [28, 18].
These methods align with the *hypersphere* concept in HyperCore:
Like one-class approaches, HyperCore identifies “inlier” samples with small norm while pushing outliers away.
Unlike hypersphere classifiers, HyperCore uses simpler per-class MLPs with Youden’s J thresholds, and a modified loss function.

#### Norm-Based Confidence Scores in Distillation and Noise Removal.

Several approaches for dataset distillation or data pruning measure “confidence” using norms in feature space.
For example, Lee et al. [17] compute class-conditional Gaussians on deep embeddings (Mahalanobis distance) and exclude points far from the nearest class center.
Similarly, Pleiss et al. [26] track margin-based criteria to detect possible mislabels, effectively removing outlier examples.
In coreset selection under noisy labels, Kang et al. [12] highlight that samples near the class centroid are typically correct, while label errors lie on the distribution fringe.
This principle resonates with HyperCore’s geometry-driven approach:
we train a *binary in/out* classifier per class to separate inlier vs. out-class data, then threshold based on distance.

#### Lightweight vs. Full-Model Coreset Approaches.

While gradient or influence-based selection can yield high-quality coresets [25, 14], such methods typically incur substantial overhead:
they require partial or entire model training to compute per-sample gradients or forgetting events [33].
By contrast, HyperCore remains *partially trained* (it fits a class-wise MLP to discriminate in/out), but each MLP sees only a subset of the dataset and does not require a large architecture or global alignment.
This design reduces computation while flexibly adapting to each class’s unique geometry.

#### Prototype Selection and Continual Learning.

Prototype-based selection methods identify exemplars that approximate the class mean.
For instance, iCaRL [27] selects a small set of class representatives that minimize the distance to that class’s mean embedding.
When data are mislabeled or heavily imbalanced, however, simple centroid-based picks can inadvertently keep outliers if they exhibit subtle bias in embedding space.
HyperCore addresses such issues by *actively* learning a boundary between inliers and outliers for each class, producing a more robust subset.
In continual learning, HyperCore could replace iCaRL’s herding by selecting reliably central class samples.

#### Relation to Our HyperCore Approach.

We draw on the success of minimal, class-wise boundaries (like SVDD), but unify them with a simple *per-class in/out MLP* plus a *Youden’s $J$ threshold* to auto-prune ambiguous points.
As a result, HyperCore effectively discards label noise and yields a representative coreset *without* requiring a large network or full-model backprop.
This blend of geometry-driven inlier detection, threshold-based selection, and partial learning stands in contrast to existing coreset methods that rely on global fractions or heavy optimization of large models.
HyperCore complements existing methods by providing a lightweight, noise-robust, class-specific alternative.

## 6 Limitations

HyperCore has several limitations that are worth noting despite its significant strengths.
The hypersphere models depend heavily on learning meaningful embeddings from relatively small per-class data subsets.
In classes with extremely limited or highly imbalanced data, the learned boundaries might degrade, reducing the robustness of HyperCore’s adaptive pruning.
Also, training separate hypersphere models per class could become computationally expensive as the number of classes scales (e.g., beyond thousands of classes) and the amount of GPUs/CPUs is limited, although HyperCore significantly reduces computational overhead compared to full-model coreset methods.

In addition, Youden’s $J$ implicitly treats false positives and false negatives as equally costly.
In domains with highly asymmetric costs or constraints (e.g., extreme class imbalance or safety-critical false negatives), alternative thresholding rules may be preferable—such as optimizing a weighted $J$ (cost-sensitive TPR/FPR), setting a target precision/recall operating point, or calibrating thresholds via validation risk minimization.

## 7 Conclusion & Future Work

We introduced HyperCore, a robust coreset selection framework leveraging hypersphere models.
Unlike existing methods, HyperCore utilizes class-conditional embeddings with adaptive pruning thresholds determined by Youden’s J statistic, enabling automatic and noise-aware subset selection without extensive hyperparameter tuning.
HyperCore raises the bar for robust coreset selection, setting new benchmarks for pruning accuracy and label noise tolerance.
By effectively discarding mislabeled or ambiguous data points, HyperCore ensures that the retained coresets are compact yet highly representative, thereby promoting efficient and robust model training.

Future work includes applying HyperCore to semi-supervised, continual learning, and large-scale tasks.
Additionally, analyzing dynamic updates to hypersphere boundaries could further enhance HyperCore’s versatility.

## References

- [1]
Agarwal, P.K., Har-Peled, S., Varadarajan, K.R., et al.: Geometric approximation via coresets. Combinatorial and computational geometry 52(1), 1–30 (2005)
- [2]
Bengio, S., Dembczynski, K., Joachims, T., Kloft, M., Varma, M.: Extreme classification (dagstuhl seminar 18291). In: Dagstuhl Reports. vol. 8. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2019)
- [3]
Bhalerao, M.M.: On Fine-Tuning Submodular Functions for Data Subset Selection. Master’s thesis, University of Washington (2024)
- [4]
Csiba, D., Richtárik, P.: Importance sampling for minibatches. The Journal of Machine Learning Research 19(1), 962–982 (2018)
- [5]
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: CVPR. pp. 248–255. Ieee (2009)
- [6]
Ding, Q., Liang, Z., Wang, L., Wang, Y., Yang, J.: Not all patches are equal: Hierarchical dataset condensation for single image super-resolution. IEEE Signal Processing Letters (2023)
- [7]
Ducoffe, M., Precioso, F.: Adversarial active learning for deep networks: a margin based approach. arXiv preprint arXiv:1802.09841 (2018)
- [8]
Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., et al.: Predictability and surprise in large generative models. In: 2022 ACM Conference on Fairness, Accountability, and Transparency (2022)
- [9]
Guo, C., Zhao, B., Bai, Y.: Deepcore: A comprehensive library for coreset selection in deep learning. In: International Conference on Database and Expert Systems Applications. pp. 181–195. Springer (2022)
- [10]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR. pp. 770–778 (2016)
- [11]
Huber, P.J.: Robust estimation of a location parameter. In: Breakthroughs in statistics: Methodology and distribution, pp. 492–518. Springer (1992)
- [12]
Kang, B., Xie, S., Rohrbach, M., Yan, Z., Gordo, A., Feng, J., Kalantidis, Y.: Decoupling representation and classifier for long-tailed recognition. arXiv preprint arXiv:1910.09217 (2019)
- [13]
Katharopoulos, A., Fleuret, F.: Not all samples are created equal: Deep learning with importance sampling. In: ICML. pp. 2525–2534. PMLR (2018)
- [14]
Killamsetty, K., Durga, S., Ramakrishnan, G., De, A., Iyer, R.: Grad-match: Gradient matching based data subset selection for efficient deep model training. In: ICML. pp. 5464–5474. PMLR (2021)
- [15]
Killamsetty, K., Sivasubramanian, D., Ramakrishnan, G., Iyer, R.: Glister: Generalization based data subset selection for efficient and robust learning. In: AAAI. vol. 35, pp. 8110–8118 (2021)
- [16]
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
- [17]
Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurIPS 31 (2018)
- [18]
Liznerski, P., Ruff, L., Vandermeulen, R.A., Franks, B.J., Kloft, M., Müller, K.R.: Explainable deep one-class classification. arXiv preprint arXiv:2007.01760 (2020)
- [19]
Margatina, K., Vernikos, G., Barrault, L., Aletras, N.: Active learning by acquiring contrastive examples. arXiv preprint arXiv:2109.03764 (2021)
- [20]
Mirzasoleiman, B., Bilmes, J., Leskovec, J.: Coresets for data-efficient training of machine learning models. In: ICML. pp. 6950–6960. PMLR (2020)
- [21]
Moser, B., Raue, F., Hees, J., Dengel, A.: Less is more: Proxy datasets in nas approaches. In: CVPR. pp. 1953–1961 (2022)
- [22]
Moser, B.B., Raue, F., Dengel, A.: A study in dataset pruning for image super-resolution. In: International Conference on Artificial Neural Networks. pp. 351–363. Springer (2024)
- [23]
Na, B., Mok, J., Choe, H., Yoon, S.: Accelerating neural architecture search via proxy data. arXiv preprint arXiv:2106.04784 (2021)
- [24]
Nogueira, S., Sechidis, K., Brown, G.: On the stability of feature selection algorithms. Journal of Machine Learning Research 18(174), 1–54 (2018)
- [25]
Paul, M., Ganguli, S., Dziugaite, G.K.: Deep learning on a data diet: Finding important examples early in training. NeurIPS 34, 20596–20607 (2021)
- [26]
Pleiss, G., Zhang, T., Elenberg, E., Weinberger, K.Q.: Identifying mislabeled data using the area under the margin ranking. NeurIPS 33, 17044–17056 (2020)
- [27]
Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: CVPR. pp. 2001–2010 (2017)
- [28]
Ruff, L., Vandermeulen, R., Goernitz, N., Deecke, L., Siddiqui, S.A., Binder, A., Müller, E., Kloft, M.: Deep one-class classification. In: ICML. pp. 4393–4402. PMLR (2018)
- [29]
Sener, O., Savarese, S.: Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489 (2017)
- [30]
Song, L., Zhang, J., Yang, T., Goto, M.: Adaptive ranking-based sample selection for weakly supervised class-imbalanced text classification. arXiv preprint arXiv:2210.03092 (2022)
- [31]
Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., Morcos, A.: Beyond neural scaling laws: beating power law scaling via data pruning. NeurIPS 35, 19523–19536 (2022)
- [32]
Tax, D.M., Duin, R.P.: Support vector data description. Machine learning 54, 45–66 (2004)
- [33]
Toneva, M., Sordoni, A., Combes, R.T.d., Trischler, A., Bengio, Y., Gordon, G.J.: An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159 (2018)
- [34]
Wang, T., Zhu, J.Y., Torralba, A., Efros, A.A.: Dataset distillation. arXiv preprint arXiv:1811.10959 (2018)
- [35]
Welling, M.: Herding dynamical weights to learn. In: ICML. pp. 1121–1128 (2009)
- [36]
Xiao, L., Liu, S., He, Y., Wang, X.: Rethinking large-scale dataset compression: Shifting focus from labels to images. arXiv preprint arXiv:2502.06434 (2025)
- [37]
Yang, R., Su, L.: Data-efficient trajectory prediction via coreset selection. arXiv preprint arXiv:2409.17385 (2024)
- [38]
Yao, P., Liao, C., Jia, J., Tan, J., Chen, B., Song, C., Zhang, D.: Asp: Automatic selection of proxy dataset for efficient automl. arXiv preprint arXiv:2310.11478 (2023)
- [39]
Youden, W.J.: Index for rating diagnostic tests. Cancer 3(1), 32–35 (1950)
- [40]
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning (still) requires rethinking generalization. Communications of the ACM 64(3), 107–115 (2021)
- [41]
Zheng, H., Liu, R., Lai, F., Prakash, A.: Coverage-centric coreset selection for high pruning rates. ICLR (2022)

## Appendix 0.A Hardware and Software

All experiments were run on a workstation equipped with an NVIDIA RTX A6000 GPU (48 GB VRAM). Our implementation uses PyTorch 1.10.1 with torchvision 0.11.2, and we build upon the DeepCore library for coreset selection routines.

## Appendix 0.B Datasets

We evaluated HyperCore on two widely used image‐classification benchmarks:

- •
CIFAR-10: 50,000 training and 10,000 test images across 10 classes, each of size $32\times 32$ pixels. Its small, balanced format makes it ideal for studying the behavior of coreset methods at extreme pruning levels.
- •
ImageNet-1K: 1,281,167 training and 50,000 validation images spanning 1,000 object categories. Following standard practice, we resize all images to $224\times 224$ pixels before selection and training.

## Appendix 0.C Additional Cross-Architecture Evaluations

**Table 5: Coreset selection performances on CIFAR-10 with five randomly initialized VGG-16 models. The best results are marked in bold.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 11.6$\pm$1.2 | 12.7$\pm$0.6 | 16.0$\pm$3.9 | 54.1$\pm$3.1 | 66.0$\pm$3.5 | 71.5$\pm$1.5 | 78.3$\pm$1.0 | 80.9$\pm$0.5 | 85.7$\pm$1.5 | 88.2$\pm$0.7 | 93.2$\pm$0.3 | 94.3$\pm$0.0 |
| k-Center Greedy | 12.3$\pm$1.0 | 14.5$\pm$2.9 | 14.7$\pm$1.3 | 38.7$\pm$12.0 | 75.6$\pm$1.3 | 85.5$\pm$0.3 | 89.0$\pm$0.4 | 91.0$\pm$0.2 | 91.8$\pm$0.3 | 92.7$\pm$0.2 | 94.1$\pm$0.1 | 94.3$\pm$0.0 |
| Forgetting | 12.5$\pm$0.8 | 13.8$\pm$2.8 | 25.6$\pm$4.3 | 55.9$\pm$1.3 | 75.2$\pm$1.4 | 86.2$\pm$0.2 | 89.0$\pm$0.2 | 91.3$\pm$0.2 | 91.9$\pm$0.2 | 92.8$\pm$0.0 | 94.1$\pm$0.2 | 94.3$\pm$0.0 |
| GraNd | 11.7$\pm$0.6 | 12.5$\pm$0.0 | 16.6$\pm$3.0 | 28.1$\pm$0.0 | 42.4$\pm$0.9 | 73.4$\pm$0.7 | 86.5$\pm$0.7 | 91.2$\pm$0.0 | 93.1$\pm$0.4 | 93.6$\pm$0.2 | 94.1$\pm$0.2 | 94.3$\pm$0.0 |
| CAL | 12.9$\pm$2.0 | 15.8$\pm$4.1 | 19.2$\pm$4.5 | 63.0$\pm$1.4 | 72.2$\pm$0.2 | 78.1$\pm$0.9 | 82.6$\pm$0.5 | 85.0$\pm$0.2 | 87.6$\pm$0.7 | 89.6$\pm$0.4 | 93.0$\pm$0.2 | 94.3$\pm$0.0 |
| DeepFool | 11.6$\pm$0.7 | 14.6$\pm$1.3 | 15.6$\pm$4.1 | 47.3$\pm$6.8 | 71.7$\pm$1.1 | 84.2$\pm$0.4 | 90.0$\pm$0.2 | 91.6$\pm$0.1 | 92.4$\pm$0.2 | 93.2$\pm$0.2 | 94.1$\pm$0.1 | 94.3$\pm$0.0 |
| Craig | 13.3$\pm$1.4 | 14.2$\pm$1.4 | 13.6$\pm$0.0 | 46.6$\pm$0.0 | 67.3$\pm$3.1 | 82.4$\pm$0.6 | 88.8$\pm$0.3 | 91.4$\pm$0.0 | 92.7$\pm$0.2 | 93.4$\pm$0.1 | 94.2$\pm$0.1 | 94.3$\pm$0.0 |
| GradMatch | 13.4$\pm$2.0 | 13.0$\pm$2.3 | 13.0$\pm$0.0 | 44.6$\pm$3.7 | 53.9$\pm$1.4 | 77.9$\pm$0.4 | 87.2$\pm$0.3 | 90.2$\pm$0.6 | 92.6$\pm$0.0 | 93.4$\pm$0.0 | 94.2$\pm$0.1 | 94.3$\pm$0.0 |
| Glister | 15.7$\pm$0.0 | 13.9$\pm$3.0 | 18.9$\pm$5.0 | 44.0$\pm$4.0 | 53.5$\pm$0.9 | 78.2$\pm$2.2 | 86.4$\pm$0.1 | 89.7$\pm$0.0 | 92.5$\pm$0.2 | 93.3$\pm$0.2 | 94.2$\pm$0.1 | 94.3$\pm$0.0 |
| HyperCore (ours) | 15.7$\pm$0.6 | 16.6$\pm$1.2 | 24.2$\pm$2.2 | 59.7$\pm$0.3 | 73.3$\pm$0.4 | 83.5$\pm$0.1 | 87.2$\pm$0.2 | 89.3$\pm$0.2 | 90.6$\pm$0.2 | 91.5$\pm$0.1 | 93.7$\pm$0.1 | 94.3$\pm$0.0 |

**Table 6: Coreset selection performances on CIFAR-10 with five randomly initialized ResNet-50 models. The best results are marked in bold.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 16.2$\pm$2.4 | 23.0$\pm$2.4 | 29.5$\pm$2.8 | 46.3$\pm$2.8 | 58.5$\pm$5.2 | 73.1$\pm$2.1 | 80.0$\pm$0.9 | 84.1$\pm$0.5 | 86.4$\pm$0.9 | 89.6$\pm$0.9 | 94.6$\pm$0.3 | 95.4$\pm$0.3 |
| k-Center Greedy | 15.4$\pm$1.8 | 18.8$\pm$3.1 | 25.0$\pm$4.6 | 53.1$\pm$1.9 | 68.8$\pm$2.1 | 86.6$\pm$0.4 | 90.2$\pm$0.6 | 91.7$\pm$0.2 | 93.4$\pm$0.3 | 94.2$\pm$0.3 | 95.3$\pm$0.3 | 95.4$\pm$0.3 |
| Forgetting | 17.0$\pm$2.2 | 26.9$\pm$2.7 | 34.0$\pm$1.0 | 49.5$\pm$2.9 | 67.5$\pm$2.6 | 86.2$\pm$0.7 | 89.9$\pm$0.4 | 92.0$\pm$0.3 | 92.8$\pm$0.2 | 93.6$\pm$0.1 | 95.2$\pm$0.1 | 95.4$\pm$0.3 |
| GraNd | 15.1$\pm$1.7 | 17.2$\pm$1.4 | 19.7$\pm$1.2 | 23.6$\pm$1.6 | 34.9$\pm$2.4 | 70.3$\pm$5.0 | 85.5$\pm$1.4 | 92.0$\pm$0.3 | 94.2$\pm$0.4 | 94.9$\pm$0.4 | 95.4$\pm$0.2 | 95.4$\pm$0.3 |
| CAL | 19.9$\pm$1.3 | 27.5$\pm$0.6 | 35.9$\pm$1.2 | 56.4$\pm$1.4 | 66.2$\pm$1.1 | 77.6$\pm$0.8 | 83.0$\pm$0.4 | 85.5$\pm$0.5 | 88.2$\pm$0.5 | 90.5$\pm$0.6 | 94.2$\pm$0.2 | 95.4$\pm$0.3 |
| DeepFool | 14.4$\pm$2.0 | 19.8$\pm$1.4 | 26.4$\pm$1.1 | 42.7$\pm$2.9 | 65.7$\pm$2.8 | 86.3$\pm$0.9 | 91.1$\pm$0.5 | 92.9$\pm$0.3 | 93.9$\pm$0.2 | 94.5$\pm$0.2 | 95.3$\pm$0.2 | 95.4$\pm$0.3 |
| Craig | 17.0$\pm$1.1 | 21.5$\pm$1.4 | 28.1$\pm$2.6 | 42.0$\pm$1.5 | 55.1$\pm$2.7 | 80.8$\pm$2.8 | 89.9$\pm$0.7 | 92.5$\pm$0.7 | 93.5$\pm$0.2 | 94.7$\pm$0.1 | 95.4$\pm$0.2 | 95.4$\pm$0.3 |
| GradMatch | 14.3$\pm$2.8 | 21.7$\pm$0.9 | 28.2$\pm$0.9 | 37.4$\pm$2.6 | 50.5$\pm$3.5 | 74.0$\pm$2.3 | 87.6$\pm$1.1 | 92.3$\pm$0.4 | 93.8$\pm$0.3 | 94.7$\pm$0.5 | 95.2$\pm$0.2 | 95.4$\pm$0.3 |
| Glister | 14.9$\pm$0.9 | 20.4$\pm$1.5 | 24.2$\pm$3.3 | 37.0$\pm$1.9 | 50.1$\pm$3.0 | 79.6$\pm$2.5 | 88.6$\pm$0.8 | 92.0$\pm$0.3 | 93.5$\pm$0.2 | 94.5$\pm$0.3 | 95.2$\pm$0.2 | 95.4$\pm$0.3 |
| HyperCore (ours) | 20.3$\pm$1.8 | 23.0$\pm$1.6 | 36.9$\pm$0.5 | 56.4$\pm$2.7 | 69.2$\pm$1.4 | 82.4$\pm$1.0 | 88.0$\pm$0.4 | 90.4$\pm$0.2 | 91.6$\pm$0.2 | 92.9$\pm$0.3 | 94.8$\pm$0.2 | 95.4$\pm$0.3 |

**Table 7: Coreset selection performances on CIFAR-10 with five randomly initialized InceptionNetV3 models. The best results are marked in bold.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 14.6$\pm$0.7 | 21.8$\pm$1.7 | 28.5$\pm$1.4 | 42.9$\pm$4.0 | 61.1$\pm$2.2 | 74.0$\pm$1.3 | 80.5$\pm$1.0 | 84.7$\pm$0.4 | 87.7$\pm$0.9 | 90.3$\pm$0.8 | 94.8$\pm$0.2 | 95.6$\pm$0.1 |
| k-Center Greedy | 15.1$\pm$2.0 | 22.1$\pm$3.0 | 26.6$\pm$1.4 | 51.2$\pm$2.3 | 72.9$\pm$1.7 | 85.8$\pm$0.4 | 89.8$\pm$0.4 | 92.2$\pm$0.3 | 93.6$\pm$0.3 | 94.4$\pm$0.4 | 95.4$\pm$0.2 | 95.6$\pm$0.1 |
| Forgetting | 18.5$\pm$0.6 | 28.5$\pm$1.0 | 32.2$\pm$0.9 | 50.9$\pm$1.5 | 69.3$\pm$1.3 | 85.4$\pm$0.9 | 90.4$\pm$0.4 | 92.4$\pm$0.1 | 93.4$\pm$0.2 | 94.3$\pm$0.3 | 95.4$\pm$0.2 | 95.6$\pm$0.1 |
| GraNd | 14.8$\pm$2.3 | 17.4$\pm$1.2 | 20.1$\pm$0.8 | 27.1$\pm$1.1 | 38.1$\pm$1.4 | 70.0$\pm$2.0 | 86.8$\pm$0.6 | 92.9$\pm$0.4 | 94.4$\pm$0.3 | 95.0$\pm$0.3 | 95.5$\pm$0.1 | 95.6$\pm$0.1 |
| CAL | 17.9$\pm$1.2 | 31.6$\pm$1.5 | 36.3$\pm$1.1 | 60.2$\pm$2.0 | 70.6$\pm$0.6 | 78.7$\pm$0.7 | 83.1$\pm$0.5 | 86.3$\pm$0.8 | 88.9$\pm$0.4 | 90.3$\pm$0.3 | 94.3$\pm$0.1 | 95.6$\pm$0.1 |
| DeepFool | 14.5$\pm$1.6 | 22.3$\pm$1.1 | 26.8$\pm$1.6 | 47.7$\pm$3.0 | 70.2$\pm$2.1 | 83.6$\pm$1.2 | 90.2$\pm$0.4 | 92.8$\pm$0.2 | 94.0$\pm$0.2 | 94.6$\pm$0.3 | 95.6$\pm$0.2 | 95.6$\pm$0.1 |
| Craig | 16.5$\pm$2.5 | 24.7$\pm$1.3 | 27.8$\pm$2.0 | 44.5$\pm$1.8 | 58.8$\pm$2.5 | 79.5$\pm$1.4 | 88.0$\pm$1.1 | 92.3$\pm$0.2 | 94.0$\pm$0.2 | 95.0$\pm$0.2 | 95.6$\pm$0.1 | 95.6$\pm$0.1 |
| GradMatch | 16.5$\pm$1.2 | 24.2$\pm$0.8 | 29.1$\pm$1.0 | 40.1$\pm$2.7 | 53.1$\pm$2.2 | 76.9$\pm$2.6 | 87.1$\pm$0.3 | 91.8$\pm$0.7 | 94.2$\pm$0.5 | 94.9$\pm$0.3 | 95.4$\pm$0.2 | 95.6$\pm$0.1 |
| Glister | 14.8$\pm$1.5 | 22.9$\pm$1.8 | 29.3$\pm$1.8 | 41.3$\pm$1.8 | 52.4$\pm$3.1 | 77.5$\pm$1.7 | 87.8$\pm$0.7 | 91.8$\pm$0.4 | 94.0$\pm$0.1 | 94.9$\pm$0.1 | 95.6$\pm$0.1 | 95.6$\pm$0.1 |
| HyperCore (ours) | 20.8$\pm$0.7 | 32.6$\pm$0.8 | 37.9$\pm$0.6 | 57.2$\pm$1.6 | 71.0$\pm$0.8 | 83.2$\pm$0.9 | 87.5$\pm$0.4 | 90.2$\pm$0.2 | 92.4$\pm$0.2 | 93.4$\pm$0.3 | 95.2$\pm$0.1 | 95.6$\pm$0.1 |

**Table 8: Coreset selection performances on CIFAR-10 with five randomly initialized WRN-16-8 models. The best results are marked in bold.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 23.8$\pm$1.4 | 31.3$\pm$3.5 | 39.6$\pm$2.8 | 60.2$\pm$1.2 | 66.9$\pm$2.3 | 77.3$\pm$1.0 | 81.9$\pm$0.7 | 86.4$\pm$1.3 | 89.0$\pm$0.6 | 92.3$\pm$1.0 | 95.2$\pm$0.1 | 96.0$\pm$0.1 |
| k-Center Greedy | 19.2$\pm$0.6 | 27.7$\pm$0.6 | 34.6$\pm$0.2 | 67.9$\pm$0.8 | 81.6$\pm$0.5 | 89.4$\pm$0.2 | 92.1$\pm$0.3 | 93.6$\pm$0.2 | 94.3$\pm$0.1 | 94.9$\pm$0.1 | 95.9$\pm$0.1 | 96.0$\pm$0.1 |
| Forgetting | 21.5$\pm$1.0 | 30.1$\pm$0.9 | 36.3$\pm$0.6 | 58.5$\pm$1.0 | 74.9$\pm$0.8 | 88.8$\pm$0.5 | 93.3$\pm$0.1 | 94.5$\pm$0.1 | 95.0$\pm$0.1 | 95.4$\pm$0.1 | 95.9$\pm$0.1 | 96.0$\pm$0.1 |
| GraNd | 15.0$\pm$1.8 | 19.5$\pm$0.7 | 21.4$\pm$0.3 | 37.8$\pm$1.0 | 58.7$\pm$1.0 | 81.8$\pm$0.7 | 92.1$\pm$0.1 | 94.3$\pm$0.2 | 95.3$\pm$0.1 | 95.7$\pm$0.2 | 96.0$\pm$0.2 | 96.0$\pm$0.1 |
| CAL | 22.2$\pm$2.4 | 37.1$\pm$2.5 | 45.5$\pm$1.4 | 66.2$\pm$0.6 | 74.2$\pm$0.9 | 82.7$\pm$0.5 | 86.3$\pm$0.8 | 89.2$\pm$0.6 | 91.0$\pm$0.3 | 92.4$\pm$0.2 | 95.3$\pm$0.1 | 96.0$\pm$0.1 |
| DeepFool | 18.9$\pm$1.9 | 29.2$\pm$1.0 | 35.0$\pm$1.8 | 57.4$\pm$3.2 | 74.0$\pm$1.8 | 87.5$\pm$0.4 | 92.0$\pm$0.3 | 93.5$\pm$0.2 | 94.6$\pm$0.1 | 95.2$\pm$0.1 | 96.0$\pm$0.1 | 96.0$\pm$0.1 |
| Craig | 21.9$\pm$1.5 | 30.0$\pm$0.8 | 38.1$\pm$1.4 | 60.1$\pm$1.4 | 69.2$\pm$0.8 | 86.5$\pm$0.4 | 91.4$\pm$0.2 | 93.8$\pm$0.1 | 94.8$\pm$0.1 | 95.4$\pm$0.1 | 95.9$\pm$0.1 | 96.0$\pm$0.1 |
| GradMatch | 20.1$\pm$1.8 | 27.8$\pm$1.2 | 31.4$\pm$2.1 | 54.4$\pm$2.6 | 70.5$\pm$2.0 | 84.6$\pm$0.7 | 90.7$\pm$0.5 | 93.3$\pm$0.3 | 94.7$\pm$0.4 | 95.3$\pm$0.1 | 96.0$\pm$0.1 | 96.0$\pm$0.1 |
| Glister | 20.1$\pm$1.2 | 28.0$\pm$1.4 | 31.6$\pm$0.8 | 49.3$\pm$3.3 | 67.9$\pm$1.7 | 83.6$\pm$0.8 | 90.5$\pm$0.9 | 93.6$\pm$0.1 | 94.6$\pm$0.1 | 95.2$\pm$0.2 | 96.0$\pm$0.2 | 96.0$\pm$0.1 |
| HyperCore (ours) | 26.9$\pm$0.7 | 39.3$\pm$0.2 | 45.9$\pm$0.2 | 67.4$\pm$0.3 | 79.1$\pm$0.3 | 86.5$\pm$0.2 | 90.3$\pm$0.1 | 92.2$\pm$0.2 | 93.2$\pm$0.2 | 94.0$\pm$0.2 | 95.6$\pm$0.1 | 96.0$\pm$0.1 |

**Table 9: Fixed coreset selection performance of five randomly initialized VGG-16 under label noise on CIFAR-10, where 10% of the training labels are randomly corrupted by assigning them to incorrect classes. Bold entries indicate the highest performance at each data fraction.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 12.5$\pm$0.7 | 11.7$\pm$1.6 | 11.8$\pm$1.4 | 10.5$\pm$0.8 | 14.6$\pm$2.8 | 33.7$\pm$4.3 | 57.2$\pm$2.0 | 71.9$\pm$1.8 | 81.1$\pm$0.7 | 83.3$\pm$0.4 | 88.6$\pm$0.3 | 88.8$\pm$0.2 |
|  | +0.9 | -1.0 | -4.2 | -43.6 | -51.4 | -37.9 | -21.1 | -9.0 | -4.6 | -5.0 | -4.5 | -5.5 |
| k-Center Greedy | 12.0$\pm$1.4 | 12.2$\pm$1.3 | 12.6$\pm$1.2 | 17.0$\pm$4.7 | 40.6$\pm$3.8 | 68.2$\pm$1.7 | 76.0$\pm$0.6 | 79.6$\pm$0.5 | 82.6$\pm$0.3 | 85.1$\pm$0.3 | 88.3$\pm$0.2 | 88.8$\pm$0.2 |
|  | -0.4 | -2.2 | -2.1 | -21.6 | -35.0 | -17.4 | -13.1 | -11.3 | -9.2 | -7.6 | -5.8 | -5.5 |
| Forgetting | 12.6$\pm$0.9 | 14.4$\pm$1.2 | 21.5$\pm$5.0 | 55.8$\pm$1.0 | 72.5$\pm$2.0 | 83.0$\pm$0.8 | 86.0$\pm$0.8 | 87.8$\pm$0.2 | 88.1$\pm$0.4 | 88.2$\pm$0.3 | 88.6$\pm$0.1 | 88.8$\pm$0.2 |
|  | +0.1 | +0.6 | -4.1 | -0.1 | -2.8 | -3.1 | -2.9 | -3.5 | -3.8 | -4.6 | -5.4 | -5.5 |
| GraNd | 12.8$\pm$1.2 | 11.4$\pm$0.5 | 12.1$\pm$1.1 | 10.9$\pm$0.4 | 11.1$\pm$0.4 | 24.9$\pm$0.8 | 45.4$\pm$0.6 | 64.1$\pm$2.2 | 76.2$\pm$0.9 | 82.9$\pm$0.6 | 88.8$\pm$0.2 | 88.8$\pm$0.2 |
|  | +1.1 | -1.1 | -4.5 | -17.3 | -31.3 | -48.5 | -41.1 | -27.1 | -17.0 | -10.8 | -5.3 | -5.5 |
| CAL | 13.9$\pm$1.5 | 13.2$\pm$1.3 | 18.0$\pm$5.3 | 57.6$\pm$2.0 | 69.6$\pm$2.1 | 76.1$\pm$0.7 | 81.8$\pm$0.5 | 84.1$\pm$0.5 | 86.3$\pm$0.4 | 88.0$\pm$0.3 | 89.6$\pm$0.2 | 88.8$\pm$0.2 |
|  | +0.9 | -2.6 | -1.2 | -5.4 | -2.6 | -1.9 | -0.8 | -0.9 | -1.3 | -1.6 | -3.4 | -5.5 |
| DeepFool | 13.1$\pm$2.0 | 13.2$\pm$1.0 | 17.4$\pm$2.2 | 36.1$\pm$4.3 | 62.5$\pm$2.9 | 74.5$\pm$0.7 | 82.0$\pm$0.4 | 84.8$\pm$0.3 | 86.5$\pm$0.4 | 87.1$\pm$0.2 | 88.6$\pm$0.1 | 88.8$\pm$0.2 |
|  | +1.5 | -1.4 | +1.8 | -11.2 | -9.2 | -9.6 | -8.0 | -6.8 | -6.0 | -6.0 | -5.5 | -5.5 |
| Craig | 13.7$\pm$2.0 | 11.5$\pm$0.3 | 13.0$\pm$1.5 | 22.8$\pm$2.1 | 36.0$\pm$3.3 | 55.2$\pm$4.6 | 67.3$\pm$1.8 | 78.5$\pm$1.5 | 82.2$\pm$0.5 | 85.3$\pm$0.4 | 88.7$\pm$0.3 | 88.8$\pm$0.2 |
|  | +0.3 | -2.7 | -0.7 | -23.8 | -31.3 | -27.1 | -21.4 | -12.9 | -10.5 | -8.1 | -5.5 | -5.5 |
| GradMatch | 12.1$\pm$0.8 | 12.7$\pm$1.4 | 13.7$\pm$2.9 | 31.6$\pm$1.9 | 41.5$\pm$2.6 | 48.3$\pm$2.3 | 61.4$\pm$1.4 | 72.7$\pm$1.1 | 79.0$\pm$1.5 | 84.6$\pm$0.4 | 88.8$\pm$0.3 | 88.8$\pm$0.2 |
|  | -1.3 | -0.4 | +0.7 | -13.0 | -12.4 | -29.6 | -25.9 | -17.5 | -13.6 | -8.8 | -5.5 | -5.5 |
| Glister | 12.7$\pm$1.4 | 13.0$\pm$0.7 | 13.7$\pm$2.4 | 30.4$\pm$1.0 | 42.6$\pm$3.2 | 49.3$\pm$3.8 | 61.5$\pm$1.0 | 71.1$\pm$2.3 | 79.3$\pm$1.5 | 84.5$\pm$0.4 | 88.7$\pm$0.3 | 88.8$\pm$0.2 |
|  | -2.9 | -0.9 | -5.2 | -13.6 | -11.0 | -28.9 | -24.9 | -18.6 | -13.2 | -8.8 | -5.5 | -5.5 |
| HyperCore (ours) | 14.0$\pm$0.6 | 21.9$\pm$3.4 | 26.0$\pm$6.1 | 62.6$\pm$1.9 | 70.7$\pm$3.4 | 83.4$\pm$0.3 | 87.3$\pm$0.3 | 89.5$\pm$0.3 | 91.0$\pm$0.3 | 92.1$\pm$0.1 | 92.2$\pm$0.1 | 88.8$\pm$0.2 |
|  | -1.8 | 5.3 | 1.7 | 2.9 | -2.6 | -0.1 | 0.0 | 0.2 | 0.4 | 0.6 | -1.5 | -5.5 |

**Table 10: Fixed coreset selection performance of five randomly initialized ResNet-50 under label noise on CIFAR-10, where 10% of the training labels are randomly corrupted by assigning them to incorrect classes. Bold entries indicate the highest performance at each data fraction.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 11.4$\pm$1.1 | 12.4$\pm$1.9 | 10.6$\pm$0.7 | 11.4$\pm$0.8 | 11.3$\pm$0.7 | 36.5$\pm$3.8 | 56.6$\pm$2.1 | 71.5$\pm$2.6 | 79.3$\pm$1.0 | 84.5$\pm$0.8 | 89.7$\pm$0.3 | 89.8$\pm$0.4 |
|  | -4.8 | -10.6 | -18.9 | -34.9 | -47.2 | -36.7 | -23.4 | -12.6 | -7.1 | -5.1 | -4.9 | -5.6 |
| k-Center Greedy | 11.0$\pm$0.6 | 14.0$\pm$0.9 | 14.8$\pm$1.8 | 27.7$\pm$1.4 | 42.5$\pm$3.1 | 64.6$\pm$2.9 | 75.7$\pm$1.7 | 81.6$\pm$0.7 | 84.6$\pm$0.6 | 86.1$\pm$0.4 | 89.4$\pm$0.4 | 89.8$\pm$0.4 |
|  | -4.4 | -4.8 | -10.3 | -25.4 | -26.3 | -22.0 | -14.4 | -10.1 | -8.9 | -8.1 | -5.9 | -5.6 |
| Forgetting | 17.6$\pm$3.3 | 26.0$\pm$4.1 | 32.6$\pm$2.1 | 53.8$\pm$2.5 | 68.9$\pm$3.2 | 82.4$\pm$0.8 | 87.1$\pm$0.7 | 88.5$\pm$0.3 | 89.9$\pm$0.3 | 90.2$\pm$0.3 | 90.1$\pm$0.4 | 89.8$\pm$0.4 |
|  | +0.6 | -0.9 | -1.4 | +4.3 | +1.4 | -3.8 | -2.8 | -3.5 | -2.9 | -3.4 | -5.2 | -5.6 |
| GraNd | 13.5$\pm$1.3 | 13.5$\pm$0.8 | 12.8$\pm$1.7 | 11.3$\pm$1.1 | 12.0$\pm$1.2 | 22.4$\pm$1.3 | 44.8$\pm$1.5 | 62.9$\pm$1.4 | 77.1$\pm$0.2 | 84.6$\pm$1.0 | 89.5$\pm$0.4 | 89.8$\pm$0.4 |
|  | -1.6 | -3.6 | -6.9 | -12.3 | -22.9 | -48.0 | -40.7 | -29.2 | -17.1 | -10.3 | -6.0 | -5.6 |
| CAL | 16.9$\pm$4.1 | 27.0$\pm$2.2 | 34.5$\pm$2.4 | 52.2$\pm$2.9 | 65.2$\pm$0.3 | 76.8$\pm$1.2 | 81.7$\pm$1.0 | 84.5$\pm$0.5 | 86.9$\pm$0.4 | 88.5$\pm$0.5 | 90.5$\pm$0.1 | 89.8$\pm$0.4 |
|  | -3.0 | -2.5 | -2.4 | -4.1 | -1.0 | -0.8 | -1.3 | -1.1 | -1.3 | -2.0 | -3.7 | -5.6 |
| DeepFool | 16.0$\pm$3.4 | 19.8$\pm$1.2 | 24.5$\pm$1.6 | 38.0$\pm$1.2 | 52.4$\pm$4.1 | 75.0$\pm$2.6 | 84.1$\pm$0.5 | 86.2$\pm$0.5 | 87.8$\pm$0.5 | 88.7$\pm$0.4 | 90.1$\pm$0.2 | 89.8$\pm$0.4 |
|  | +1.6 | 0.0 | -1.9 | -4.7 | -13.4 | -11.3 | -7.0 | -6.7 | -6.1 | -5.8 | -5.2 | -5.6 |
| Craig | 17.0$\pm$1.0 | 15.5$\pm$2.3 | 20.6$\pm$2.1 | 31.9$\pm$0.9 | 33.8$\pm$3.0 | 46.7$\pm$2.4 | 65.9$\pm$3.5 | 75.6$\pm$2.1 | 83.3$\pm$0.3 | 85.6$\pm$0.3 | 89.7$\pm$0.2 | 89.8$\pm$0.4 |
|  | -0.1 | -6.0 | -7.5 | -10.1 | -21.3 | -34.0 | -24.0 | -16.8 | -10.2 | -9.1 | -5.7 | -5.6 |
| GradMatch | 13.9$\pm$1.7 | 17.7$\pm$1.8 | 24.1$\pm$2.0 | 29.2$\pm$0.5 | 35.7$\pm$2.7 | 46.9$\pm$6.1 | 59.3$\pm$2.6 | 73.1$\pm$1.1 | 80.9$\pm$1.0 | 84.4$\pm$0.3 | 89.6$\pm$0.4 | 89.8$\pm$0.4 |
|  | -0.4 | -4.0 | -4.1 | -8.3 | -14.8 | -27.0 | -28.4 | -19.1 | -12.9 | -10.3 | -5.6 | -5.6 |
| Glister | 13.6$\pm$0.9 | 18.9$\pm$1.9 | 22.0$\pm$2.1 | 28.6$\pm$2.0 | 34.5$\pm$2.1 | 47.4$\pm$4.1 | 58.6$\pm$0.9 | 71.5$\pm$2.6 | 80.3$\pm$0.6 | 84.6$\pm$1.2 | 89.2$\pm$0.5 | 89.8$\pm$0.4 |
|  | -1.3 | -1.6 | -2.2 | -8.4 | -15.6 | -32.2 | -30.0 | -20.5 | -13.2 | -9.9 | -6.1 | -5.6 |
| HyperCore (ours) | 20.3$\pm$1.4 | 28.8$\pm$1.7 | 36.4$\pm$1.6 | 56.1$\pm$2.4 | 68.5$\pm$0.7 | 83.5$\pm$0.7 | 87.8$\pm$0.3 | 90.7$\pm$0.3 | 92.1$\pm$0.1 | 93.1$\pm$0.2 | 93.2$\pm$0.3 | 89.8$\pm$0.4 |
|  | -0.1 | -1.2 | -0.5 | -0.2 | -0.8 | +1.1 | -0.2 | +0.3 | +0.5 | +0.1 | -1.6 | -5.6 |

**Table 11: Fixed coreset selection performance of five randomly initialized InceptionNetV3 under label noise on CIFAR-10, where 10% of the training labels are randomly corrupted by assigning them to incorrect classes. Bold entries indicate the highest performance at each data fraction.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 12.4$\pm$1.2 | 12.1$\pm$0.6 | 10.8$\pm$0.7 | 10.1$\pm$0.2 | 10.6$\pm$0.6 | 34.5$\pm$4.0 | 54.5$\pm$2.3 | 69.3$\pm$1.8 | 77.9$\pm$1.0 | 83.8$\pm$1.1 | 90.9$\pm$0.4 | 90.9$\pm$0.4 |
|  | -2.2 | -9.7 | -17.7 | -32.9 | -50.5 | -39.5 | -26.1 | -15.4 | -9.9 | -6.5 | -3.9 | -4.6 |
| k-Center Greedy | 12.7$\pm$1.8 | 13.6$\pm$0.8 | 14.2$\pm$0.8 | 27.6$\pm$1.9 | 45.7$\pm$3.8 | 66.1$\pm$1.4 | 73.1$\pm$1.0 | 81.0$\pm$0.5 | 83.7$\pm$0.8 | 86.4$\pm$0.7 | 90.2$\pm$0.3 | 90.9$\pm$0.4 |
|  | -2.4 | -8.5 | -12.4 | -23.6 | -27.1 | -19.7 | -16.6 | -11.2 | -9.9 | -7.9 | -5.2 | -4.6 |
| Forgetting | 17.1$\pm$2.4 | 28.6$\pm$0.7 | 33.1$\pm$1.7 | 50.0$\pm$1.0 | 71.2$\pm$0.8 | 83.1$\pm$1.2 | 87.3$\pm$0.7 | 89.4$\pm$0.1 | 90.2$\pm$0.3 | 90.6$\pm$0.3 | 90.8$\pm$0.3 | 90.9$\pm$0.4 |
|  | -1.4 | 0.0 | +0.9 | -1.0 | +1.9 | -2.3 | -3.1 | -2.9 | -3.1 | -3.7 | -4.6 | -4.6 |
| GraNd | 12.9$\pm$1.0 | 12.5$\pm$1.0 | 11.2$\pm$0.6 | 10.9$\pm$0.5 | 11.3$\pm$0.3 | 25.0$\pm$1.0 | 44.9$\pm$1.0 | 61.8$\pm$1.9 | 77.0$\pm$0.9 | 83.3$\pm$0.9 | 90.6$\pm$0.3 | 90.9$\pm$0.4 |
|  | -1.9 | -4.9 | -8.9 | -16.2 | -26.8 | -45.0 | -41.9 | -31.0 | -17.4 | -11.6 | -5.0 | -4.6 |
| CAL | 15.9$\pm$1.5 | 26.8$\pm$1.9 | 30.9$\pm$1.6 | 56.3$\pm$3.5 | 67.8$\pm$2.7 | 77.0$\pm$0.7 | 82.3$\pm$0.5 | 84.5$\pm$0.5 | 87.1$\pm$0.3 | 89.1$\pm$0.4 | 91.5$\pm$0.3 | 90.9$\pm$0.4 |
|  | -2.1 | -4.8 | -5.4 | -3.9 | -2.7 | -1.7 | -0.7 | -1.8 | -1.8 | -1.2 | -2.8 | -4.6 |
| DeepFool | 17.0$\pm$1.0 | 19.9$\pm$1.5 | 24.7$\pm$2.2 | 38.1$\pm$1.0 | 54.7$\pm$5.2 | 76.1$\pm$0.7 | 83.2$\pm$0.6 | 87.2$\pm$0.5 | 89.1$\pm$0.3 | 89.9$\pm$0.4 | 91.3$\pm$0.2 | 90.9$\pm$0.4 |
|  | +2.4 | -2.4 | -2.1 | -9.7 | -15.5 | -7.5 | -7.0 | -5.5 | -5.0 | -4.7 | -4.2 | -4.6 |
| Craig | 17.8$\pm$2.5 | 21.0$\pm$1.1 | 23.4$\pm$1.4 | 29.8$\pm$2.0 | 36.2$\pm$3.2 | 45.0$\pm$3.5 | 58.6$\pm$2.2 | 71.4$\pm$1.1 | 78.5$\pm$1.1 | 84.3$\pm$0.8 | 90.7$\pm$0.4 | 90.9$\pm$0.4 |
|  | +1.3 | -3.7 | -4.4 | -14.7 | -22.6 | -34.4 | -29.4 | -20.9 | -15.5 | -10.6 | -4.9 | -4.6 |
| GradMatch | 14.4$\pm$1.6 | 21.5$\pm$1.0 | 22.2$\pm$1.0 | 29.3$\pm$2.4 | 38.3$\pm$3.3 | 42.7$\pm$3.3 | 57.4$\pm$0.9 | 68.3$\pm$0.8 | 75.3$\pm$2.0 | 83.1$\pm$0.8 | 90.8$\pm$0.2 | 90.9$\pm$0.4 |
|  | -2.1 | -2.7 | -6.9 | -10.8 | -14.7 | -34.2 | -29.7 | -23.5 | -19.0 | -11.8 | -5.0 | -4.6 |
| Glister | 15.0$\pm$1.4 | 20.2$\pm$1.4 | 22.9$\pm$0.7 | 32.3$\pm$1.7 | 38.3$\pm$3.4 | 43.1$\pm$2.3 | 54.8$\pm$3.6 | 67.4$\pm$2.9 | 77.0$\pm$1.0 | 82.9$\pm$0.6 | 90.9$\pm$0.2 | 90.9$\pm$0.4 |
|  | +0.2 | -2.7 | -6.3 | -9.0 | -14.0 | -34.4 | -33.0 | -24.4 | -17.1 | -12.0 | -4.7 | -4.6 |
| HyperCore (ours) | 20.1$\pm$1.8 | 31.2$\pm$0.2 | 37.4$\pm$0.7 | 57.3$\pm$1.8 | 72.8$\pm$1.6 | 83.5$\pm$0.3 | 87.6$\pm$0.3 | 90.4$\pm$0.2 | 92.3$\pm$0.3 | 93.5$\pm$0.3 | 93.6$\pm$0.3 | 90.9$\pm$0.4 |
|  | -0.7 | -1.3 | -0.5 | 0.0 | +1.8 | +0.2 | 0.0 | +0.3 | -0.1 | +0.1 | -1.6 | -4.6 |

**Table 12: Fixed coreset selection performance of five randomly initialized WRN-16-8 under label noise on CIFAR-10, where 10% of the training labels are randomly corrupted by assigning them to incorrect classes. Bold entries indicate the highest performance at each data fraction.**
| Fraction $(1-\alpha)$ | 0.1% | 0.5% | 1% | 5% | 10% | 20% | 30% | 40% | 50% | 60% | 90% | 100% |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Herding | 11.3$\pm$0.8 | 10.7$\pm$1.6 | 8.9$\pm$1.7 | 8.6$\pm$1.4 | 12.9$\pm$2.5 | 32.8$\pm$1.7 | 56.0$\pm$1.5 | 71.7$\pm$1.6 | 79.5$\pm$1.0 | 85.1$\pm$0.8 | 91.3$\pm$0.3 | 91.6$\pm$0.2 |
|  | -12.5 | -20.5 | -30.7 | -51.6 | -53.9 | -44.5 | -25.9 | -14.7 | -9.6 | -7.2 | -3.9 | -4.4 |
| k-Center Greedy | 12.6$\pm$1.0 | 15.0$\pm$0.9 | 18.3$\pm$0.7 | 31.4$\pm$1.6 | 47.8$\pm$3.9 | 68.8$\pm$2.3 | 78.3$\pm$0.9 | 82.4$\pm$0.6 | 85.5$\pm$0.3 | 87.8$\pm$0.2 | 91.1$\pm$0.2 | 91.6$\pm$0.2 |
|  | -6.6 | -12.7 | -16.3 | -36.5 | -33.8 | -20.6 | -13.8 | -11.2 | -8.8 | -7.2 | -4.8 | -4.4 |
| Forgetting | 21.7$\pm$2.2 | 31.7$\pm$1.6 | 36.0$\pm$1.5 | 62.0$\pm$0.7 | 77.4$\pm$0.7 | 88.1$\pm$0.2 | 91.0$\pm$0.1 | 91.5$\pm$0.3 | 91.8$\pm$0.3 | 91.7$\pm$0.2 | 91.5$\pm$0.3 | 91.6$\pm$0.2 |
|  | +0.2 | +1.6 | -0.4 | +3.5 | +2.6 | -1.7 | -2.4 | -3.0 | -3.2 | -3.7 | -4.4 | -4.4 |
| GraNd | 12.7$\pm$1.7 | 10.8$\pm$0.5 | 8.7$\pm$0.9 | 8.6$\pm$0.7 | 10.5$\pm$0.8 | 29.2$\pm$1.1 | 50.5$\pm$1.2 | 67.1$\pm$0.7 | 79.5$\pm$0.6 | 84.9$\pm$0.4 | 91.4$\pm$0.2 | 91.6$\pm$0.2 |
|  | -2.3 | -8.8 | -12.8 | -29.1 | -48.2 | -52.5 | -41.6 | -27.2 | -15.8 | -10.7 | -4.6 | -4.4 |
| CAL | 24.9$\pm$2.0 | 36.2$\pm$1.3 | 44.2$\pm$2.2 | 67.4$\pm$1.3 | 73.9$\pm$0.9 | 82.5$\pm$0.5 | 85.5$\pm$0.6 | 88.7$\pm$0.3 | 90.6$\pm$0.3 | 91.6$\pm$0.3 | 92.8$\pm$0.2 | 91.6$\pm$0.2 |
|  | +2.6 | -1.0 | -1.3 | +1.2 | -0.2 | -0.2 | -0.7 | -0.5 | -0.4 | -0.9 | -2.6 | -4.4 |
| DeepFool | 17.2$\pm$1.7 | 22.2$\pm$1.4 | 27.3$\pm$2.3 | 43.2$\pm$1.7 | 54.7$\pm$4.1 | 70.4$\pm$1.7 | 80.4$\pm$1.5 | 84.7$\pm$0.6 | 87.7$\pm$0.4 | 89.3$\pm$0.2 | 91.3$\pm$0.2 | 91.6$\pm$0.2 |
|  | -1.7 | -7.0 | -7.7 | -14.2 | -19.4 | -17.1 | -11.6 | -8.8 | -6.9 | -5.9 | -4.7 | -4.4 |
| Craig | 20.6$\pm$0.9 | 24.6$\pm$2.3 | 27.0$\pm$2.5 | 28.5$\pm$1.8 | 33.2$\pm$1.8 | 46.8$\pm$4.0 | 62.5$\pm$1.8 | 74.6$\pm$0.6 | 82.2$\pm$0.5 | 86.5$\pm$1.0 | 91.2$\pm$0.2 | 91.6$\pm$0.2 |
|  | -1.3 | -5.4 | -11.1 | -31.6 | -36.0 | -39.6 | -28.8 | -19.2 | -12.5 | -8.9 | -4.7 | -4.4 |
| GradMatch | 16.4$\pm$1.7 | 23.6$\pm$1.8 | 24.9$\pm$2.6 | 29.9$\pm$2.6 | 36.9$\pm$5.3 | 49.7$\pm$3.7 | 63.9$\pm$4.2 | 75.7$\pm$2.0 | 80.9$\pm$0.9 | 86.1$\pm$0.3 | 91.1$\pm$0.1 | 91.6$\pm$0.2 |
|  | -3.8 | -4.2 | -6.6 | -24.5 | -33.5 | -34.9 | -26.8 | -17.6 | -13.8 | -9.3 | -4.8 | -4.4 |
| Glister | 17.1$\pm$0.7 | 24.8$\pm$1.4 | 28.8$\pm$2.6 | 28.3$\pm$2.9 | 39.3$\pm$2.7 | 47.8$\pm$3.8 | 63.4$\pm$5.0 | 74.0$\pm$1.2 | 81.5$\pm$0.4 | 86.5$\pm$0.2 | 91.0$\pm$0.1 | 91.6$\pm$0.2 |
|  | -3.0 | -3.3 | -2.8 | -21.0 | -28.5 | -35.8 | -27.1 | -19.5 | -13.2 | -8.7 | -4.9 | -4.4 |
| HyperCore (ours) | 22.7$\pm$0.8 | 39.6$\pm$0.8 | 47.4$\pm$0.5 | 70.2$\pm$0.7 | 79.2$\pm$0.5 | 86.7$\pm$0.2 | 90.0$\pm$0.3 | 92.2$\pm$0.1 | 93.4$\pm$0.1 | 94.2$\pm$0.2 | 94.1$\pm$0.1 | 91.6$\pm$0.2 |
|  | -4.3 | +0.4 | +1.5 | +2.8 | +0.1 | +0.2 | -0.2 | 0.0 | +0.2 | +0.2 | -1.5 | -4.4 |

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: See Weaknesses
soundness: 2
strengths: 1) The paper is well written and is easy to follow. The problem of subset selection for training ML models is a very important problem and of interest to community. 
2) The algorithm is intuitive, and its parallelizability makes it very efficient
3) Empirically the algorithm is compared with a bunch of baselines in terms of time and scalability both in noisy and noise free settings and the results are encouraging.
summary: The paper provides a subset selection algorithm for data especially when there is noise in the data in form of labelling errors. Their algorithm learns a hypersphere for every class separately and uses the distance of every point to the center of the hypersphere to decide if the sample should be in the subset. The distance is compared with an adaptive threshold determined automatically using Youden’s J statistic. Extensive empirical evaluations show the effectiveness and scalability of their method especially for noisy environments.
weaknesses: I appreciate that the authors have listed out most limitations of their method. However, one key limitation is that the method looks more heuristic in nature. some of the baselines while performing slightly weaker empirically do come with guarantees about the quality of sampling. 

The experiments are compared with a variety of baselines, however, as pointed out in the point above, since this is less theoretically strong method compared to few other baselines, it would be useful to test on a couple of other datasets/ downstream tasks too. 

Question: Elaborate on the effect of using $\mathbf{c =0} $ as center. Does it change quality of hypersphere obtained or effect sample size other than making the method computationally efficient?

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: - Q1: Although the hypershere model seems useful as it is relatively straighforward to fit, one wonders whether an even slightly more complex model that can adapt to each dimension would be a better performer (although increase fitting complexity Eq 2).  In these high dimensions
- Q2: Want to confirm that $\mathbf{x}$ is always the raw input and not some embedded form of it.
- Q3: What is the actual comparative wall-clock time of training these small models?  What is the size of these models?
soundness: 3
strengths: - S1: The notion of adapting parameters per class (if you have the labels) is interesting and relevant.  And the use of classical signal detection theory ideas to adapt each threshold is clear.

- S2: Likewise explicitly capturing noise in the data and analyzing robustness to such noise is a nice addition to the coreset literature.

- S3: The method is relatively simple leading to what seem to be low computational costs, although no comparative wallclock time is given, the networks trained are "small."

- S4: The exposition is clear and the paper is understandable.
summary: HyperCore introduces a coreset selection method leveraging class-wise hypersphere models, which effectively identify and retain representative data samples under noisy conditions. Assuming the data are fully labels, hypercore trains per-class models and then adapts a per-class threshold with Youden’s J statistic.  It seems able to efficiently discards mislabeled or outlier points without manual parameter tuning. The approach is analyzed on CIFAR-10 and on ImageNet-1K.  A compelling result that shows robustness to noise in the data is alao included.
weaknesses: - W1: The basic assumption in the paper --- that this is a fully supervised problem with a label for all samples --- is barely tacitly stated (L078).  Yet, this problem setup differs from the original idea in Sener and Saverese 2017 which is an active learning setup where only a subset of the labels exist at any iteration and the goal is to find the next subset of unlabeled samples to label.  Although numerous papers in the coreset literature make this same questionable deviation from the original problem, it is highly questionable.  The motivation in the introduction does not use the motivation from Sener and Saverese (namely that labeling very large datasets is prohibilitively expensive and complicated), but rather shifts to the notion that it is possible to get better performance from a best core subset.  This shift renders the problem mostly irrelevant from an impact standpoint.  

- W2: Critical competitive references are not included -- e.g., Temporal Dual-Depth Scoring Zhang et al. CVPR 2024.  In fact, no comparative result in the tables exist for papers after 2021???  

- W3: Coreset selection is a challenging problem (with or without class labels).  Interestingly, reports (e.g., https://arxiv.org/pdf/2411.15349) have shown that uniform sampling is a very competitive baseline.  This paper has no such analysis against the competitive baseline.  EG. Looking at the prune rate of 50% for CIFAR10 in that paper and this paper (both using ResNet18 as the downstream detector), the uniform sampler outperforms this HyperCore (AND the uniform sampler uses no class labels).

- W4: The static pruning baseline is derived in Sec 3.2. But it is not included in any of the tables.
  

Other things
- L033 -- Missing Reference?

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 2
questions: - Please refer to the weaknesses section of this review.
soundness: 2
strengths: 1. Using a hypersphere is a straightforward concept and is a variation of Moderate [A] where samples lying in the median between class centres are picked as the coreset. The formulation depends upon the intuition that mislabelled samples lie farther from class centres in embedding space.
2. Expensive gradient and influence-based computations are avoided.
3. The use of adaptive threshold selection through Youden’s J statistics handles class imbalance.
summary: - The paper introduces coreset selection for dataset with annotation errors/noisy environments. 
- The proposed approach depends on lightweight hypersphere models learned per class, embedding in-class samples close a hypersphere center while segregating out of class samples based on their distance.
- In more details, the method trains lightweight hypersphere models that learn to separate in-class from out-of-class (including erroneous labeled samples). It’s a process of separating samples that appear atypical when measured against a given class distribution, which are treated as outliers. Measure the distance of each point to its class-specific hypersphere centre and obtain an interpretable conformity score.
- J statistics is used to decide the hypersphere’s decision boundary.
weaknesses: **1. Missing baselines:**

Several important coreset selection baselines are missing. All the methods compared are part of DeepCore which was released in 2022 and there have been some influential works published in the field of coreset selection in the past 3 years.
A) Moderate Coreset: A Universal Method of Data Selection for Real-world Data-efficient Deep Learning (ICLR 2023)
B) Robust Data Pruning under Label Noise via Maximizing Re-labeling Accuracy (NeurIPS, 2023)
C) Coverage-Centric Coreset Selection for High Pruning Rates (ICLR, 2023)
D) Data Pruning via Moving-one-Sample-out (NeurIPS 2023)
E) Noise-free Loss Gradients: A Surprisingly Effective Baseline for Coreset Selection (TMLR, 2025)

**2. Limited Experiments**

Results are provided only for CIFAR-10 and ImageNet-1K, while other standard datasets such as CIFAR-100 and Tiny ImageNet are not utilized. These would have provided insight into how the methodology behaves with varying number of classes.

**3. Timing comparison**

Timing comparison seems misleading, as all the other methods being compared , select coreset for the entire dataset, while the proposed method selects coreset in a class wise manner. Assumption that sufficient number of parallel cores and GPUs are available defeats the core intention of coreset selection i.e. efficient deep learning method in a compute-constrained environment.
Timing analysis is provided only for CIFAR-10 dataset. The scale up required for 1000 class dataset of ImageNet-1K would definitely be very time intensive.

**4. Performance of the method**

As reported in Table 1 and Table 2, the proposed method barely outperforms existing methods for only a few of the selection fraction. The trade-off of compute time due to the nature of per-class selection against very minimal performance gain (for a very few selected fractions) seems trivial.

**5. Dataset not unexplored**

As the paper claims that the method is suitable specifically in cases of datasets with label noises, it would have been helpful to understand the performance of the method on datasets such as CIFAR-100N and CIFAR-10N, which are datasets with realistic label noise due to human annotation error.

**6.  Lack of theoretical and empirical analysis of hyperspheres for coreset selection**

The paper does not provide any supporting proof towards hyperspheres being able to select representative samples as compared to samples with label noises. An analysis of percentage of clean samples and percentage of noisy samples being selected in the coreset would be helpful.

**7. Alternate thresholding criteria [MINOR]**

The paper does not explore (or atleast discuss) alternative thresholding criteria other than the J statistics. An ablation of alternate thresholding criteria would help in highlighting the impact of J statistics.
 
**8. Formatting issues [MINOR]**

There is a “?” mark in line 034, which seems to be a missing reference.

---

confidence: 4
contribution: 4
initial_rating: 4
presentation: 3
questions: 1. Could the authors provide additional experiments on CIFAR-100 or ImageNet-1K to demonstrate scalability and robustness under varying noise conditions?

2. Please clarify the baseline setup in Table 4 and define what "static pruning" performance refers to.
soundness: 1
strengths: 1. The motivation is clear. The paper correctly identifies the need for a method that is both noise-robust and adaptive, which is a critical gap in the current literature.

2. The use of hypersphere-based class embeddings combined with Youden's J statistic for adaptive thresholding is an elegant idea with solid theoretical grounding. It effectively reduces heuristic tuning.

3. The paper is generally clear and easy to follow, with well-organized technical sections.
summary: This paper introduces HyperCore, a robust and adaptive coreset selection framework designed explicitly for noisy environments. Unlike existing methods, HyperCore utilizes class-conditional embeddings derived by hypersphere models with adaptive pruning thresholds determined by Youden's J statistic, enabling automatic and noise-aware subset selection without extensive hyperparameter tuning. Experiments on CIFAR-10 with 10% label noise show the effectiveness of the HyperCore.
weaknesses: (Major)

The experimental evidence is not strong enough to substantiate the claims of robustness and generalization. Critical evaluations are missing:

1. No comparison with Random selection, which is a standard and strong baseline in DeepCore[Ref 1]).

2. While the paper clained that HyperCore raises the bar for robust coreset selection, setting new benchmarks for pruning accuracy and label noise tolerance, they only tested TyperCore on CIFAR-10 with 10% label noise. This paper should test: 

(1)  on additional datasets, such as CIFAR-100 and ImageNet-1K, 

(2) under different noise level, such as 20%, 40%, and 80% symmetric noise and 20% and 40% asymmetric noise, 

(3) against recent robust coreset selection methods, such as [Ref 2-4].

3. The static pruning algorithm (Sec. 3.2) is not analyzed. Its standalone performance should be reported.

4. Table 3: The meaning of red and green results is unclear.

5. Table 4: The corresponding experimental settings are not described.

(Minor)

1. Citation error: Lines 33-34 "... informative subset that preserves the performance of training on the full dataset (?Sorscher et al., 2022 ..." (misformatted reference)

2. Some figures could benefit from improved captions and descriptions to clarify the setup.

More comments that do not impact the score:

- The method's core principle is to select the most "central" or "prototypical" samples (those with the smallest norm). As discussed by the authors in the limitations, and as is common with centroid-based methods/prototype selection methods, this might systematically discard valuable, informative outliers or samples that lie on the decision boundary, potentially limiting samples diversity, and this also ignores the distribution diversity. This could be discussed more.


[Ref 1] Guo C, Zhao B, Bai Y. Deepcore: A comprehensive library for coreset selection in deep learning[C]//International Conference on Database and Expert Systems Applications. Cham: Springer International Publishing, 2022: 181-195.

[Ref 2] Xia X, Liu J, Yu J, et al. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning[C]//The Eleventh International Conference on Learning Representations. 2022.

[Ref 3] Xia X, Liu J, Zhang S, et al. Refined coreset selection: Towards minimal coreset size under model performance constraints[C]. ICML2024.

[Ref 4] Mohanty S, Anudeep C, Mopuri K R. Noise-free Loss Gradients: A Surprisingly Effective Baseline for Coreset Selection[J]. Transactions on Machine Learning Research, 2025.