DRoP: Distributionally Robust Data Pruning
Authors: Artem M Vysogorets
Kartik Ahuja
Julia Kempe
Venue: ICLR 2025 Spotlight (2025)
Keywords: Data Pruning
Classification Bias
Robustness
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We find that existing data pruning algorithms exacerbate classification bias and propose a simple robustness-aware random pruning protocol to address this deficiency.

Abstract: In the era of exceptionally data-hungry models, careful selection of the training data is essential to mitigate the extensive costs of deep learning. Data pruning offers a solution by removing redundant or uninformative samples from the dataset, which yields faster convergence and improved neural scaling laws. However, little is known about its impact on classification bias of the trained models. We conduct the first systematic study of this effect and reveal that existing data pruning algorithms can produce highly biased classifiers. We present theoretical analysis of the classification risk in a mixture of Gaussians to argue that choosing appropriate class pruning ratios, coupled with random pruning within classes has potential to improve worst-class performance. We thus propose DRoP, a distributionally robust approach to pruning and empirically demonstrate its performance on standard computer vision benchmarks. In sharp contrast to existing algorithms, our proposed method continues improving distributional robustness at a tolerable drop of average performance as we prune more from the datasets.

Published as a conference paper at ICLR 2025
DROP: DISTRIBUTIONALII ROBUST DATA PRUNING
Artem Vysogorets
Kartik Ahuja Meta FAIR
Julia Kempe
Data Science Platform Rockefeller University
New York University
Meta FAIR
mmv4580nyu. edu
ABSTRACT
In the era of exceptionally data-hungry models, careful selection of the training data is essential to mitigate the extensive costs of deep learning. Data pruning offers a solution by removing redundant or uninformative samples from the dataset, which yields faster convergence and improved neural scaling laws. However, little is known about its impact on classification bias of the trained models. We conduct the first systematic study of this effect and reveal that existing data pruning algorithms can produce highly biased classifiers. We present theoretical analysis the classification risk in mixture of Gaussians argue that choosing appropriate class pruning ratios, coupled with random pruning within classes has potential to improve worst-class performance. We thus propose DRoP, distributionally robust approach to pruning and empirically demonstrate its performance on standard computer vision benchmarks. In sharp contrast to existing algorithms, proposed method continues improving distributional robustness at a tolerable drop of average performance we prune more from the datasets.
INTRODUCTION
The ever-increasing state-of-the-art performance of deep learning models requires exponentially larger volumes of training data according neural scaling laws (Hestness et al. 2017 Kaplan et al.) 2020 Roseenelld et al. 2220;; Gordon et al.) 2021). However, not all collected data is equally important for learning as it contains noisy, repetitive, or uninformative samples. recent research thread on data pruning is concerned with removing those unnecessary data, resulting in improved convergence speed, scaling, and resource efficiency (Toneva et al., 2019 Paul al. 2021 He et al.. 2023). These methods design scoring mechanisms to assess the utility of each sample, often measured by its difficulty or uncertainty as approximated during a preliminary training round, that guides pruning. Sorscher al. (2022) report that selecting high-quality data using these techniques can trace Pareto optimal frontier, beating the notorious power scaling laws; Kolossov et al. (2024) demonstrate that data selection can improve training.
recent study by (Pote et al. 2023) hints that data pruning may mitigate distributional bias in trained models, which is well-established issue in Al systems concerning the performance disparity across classes (Dwork et al.) 2012) Hardt et af.; (016b) or protected, minority groups of the population (e.g., race or gender). In Section 3 we conduct the first systematic evaluation of various pruning algorithms with respect to classification bias-a phenomenon characterized by highly disparate model performance across different classes-on variety of benchmarks and conclude otherwise. For example, we find that Dynamic Uncertainty He et al. (2023) achieves superior average test performance with VGG-19 on CIFAR-100 and ResNet-50 on ImageNet but fails miserably in terms of worst-class accuracy. Thus, we argue that it is imperative to benchmark pruning algorithms using a more comprehensive suite of metrics that reflect classification bias, and to develop solutions that address distributional robustness directly.
To understand the fundamental principles that exacerbate classification bias for various pruning methods, we present theoretical analysis of linear decision rules for mixture of two isotropic Gaussians in Section It also illustrates how simple random subsampling with difficulty-awan class-wise pruning guotas may yield better worst-class performance compared to existing, finer pruning algorithms that operate on a sample level. Based on these observations, we design a first teauustness-wware data pruning protocol, coined Distributionally Robust Pruning (DRoP). It selects

---

Published as a conference paper at ICLR 2025
target class proportions based on the corresponding class-wise enror rates computed hold-out validation set after a preliminary training round on the full dataset. While these target quotas already improve robustness when combined with existing pruning algorithms, they particularly shine when applied together with random pruning in accordance with our theoretical analysis. Pruning with DRoP substantially reduces the classification bias of models even compared to training the full dataset while offering enhanced data efficiency. We further verified the effectiveness of our method for initially imbalanced datasets and in the context of group robustness.
In hindsight, DRoP carries some analogies with distributionally robust optimization methods that address class/group bias, such as subsampling (Barua et al.) 2012] Cui et al.) 2019 Tan et al. 2020
or cost-sensitive learning that constructs error- error-based based importance scores to reweigh the training loss.
emphasizing data from the more di difficult or minority classes (Sinha et al. 2022 Sagawa et al. 2020
Liu et al. 2021 Idrissi et al.) 2022 Wang et al. [023] Lukasik et al. [022 Chen et al. 2017). DRoP
emulates this behavior with data pruning, retaining more samples from the more difficult classes. We
discuss these similarities and compare DRoP to prototypical cost-sensitive method, CDB-W (Sinha
ct al. 2022). on a variety of dataset types to further illustrate the effectiveness of our method.
The summary of our contributions and the structure of the remainder of the paper are as follows.
using standard computer vision testbed, we conduct the first comprehensive evaluation of existing data pruning algorithms through the lens of classification bias for a variety of datasets and architectures;
In Section 3
In Section 4
we provide our theoretical analysis in the Gaussian mixture model to illustrate increased bias of current pruning methods, show how to optimize for worst-class risk and thus motivate our proposed solution;
In Section 5 we propose a random pruning procedure with error-based class ratios coined DRoP, and verify its effectiveness in drastically reducing the classification bias. We also provide ablations to illustrate the strength of DRoP as a pruming method, which in some range is even able to improve over full-dataset cost-sensitive learning.
2 RELATED WORK
In the era where the training corpora of contemporary models are of web-scale size, improving data efficiency has become the focus of practitioners and researchers alike. The corresponding literature is exceptionally rich, with a few fruitful and relevant research threads. Dataset distillation replaces the original samples with synthetically generated ones that bear compressed, albeit not as much interpretable, training signal (Sucholutsky & Schonlau 2021 Cazenavette et al., [202] Such jet al., 2020; Zhao & Bilen 2023 Nguyen et al. [202 Feng et al. 2024). CoreSel methods select representative samples that jointly capture the entire data manifold (Sener & Savarese) 2018 Guo
cct al.) 2022 Zheng et al. 22023] Agarwal et al. 2020 Mirzasoleiman et al. | [0l]]22220/]) Welling 2009 | they
yield weak generalization guarantees for non-convex problems and are not too effective in practice, especially on larger datasets (Feldman 2020: Paul et al. 2021). p.. Active learning iteratively selects an informative subset of a larger pool of unlabeled data for annotation, which is ultimately used for supervised learning (Tharwat & Schenck, 2023 Ren et al. 2021 Beluch et al.) [0018] Kirsch et al.) 2019). Subsampling deletes instances of certain groups or classes when datasets are imbalanced, aiming to reduce bias and improve robustness of the downstream classifiers (Chawla et al. 2002
Barua et al. 2012 Chaudhuri al. 2023).
Data Pruning. More recently, data pruning emerged as a new research direction that simply removes parts of the dataset while maintaining strong model performance. In contrast to previous techniques, data pruning selects a subset of the original, fully labeled, and not necessarily imbalanced dataset, all while enjoying strong results in deep learning applications. Data pruning algorithms use the entire dataset = (Xi, 9/ J to optimize a preliminary query model t/e parameterized by 0 that most often assigns "utility" scores A(ú, X) to each training sample X; then, the desired fraction s of the least useful instances is pruned from the dataset, yielding a sparse subset D, = {X: A(v, X) 2 quantile[A(t, s]}. In their seminal work, Toneva al. (2019) let X)) be the number of times (X, 3) is both learned and forgotten while training the query model. Paul et al. (2021) design "difficulty" measure A X) (x)(() vll2 where denotes the softmax
2

---

Published as a conference paper at ICLR 2025
function and y is one-hot. These scores, coined EL2N, are designed to approximate the GraNd metric A(e', X) = (oL(v(X)) yl11.. which is simply the f2-norm of the parameter gradient of the loss L computed at (X, y). Both EL2N and GraNd scores require only short training round (e.g.. 10 epochs) of the query model. [e et al. (2023) propose to select samples according to their dynamic uncertainty throughout training of the query model. For each training epoch k, they estimate the variance of the target probability By [x)) across a fixed window of J previous epochs, and finally average those scores across k. Since CoreSet approaches are highly relevant for data pruning, we also consider one such label-agnostic procedure that greedily selects training samples that best jointly cover the data embeddings extracted from the penultimate layer of the trained query model vi (Sener & Savarese, 2018). While all these methods come from various contexts and with different motivations, several studies show that the scores computed by many of them exhibit high cross-correlation (Sorscher et al. 2022 K Kwok et al. 2024).
Robustness & Evaluation Metries. Distributional robustness in machine learning concerns the issue of non-uniform accuracy over the data distribution (Sagawa et al. 2020). The majority of the vast research in this area focuses on group robustness where certain, often under--epresented or sensitive, groups of the population have worse predictive performance (Hashimoto et al., 2018)]
Thomas McCoy et al., 2020). In general, groups can be subsets of classes, and worst-group accuracy is standard optimization criterion in this domain (Sagawa* et al.) 2020; Kirichenko et al. 2023
Rudner et al.] 2024). As a special case, fairness in machine learning aims to mitigate disparity of model performance across society and demographic groups and uses specific criteria such as equal opportunity and equalized odds (Hardt et al. 2016a Caton & Haas 2023). Many well-established algorithms in the space of distributional robustness apply importance weighting to effectively re- balance the long-tailed data distributions (cf. cost-sensitive learning (Elkan, (001))
). Thus, |Saaway) et al. 2020) optimize an approximate minimax DRO (distributionally robust optimization) objective using a weighted sum of group-wise losses, putting higher mass on high-loss groups; [iihaa et al. (2022) weigh samples by the current class-wise misclassification rates measured on holdout validation set: Liu et al. (2021) pre-train a reference model to estimate the importance factors of groups for subsequent re- re-training. Similar cost- weighttng strategies are adopted for robust knowledge distillation (Wang et al.) 2023 Lukasik et al. 2022) and online batch selection (Kawaguchi & Lu
2020 Mindermann et al. 2022). Note that these techniques compute weights at the level of classes or sensitive groups and not for individual training samples. The focus of our study is special case of group robustness, classification bias, where groups are directly defined by class attributions. Classification bias commonly arises in the context of imbalanced datasets where tail classes require upsampling or reweighting to produce models with strong worst-class performance (Cui et al. 2019)
Barua al., 2012 T aa et al. 20200 (Caadhhuri et al., 2023). However, classification bias is also studied for balanced datasets, and is found to be exacerbated by adversarial training (Li & Liu 2023
Benz et al.
2021
Xu | 2021 Nanda al.) [221] Ma et al. (002) and network parameter pruning Joseph et al. 2020 Tran et al. [0222] Good et al. 2022). Following mainstream
(Paganini 2020
Li & Liu| (2023): Zayed al. (2023). we will use natural suite of metrics to measure classification bias. Given accuracy (recall) Fik for each class & E [K], we report (I) worst-class accuracy minę rk. (2) difference between the maximum and the minimum recall, max; T%k ming F'k
prior work
(Joseph et al. 2020), and (3) standard deviation of recalls std,rk (Ma et al. 2022).
Data Pruning Meets Robustness. Although existing data pruning techniques have proven to achieve strong average generalization performance, to our knowledge, our work contains the first comprehensive comparative study to call the extensive worsening of classification bias of state-of- the-art data pruning methods. Among related works, Pote et al.] (2023) studied how EL2N pruning affects the class-wise performance and found that, at high data density levels (c.g., 80-90% remaining data). under- perforrinfoningggggg classes improve their accuracy compared to training on the full dataset. Zayed et al.) (202)) propose a modification of EL.2N to achieve robustness across protected groups with two attributes (e.g., male and female) in datasets with counterfactual augmentation, which is rather specific context. In this study, we eliminate this blind spot in the data pruning literature and analyze the trade-off between the average performance and classification bias exhibited by some of the most common data pruning methods: EL.2N and GraNd (Paul et al., 2021). Forgetting (Toneva) let al. 2019), Dynamic Uncertainty (He et al., (023), ) and CoreSet (Sener & Savarese 2018). We also benchmark random pruning (Random), which is regarded as a notoriously strong baseline in active learning and data pruning, especially when pruning large fractions of data.
3

---

Published as a conference paper at ICLR 2025
3 DATA PRUNING IS NOT ROBUST
Figure 2 illustrates our evaluation of data prun- ing through the lens of wiss robustness for two vision benchmarks. CIFAR-100 and Tinylm-
10
class accuraty (Ful)
1.0
dongty
ageNet (additional plots for other metrics can be
0.6
found in Appendix C). We present the experimen- tal details and hyperparameters in Appendix A First, we note that no pruming method uniformly state-of-the-art, and the results vary considerably across model-dee -ottss- pairs. Among all algorithms, Dynamic Uncertainty with VGG-19 on CIFAR- 100 and ResNet-50 on ImageNet presents partic- ularly interesting and instructive case. While it is arguably the best with respect to the average test performance, fails miserably across all robust- ness metrics. Figure Ieeft reveals that it actually removes entire classes already at 10% of CIFAR- 100. Therefore, it seems plausible that Dynamic Uncertainty simply sacrifices the most difficult classes to retain strong performance on the eas- ier ones. Figure T6 confirms our hypothesis: contrast to other algorithms, low density levels,
min SPC
10%:
correlation: 0.25
0.0
0.2
class index isomed density!
dass density
0.7 dataset density
dataset density
Figure I: Pruning Exacerbates Bias: Dynamic Un- certainty applied to CIFAR-100 See Appendix G for similar plots for other pruning methods and models. Left: Sorted class densities at different dataset density levels. We also report the minimum number of sam- ples per class (SPC) at 10% dataset density. Right: Full dataset test class-wise accuracy against dataset density, We also report the correlation coefficient be- tween these two quantities across classes, averaged over dataset densities.
Dynamic Uncertainty tends to prune classes with lower baseline accuracy (obtained from training on the full dataset) a deceptively strong average accuracy. This observation presents compelling argument for using criteria beyond average test performance for data pruning algorithms, particularly emphasizing classification bias.
aggressively, which entails catastrophic classification bias hidden underneath
VGG 1A66-16+IIAARN. 0.900
VGG-19+ CIFAR-00
0.70
0.50
0.955
25%
0.53
0000000 100003330009)
0.6
45%
32%
0.50
0.915
3.60
05%
02%
0.75
0.92
0.900
035
75%
0.5
0.50
02. 0.1 8.6
05. 85 0J on 69
1.0
0.78
0.30
0.2
0.34
036
0.2
0.5
044
255
B.
0.7.
04 09 L0
0.0
0.1
0.2
633
044
consity
acouracy
ResNet-18- TinylmageNet
0.0
0.02
0.60
0.4
50%
0.3
0.02
0.02
60%
10%6
0.50
1074
hill
0.3
0.56
detamet demily
WAPRbCanA aCCursty
dstunet densiyy
wond-clas acEuTICy
Random
GraNd
Dynamic Uncertainty
EL2N
Forgetting
CoreSet
The average test performance of various data pruning algorithms against dataset density (fraction of samples remaining after pruning) and worst-class accuracy. All results averaged over 3 random seeds. Error bands represent min/max. Additional plots can be found Appendix
Figure 2:
Overall, all studied algorithms exhibit poor robustness to bias, although several of them improve ever so slightly over the full dataset. In particular, EL.2N and GraNd achieve relatively low classification bias, closely followed by Forgetting. At the same time, Forgetting has substantially stronger average test accuracy compared to these three methods, falling short of Random only after pruning more than 60% from CIFAR- 100. As seen from additional plots Appendix G Forgetting produces more balanced datasets than EL2N and GraNd low densities (Figure 17p. and tends to prune "easier" classes more aggressively compared to all other methods (Figure 16). These two properties seem to be beneficial, especially when the available training data is scarce.
4

---

Published as a conference paper at ICLR 2025
after pruning
optimal (average)
optimal (warst-cless)
Ru7) 10.2%
probablity density
(x() 6%
Ru(h) - 12, .5%
R, - 8.3%
0.6-
d 33
0.6
- 36x2%
DE-
3622% 6422%
00137% 77%
0.4
3.0
0.0
-2.
-1.
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
-2.0
-15
-10
-0.5
0.5
3.0
1.5
2.0
11.
1.0
Figure 3: The effect of different pruning procedures on the solution mixture of Gaussians problem with μo -1,
Random pruning with the optimal class-wise densities that satisfy di 10 dodgo Middle: SSP. Right: Random pruning with respect to class ratios provided by the SSP algorithm. All results averaged across 10 datasets [Di) each with 400
Aii 1, on 0.5, σi 1, and Φo d1. Pruning dataset density = 50%. Left:
points. The average ERM is T 10 Σ T(D)) fitted to pruned datasets D4. The class risks of the average and worst-class optimal decisions for this Gaussian mixture are Ry (f" (1) 4.8%, R1 [t" (1) = 12.1%, and
(i( R. (i) 9.1%.
4 THEORETICAL ANALYSIS
In this section, we derive analytical results for data pruming in a toy model of binary classification for mixture of two univariate Gaussians with linear classifiers. Perhaps surprisingly, in this framework we can derive worst-class optimal densities as well as demonstrate how prototypical pruning algorithms fail with respect to class robustness.
Let M be a mixture of two univariate Gaussians with conditional density functions p(rly = 0)
N Nuo, oū) and p(zly = 1) = (f,,,of,, priors do (P(y = 0) and Φi = P(y 1) (@o + Φ1 = 1), and øn 01 Without loss of generality, we assume 140 < μ41 Consider linear decision rules t E R U [ with a prediction function Gc(x) = 1(x > t}. The statistical 0-1 risks of the two
classes are
Ro(() = ( [o (10- (10-1)
1 7- M1
(t) +b
where @ is the standard normal cumulative distribution function. Under some reasonable but nuanced conditions on the means, variances, and priors discussed in Appendix [.I the optimal decision rule minimizing the average risk
R(t) do (t) + 01 (t)
computed by taking the larger of the two solutions to quadratic equation 0R/ôt 0, which denote by
mooi -mio 000
(no m1)2 2(0 - of) log dn01 01
of -
Note that ("(co/01) the rightmost intersection point of fo(() and & ft)((t) where fo and fi the corresponding probability density functions (Cavalli 1945). Note that in the balanced case (øn = Φ1) ) the heavier-tailed class more difficult and Ri [t (1) [1)]
We now turn the standard (class-) distributionally robust objective: minimizing worst-class statistical risk gives rise to the decision threshold denoted by 8 argmin, max (o R, (t)). As we show in Appendix B.2 i satisfies (t) = Rí (f), and Equation ] then immediately yields
i= (oor) pi plop)/(0o + t1).
Note that & (1). This means that in the balanced case i is closer to μoo the mean of the "easier" class. To understand how we should prune to achieve optimal worst-class accuracy, we compute class priors du and Φi that guarantee that the average risk minimization (Equation 2) achieves the best worst-class risk. These priors can be seen as the optimal class proportions in a "robustness-awann dataset. Observe that, from Equation 3 1t (00//1)
because the logarithm in the discriminant
5

---

Published as a conference paper at ICLR 2025
vanishes and we obtain Equation Therefore, the optimal average risk minimizer coincides with the solution to the worst-case error over classes, and is achieved when the class priors @u, @1 satisfy
óo /1 - 00/
(5)
Intuitively, sufficiently large datasets sampled from M with class-conditionall. independent samples mixed in proportions N, and N₂ should have their empirical 0-1 risk minimizers close t* (No/N That is, randomly retaining da fraction of samples in class k = 0, 1 where do dj N (see Equation P yields robust solutions with equal statistical risk across classes and, hence, optimal worst-class error. Figure 3 (left) confirms analysis: the average empirical risk minimizer (ERM) fitted to datasets pruned in this fashion lands near t.
The class-conditions independence assumption made above is crucial. While it is respected when subsampling randomly within each class, it clearly does not hold for existing, more sophisticated, data pruning algorithms. Therefore, even though they tend prune easier classes more aggressively as evident from Figure [6] they rarely enjoy any improvement of the worst-class performance compared the original dataset. We further illustrate this observation by replicating a supervised variant of the Self-Superve Pruning (SSP) developed by SSorscher et al. (2022) for ImageNet. In this algorithm, we remove samples globally (i.c., irrespective of their class membership) located within a certain margin M > 0 of their class means. Intuitively, this algorithm discards the easiest or the most representative samples from the most densely populated regions of the distribution. As illustrated in Figure 3 (middle). this method fortuitously prunes the easier class more aggressively; indeed, the amount of samples removed from class with mean j and variance o² proportional to the area under the probability density function over the pruning interval [M M, / + M). which is larger for smaller values of a. Nevertheless, if the original dataset has classes mixed in proportions Ny and N1. the solution remains close to t" (No/ even after pruning, as show formally in Appendix B.3 On the other hand, random subsampling according to class-wise pruning proportions defined by SSP does improve worst-class accuracy, illustrated in the right plots of Figure This corresponds to our observation in Figure 5 that random pruning respecting the class proportions discovered by GraNd and Forgetting often improves robustness compared to these methods themselves.
10a
0.8
fg(7 - 10.9%
0.8
RaTh
n.(T
RiTh - 0.2%
Rolt')/R((e))
0.6
de - 50.0%
0.6
m 29:00% 71:00%
di 100
109
0.2
10
0.0
0.0
10
10a
10
-1.5
0.0
1.5
-2.0
-1.5
-0.5
0.0
0.5
1.0
1.5
2.0
[dt0ml/[dnce
T: after pruning
optimal (average)
8: optimal (worst-class))
(a)
(b)
Figure 4: (a): Class-wise risk ratios of the optimal solution t" = t- =d"(00/b11 vs. optimal ratios hased on Equation
computed for various ao <, ai drawn uniformly from [10 2 10") and do U/0 1] and Φ1
do- The
results are independent of po. /11. (b): Random pruming with DRoP. Left: 75%; Right: d 50%.
A priori, it is unclear how exactly the variance-based worst-case optimal pruning quotas in Equation gennralle deep learning, and in particular how they can lead to practical pruning algorithms. One could connect our theory to the feature distribution in the penultimate layer of neural networks and determine class densities from cluster variances around class means. In fact, SSP (Sorscher et al. 2022) uses such metrics to determine pruning scores for samples, using a pretrained model. However, such variances are noisy, especially for smaller class sizes, and hard connect to class accuracies. Therefore, instead, we propose to use class errors as a proxy for these variances in our DRoP method introduced in Section 5 More formally, given a dataset originally with class sizes Ng and N1, we replace the worst-class optimal condition da No = d2 No with do Ri [t (No = di Rolt" (No,NN() To motivate this practical approach, Figure 44 shows that these new error-based class densities approximate the optimal variance- ones fairly well, especially when σ1/ 01/0u ou is small. Figure 4p demonstrates that random pruning according to DRoP class proportions lands the average ERM near the worst-class optimal value. Thus, even though error- error-based class quotas do not enjoy simple theoretical guarantees, they still operate optimally in this toy setup.

---

Published as a conference paper at ICLR 2025
5 DROP: DISTRIBUTIONALLY ROBUST PRUNING
We are ready to propose our "robustness-awand data pruning method, which consists in random subsampling according to carefully selected target class-wise sizes, incorporating lessons learned from our theoretical analysis. We propose to select the pruning fraction of each class based on its validation performance given a preliminary model to trained on the whole dataset. Such a query model is still required by all existing data pruning algorithms to compute scores, so we introduce no additional resource overhead (see Appendix B for more discussion).
Consider pruning a K-way classification dataset origi- nally with N samples down to density 0 S d S 1, so the target dataset size is dNN (prior literature sometimes refers to 1 - d as the pruning fraction). Likewise, for each class & E [ define N₂ to be the original number of samples so N = Σ Nr. and let 0 < d2 S be the desired density of that class after pruning. Then, we set de x TA where denotes recall (accuracy) of class computed by 1/ on a hold-out validation set. In particular, we define DRoP quotas as d2 = dd1 r
Algorithm 1: DRoP
Input: Target dataset density d E [0. 1]. For each class k E [K]: original size Ng. validation recall rı E [0, 1].
Initialize: Unsaturated set of classes U t [K], excess < dN, class densities d₃ + 0 A/k E [K].
while F > 0 do
Z + 1 EkeU N₂(1 rk):
for le E U do
where Z = (1 - N N is a normalizing fac-
d/ + (1 rw)/Z:
tor to ensure that the target density is respected, i.e..
d2 < d: + di;
dN = ΣK= Nz. Alas, not all dataset densities E [0, can be associated with a valid DRoP col- lection; indeed, for large enough d, the required class proportions may demand d3 > for some k E [K]. In such case, we do not prune such classes and redis- tribute the excess density across unsaturated (ds < 1) classes according their DRoP proportions. The full procedure is described in Algorithm [ This algorithm is guaranteed terminate as the excess decreases in each iteration of the outer loop.
E E --Ndk
d2 > then
U < U \ {k);
E < E + Nx 1);
du + 1
end
end
end Return: [dx )
Evaluation. To validate the effectiveness of random pruning with DRoP (Random+DRoP) in reducing classification bias, we compare it to various baselines derived from the strongest pruning algorithms: EL2N (Paul et al., 2021) (for CI and ImageNet), GraNd (Paul et al.) 2021) and Forgetting (Toneva et al.] 20199 (for CIFAR- 100 and TinyImageNet). In addition to plain random pruning (Random)-removinng a random subset all training samples-for each of these two strategies. we consider (1) random pruning that respects class-wise ratios automatically determined by the strategy (Random+Strategy) sample quotas across classes according to DRoP DRoP), and (3) the strategy itself. The motivation for (1) is the anecdotal observation that existing pruning algorithms automatically balance classes according to their validation errors fairly well (Figure [66. while their overly scrupulous filtering within classes may be suboptimal. In support of this view. (Aydd & Hayou (2023) formally show that integrating random sampling into score-based pruning procedures improves their average performance. Next, (2) serves as a reasonable ablation, while (3) is the benchmark we compare to. The implementation details are listed in Appendix A
(2) applying the strategy for pruning within classes but distributing
Results. Figure 5 presents our empirical results (additional plots showcasing other robustness metrics can be found in Appendix D). Overall, DRoP with random pruning consistently exhibits a significant improvement in distributional robustness of the trained models. In contrast to all other baselines that arguably achieve their highest worst-class accuracy at high dataset densities (80-90%) our method reduces classification bias induced by the datasets as pruning continues, e.g., up to 30-40% dataset density of Tiny ImageNet. Notably, Random+DRoP improves all robustness metrics compared to the full dataset on all model-dataset pairs, offering both robustness and data efficiency at the same time. For example, when pruning half of CIFAR-100, we achieve an increase in the worst-class accuracy of VGG-19 from 35.8% to 45.4% an almost 10% change at a price of under 6% of the average performance. The leftmost plots in 3 that Random+DRoP does suffer slightly larger degradation of the average accuracy as dataset density decreases compared to global
7

---

Published as conference paper at ICLR 2025
VGG-19 CIFAR-100
ResNet-18 TinylmageNet
067
34500 SCO687
3.90
0.66
GraNe
0.62
GraNid
0.60
0.35
0.59
0.60
0.45
0.00
099 090-920 600A000
0.65
Forgetiigg
0.5
Forgetting
641
0.62
0.60
0.99
0:55
6.50
02 0.3
04
055 06 DJ 688 0.5
0.6
0.25
0.30
0.35
0.40
0.45
0.2
0.
04
05 0.6
08
6.9
1.0
0.4
0.075
0.106
0.125
0.150
6.375 6:200
dotssetdericyy
accuraty
density
VGG-16
CIFAR-10
ResNet-50 ImageNet
093
0.05
3.2
0:93
2.42
0.66
ELON
0.92
3.00
3.00
0.65
0.00
0.75
0.92
0.54
677
08
0.80
0.82
3.58
0.5
0.8
0.7
0.8
0.9
3.5
0.50
0.03
0.19
0.12
0.14
wombb accurnsy
Random- StrategyO
Random- DRoP
Strategy
Strategy Dllop
Figure The average test performance various data pruning protocols against dataset density and worst-class accuracy. All results averaged over random seeds. Error hands represent min/max.
random pruning, which is unavoidable given the natural trade-off between robustness and average performance. Yet these low densities, the average accuracy of Random+DRoP exceeds that of all other pruning algorithms.
As demonstrated in Figure 6 DRoP produces exceptionally imbalanced datasets unless the density is too low by heavily pruming easy classes while leaving the more difficult ones intact. As expected from its design, the negative correlation between the full-dataset accuracy and density of each class is much more pronounced for DRoP compared to existing pruning methods (cf. Figure [6).. Based on our examination of these meth- ods in Section 3 we conjectured that these two properties are associated with smaller classification bias, which is well-supported by DRoP. Not only does it achieve unpar- alleled performance with random pruning, but it also enhances robustmess of GraNd and Forgetting: Strategy+DRoP curves of- ten trace a much better trade-off' between the average and worst-elass accuracies than their original counterparts (Strategy). At the same time, Random+Straateyy fares similarly well, surpassing the vanilla algo- rithms, too. This indicates that robustness is achieved not only from the appropriate class ratios but also from pruning randomly opposed to cherry-picking hard samples.
cless accuracy (Fail)
1.0
0.8
0.8
0.6
0.6
0.4
min SPC
10%::
Javg. comreaaiinn -0.71
0.0
0.25
0.3
0.9
0.9
0.7
dataset density
dataset density
Figure DRoP. Left: Sorted class densities different dataset density levels. We report the minimum number of samples per class (SPC) at 10% dataset density. Right: Full dataset test class-wise accuracy against dataset density. We also report the correlation coefficient between these two quan- tities across classes, averaged over dataset densities.
A DRO baseline. The results in Figure provide solid evidence that Random+DRoP is by far the state-of-the-art data pruning algorithm in the robustness framework. Not only superior to prior pruning baselines, it in fact produces significantly more robust models compared to the full dataset, too. Thus, we go further and test our method against one representative cost-sensitive learning method from the DRO literature. Class-wise Difficulty Based Weighted loss (CDB-W) (Sinha et al.) 2022). In its simplest form adopted in this study, CDB-W W dynamically updates class-specific weights Wk.t Tke at every epoch by computing recalls Fket holdout validation set, which is
8

---

Published as a conference paper at ICLR 2025
EL2N
CDBNW+EL2N
EDBN+
EL2N+ DRoP
Random-DRof
0.700
0.675
0.65
0.000
0.60
0.55
0.600
0.08
0.575
0.2 0.3 0.5 0.6
G.9
1.0
0.2
0.3
0.4
0.40
0.45
0.50
0.55
0.60
0.65
0.10
0.32
0.14
0.16
dataset density
acrst-ciass accumary
best-class recall worst recall
40%
50%
60%
70%
80%
90%
(Full)
Figure 7: The average test performance of data pruning protocols against data density and measures of class robustness (VGG-19 19 on CIFAR- 100). All results averaged over 3 random seeds, Error bands represent min/max.
precisely the proportions used by DRoP. Throughout training, CDB-W uses these weights upweigh the per-sample losses based on the corresponding class labels. As part the evaluation, we test if the robustness-drive
optimization of CDB-W can help data pruning (for these experiments, use EL2N). To this end, we consider two scenarios: training the final model with CDB-W W on the EL2N-pruned dataset (EL2N+CDB-W) -), and using robust query model trained by CDB-W BB- to generate EI.2N scores (CLL-W+
Having access to the full dataset for training, CDB-W W improves the worst-class accuracy of VGG-19 on CIFAR- by 5.7% compared to standard optimization, which is almost twice as short of the increase obtained by removing 50% of the dataset with Random+DRoP (Figure 5 When applied to EL2N-pruned datasets, CDB-W maintains that original bias across sparsities, which is clearly inferior not only to Random+DRoP but also to EL2N+DRoP. Perhaps surprisingly. EL2N with scores computed by query model trained with C fails spectacularly. inducing one of the worst bias observed in this study. Thus, DRoP can compete with other existing methods that directly optimize for worst-class accuracy.
Imbalanced datasets. In the literature, robustness almost exclusively arises in the context of long-tailed distributions where certain classes or groups appear far less often than others; for example, CDB-W was evaluated in this setting. While dataset imbalance may indeed exacerbate implicit bias of the trained models towards more prevalent classes, study demonstrates that the key to robustness lies in the appropriate, difficulty-baat based class proportions rather than class balance per se. Even though the overall dataset size decreases, pruning with DRoP produce far more robust models compared to full but balanced datasets (Figure 6). Still, to promote consistency in evaluation consider long-tailed classification scenarios. We follow the approach used by Cui et al. (2019) to inject imbalance into the originally balanced Tiny ImageNet. In particular, we subsample each class [K] and retain & of its original size for some μ E (0, 1). For an initially balanced dataset, the size ratio between the largest (k 1) and the smallest (k = K) classes then becomes - which is called the imbalance factor denoted by Figure X reveals that Random+DRoo consistently beats EL.2N in terms of both average and robust performance across range of imbalance factors (1 = 2, 5, 20). Likewise, it almost always reduces bias of the unpruned imbalanced Tiny TinylmageNet even when training with a robustness-awan CDB-W procedure.
and to further validate our algorithm,
Random+DRoP
1=2
EL2N
0.4
overge
0.3
0.55
0.45
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.6
0.00
0.05
0.10
0.15
0.6
0.7
0.8
0.9
0.14
0.16
0.18 0.20
0.2
deteset density
warni-class accunty
Dench wwrdeclaae meail
30%
40%
50%
60%
70%
90%
Full
CDB-w (Full)
Figure 8: The average test performance of Random+DRoP (red-toned curves) and EL2N (bluc-toned curves) against dataset density and measures of class robustness across dataset imbalance factors = 2, 5, 20. ResNet- on imbalanced TinyImageNet Results averaged over random seeds. Error bands represent min/max.

---

Published as a conference paper at ICLR 2025
Group Distributional Robustness: Waterbirds. While we scoped our study around robustness to classification bias, a vast majority of research is concerned with strong model performance across sensitive or minority groups within the distribution. One common benchmark dataset in this area is Waterbirds, which is a binary image classification of bird types where ground truth is spuriously correlated with image background (forest/water) for the majority of samples (Sagawa* al. (2020). Thus, standard optimization techniques produce models that often make predictions based on the background type rather than bird features, yielding poor accuracy for images from waterbird++freet or landbird+water groups. We use the original learning setup introduced by Sagawa" et al. (2020) to evaluate DRoP against other pruning baselines as well CDB-W, W. Figure(Jjhows that Random+DRoP and EL2N+DRoP achieve significant improvement of the worst-group accuracy and other metrics. For worst-class accuracy, we adopt a handful of prior art including Just Train Twice (Liu al.
2021), Learning from Failure (Nam et al. 2020), Learning to Split (Bao & Barzilay] (022), and Bias Amplification (Li et al. ):23); the performance these algorithms in this setup is reported by Pezeshki et al. (2024), which
copied verbatim. As evident from Figure 9 Random+DroP is well on par with these more sophisticated techniques that operate on full datasets. Therefore, as expected, our algorithm applies not only to reduce classification bias but also to improve group-wise robustness.
Random
EL2N
EL2N+DRRP DRoP
Random+DRooP
0.975
3,880
Beerge scumncy
3.550
0.975
3.925
0.94
3,000
0.900
0.92
0.970
0.875
0.975
0.965
0.800
0.86
0.4
0.6
0.8
0.2
0.3
0.4
0.
0.075 0,100 0.125 6.150 0.175 0.200
by
worst-grber accuracy
best-group acc worst-group atc.
std group accuracy CDB-W (Full)
30%
40%
50%
60%
70%
80%
90%
*
Full
LS
XRM
BAM
Figure 9: The average test performance of data pruning protocols and existing baselines against measures of group-wise rohustness (ResNet-50 on Waterbirds). The results data pruning and CDB-W - averaged over random seeds. Error bands represent min/max. To conform with Sagawa* ct al. (2020). for this dataset, we compute average accuracy as sum of group accuracies weighted by the original training group proportions. This explains the sharp degradation of the average performance of DRoP-backed pruning at low densities (d S 0.4): these datasets are skewed towards minority groups that weigh much less than severely pruned majority groups. Please see Appendix F for an extended discussion.
6 DISCUSSION
Data pruning-remenn of uninformative samples from the training dataset-offee much needed efficiency in deep learning. However, all existing pruning algorithms are currently evaluated exclu- sively on their average performance, ignoring their potentially disparate impact on model predictions across data distribution. Through systematic study of the classification bias, we reveal that current methods often exacerbate the performance disparity across classes, which can deceptively co-occur with high average performance. This leads us to formulate error-based class-wise pruning quotas coined DRoP. At the same time, find value in pruning randomly within classes, opposed to cherry cherry-picking individual samples, which is inherent to the existing data pruning techniques. We confirm the effectiveness of our method simple pruning protocol traces the best trade-off between average and worst-class performance among all existing data pruning algorithms and related baselines. Additionally, we find theoretical justification for the phenomenal success of this simple strategy in a toy classification model.
series of standard computer vision benchmarks; our
Limitations & Future Work. In this study, we focused our empirical evaluation primarily on classification bias. Thus, we only scratched the surface of robustness in deep learning, which is often concerned with group wise model performance. Further research may attempt to understand the effect of DRoP and data pruning at large on worst-group accuracy and spurious correlations more deeply. Finally, attribute limited cross-eves eoosueevalttioon with a broad spectrum of distributionally robust optimization methods. Thus, future research is warranted to perform more detailed comparative study between these approaches.
contributions mostly to research data pruning and, therefore, present
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
AV and JK acknowledge support through the NSF under award 1922658. Part of this work was done while JK was visiting the Centre Sciences de Donnees (CSD) at the Ecole Normale Superieure in Paris, France, and JK wishes to thank the CSD for their hospitality. This work was supported through the NYU IT High Performance Computing (HPC) resources, services, and staff expertise.
REFERENCES
Sharat Agarwal, Himanshu Arora, Saket Anand, and Chetan Arora. Contextual diversity for active learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings. Part XVI 16, pp. 137-153. Springer, 2020.
Anders Johan Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roelofs. The evolution of out-of-distribution robustness throughout fine-tuning. Trunsuctions on Machine Learning Research, 2022. ISSN 2835-8856.
Fadhel Ayed and Soufiane Hayou. Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
Yujia Bao and Regina Barzilay. Learning split for automatic bias detection. arXiv preprint arXiv:2204. 13749, 2022.
Sukarna Barua, Md Monirul Islam, Xin Yao, and Kazuyuki Murase, Mwmote Mwmote-majority weighted mi- nority oversampling technique for imbalanced data set learning. IEEE Transactions on knowledge and data engineering, 26(2):405-444 425, 2012.
William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. The power of ensembles for active learning in image classification. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 9368-9377, 2018.
Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. Robustness may be at odds with fairness: An empirical study class- accuracy. In NeurlPS 2020 Workshop on Pre-registration in Machine Learning, pp. 325-325-33 342. PMLR, 2021.
Yaohui Cai, Weizhe Hua, Hongzheng Chen, G. Edward Suh, Christopher De Sa, and Zhiru Zhang. Structured pruning is all you need for pruning cnns initialization, 2022.
Simon Caton and Christian Haas. Fairness in machine learning: A survey. ACM Comput. Surv., aug 2023.
Luigi Cavalli. Alcuni problemi della analisi biometrica di popolazioni naturali, 1945.
George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A Efros, and Jun-Yan Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4750-4759, 2022.
Kamalika Chaudhuri, Kartik Ahuja, Martin Arjovsky, and David Lopez-Paz. Why does throwing away data improve gorst-group error? In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 4144-4188. PMLR, 23-29 Jul 2023.
Nitesh V Chawla, Kevin W Bowyer, Lawrence Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
Robert Chen, Brendan Lucier, Yaron Singer, and Vasilis Syrgkanis. Robust optimization for non-convex objectives. Advances in Neural Information Processing Systems, 30, 2017.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. bals-balaaced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9268-92777 2019.
11

---

Published as a conference paper at ICLR 2025
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A seage-scale&ae hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214-226, 2012.
Charles Elkan. The foundations of cost-sensitive learning. In International joint conference on artificial intelligence, volume 17, pp. 973-978. Lawrence Brlbaum Associates Ltd, 2001.
Dan Feldman. Core-sets: An updated survey. WIREs Data Mining and Knowledge Discovery, 10(1): e1335, 2020.
Yunzhen Feng, Shanmukha Ramakrishna Vedantam, and Julia Kempe, Embarrassingly simple dataset distillation. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
Benjamin Feuer, Ameya Joshi, Minh Pham, and Chinmay Hegde. Distributionally robust classification on data budget. arviv preprint Arviv:2308.00 033222,0 2023.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Pruning neural networks at initialization: Why are we missing the mark? In International Conference on Learning Representations, 2021.
Aidan Good, Jiaqi Lin, Xin Yu, Hannah Sieg, Mikey Fergurson, Shandian Zhe, Jerzy Wieczorek, and Thiago Serra, Recall distortion in neural network pruning and the undecayed pruning algorithm. Advances in Neural Information Processing Systems, 35:32762-32776, 2022.
Mitchell A Gordon, Kevin Duh, and Jared Kaplan. Data and parameter scaling laws for neural machine translation. In Maric-Francins Moens, Xuanjing Huang. Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing, pp. 5915-5922, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
Chengeheng Guo, Bo Zhao, and Yanbing Bai. Deepcore: A comprehensive library for coreset selec- tion in deep learning. International Conference on Database and Expert Systems Applications, pp. 181-195. Springer, 2022.
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 3323-3331, Curran Associates Inc., 2016a.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, Advances in neural information processing systems, 29, 2016b.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pp. 1929-1938. PMLR, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recogmition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Muyang He, Shuo Yang. Tiejun Huang, and Bo Zhao. Large-scale dataset pruning with dynamic uncertainty. arXiv preprint arXiv:2306.0517 2023.
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv: 1712 2017.
12

---

Published as a conference paper at ICLR 2025
Badr Youbi Idrissi, Martin Arjovsky. Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst-grows ceatttrooouuuurueeee In Bernhard Schölkopf, Caroline Uhler, and Kun Zhang (eds.), Proceedings of the First Conference Causal Learning and Reasoning. volume 177 of Proceedings of Machine Learning Research, pp. 336-351. PMLR, 11-13 Apr 2022.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning- Volume 37, ICML' 10MLL'1,,N pp. 448 456. JMLR.org. 2015.
Vinu Joseph, Shoaib Ahmed Siddiqui, Aditya Bhaskara, Ganesh Gopalakrishnan, Saurav Muralidha- ran, Michael Garland, Sheraz Ahmed, and Andreas Dengel. Going beyond classification accuracy metrics in model compression. arXiv preprint arXiv:2012.01604 2020.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford. Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836 2020.
Kenji Kawaguchi and Haihao Lu. Ordered sgd: A new stochastic optimization framework for empirical risk minimization. In International Conference on Artificial Intelligence and Statistics, pp. 669-679. PMLR, 2020.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustmess spurious correlations. In The Eleventh International Conference Learning Representations, 2023.
Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. Advances in neural information processing systems, 32, 2019.
Germain Kolossov, Andrea Montanari, and Pulkit Tandon. Towards statistical theory of data selec- tion under weak supervision. In The Twelfth International Conference on Learning Representations, 2024.
Alex Krizhevsky. Learning multiple layers of features from tiny images. pp. 32-33, 2009.
Devin Kwok, Nikhil Anand, Jonathan Frankle, Gintare Karolina Dziugaite, and David Rolnick. Dataset difficulty and the role of inductive bias. arXiv preprint FrXv:22001.0187,,1 2024.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Boqi Li and Weiwei Liu. Wat: improve the worst-class robustness in adversarial training. MAA/23/AAA23EAAAA
AAAI Press, 2023. ISBN 978-1-57735-880-0
Gaotang Li, Jiarui Liu, and Wei Hu, Bias amplification enhances minority group performance. arXiv
preprint arXiv:2309.0m6 0233009.7711 17, 2023.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 6781-6792. PMLR, 1 18-24 Jul 2021.
Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, and Sanjiv Kumar. Teacher's 's pet: understanding and mitigating biases in distillation. Transactions on Machine Learning Research, 2022. ISSN 2835-8856.
Xinsong Ma. Zekai Wang, and Weiwei Liu. On the tradeoff between robustness and fairness. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Sören Mindermann, Jan M Braumer, Muhammed T Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N Gomez, Adrien Morisot, Sebastian Farquhar, et al. Prioritized training on points that are learnable, worth learning, and not yet learnt. In International Conference Machine Learning, pp. 15630-15649, PMLR, 2022.
13

---

Published as a conference paper at ICLR 2025
Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning models. In International Conference on Machine Learning, pp. 1950-6960. PMLR, 2020.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
Vedant Nanda, Samuel Dooley, Sahil Singla, Soheil Feizi, and John P. Dickerson. Fairness through robustness: Investigating robustness disparity in deep learning. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAceT '21, pp. 466-477, New York, NY, USA, 2021. Association for Computing Machinery.
Timothy Nguyen, Roman Novak, Lechao Xiao, and Jachoon Lee. Dataset distillation with infinitely wide convolutional networks. In A. Beygelzimer, Y. Dauphin, P. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021.
Michela Paganini. Prune responsibly, 2020.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on data diet: Finding important examples early in training. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021.
Mohammad Pezeshki, Diane Bouchacourt, Mark Ibrahim, Nicolas Ballas, Pascal Vincent, and David Lopez-Paz. Discovering environments with XRM. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 4/st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 40551 1-569, PMLR, 21-27 Jul 2024.
Tejas Pote, Mohammed Adnan, Yigit Yargic, and Yani loannou. Classification bias on a data diet. In Conference Parsimony and Learning (Recent Spotlight Track), 2023.
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po- Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM computing surveys (CSUR), 54(9):1-40, 2021.
Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit. constructive prediction of the generalization error across scales. In International Conference on Learning Representations (ICLR) 2020, 2020.
Tim GJ Rudner, Ya Shi Zhang, Andrew Gordon Wilson, and Julia Kempe. Mind the gap: Improving robustness to subpopulation shifts with group-aware priors. In International Conference Artificial Intelligence and Statistics (AISTATS), 2024.
Shiori Sagawa*, * Pang Wei Koh*, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Saptarshi Sinha, Hiroki Ohashi, and Katsuyuki Nakamura. Class-difficulty based methods for long-tailed visual recognition. International Journal of Computer Vision, 130(10):2517-2531 2022.
14

---

Published as a conference paper at ICLR 2025
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari S. Morcos. Beyond neural scaling laws: beating power law scaling via data pruning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Felipe Petroski Such, Aditya Rawal, Joel Lehman, Kenneth Stanley, and Jeffrey Clune. Generative teaching networks: Accelerating neural architecture search by learning to generate synthetic training data. In International Conference on Machine Learning, pp. 9206-9216. PMLR, 2020.
Ilia Sucholutsky and Matthias Schonlau. Soft-label dataset distillation and text dataset distillation. In 2021 International Joint Conference on Neural Networks (ICNN), pp. 1-8. IEEE, 2021.
Jingru Tan, Changbao Wang. Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, and Junjie Yan. Equalization loss for long-tailed object recognition. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1 11662- 11671, 2020.
Alaa Tharwat and Wolfram Schenck. A survey on active learning: State-of-the-art, practical chal- lenges and research directions. Mathematics, (1)),, 2023.
R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. ACI. 2019 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, ACL 2019 57th Annual Meeting of the Association for Computational Linguistics. Proceedings the Conference, pp. -428-3448. Association for Computational Linguistics (ACL), 2020.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. An empirical study of example forgetting during deep neural network learning. In International Conference on Learning Representations, 2019.
Cuong Tran, Ferdinando Fioretto, Jung-Eun Kim, and Rakshit Naidu. Pruning has a disparate impact on model accuracy. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Chaoqi Wang, Guodong Zhang, and Roger Grosse.
Picking winning tickets before training by preserving gradient flow. In International Conference on Learning Representations, 2020.
Serena Wang, Harikrishna Narasimhan, Yichen Zhou, Sara Hooker, Michal Lukasik, and Aditya Kr- ishna Menon. Robust distillation for worst-class performance: on the interplay between teacher and student objectives. In Robin Evans and Ilya Shpitser (eds.). Proceedings of the Thirty-Nine Conference on Uncertainty in Artificial Intelligence, volume 216 of Proceedings of Machine Learning Research, pp. 2237-2247. PMLR, 31 Jul-04 Aug 2023.
Max Welling. Herding dynamical weights to learn. In Proceedings of the 26th Annual Interna- tional Conference Machine Learning, ICML '09, pp. 1121-1128, New York, NY, USA, 2009. Association for Computing Machinery.
Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. To be robust or to be fair: Towards fairness in adversarial training. In International conference on machine learning, pp. 1192-11501.
PMLR, 2021.
Sergey Zagoruyko. Wide residual networks. arXiv preprint arXiv:1605.07140 2016.
AbdeIrahman Zayed, Prasanna Parthasarathi, Gonçalo Mordido, Hamid Palangi. Samira Shabanian, and Sarath Chandar. Deep learning on a healthy data diet: finding important examples for fairness. In Proceedings of the Thirty-Sevntth AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence, AAAT23ЛAААΣ23ЕAC 23/AAAT 23. AAAI Press, 2023.
Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pp.
2023.
Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centrie coreset selection for high pruning rates. In The Eleventh International Conference on Learning Representations, 2023.
15

---

Published as a conference paper at ICLR 2025
A IMPLEMENTATION DETAILS
Our empirical work encompasses three standard computer vision benchmarks (Table All code is implemented in PyTorch (Paszke et al. (0117) and run on an internal cluster equipped with NVIDIA RTX8000 GPUs. The total runtime of the empirical work presented in this paper is approximately 45, 000 GPU hours, We make our code available at at/htps://// /tttt///g/ihbb...o// com/ foom/aan avysogorets////// Stop-data- prrniig
Data Pruning. Data pruning methods require different procedures for training the query model and extracting scores for the training data. For EL2N and GraNd, we use 10% the full training length reported in [able|[]eeoor calculating the importance scores, which is than the minimum of 10 epochs recommended by Paul et al. (022). To improve the score estimates, we repeat the procedure across 5 random seeds and average the scores before pruning. Forgetting and Dynamic Uncertainty operate during training, so we execute a full optimization cycle the query model but only do so once. Likewise, CoreSet applied once on the fully trained embeddings. We use the greedy k-center variant of CoreSet. Since some of the methods require a hold-out validation set (e.g., DRoP, CDB-W), we reserve 50% of the test set for this purpose. This split never used when reporting the final model performance.
Data Augmentation. We employ data augmentation only when optimizing the final model. The same augmentation strategy is used for all datasets except for Waterbirds where we used none. In particular, we normalize examples per-channel and randomly apply shifts by at most pixels in any direction and horizontal flips.
Models & Datasets.
As shown Table [ we use the following model-dataset pairs: VGG-16
and VGG-19 (Simonyan & Zisserman, 2015) on CIFAR-10 and CIFAR-100 (Kizhevsky 2009
respectively, ResNet-18 (He al., 2016)) on TinylmageNer (MIT License) (Le & Yang 2015).
ImageNet pre-trained ResNet-50 on Waterbirds (Sagawa* et al. 2020) (MIT License), and ResNet-50
on ImageNet (Deng et al. 2009). We also use a slight adaptation Wide-RessNet1101 Zagoruyko
2016) CIFAR-100 with a few downsampling layers removed.
Model
Dataset
Epochs
Drop Epochs
Batch
Decay
VGG-16
CIFAR-10
160
80/120
le-4
VGG-19
CIFAR-100
160
80/120
128
0.1
5e-4
ResNet-18 18
Tiny ImageNet
200
100/150
256
0.2
le-4
ResNet-50
ImageNet
90
30/60
512
0.2
1e-4
ResNet-50
Waterbirds
300
None
128
0.0001
1e-4
Wide-ResNet-101
CIFAR-100
200
60/120/160
128
0.1
1e-4
Table 1: Summary of experimental work and hyperparameters. All architectures include batch normalization (Toffe & Szegedy (205)) layers followed by ReLU activations. Models are initialized with Kaiming normal (He al. (2015) and optimized by SGD (momentum 0.9) with a stepwise LR schedule (0.2x drop factor applied on specified Drop Epochs) and categorical cross-entropy. These
hyperparameters are adopted from prior studies (Frankle et al. 2021 Wang
2020
Sagawa
et al. 2020 Cai et al. (022)..
B THEORETICAL ANALYSIS FOR A MIXTURE OF GAUSSIANS
Consider the Gaussian mixture model and the hypothesis class of linear decision rules introduced in Section Here we give more formal treatment of the assumptions and claims made that section.
B. AVERAGE RISK MINIMIZATION
Recall that we consider the average risk of the linear decision (x(x) = 1(x >} as R(t) = Ex, (f (x). where the expectation is over (x, p(x, and class-conditional risk as
16

---

Published as a conference paper at ICLR 2025
(t) = (xln(((((())) y)1, where the expectation is over p(zly) for y {0,1}. If l is the 0-1 loss, we thus obtain Equations|
[ and Recall that we have assumed do < oi and du < P1.
We give the precise conditions under which the average risk minimizer takes the form of t" (" (1) from Equation 3 the larger intersection point between the graphs of scaled probability density functions fo (t) and @1 (t)..
First, we make an assumption that this intersection exists, i.e., the expression under the square root in Equation 3 is non-negative. That is, we require
Φ
do exp
1 (P0 - 11) [ of - oB
(6)
Φ1
This assumption simply guarantees the existence of intersection points of the two scaled density functions; provided this holds, we establish an additional condition on priors necessary for (Øo/ (01/ to be the average risk minimizer.
Theorem B.1. If Equation 6 holds, define l' (doo
(1)) statistical risk minimizer for the Gaussian mixture model if
in Equation
Then, 1 (1) is the
@ > ②b 11
t (1((00/0 (1(00/01)-A)) -
/
(t(00//0)
(1) μio
O0
Proof. For a decision rule / E IR U (+xc), the statistical risk in the given Gaussian mixture model is given in Equation [ minimizer is solution to @R(t)/Ot = 0:
The global minimum achieved either at +o00 or on the latter case, the
aR(()
14
)
2
Φi V
t mi a1
2
exp
-
exp
0t
V2TOO
2
Rearranging and taking the logarithm on both sides yields
Φ
M1
2
140
2
0 = log
+
(8)
Φ100
T
which is quadratic equation in & with solutions
t pooe f + 0o0
/ (HD 21) =2( - log do 0100
(9)
of o3
By repeating the same steps for an inequality rather than equality, conclude that 0 < aR(t)/Ot if and only
poo0 $100
- 1--
M1
2
+ P40 - -)) 2
0 < 22g
(10)
similarly to Equation g This identity holds because the logarithm is a monotonically increasing function, preserving the inequality. Further expanding the right-hand side of Equation IO and collecting similar terms, we arrive at quadratic equation in t with the leading (quadratic) coefficient 0 01 O. Hence, the right-hand side defines an poward-branching parabola with zeros given 9 when they exist (we assume they do owing to assumption in Equation 6). The derivative of the statistical risk positive whenever the right-hand side of Equation T0 is, f.e., on intervals t-) and (0). Hence, the risk R(t) must be increasing on the interval t-), and so t_ can never be global minimizer. Likewise, the risk is increasing on the interval (t4, oo), which rules out (- Therefore, just need establish that R( ((-00) 2 R(f+), which is equivalent to Equation since t" (0o/01) = =4
in Equation
Remark. We have considered unequal variances of < of as a natural way to model classes with different difficulty. Yet note that our analysis still holds with slight modifications when op ai
17

---

Published as a conference paper at ICLR 2025
The difference in this case is that for any choice of priors, there is exactly one solution Equation (and exactly one intersection point of the scaled density functions), given by
20 log
2n
+ (pī m5)
t" (00/01)
2(M1
(o)
In particular, no additional assumptions as in Equation to guarantee the existence of an intersection point need to be made. Furthermore, similar derivative analysis above implies that the risk is decreasing on the interval (-xo, (0o/ /21)) )) and increasing on the interval (t" (((o/ó1) +o0), so that 1" (dbo// (1) must be the statistical risk minimizer and the assumption in Equation 7 is. in fact, always satisfied. With these simplifications, the rest of the analysis presented in this section and in Section 4 holds for 0g 01.
B.2 WORST-CLASS OPTIMAL PRIORS
We wish to formally establish that t" (0)//r)) as defined in Equation minimizes both the average and worst-class risks when do x on and Φi x o1. For the first part, as we argued in Section B. these priors must satisfy the assumption in Equation 7 (Equation 6 is trivially satisfied). In this case, we can equivalently rewrite as
σ0 01
14001 1100 11000 11
14001 + 1100 Ho00 11001
>
01 + σ1)
01 to)
(4 M1
#1 M4
M1 [11 pI0
P1 MO
1 +
00 + σ1
TO + 01
00 + σ
0o + +
Defining 2 = ①b (سーui
we arrive at 002 > 01 (1 2). By rearranging and collecting similar which is equivalent
ootor
terms, we get ootai
11 po > (oo + + 01) Ou 01
(11)
since T- is monotonically increasing. Hence, the assumption in Equation can be interpreted as a lower bound on the separation between the two means Po and μن. When this condition holds, 1r (aa//o) /1) minimizes the average statistical risk, as desired.
For the second part, we start by proving the following lemma.
Lemma B.2. Suppose f R j0, 1] is a strictly increasing continuous function and g R [0, a strictly decreasing continuous function, satisfying
[lim---00 00 f(x) = lim g(x) = 0
(12)
lim fxx = lim g(z) =
The solution x² min, max [f({((( (x), is unique and satisfies f(x)) = g(x*).
Proof. Define =(x)= = f(x) g(r). Observe that h(x)) is strictly increasing continuous function
as f and -g are strictly increasing. Also, lim h1(x)) = -1 and lim h(x) 1. From the Intermediate Value Theorem, there exists point x², where h(x*) = 0, i... f(x)) =x(x") Observe that x < x= implies (fxx)(((x)) g(x)} = g(x) and 2 x* implies max, (f()).((x)) = f(x).
Therefore, the objective function decreases up to x² and then increases. Hence, x" is the minimizer of the objective min, max (x)) g(x)).
Since f(t) (t) and g(t) Ro(t) meet the conditions of Lemma B.2 the minimizer of the worst-class risk 7 satisfies (i( R, (1). Equating the risks in Equation then immediately proves the formula for (Equation 4 Finally, note that 1 (an/a1) = i. because of the vanishing logarithm. Therefore, 1" (ap/a) /0i) minimizes both the average and worst-class statistical risks for mixture with ai provided that Equation I] (reformulated assumption in Equation priors do aotor and 61 a0-o; for the given choice of priors) holds.
Finally, note that if o = oi. f = to (0 = (An + 412/2 for the optimal priors do = ~1, as expected.
18

---

Published as a conference paper at ICLR 2025
B.3 THE EFFECT OF SSP
Our setting allows us to adapt and analyze a state-of-the-art baseline pruning algorithm, Self- Supervised Pruning (SSP) (Sorscher al. 2022). Recall that, in Section 4 we adopted variant of SSP that removes samples within a margin i M > 0 of their class means. SSP performs k-means clustering in the embedding space of an ImageNet pre-trained self-supervised model and defines the difficulty of each data point by the cosine distance to its nearest cluster centroid. or prototype. In the case of two univariate Gaussians, this score corresponds to measuring the distance to the closest mean.
We claimed and demonstrated in Figure 3 that the optimal risk minimizer remains around (Øo/ (1) even after pruning. In this section, we make these claims more precise. To this end, note that, for Gaussian variable with variance x², the removed probability mass is
M +
(μ MM
M
+
+
24
1
(13)
Now, for sufficiently small M. we can assume that the average risk minimizer lies within the interval (Po + M1 M) (recall that the average risk minimizer of the original mixture lies between Po and μu owing to the assumption
Equation 7). In this case, the right tail of the easier class and the left tail of the more difficult class (these tails are misclassified for the two classes) are unaffected by pruning and, hence, the average risk after SSP (1) should remain proportional to the original risk R(t). More formally, consider the class-wise risks after SSP:
[40
Ro (t)
(t) 2Φ (M/oo) 00)
2-20 (M/o0)
(-
R1 (1)
(t) - 2 (M/o1)
2 24 (M/o1)
where the denominators are the normalizing factors based on Equation 13
To compute the average
risk (t) % (t) + of R2 (f), shall identify the modified class priors % and of after pruning.
Again, from Equation 13 we obtain
. 0
M
%
Φ0
24
2
2Φ
(14)
global up to normalizing constant that ensures % + 0j = Therefore, the average risk is indeed proportional (t) Φ1 Ri (t) R(tt, desired, so the average risk minimizer after pruning coincides with the original one.
B.4 MULTIVARIATE ISOTROPIC GAUSSIANS
While the analysis of general multivariate Gaussians is beyond the scope of this paper. we show here that in the case of two isotropic multivariate Gaussians (i.e., Var(æry) og for y = 0, 1), the arguments in Sections B.I B.3 apply. In partieular, we demonstrate that this scenario reduces to univariate case.
For simplicity, assume that the means of the two Gaussians are located -44 and μ for some μ E Rd for classes y = 0 and y = 1, respectively, which can always be achieved by prrseric- transformation (translation and/or rotation). Generalizing the linear classifier from Section [ we now consider a hyperplane defined by a vector
Rdd of unit ---momm and threshold E R (=oo), classifying a point x E Rd as 1( x I t). Note that for a fixed vector w E Rd and x ~ N (N,, L have (w w. T 1 ~ (0, 1) linear transformation of the isotropic Gaussian. We can now compute the class risks as
Rí (w, t)
P(w7
x S tly = 1)
= PP
w T E. w T
P <
t w /
==
= Φb
t w
T1
T1
σ1
Ro(w,t) = P(wT T x 2 tly = 0)
w x. + w
I + w T μ
- w
do
du
19

---

Published as a conference paper at ICLR 2025
The average risk given the class priors is thus
R(w,t) )) = Φ1 ("
01 /
+ o (- w
We will now show that for all the risk minimization problems we have considered our theoretical analysis, we can equivalently minimize over a one-dimensional problem. It suffices to observe that for fixed
R. both Ro(w, and R1 (w. t) are minimized when w (with || coincides with μ/|lØli because of monotonicity of ④. This leads to the expressions in Equation | with 11.0 = + /|/l Thus, to compute the minimum of R(w, I) with respect to w and I, we can equivalently minimize ([///////2.1 i) with respect to I. For the worst-class optimization
min max Rı (w, t), R)( |
observe that
min min max(RI(w: t), Ro(w,))) t)) min max (Rx(l/////ll Ro(m/Ilelll.) -=
Therefore, this problem also reduces to the minimization in the univariate case from Section B.2
C DATA PRUNING: FULL RESULTS
Random
GraNd
Dynamic Uncertainty
EL2N
Forgetting
CoreSet
6:900
0.00
20%
20%
0.03
40%
0.00
0.93
6.915
50%
60%
70%
0.75
0.02
10%
0.70
0.4
05
0.6
07
0.000
22%
0.63
0.65
40%
30%
0.00
0.60
50%
60%
70%
035
005
0.00
hull
0.50
3.50
02 0.3
04
05
06
07 D. 0.
1.0
0.0
0.1
0.2
0.3
0.6
03
0.6
0.7
0.8
0.9
0.12 0.14 0.16
8.18 0.70
0.32
0.2
0.60
0.50
0.54
00%
90%
FUll
0.58
0.2 033
0.
05
0. 0.7
0.14
017
50%
60%
0.62
0.69
80%
70%
0.02
0.82
0.6
80%
Tull
0.58
0.6
0.7
0.61
0.5
0.9
1.5
0.58
0.04
0.08
0.03
0.10
0.66 0.88
5,00
0.9
0.94
0.100
0.183
0.170
0.135
dersity
worsi-lass accuracy
Figure 10: The average test performance of various data pruning algorithms against dataset density (fraction of samples remaining after pruning) and metries of class robustness. Top to Bottom: VGG- 16 on CIFAR- 10: VGG- 19 CIFAR-10K ResNet-18 Imngln ResNet-50 ImageNet. All results averaged over 3 random seeds. Error bands represent min/max.
20

---

Published as a conference paper at ICLR 2025
D DROP: FULL RESULTS
Random
Random+ Strategyy
Random DRoP
Strategy
Strakegy DRoP
0.02
Gra
0.90
0.1
0.00
0.92
0.75
0.89
0.83
0.03
Forgetting
0.80
0.93
0.91
0.88
0.7
0.78
0.80
0.075 0.125 6.175
0.33
0.04
dataset density
worst-class accuracy
best-da66 recall - worst-ciass recall
recal
(a) VGG-16 CIFAR-10
0.70
0.65
0.68
0.60
0.66
GraNd
0.64
0.55
0.60
0.62
0.50
0.80
0.70
0.70
0.65
0.68
Forgatting
0.60
0.70
0.65
0.64
0.68
0.62
0.3
D.
accuraty
(b) VGG-19 CIFAR-100
0.5-
0.60
0.4
0.5
Grald
3.80
0.2
0.5
Forgetting
0.6
0.62
0.50
0.58
0.2 0.3 0.4 0.5 0.6 0.7
088
0.9
1.0
0.45
075
100
0.125 0.550 0:200
55
0.60
0.55
0.70
0.75
0.80
0.12
0.14
0.6
0.10
datacel density
worst diss accuracy
Bast-class recall
worst-clss recail
recal std
40%
50%
60%
70%
80%
90%
* RJll
(c) ResNet-18 TinyImageNet
Figure 11: The average test performance of various data pruning protocols against dataset density and measures of class robustness. Random+DroP consistently outperforms all baselines with respect to these additional measures of distributional robustness besides worst-class accuracy reported in Figure 5 too, confirming its effectiveness reducing bias. All results averaged over random seeds, Error bands represent min/max.
21

---

Published as a conference paper at ICLR 2025
Random
Random Strateyyy
Random Handom+DRoP
Stretegy
Stratagy DRoP
0.66
2.66
0.64
0.66
EL.N
3.62
0.80
0.60
0.64
0.08
0.10
0.12
0.800
d50 875 0,900
accuracy
class
50%
60%
70%
80%
90%
*
Full
Figure 12: The average test performance of various data pruning protocols against dataset density and measures of class robustness. ResNet-50 ImageNet with EL2N Strategy. All results averaged over 3 random seeds. Error bands represent min/max.
Table 2: Average and worst-class accuracy (in %) when pruning datasets to d 0.3, 0.5, 0.8 by various methods with or without DRoP. For each column, we boldface the best worst-class accuracy across all three methods. In all cases, DRoP achieves superior worst-class accuracy, especially so when used together with Random. Furthermore, Random+DRoP always improves robustness of full datasets when halving them 0.5).
CIFAR-10
CIFAR- 100
TinyImageNet
Method
DRoP
Acc.
0.3
0.5
0.8
0.3
0.5
0.8
0.3
0.5
0.8
x
avg
86.8
89.6
91.7
59.1
64.8
68.2
47.8
53.6
59.7
worst
75.5
80.1
82.2
24.7
30.1
33.0
8.7
10.0
17.3
Random
avg
86.5
89.5
91.8
57.2
63.7
68.0
45.8
51.3
58.1
worst
82.1
84.9
34.3
45.3
40.3
16.2
20.0
16.9
x
avg
87.6
91.9
92.7
40.4
58.5
69.3
30.4
48.7
60.0
GraNd
worst
77.0
83.6
85.1
12.3
31.7
39.0
1.0
12.4
15.8
avg
85.9
91.6
92.6
35.5
55.9
68.7
29.9
48.6
59.4
worst
78.0
83.8
81.7
15.0
30.1
36.3
4.7
10.7
16.7
87.2
91.5
92.4
56.1
65.2
69.4
25.9
48, 1
60.2
Forgetting avg
worst
79.1
84.2
85.0
26.3
34.0
33.3
1.0
9.5
15.2
85.7
91.4
92.6
58.2
64.8
68.9
22.1
42.8
58.2
worst
78.7
84.6
85.1
32.0
34.0
38.3
1.3
12.7
18.0
X
avg
87.2
91..0
92.3
51.9
62.8
69.0
34.7
50.1
60.0
Average
worst
77.2
82.6
84.1
21.1
31.9
3.6
16.1
avg
86.0
90.8
92.3
50.3
61.5
68.5
32.6
47.6
58.6
worst
79.6
84.8
84.9
27.1
36.5
38.3
7.4
14.5
17.2
Full Dataset
avg
92.8
70.0
61.6
worst
84.8
35.8
17.8
Random
Random StrategyO
DRte
Strategy
Strategy DRopP
0.76
overige acCUaSy
0.6
0.72
0.5
0.75
0.70
0.70
0.2 0.3 04 0.5 0.6 0.7 0.8
0.
1.0
0.35
0.40
0.45
0.45
0.50
0.55
0.60
0.10
0.32
0.14
0.36
dataset density
ward--iwss accurwcy
best-clans necwl warsh-llass recall
recall skd
40%
50%
60%
70%
80%
90%
Figure 13: The average test performance of various data pruning protocols against datasct density and measures of class robustness. Wide ResNet- 101 CIFAR- 100 with EL2N as Strategy. All results averaged over 3 random soeds. Error bands represent min/max.
22

---

Published as a conference paper at ICLR 2025
Larger Models. Some previous studies argue that larger models enjoy better distributional robust- ness compared to smaller ones (Andreassen et al. 2022 Feuer et al. 2023). To see how model size could affect the applicability and effectiveness of DRoP, 2016) with expansion factor of 2 on CIFAR-100, which is an unusually high caliber architecture for such relatively small dataset. In fact, had to remove a few early downsampling layers from the original implementation of this model to account for the resolution of CIFAR-100. On full dataset, Wide-ResNet-100 101 with 125M parameters achieves 4% higher average and almost 10% higher worst-class accuracy compared to VGG-19 with just 20M parameters (see Figures 11p and 13). Even in this more competitive setup, Random+DRoP continues to show considerable improvement in all robustness metrics over other pruning baselines as well as the full dataset (Figure 13
train Wide ResNet--NN1 (Zagoruyko
E DRoP ABLATION: PRE-TRAINING LENGTH
As mentioned in Section 5 DRoP requires a pre-trained query model to compute class-wise validation errors. All existing data pruning methods also require pre-training compute sample-wise pruning scores. However, as Paul et al. (2021) shows, it suffices to run optimization for as little as 10 epochs to achieve accurate enough scores for EL2N. In our experimental work, we used 10% of the full training cycle to train the query model for EL2N, GraNd, and DRoP, which is at least the recommended 10 epochs for all model-dataset pairs tested in this study. this section, we verify that DRoP reaches its full performance with this much pre- wre-training and, hence, it does not introduce any additional computational overhead already needed by the current data pruning algorithms. Indeed, Figure [4] shows that the downstream performance of DRoP saturates at 10 pre-training epochs or earlier, with lower pruning ratios allowing even less training than more aggressive ones.
d = 0.4
0.5
d 0.8
50
0.70
everoge sicundC
0.60
35
30
0.66
20
0.64
10
0.62
0.60
0.30
0.6
0.10
0.12
30
average accuracy
0.60
25
0.55
15
8
0.50
0.45
0.125
0.225
0.65 best-class recail wonst- recall
0.13
0.34
worsb-class recail
Figure 14: Final model performance after DRoP with different lengths of the query model training. Top: VGG-19 on CIFAR-100, Bottom: ResNer-18 on TinylmageNet. Repeated over random seeds and three dataset densities (d = 0.4, 0.5, 0.8). In our study, we use 16 and 9 pre-training epochs for VGG-19 on CIFAR-100 and 18 on TinyImageNet (10% of the full training cycle). respectively.
F RESNET-50 ON WATERBIRDS
As stated in the caption of Figure Sagawa et al. (220)) compute average accuracy on Waterbirds as sum of group accuracy weighted by the group proportions in the training dataset, which is highly skewed towards majority classes (landbird+forest and waterbird+ see Figure [[p). We adhere to this strategy in Figure to ensure fair comparison with prior algorithms. Note, however, that this places DRoP at a disadvantage since it is designed to prune easier groups more aggressively and protect underperforming minority groups (see left barplots for each method in Figure T5p). Ultimately,
23

---

Published as a conference paper at ICLR 2025
this leads to rapid degradation of the average aceuracy at low dataset densities, which observed in Figure (left. In Figure [3] (bottom) we switch to a more standard accuracy computation as fraction of conrectly classified samples regardless of group identities. In this framework, Random+DRoP remains strong relative to other methods. We should note that the test split is also slightly imbalanced with aandbirdffores and landbird+ water represented by 2255 images each, and waterbird+forest and snterbird+water---e 642.
Random
EL2N
1.2N-DRPP
Random+DRoP
0.975
250
1.0
0.950
300
350
0.925
0.6
0.900
0.280
0.2
0.1
0.3
0.5
0.7
0.9
1.0
3500
3.0
0.90
150
3000
1.5
0.0
0.85
0.90
1000
0.75
gvertidd
0.3
0.7
dbt0set dons
in
(a)
(b)
Figure 15: (a): Average test accuracy against dataset density across different pruning methods. Top: Average
accuracy computed as a weighted sum of group-wise accuracies according to training proportions. Bottom: Average accuracy is computed as fraction of correctly classified samples of the test sct. (b): Group sizes and group accuracy for all four groups in Waterbirds d = 0.1 for different pruning methods, as well as on the full dataset (bottom right barplots).
G
DATA PRUNING: CLASS- WISE ANALYSIS
Grahds
Forgetting
1.00
2.95
0.85
3.0
6.5
0.0
8.5
1.0
0.5
1.0
0.5
3.5
1.0
9.0
0.5
01
0.
Dog. carrlation 0.32
Sxg. COTT
0.5
10
0.0
2vg COmTECSOT
[Agp correlatios
[Pg.
05
1.0
EWQ COmTiaatinn 1.04
[Ng. DOTTEASION -0.22
3v9 CorTRESOO -.41
PKg ComTiction: -2.1
D42 CONTEDAIOO -0.15
370 COT 00 0Z
1.5
68
1.0
denstty
Figure 16: Full dataset test accuracy against density, across all classes, after pruning with different methods. We use the full dataset accuracy to capture the baseline "difficulty" of each class. On each plot, we report the correlation coefficient between these two quantities across classes, averaged over data density levels (0.1, 0.3, 0.5, 0.7, 0.9). Top: VGG-16 on CIFAR- Center: VGG-19 on CIFAR- 100, Bottom: ResNet- 18 TinylmageNet. Experiments repeated over random seeds.
24

---

Published as a conference paper at ICLR 2025
Dynamic Uncerbaity
Forpetting
CoreSet
0.4
10%:
1.0
0.0
1ON
PER S8C
1.
30
SPC JYYCC 13
0.7
0.1
1.0
0.0
0.6
0.2
1.00
160
2003
80
320
160
2000
40
80
120
160
200
80
130
160
2000
40
320
360
2300
60
1201
160
200
(355 Indxx Isonted by density!
Figure 17: Sorted class densities in the training dataset pruned by various algorithms to different density levels. On each plot, report the minimum number of samples per class (SPC) 10% dataset density. Top: VGG-16 on CIFAR-10, Center: VGG-19 on CIFAR-100, Bottom: ResNet-18 on TinllmaeNNet. Repeated over 3 random seeds.
25


Reviews:
[{"review_id": "B65uuMiILW", "reviewer": "Reviewer_JvuS", "summary": "This paper presents a dataset pruning method that helps the models trained on the pruned datasets be more fair in their class accuracies. The proposed method is called \"DRoP\", and it selects class-specific pruning ratios based on the error rates, improving performance on underrepresented classes.\nThe emprical evaluations are performed using VGGs, ResNets on CIFAR-10, CIFAR-100, TinyImageNet, and WaterBirds datasets.\nMost of the evaluations are very favourable for the proposed method.", "questions": "Q1- In Figure 9 (leftmost plot), when dataset sparisty is more than 50% (density less than 0.5) why is the proposed DRoP significantly less stable than all the other methods? Some explaination for this would be very helpful.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 1, "presentation": 1, "contribution": 3, "strengths": "The idea of DRoP is quite interesting and to the best of my understanding very novel. \n\nThe method seems to be simple and straightforward.\n\nThe paper provides detailed theoritical insights and this helped understand the method.", "weaknesses": "The framing of the paper is currently unclear to me. \n(W1) Firstly, It would significantly help in include \"Dataset Pruning\" rather than just \"Pruning\" in the title of the paper, because given the dominence of \"model pruning\" methods, it is very understandable that one might get confused by the title. One suggestion would be DRoPD (.... Pruning Datasets).\n\n(W2) Secondly, the problem of \"classification bias\" is never really defined in the paper, that is, it is never explained what is meant by \"classification bias\" and why is it a problem and how datasets affect this so-called \"classification bias\".\n\n(W3) Next, the use of the word \"Robustness\" seems incorrect in this context. It is not really \"robustness\" being evaluated but class-wise fairness. Might be prudent to frame the paper differently around this.\n\n(W4) Lastly, the evaluations are limited to non-robust small models like VGG-16, VGG-19, ResNets up to ResNet50. However, it has been seen that large models like ResNet101, ConvNeXt-B onwards, and ViT based large models are more robust to distribution changes. They might be more robust to underrepresentation of classes as well, or might not show gains when using the proposed \"DRoP\" method. Thus having experimental evaluations with these models would help better understand the applicability of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xMDu3HQknb", "reviewer": "Reviewer_XsKC", "summary": "The paper studies the effect of dataset pruning on classification bias for several existing methods using the metrics of worst-class accuracy, difference between best and worst class accuracy, and standard deviation. Due to the purported worsening of worst-class accuracy at the cost of better average performance, the paper proposes random pruning based on error rates on a holdout set which is motivated by a theoretical analysis of a simple 1-D mixture of two Gaussians. Experimental analysis is conducted on CIFAR10, CIFAR100, TinyImageNet and ImageNet datasets for the various settings.", "questions": "1. in Figure 6, right, why does a higher negative correlation of class accuracy to class density indicate more robustness? Ideally class accuracy should have zero correlation with class density for a model invariant to class size.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The presented theoretical analysis on the GMM for minimizing worst-case statistical risk and average risk is good and aligns with the proposed pruning method.\n2. The experimental analysis is comprehensive and convincing.", "weaknesses": "1. While the Random+DRoP method does reduce classification bias compared to existing dataset pruning strategies, it generally seems to do slightly worse on average accuracy (Figure 5). The combined Strategy+DRoP seems to mitigate this a bit but the trend is not clear.\n2. The experiments do not always indicate such a clear and striking trend towards robustness as the authors suggest. Some aggregate scalar numbers might be beneficial to understand which strategy does better across all data densities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gzIE6nzLL5", "reviewer": "Reviewer_kAs6", "summary": "The paper studies classification bias in data pruning. It first provides a systematic study of the existing methods and shows that they are all biased to some extent. To this end, the authors propose DRoP, a distributionally robust approach for pruning that preserves good robustness against classification bias. Their method achieves state-of-the-art robustness performance when combined with random subsampling across different settings.", "questions": "- It is quite impressive that Random + DRoP outperforms other methods when the overall dataset density is low (e.g. in Figure 5). But do the authors have an idea why in many cases, it gives inferior worst-class performance when the overall dataset density is high? For example, for ResNet18 + TinyImageNet at 90% density, Random + DRoP is the second worst performance (only better than Forgetting) in terms of the worst-class performance in Figure 5.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method DRoP is quite concise, which is highly appreciated since it can potentially increase the accessibility and applicability of the method beyond the current scope.\n- Despite its simplicity, DRoP + random sampling achieves superior performance across various settings.\n- The paper conducts a systematic evaluation of the existing data pruning methods and reveals a valuable conclusion that none of the existing methods actually has good robustness in terms of classification bias.\n- The evaluation covers a wide range of scenarios beyond the classic data pruning, which enhances the importance of the method as it could lead to potential influence that extends beyond traditional use cases.\n- The discussion of the limitations and future work is sound and makes sense.", "weaknesses": "- The flow of the paper can be improved. At the moment it feels that the related work is scattered throughout the paper, which disrupts readability and makes it challenging to grasp. It would be clearer to bring all of the related work into one section and then refer back to it when needed in the experiment section. For example, the authors may group related work into categories like \"Data Pruning Methods\", \"Robustness and Fairness\", \"Long-Tailed Recognition Techniques\", and \"Robust Data Pruning Methods\", and place this consolidated section after the introduction.\n\n- Since there is no standalone related work section, it is difficult to assess the novelty of the proposed method and the relation with existing data pruning techniques. It would be great to also include a brief discussion of the most recent advancements in data pruning and how DRoP is similar to/different from them.\n\n- The robustness evaluation on imbalanced datasets and group distributional robustness is not compared with the methods from the corresponding fields. It would be great if the performance of some task-specific methods is also compared in Figures 8 and 9, which helps to better assess the method's relative performance in context. For example, [1,2] for \"imbalanced datasets\" and [3,4] for \"group distributional robustness\"\n\n[1] Kang, Bingyi, et al. \"Decoupling representation and classifier for long-tailed recognition.\" ICLR 2020.\n\n[2] Cao, Kaidi, et al. \"Learning imbalanced datasets with label-distribution-aware margin loss.\" NeurIPs 2019.\n\n[3] Liu, Evan Z., et al. \"Just train twice: Improving group robustness without training group information.\" ICML 2021.\n\n[4] Sagawa, Shiori, et al. \"Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.\" ICLR 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)