Coreset-Driven Re-Labeling: Tackling Noisy Annotations with Noise-Free Gradients
Authors: Saumyaranjan Mohanty
Konda Reddy Mopuri
Venue: Accepted by TMLR ()

Abstract: Large-scale datasets invariably contain annotation noise. Re-labeling methods have been developed to handle annotation noise in large-scale datasets. Though various methodologies to alleviate annotation noise have been developed, these are particularly time-consuming and computationally intensive. The requirement of high computational power and longer time duration can be drastically reduced by selecting a representative coreset. In this work, we adapt a noise-free gradient-based coreset selection method towards re-labeling applications for noisy datasets with erroneous labels. We introduce ‘confidence score’ to the coreset selection method to cater for the presence of noisy labels. Through extensive evaluation over CIFAR-100N, Web Vision, and ImageNet-1K Datasets, we demonstrate that our method outperforms the SOTA coreset selection for re-labeling methods (DivideMix and SOP+). We have provided the codebase at URL.

Published in Transactions on Machine Learning Research (10/2025)
Coreset-Driven Re-Labeling: Tackling Noisy Annotations with Noise-Free Gradients
Saumyaranjan Mohanty
ai23resch04001 @att000@ithhh.....
Deportment of Artificial Intelligence Indian Institute of Technology Hyderobud
Konda Reddy Mopuri
Armopuri@ai.iict acth. ac.in
Deportment of Artificial Intelligence Indian Institute of Technology Hyderobad
Reviewed on OpenReview: https: openrevieew. net/ htrt/frmm?? id= 7x78vb2047
Abstract
Large-scale datasets invariably contain annotation noise. Re-labeling methods have been developed to handle annotation noise large-scale datasets. Though various methodologies to alleviate annotation noise have been developed, these are particularly time-consuming and computationally intensive. The requirement of high computational power and longer time duration can be drastically reduced by selecting representative coreset. In this work, adapt noise-free gradient-based coreset selection method towards re-labeling applications for noisy datasets with erroncous labels. We introduce 'confidence score' to the coresct selection method to cater for the presence of noisy labels. Through exlensive evaluation over CIFAR-10ON, Web Vision, and ImageNet- 1K Datasets, we demonstrate that our method outperforms the SOTA coresct selection for re-labeling methods (DivideMix and SOP+). We have provided the codebase at URL
1
Introduction
The rise in the size and complexity of modern datasets and deep learning models has resulted in the use of extensive computational resources. Modern deep-learning tasks depend upon very large datasets to achieve State-OE-Tho-Art (SOTA) performance (Pooladzandi et al. 2022 Yang et al. 2023). Supervised learning tasks such as image classification require labels for each image the training and Lest sels. Dataset annola- tion tion/labeling is usually done through human annotators. The human factor invariably leads to noisy labels in the datascts. In turn, these noisy labels pose significant challonge to training deep lcarning models and
severely degrade the generalization performance of deep neural networks (Song et al. 2022). Some of the
popular approaches towards dealing with noisy labels involve the development of robust architectures (iao
et al. 2015), unbiased estimators and weighted loss functions @Liu & Tao 2016
Natarajan et al. 2013
robust regularizers (Shorten Khoshgoftaar 2019). ) loss correction (Liu & Guo 2020 Patrini et al.
2017
sample selection-aided (Han al. 2018b) Jiang al., 2018 Yu et al. 2019), etc.
Re-labeling (Song et al. 2019) is family of robust training methods to identify wrong labels and correct these
labels during training time. Consistency training methods (Xie et al. 2020a) apply regularization to model prediction to ensure that the model is invariant to sample noise. These self-consistency regularisations have been shown to achieve SOTA performance on the re-labeling tasks. Li et al. (2020b) introduced DivideMix, a framework for learning with noisy labels that leverages semi-supervised learning techniques. Sparse Over- Parameterization (Liu et al.) 2022a) proposes a principled approach for robust training of deep learning models for noisy training labels. This method assumes label and model noise are sparse and trains separate noise from data. Due to various factors such as augmentations and multiple backbones, these methods require high computational time.

---

Published in Transactions on Machine Learning Research (10/2025)
Original Dataset with label noise
Re-labeling method
Label corrected data
Test dataset
Training
Comparable
Coreset selection
Re-labeling method
Label corrected coreset
Test dataset
Coreset
Method
Training
Figure 1: Block diagram of the entire process. The re-labeling method is used as a black-box. We have empirically shown that selection a much smaller corcset through our proposed mothod improves training speed while not significantly impacting the generalization performance of the output of the re-labeling method over the coreset.
To reduce the computational burden, Park et al. (2023) have introduced utilization of coreset with re-labeling for training a deep neural network over a dataset with noisy labels. They have shown that existing data- pruning/core coresct selection algorithms don't work well with re-labeling task because they don't take re-labeling into account. They have proposed a method called PruneçRel, which selects subset that maximizes the re-labeling capability.
Coresets aim to address the extensive compute and storage requirements for training complex deep learn- ing models (CNNs, transformers, foundation models, etc.) on large-scale datasets. Training these complex models on large datasets requires very high computational power and is very time-consuming as well (Dong et al. 2021b. Hyperparameter tuning of these complex deep-learning models is also computationally inten- sive Zoph al. 2018) and requires a lot of time to learn a set of hyperparameters that optimize the model for the task at hand. For instance, the storage requirement for large-scale visual benchmark datasets like ImageNet-22k (Deng et al.) 2009) is in the order of TBs (Xia et al. 2023 Furthermore, the advent of complex models and very large datasets has led to exponential rise in carbon footprint (tec Killansetty
et al. 2021c
Coresct Selection aims to mitigate these issues by finding the most representative subset of the original larger dataset. In particular, coreset selection methods atlempt to approximate the learning characteristics of the complete dataset (e.g., gradient information) (Feldman) 2020). A representative coreset with the cardinality of a fraction of the entire dataset would drastically reduce training duration and computational requirements for end-to-end training while delivering the desired generalization performance. [Sorscher et al. (2022) have shown that the discovery of good data-pruning metrics may provide a viable path forward to substantially improved neural scaling laws, thereby reducing the resource cost of modern deep learning.
To study the effectiveness of the coresct approach for datasets with noisy labels, we begin with analysis of CIFAR-10ON (Wei et al., 2022), a dataset specifically created to underscore the presence noisy labels due human annotation errors. Figure 2 depicts a bar plot of the number of mislabeled samples in class 'Apple' and the original class they belong to. It can be seen that, of all the mislabeled samples, 25% belong to the class 'pear' and 21% belong to the class 'sweet pepper', two of the visually similar classes.
This similarity has motivated us to utilize an intuitive coresct selection method based on 'Noisc-free Loss Gradients' introduced by Mohanty el al.] (2025) and apply this to the task re-labeling on a noisy dataset. This approach uses the similarity of loss gradients to compose the coreset. We have adapted their approach by introducing confidence score-based weights calculate gradient similarities. Through extensive experi- mentation over various complex, noisy datasets, we have shown that the coresct selected through our method outperforms all other coreset selection methods the re-labeling model training task (including Prane/ Rel,
2

---

Published in Transactions on Machine Learning Research (10/2025)
Fraction of mislabeled samples in class' 'ppple' and the original class which they belong
0.25
0.16
0.1
0.06
Poar
Sweet
Orange
Lobeter
Bowl
pepper
Figure 2: Fraction of mislabeled samples in class 'Apple' and the original class they belong to. The top two mislabeled classes are 'Pear' and 'Sweet Pepper', which are visually similar and can be mistaken by human annotator.
which is specifically designed for this task). The re-labeling method is treated as a black-box, with compar- ison being carried out between the performance of a model trained on label-corrected data from the original dataset and model trained on label-corrected coreset from coreset composed from the original dataset through our method. Figure shows the block diagram of the process.
In summaary the major contributions of our work are as follows.
Repurpose an existing gradient similarity- basod coreset selection method achieve generalization in the presence of noise (via re-labeling)-
Leverage model confidence as proxy for reliability of the samples, enhancing the robustness of the selected coreset.
Thoroughly evaluate the proposed method over multiple noisy datasets for the re-labeling task on image classification.
2
Related Works
In this section, we briefly discuss the prominent works in the field of coresct selection and re-labeling.
2.1 Re-labeling
The massive labeled datasets are the backbone of the impressive performance of deep neural networks. But using crowd-sourcing marketplaces such as Amazon Mechanical Turk (Turk) to reduce labeling cost has resulted in unreliable labels (Scott et al. 2013 These noisy labels lead to poor generalizability on the test dataset (Zhang et al. 2017). Song et al.
20200 has presented a detailed survey of deep learning approaches with noisy labels. The ratio corrupted labels in real-world datasets ranges from 8.0% to 38.5%.
Goldberger et al. GGolbbegger Ben-Reuven 2017) introducod noise adaptation layer into the deep neural network, additional softmax layer that explicitly models the correlation between correct labels and noisy ones. A human-assisted approach called "Masking" was introduced by Han et al. (2018a) for estimating the noise transition matrix. This method incorporates a structurc-awart model, making the transition matrix estimation more effective. A quality-embedding model was proposed by Yao et al. 22019 to improve the deep learning model's ability to learn from datasets with unreliable annotations.
While the methods mentioned in the previous paragraph explored architecture modifications, Ghosh et al. (2017) proposed a robust loss function mitigate the effects of label noise in deep learning. Hendrycks al. 2018 proposed that small subset of trusted training data would be able to achieve substantial robustness
3

---

Published in Transactions on Machine Learning Research (10/2025)
performance gain noisy labels. A normalized loss function was introduced by Ma et al. (2200) to make the loss function robust to noisy labels. Termed as "Active Passive Loss", it combines two robust loss functions that mutually boost each other.
Zhou et al. (2021) introduced a dynamic curriculum learning approach, which dynamically transitions from learning clean labels to self-supervised learning with pseudo labels. The approach improves label selection and generalisation by tracking training dynamics across multiple steps and augmentations. These 'Re-labeling' approaches try to recover correct labels from noisy labels through hcuristic rule (Song et al., 2019 Two recent SOTA works on re-labeling are DivideMix (Li et al. 2020a) and SOP+ (Liu et al, 2022b). DivideMix simultancously trains two divergent neural networks using dataset division from the other network. SOP+ exploits the sparsity of the label noise by modelling the label noise via another sparse over- peeemaaramttiiitioon term. All these approaches utilize self- consisenny loss to leverage strong augmentation to implicitly correct noisy labels. Due to the introduction of additional architectures and the requirement of strong data augmentations, re-labeling models require more computation power and exccution time (Chen et al. 2019
2.2
Coreset selection
In this sub-section, we briefly review some influential works selecting representative subsets to achieve better generalization performance. Various works have proposed different approaches to selecting coreset.
Cncertainty-based methods propose that samples with lower confidence scores on a model are more impactful for model generalization than samples with higher confidence. Selection via Proxy method |Coleman et al. 2020) uses a smaller proxy model instead of full-scale target models. These smaller and faster models provide useful signals for coresct selection based upon uncortainty and representativenoss. Other metrics used to calculate sample uncertainty are least confidence (Shen et al. 2018 and entropy (Settles 2012).
poss//Eror-baset methods select coresets based each sample's contribution towards the loss function (2019a) to select a coresct composed during model training. Catastrophic forgetting was used by Toneva et al.
of tabgettable samples. If the model correctly classifies a sample through multiple epochs, il can be removed from the dataset with minimal performance drop. Two metrics called GRAND and EL2N (Paul et al. 2023) measure the average contribution from each sample. This helps in pruning significant fractions of training data without sacrificing test accuracy.
Decision boundary-based methods choose the data points closest to the decision boundary as the coresct. Adversarial Deepfool (Ducoffe & Precioso 2018) introduces closeness to the decision boundary by introducing perturbations to the samples that result in a change in label predictions. Contrastive Active Learning (Mar- gatina et al. 2021) measures closeness through divergence of predictive likelihood.
Geometry-based methods select coresets by removing data points clustered in the feature space. Herd- ing (Chen et al. 2010) selects a coreset by greedily adding samples to the coreset that minimize the dis- tance between the coreset center and the dataset center in the feature space. K-Center Greedy approxima- tion (Sener& & Savarese 2018) attempts to solve the minimax facility location problem to select coresets from large dataset such That the maximum distance between points in the non-coreset and their closest point in the coreset is minimized.
Gradient Matching based methods such as CRAIG (Mirzasoleiman et al., 2020 and GRADMATCH Kiil- lamsetty et al., (2021a) utilize gradients produced by the full training dataset and select a coreset whose weighted gradients would result in the minimal difference. RETRIEVE (Killamsetty et al. 2021c) and GLISTER (Killamsetly et al. (2021b) pose the coreset selection problem as a bi-level optimization problem, which treats the selection of subset the outer objective and optimization of model parameters as the inner objective. An intuitive coresct selection method termed 'Noiso-free Loss Gradients' based on the sim- ilarity of loss gradients was proposed by Mohanty et al. (005),, which composes a coreset of samples that have the maximum number of neighbours with higher cosine similarity among their gradients.
Submodular functions [[yer Bilmes 2013) have been utilized to measure diversity and information and have been incorporated for coreset selection by various methods such as graph cut and facility location.
Moderate method (Xia et. al.) 2023) calculates the distance between the hidden representation of samples
4

---

Published in Transactions on Machine Learning Research (10/2025)
and the representational class centres. Based on these Euclidean distances, they rank the data points in ascending order and select the data points closest to the distance median coreset. An influence function- based iterative method was proposed by Yang et al. (2023), which calculates the influence of each sample on the model's parameter training and composes the coreset with samples that have the most influence.
All these works have not considered the label noise inherent in larger datasets. Prune4Rel (Park et al. 2023) proposes a data pruning algorithm based on the maximization of total neighbourhood confidence of all training examples to ensure the method selects the samples that maximize the re-labeling accuracy, thereby resulting in better generalization performance.
3
Methodology
3.1 Detailed Methodology
This section provides detailed description of the proposed approach. We adapt the concept of 'Noise-free Loss Gradients' (Mohanty al. 2025) and introduce additional per-sample confidence score top of the existing coreset selection method to capture the representation ability of the samples of the dataset. Confidence score refers to the probability assigned by the model to each sample of belonging to given class.
As we have brought out in Section 1 major annotation mistakes happen within visually similar classes. Table [ presents a few classes from the CIFAR-10ON dataset, with the two dominant classes to which most mislabeled samples belong. This analysis shows that most mislabeling occurs because these classes are visually similar to the incorrect class. A model trained confidence to mislabeled images that are visually similar to the incorrect class than to those that are not (Figure B) Therefore, if we can select a representative coreset that includes samples belonging to these dominant mislabeled classes, they would have a higher chance of getting corrected by the re-labeling models.
noisy dataset tends to assign higher
mislabeled class
pear
0.8
sweet_pepper
sweet_ pepper
orange
Model Score
0.6
0.4
0.2
0.0
Figure 3: Confidence score assigned by a model mislabeled samples belonging to class 'Apple. The bar plot shows the top 20 mislabeled samples and their confidence score in descending order. The samples are color-coded by the real class they belong to. As we can notice, the model assigns higher confidence scores to mislabeled samples that are visually similar to 'apple..
Table 2 provides the notation used throughout the paper.
V is the original training dataset with noisy labels. A re-labeling method trics to correct the noisy labels through minimizing the re-labeling loss function Lrelabeling- As the re-labeling methods are quite computa- tion intensive, our objective is to select a coreset S from V, such that, the classification performance on the coreset after re-labeling closely matches the performance achieved by re-labeling the entire dataset.
Let (Iz, Vs) V be the original image and label pair and (Is, vi be the corresponding ground-true label. Let (F,, (s) E S be the coreset composed from V and (Zi, v/) be the corresponding ground-truth label. The
5

---

Published in Transactions on Machine Learning Research (10/2025)
Table I: Analysis of the top two classes to which most mislabeled examples belong, For samples belonging the class Aqqariumm Fish', most of the mislabeled samples belong to the "Trout' and 'Flatfish' classes, which are inherently similar and can be mistaken by human annotator.
Class name
Top two mislabeled classes
Apple
Pear(25%), Sweet pepper(21%) T35ut(35%) Flatfish(12%) (28)) Table(22%) M3t0 Lawn mower(22%) H3usc(35%), Sky5c
Aquarium fish
Bed
Bicycle
Castle
Table 2: Notation
Symbol
Description
{ v+)
Large Training Dataset. Coreset of V (target)
Parameters of the classifier
A
Strong data angmentation
Liee
Cross entropy loss function for classification
Lrelabelling
Loss function for the re-labeling method
Threshold on the gradient similarity for neighbourhood identification
V
Training data belonging to class
gz.
Loss gradients computed for data sample (r,, y:) at the last fully connctta((clasificattio)) layer norm of 4 norm of
|
M
Cardinality set Indicator function
<..->
Dot product operator
re-labeling loss function with self self-consistency regularization
Xie et al. 2020b on coreset given by:
Lreetabling (Setabeling((;0,A)) =
Σ (B.)))
+
Σ ZES
Leeg (ā; 0, 0,
(1)
where A is a strong data augmentation, Ca(x) provides the prediction confidence score, and ô is the threshold for prediction to switch from wrong label to correct label upon administration of strong augmentation A. Let. S denotes the label corrected coreset. Our objective is to select a coreset S, that will maximize performance of a classification model trained on label corrected coreset S*.
"Noise-free Loss Gradients' (Mohanty et al. 2025 measures representation ability each sample (z,, v.) in the dataset V as:
[(x vi) Fe
Σ
p(re, Vi. Uz Uj. 0)
(2)
(K jti
Where p is the normalized cosine similarity between gradients of two samples given in Eq.
3
p(z;, Vi, 2,, Uj. = [ Illl II
(3)
A nearest neighbor search algorithm identifies the number of samples within specified radius of each sample, that is, samples whose similarity exceeds given threshold &, and assigns it as its score. Samples are then ranked according to their aggregated scores across multiple checkpoints. For a given class e, the top-ranked samples (zg, yx) are selected based on their scores.
(Ik. wk) arg max
ΣΣ JRL
11 (p(ss, 24, b)
(4)
(2.0:) V.

---

Published in Transactions on Machine Learning Research (10/2025)
We introduce weighted representation ability based on sample confidence score. For the model parameter O at given checkpoint, confidence score of sample belonging to class e is given as:
Co(x:) (z))(c
where (24)) the lelenh output of the model parametrized by 0 and the number of classes in the dataset.
We modify Eq.
(Ik: Ukb) = arg max Hloo 2,, 0) > Φ)
(6)
(m:0))e Ve 8 f#1
Using this weighted measure, we compute the suitability of every sample in become an element of the coreset in class-wise manner. Essentially, this translates to sorting the datasel samples in each class in decreasing order of this weighted measure and composing the coreset of desired cardinality.
Algorithm
! presents our approach more formally.
Algorithm Modified Noiso-free Gradients for Re-labeling algorithm
Require: Train set: V; Total epochs: T; number classes: C; number of coreset images per class: N;
Model checkpoint at all initial epochs up to T
Ensure: Coreset $
1: for class in 1,..., do for (Iz, V.) E Ve do for epochs in 1,..., T do 4: compute - Co, (Ix))
5: end for
((x)) = Σ Σ
Co. (p(0,)))(( v)
Oe (o).8))EVeJst
Store f(x;)
end for end for
10: S = 0
11: for class è in 1...., C do
12: S fex
> Descending order
13: end for
3.2 Comparison with Prune4Rel
Mechanism: Prune4Rel maximizes total neighborhood confidence to select coresets, focusing on local label consistency. Our mothod adapts the noise-free gradient framework for coresct selection, weighted by per- sample confidence scores.
Confidence definition: Prune4Rel uses neighborhood-baae confidence, while we use softmax confidence (i.c., model confidence on each sample).
4 Experimentation
4.1 Experimental setup
Applications. We have evaluated our coreset selection methodology on re-labeling application for image classification over datasets with noisy annotations.
Datasets. We have evaluated the effectiveness of our method three benchmark datasets.

---

Published in Transactions on Machine Learning Research (10/2025)
CIFAR-10ON (Wei et al. 2022 is the CIFAR-100 dataset with human-annotated real-world noisy labels collected from Amazon Mechanical Turk. This specialised dataset incorporates human-annoiated real-world noisy labels. It consists of 50, 000 colour images of dimension 32 x 32 x 3 from 100 different classes, each class having 500 images.
WebVision [Li al. 2017) contains 2.4M/ images crawled from the Web using the 1,000 concepts ImageNet-1K (Deng al., 2009). Similar to prior works (Chen et al. 2019 Park al. 2023 we use the mini-Web Vision version consisting of the first 50 classes of the Google image subset with approximately 66, 000 training images.
Following the approach in Park et al. 2023), we introduced 20% asymmetric noise to the ImageNet-lk (Deng et al. 2009) dataset. The noisy datasel is constructed by randomly selecting 20% of images from each class c and flipping their labels to c+ is a subset the larger dataset ImageNet, an image dataset organised according to the WordNet hicrarchy. ImageNct-1K consists of 1000 classes, with 1, 281, 167 training images
and 50, 000 validation images.
Re-labeling methods. DivideMix Li et al. 2020a and SOP+ (Lu et al. 2022b used as the re-labeling methods.
Coreset. Methods. We have carried out experimentation with various leading coreset selection methods including k- &-CenterGreedy (Sener & Savarese 2018), GraNd Pauul et al.) 2021), Forgetting (Toneva et al, 2019b), SmallLoss (Jiang et al.] 2018), Moderate (Xia et al.
2023 and Prune4Rel (Park et al. 2023). We for conducting our experiments.
have utilised the publicly available codebase of the Prune4Rel method
Classifier. Following prior works Li ct al. 2020a Liu ct al.
2022b Park ct al.
have used JeeeptionResNNelV (Szegedy el al.,
2023
PreAct ResNet-18 (He et al. 2016b) architecture for CIFAR- 100N, 2016) architecture for WebVision and ResNet-50 (He et al.] 2016a) architecture for ImageNet- 1K dataset.
Implementation. Experiments for CIFAR-10ON are carried out for five individual runs with different random seeds. Experiments for ImageNet-1K were carried out for two individual runs due to relatively higher computational requirements. We have used the same training strategy (optimiser, weight decay, batch size and other hyperparamcters) utilised by Prune4Rel.
syper-parameter settings. Table 3 labulates various hyperparameter settings used in the experimenta- tion for the ease of reproducibility.
Table 3: Hyper-paraneter values used across multiple datasets.
Settings
COFAR-10ON
Visiin
ILSVRC
Epochs
300
100
50
Optimizer
SGD
SGD
SGD
Momentum
0.9
0.9
0.9
Weight Decay
0.0005
0.0005
0.0005
Batch Size
128
32
64
Learning Rate
0.02
0.02
0.02
Codebase. Codebase for implementing method is provided at URL
4.2 Results CIFAR-100N
Table tabulates results obtained on the CIFAR-100N dataset with the DivideMix Re-labeling method. The first column lists various coreset selection sizes as a fraction of the full dataset. Similarly, performance results obtained on CIFAR-10ON dataset with SOP Re-labeling method given Table 5
https://github. www.hhuu...om/.aitettee htta://aaiiidddallaanneeeeaaa
8

---

Published in Transactions on Machine Learning Research (10/2025)
Table 4: Performance comparison of various coreset selection methods on CIFAR-10ON dataset with Di- videMix Re-labeling method at various selection fractions of the original dataset. The best results are shown in bold. For selection sizes ranging from 5% 50%, our proposed method able beat Prume4Rel by average 4.22%.
I-CenterGreedy
GraNd
Porgetting
SmallLoss
Uniform
Prune-Rel
Moderate
Ours
0.05
13.88
+: 1.24
6,95 + 0.86
11.18 + 1.37
14.18 = 1.13
13.56 = 1.21
18.89 = 0.23
10:05
+ 2.81
22.08 + 0.17
0.10
22.69
+:20 3.53
10.19
+ 0.32
19.64
+ 0.37
22.53
= 1.43
21.89
= 0.56
21.31
= 2.01
16.85
1.91
28.87 + 0.96
0.20
38.01
+ 1.01
15.51
+ 1.21
26.41
+ 1.31
33.31
3.22
30.52 + 1.01
39.41 + 0.81
34.21
1.42
43.57 + 1.25
0.30
44.18
+: 2.02
22.79
+ 1.78
48.44
+ 0.82
42.72
1 3.62
43.65 + 1.32
47.35 + 1.36
42.65
0.99
51.97 +: 1.62
0.40
50.07
+: 1.81
26.01
+: 1.92
54.33
+ 0.81
47.45
= 1.13
55.31 = 0.52
56.38 4 0.52
54.37
1.36
58.72 + 0.34
0.50
54.15
+: 1.47
44.06
+ 0.84
57.34
+ 0.13
53.13
= 1.49
53.97
= 0.43
55.63
= 1.08
51.77 1 0.15
59.07 + 0.23
0.60
59.74
4: 1.33
44.74
+ 1.54
63.17 + 1.30
59.41
= 0.71
57.58 = 1.92
63.59 = 0.34
56.18
0.55
60.97 = 0.19
Table 5: Performance comparison of various coreset selection methods on CIFAR-10ON dataset with SOP+ Re-laboling method at various selection fractions of the original dataset. The best results are shown in bold.
h-CenterGreedy
GraNd
Forgetting
SmallLoss
Uniform
Prune4Rel
Moderate
Ours
0.05
21.29
0.02
6.27 0.34
22.73 + 0.78
26.80 + 0.58
24.99 + 0.54
32.61 + 0.29
22.38 + 0.35
40.48 0.02
0.10
33.11
0.11
9.01 0.55
27.73 + 0.27
35.07 + 0.51
32.87 + 1.06
44.55 + 0.06
27.69 + 0.67
46.82 + 0.34
0.20
46.62
0.31
13.84 + 0.33
43.55 + 0.08
48.41 + 0.36
46.98 + 0.37
52.41 + 0.44
44.51 + 0.28
53.92 0.24
0.30
52.31
0.22
19.37 +: 0.17
51.21 +: 0.36
55.39 + 0.39
52.72 + 0.38
56.27 +: 0.07
50.82 + 0.23
58.04 + 0.25
0.40
56.73
0.14
27.47 1 0.37
56.54 1 0.11
59.72
1 0.25
56.45 1 0.13
59.33 1: 0.11
54.61 1: 0.18
60.55 + 0.17
0.50
59.48
0.14
36.32 + 0.31
60.68
+ 0.21
62.60
1 0.14
59.03
1 0.18
61.99
1 0.19
58.07
1 0.16
62.76 1 0.17
0.60
61.97
+ 0.16
45.37 + 0.07
63.38 + 0.07
64.28 + 0.25
61.11 + 0.24
63.48 + 0.51
60.21 + 0.41
64.38 + 0.05
4.3
Results on WebVision
Table tabulates results obtained on the Webvision dataset with the SOP+ Re-labeling method. The first column lists various coreset selection fractions.
Table 6: Performance comparison of various coreset selection methods on WebVision dataset with SOP+ Re-laboling mothod at various selection fractions of the original datasct. The best results are shown in bold.
k-CenterGreedy
GraNd
Forgetting
SmallL
Uniform
Prume4Rel
Moderate
Ours
0.05
39.46 + 0.58
16.82
1 0.13
40.33
1.35
23.44
+ 1.12
38.58 + 0.21
43.41 i 0.33
34.51
0.37
46.81
1.21 +
1.02
22.24
0.56
50.41
33.64
47.16
1.88
54.82
1.06
46.16
1.04
59.86
1.74
33.32
0.44
61.29
46.76
0.04
56.38 + 1.22
62.32
1.92
55.96
+
64.64
+: 1.92
43.88
1.98
65.42
56.94
+ 0.71
60.64 + 1.76
67.12 d:
61.48
1.76
66.84
0.40
67.26
+ 0.26
52.01
0.24
68.92
1.24
62.96 + 1.88
64.12 + 0.18
69.38
1 0.62
65.24
0.23
69.51
1: 0.18
0.50
69.01
+: 1.36
59.28
0.44
69.86
0.74
65.78 : 0.15
66.92 : 1.24
70.61
0.14
66.76
0.18
70.77 + 0.47
1.24
64.51
1.31
70.38
68.82
1.42
71.11 0.86
69.52
+
4.4 Results on ImageNet- -1
Tables 7 and S tabulate results obtained on the ImageNet-IK dataset with the SOP+ Re-labeling method with ResNet-50 and ViT architectures, respectively. Our proposod method is able to outperform Pruno4Rel at all the coreset selection fractions considered for both the architectures.
4.5 Accuracy vs training speed trade-off
Figure Digureddepiee depicts the trade-off between accuracy vs. training speed due to coresct selection. can be seen that coreset selection can improve training speed without significantly impacting full-data set training accuracy.

---

Published in Transactions on Machine Learning Research (10/2025)
Table 7: Performance comparison of various coreset selection methods on the ImageNet dataset with S Re-labeling method at various selection fractions of the original dataset. The best results are shown in bold. ResNet-50 architecture is used. Our proposed method outperforms Prune4Rel across all the coreset selection fractions considered.
Method
0.01
0.05
0.10
0.20
0.30
0.40
PrunelRel
1.71 + 0.04
29.20 + 0.13
42.22 + 0.34
52.03 + 0.27
57.18 + 0.83
59.09 + 0.14
Ours
2.33 + 0.23
32.97 + 0.29
47.31 + 0.47
55.42 + 0.45
57.77 + 0.75
60.17 + 0.16
Table 8: Performance comparison of various coreset selection methods on the ImageNet dataset with SOP+ Re-labeling method at various selection fractions of the original dataset. The best results are shown in bold. ViT architecture is used. Our proposed method outperforms Prune4Rel across all the coreset selection fractions considered.
Method
0.001
0.005
0.01
0.05
0.10
0.20
Prune4Rel
0.10 + 0.07
2.08 + 0.05
3.58 + 0.11
11.87 + 0.05
17.12 + 0.23
27.67 = 0.31
Ours
0.76 + 0.03
2.78 + 0.09
4.79 + 0.19
14.43 + 0.08
22.16 + 0.03
33.93 + 0.21
Dataset: WebVision, ModdelinneeplioneeeeeWeeVe22
Dataset: CIPAR- 10OM, Model: PreAotResNet18
80000
80
8000
90
28000
5000
computtatonns time (In 0e006d6)
60
4000
Acoureey (in 2
Computeliooal time (In
80
18000
40
8000
40
Acturady
30
10000
8000
R0
6000
1000
10
0
corvect 640%
corecet: 920%
ogreeet 410%%
0cresete88
Full datasei corssel 640%
curepel #20%
goreeet 010%
cor9991955
dataset
tesining
Accuracy
training time
Test Accuracy
Figure 4: Accuracy vs training speed trade-off various coresct selection percentages. Training time reduction much higher than the reduction in accuracy at a lower level of coreset selection percentage, highlighting the benefit of using coreset for re-labeling.
4.6 Ratio(%) of noisy examples in the selected subset for CIFAR-100N dataset.
Table g compares the ratio of noisy examples in the selected subset for the CIFAR-10ON data set. As can be seen, our method progressively sclects higher percentage of noisy examples as the fraction of the subset to be selected increases. Our method progressively but carefully increases the fraction of incorrectly labeled samples in the coreset, which have a higher probability of getting corrected through re-labeling process. This is not the case with methods other than Prune4Rel. SmallI. selects a very low percentage of incorrectly labeled samples and therefore sacrifices gencralizability. Other methods select a higher percentage of incorrectly labeled samples, and that too, results in poor generalivation.
10

---

Published in Transactions on Machine Learning Research (10/2025)
Table 9: Ratio noisy samples selected as part of the coreset. The percentage of noisy samples selected grows steadily with the increase in the coresct selection percentage.
Method
0.2
0.4
0.6
0.8
SmallL
3.5
8.7
16.3
27.4
Margin
61.5
56.8
51.6
46.2
Center
37.5
38.7
39.9
40.4
Forgetting
37.9
34.6
33.0
36.8
GraNd
93.9
57.3
61.2
49.3
Moderate
33.2
54.6
60.2
64.6
Prune4Rel
28.3
29.1
33.3
37.2
Ours
13.0
18.2
23.5
29.1
4.7 Impact on class- wise accuracy
We visualise the class-wis wiss accuracy of training a PreActResNet-I1. model on the CIFAR-10ON dataset with re-labeling, comparing the performance of the entire dataset and the selection of the coreset. with 20% of the dataset in Figure We observe Spearman rank-order correlation coefficient. of 0.802, indicating high correlation between class-wise accuracies obtained with the full dataset and 20% of the full dataset coreset. It can be seen that our method does not adversely impact any particular class during coreset selection. The coreset composed by our method results in class- wise accuracies strongly correlated to those of the full dataset.
Spearman Coefficient: 0.802
1.00
M PAM A M
0.80
Accuracy
0.60
0.40
0.20
0.00
21
41
51
71
Class Index
-Full Dataset -Coreset
Figure 5: Class-wise accuracy comparison with re-labeling between the full dataset and the coreset with % selection percentage. A high correlation is observed, indicating that the coreset composed by our proposed method does not impact any particular class adversely.
4.8 Visualisation of top and bottom-ranked images
Figure G shows the top and bottom-ranked images selected in the coresct. As can be seen, the top-ranked images are unambiguuus representalives of the 'ostrich' class. In contrast, the bottom-ranked images consist of mislabeled examples and images that do not represent the class.
4.9 Comparative analysis with Vision Language Models(VLMs)
To understand the impact of leveraging large multimodal models for correcting noisy annotations, we con- ducted additional experiments the ImageNet-1K dataset. using a pre-trained vision-language model (VLM,
11

---

Published in Transactions on Machine Learning Research (10/2025)
Top ranked images
Bottom ranked images
Figure 6: Top and bottom-ranked images in coresel belonging to class 'Ostrich' Top-ranked samples are umambiguously representative of their class, while bottom-ranked samples are ambiguous or erroneously labelled.
ViT-H/14) (Ilharco et al. 2021 The VLM was employed as relabeling mechanism, and its performance was compared with SOP+.
Comparative analysis of re-labelling through the VLM model and SOP+. Except for lower coreset selection fraction of 0.01, SOP+ outperforms the VLM.
Table 10:
Coresct Fraction
Accuracy with SOP+
Accuracy with VLM
0.01
2.33 = 0.23
6.87 + 0.14
0.05
32.97 + 0.29
20.21 + 0.23
0.10
47.31 + 0.47
27.79 0.92
0.20
55.42 + 0.45
37.60 0.51
From the results provided in Table 10 we observe that while the VL.M achieves higher accuracy than SOP+ at very low coreset fractions (0.01), SOP+ consistently outperforms the VLM for larger fractions (0.05 0.20). This indicates that although large multimodal models provide some advantage when very limited data is available, specialized re-labeling methods such as SOP- remain more effective and robust as the available data fraction increases. These findings reinforce our design choice treating re-labeling as black-box and focusing on the coreset contribution, while also highlighting that future work could explore hybrid strategies that combine coreset-driven relabeling with the representational strength of VLMs.
4.10
Ablation studies
4.10.1
Selection threshold
The selection threshold on the gradient similarity for neighborhood identification, Φ, important general- ization performance. Figure compares the accuracy obtained coresct selection of 5% and 10% obtained at various selection threshold values $. We use threshold of 0.4 in experiments.
12

---

Published in Transactions on Machine Learning Research (10/2025)
Accuracy Vs. Selection Threshold
Accurecy (in
40
35
0.2
0.4
0.6
0.8
Selection Threshold
5% selection
10%% selection
Figure Ablation study of accuracy vs. selection threshold. Results are obtained on the CIFAR-10ON dataset with the PreAct ResNet-18 model. Accuracy varies with selection threshold Φ, and best results are achieved in the (0.4, 0.6] range.
4.10.2 Confidence score
We have tweaked the vanilla implementation of 'Noise-free gradients' by introducing a confidence score as a weight during the selection of the coreset. Figure compares the accuracy obtained at various coreset selec- tion percentages for the vanilla implementation and with confidence score as weights. clear improvement can be noticed by incorporating the confidence score in selecting the coresct.
Effect of confidence score in selection of coreset VWebVision Dataset)
Effect of confidence score in selection of coreset (CIFAR- 1CON Dataset)
75
75
Acouracy %6
65
Sp 66
45
38
35
5
10
20
80
40
60
10
20
30
40
60
Sclection Poreentage
Selcotion Pereentage
Vanilla Implementstton Witt Confidence Score
Vanilla Implementation WWith Confidence Score
Figure 8: Impact of confidence score accuracy. A clear improvement can be noticed by incorporating the confidence score when selecting the coresct. The left image shows results for the WebVision datasct with the IeceptionnesNettV V2 archilecture. The right image shows results for the CIFAR-100N datasel with the PreAct ResNet-18 architecture.
4.10.3 Variation of softmax
Softmax outputs tend to be overconfident, even when the model's predictions incorrect (Guo et al. 2017 Nguyen et al. 2015). Our application of softmax is geared towards understanding the relative importance of sampling. Our choice of using softmax as a confidence score is motivated by its simplicity and empirical effectiveness in capturing mislabelling patterns in visually similar classes (Figure 3) We refine the selection of the coreset by multiplying the number of neighbors by the confidence score of the samples.
To understand whether the variation in softmax distribution impacts generalization accuracy significantly, we have carried out temperature scaling in the softmax caleulation (Guo et. al.] 2017) for comparative analysis.
13

---

Published in Transactions on Machine Learning Research (10/2025)
The Table 11 compares accuracy values obtained the CIFAR100 dataset with the SOP+ re-labelling method for various coreset selection percentages and temperature values.
Table 11: Impact of softmax temperature accuracy for various selections of coresct fraction for the CI- FARI00 dataset with SOP+ re-labelling method. Over wide range of temperature values, classification performance is not affected, underscoring the fact that choice of softmax as confidence score is not detri- mental in our proposed method.
Coreset Fraction
T=0.1
T=1.0
T=10.0
T=30.0
0.05
40.90 = 0.30
40.48
0.02
40.17 + 0.08
39.63
0.24
0.10
46.74 0.23
46.82 0.34
46.83 + 0.03
46.30 + 0.01
0.20
53.13 = 0.09
53.92 0.24
53.10 + 0.10
51.98 + 0.04
0.30
56.94 = 0.13
58.04 + 0.25
56.50 + 0.10
57.10 + 0.30
0.40
59.73 = 0.30
60.55 = 0.17
59.45 + 0.34
59.40 + 0.40
With wide range of temperature values, classification performance is not affected. Hence, it shows that the proposed softmax approach is effective in the current setting.
5 Conclusion
In this work, we presented an adaptation of an existing coreset selection method tailored to improve learning from noisy datasets. In this approach, we select a coreset composed of samples with the highest number of neighbours with high gradient similarity (measured by cosine similarity), weighted through their confidence score. Experimental results demonstrate that our method effectively mitigates the impact label noise, leading to improved performance compared to standard coreset approaches. Particularly, performance gains significant at lower coreset sizes, because at very small coreset size, the purity of labels and reliability of the samples strongly impact classifier accuracy. Our findings open avenues for further research into confidence-awwre sampling strategies in noisy learning environments.
Hyper-parameter tuning and other tasks that require multiple training runs over the entire dataset are time-intensive. The enhancement achieved execution time by our method makes preferred approach for these tasks. The diversity of the selected coreset an area of interest for future research. We wish explore techniques improve diversity and its impact on the generalization performance of the chosen corcset. Exploring hybrid strategies that combine coresct-driven relabeling with the reprosentational strength of VLMs, is also interesting direction of future research.
14

---

Published in Transactions on Machine Learning Research (10/2025)
References
We're getting
better
idea
of
AT's true carbon
footprint
tochnol-
ogyreview.com.
https: www. / httpp//oooo.o.ree.....o/////////
a oow
algorithmbutm. medium=emai 1uttm campaign-the_aaaee
12-04-2023 [Accessed 23-02-2025].
Pengfei Chen, Ben Ben Liao, Guangyong Chen, and Shengyu Zhang. Understanding and utilizing doep neural networks trained with noisy labels. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 1062-1070. PMLR, 09-15 Jun 2019. URL http:::////rrrcddnngg mlr html
www.mmrrr...p.p..ommomm.ommm aiess/vv7. v vtness//7/chhe1gg chen19g.
Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding. In Proceedings of the Twenty-Sirke Sinth Conference on Uncertainty in Artificial Intelligence, UAT'0, pp. 109-116, Arlington, Vir- ginia, USA, 2010. AUAI Press. ISBN 9780974903965..
Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Efficient data selection for deep learning. In International Conference on Learning Representations, 2020. URL https://openrevier: http:/rrrriiiinne//uuummrmmmaa net/forummid- HJg2boVYDr
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE Conference on Computer Vision and Pottern Recognition, pp. 248 255, 2009. doi: 10.1109/CVPRF 2VP..2009.5506848.
Shi Dong, Ping Wang, and Khushnood Abbas. A survey on deep learning and its applications. Computer Science Review, 40:100379, 2021. ISSN 1574-0137. doi: https://doi.orr/100 10. 1016/ http:/..ooovvv..00. 20ssse..021.100379
URL https://www http///rww.ccienccdree.e.c httpp///ww.....ooooaaaaaaaa/////
Melanie DucolTe and Frederic Precioso. Adversarial active learning for deep networks: margin based approach, 2018
Dan Feldman. Introduction to core-sets: updated survey, 2020.
Aritra Ghosh, Himanshu Kumar, and P. S. Sastry. Robust loss functions under label noise for deep neural networks, 2017. URL https://arxiv http:::://aaxvvooo///aa/aa org/aba/1712. 09482
Jacob Goldberger and Ehud Ben-Reuven. Training deep neural-networke. using a noise adaptation layer. International Conference on Learning Representations, 2017. URL https:://pperrrrie...ese. wwt.///uiii.eeeee/mermmree. H12GRgexg
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks.
In Doina Precup and Yee Whye Tch (eds.), Proceedings of the 94th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 06-11 Aug
2017. URL httpp:://proceedings. mlr. http://ceddnaa......rrrr///////o ptress//70/guol7a..oo
Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision, 2018a. CRL https://arxiv: www..oggg///.....oooooooo..eeeaa 1805. 1805.08193
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels, 2018b. URL https:
7/arxiv. www.iiv..og/aaee abs/ 1804. 06872
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognation (CVPR), pp. 770-778, 2016a. doi: 10. 10.1109/CVPR.2016..00
15

---

Published in Transactions on Machine Learning Research (10/2025)
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks, 2016b. URL https::/aaxiv..rt http//ass110.00000 05027
Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep net- works on labels corrupted by severe noise. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Cur- ran Associates, Inc., 2018. URL https:://proceeddngg wwwpproeeeeinnn.....i........... www..pes...pppppp.....ee......e. httpaappp..ccppppp.....eaapppp/p pfless//appr/2208///1el aa5448306666977777444Bttt
pdr
Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre, Raphael Gontijo Lopes, Jenia Jitsev, Ali Farhadi, and Ludwig Schmidt. Openelip: An open source implementation of CLIP. arXiv preprint wIXiu:2912190011 2021. URL
12017
Rishabh Iyer and Jeff Bilmes. Submodular optimization with submodular cover and submodular knapsack constraints. In Proceedings of the 27th International Conference on Neural Information Processing Systems Volume 2, NIPS 13, pp. 2436 2444, Red Hook, NY, USA, 2013. Curran Associates Inc.
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels, 2018. URL https://arxiv.. httpaaaiivvorrroa/////// abs/ 1712.05055
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, Abir De, and Rishabh Tyer. Grad- match: Gradient matching based data subset selection for efficient deep model training, 2021a.
Krishnateja Killamsetty, Durga Sivasubramanian, Ganesh Ramakrishnan, and Rishabh Iyer. Glister: Gen- eralization based data subset selection for efficient and robust learning, 2021b.
Krishnateja Killamsetty, Xujiang Zhao, Feng Chen, and Rishabh K Iyer. RETRIEVE: Coreset selection for efficient and robust semi-supervised learning. Tn A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021c. URL httpe://openrevves:
Junnan Li, Richard Socher, and Steven C. H. Hoi. Dividemix: Learning with noisy labels as semi-supervised learning, 2020a. URL /tttss://arrti:
Junnan Li, Richard Socher, and Steven C.H. Hoi. Dividemix: Learning with noisy labels semi-supervised
learning. In International Conference Learning Representations, 2020b. URL https://openreviie. met//forumiid-HJgkalll
Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data, 2017. URL https: http:///wwwwwwwo/aaa//////e/ wwtp////////..oooo//o/oooo/o/
Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over over- paraaeettttttiiiinnnn, 2022a. URL /tttps//aare arxivv org/ abs/ 220/ 14026
Sheng Lin, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over pataeamcttttiiiiiionn. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 14153-14172. PMLR, 17-23 Jul 2022b. URL https://proceedingg mlr. wwtpppsss/v///liuuup html
Tongliang Liu and Dacheng Tao. Classification with Noisy Labels by Importance Reweighting. IEEE Trans- actions on Pattern Analysis & Machine Intelligence, 38(03):447-461, March 2016. ISSN 1939-336 3539, doi: 1LLL1//TTAAM..005555566599 URL http://doo...eeecommuuerccccc/// org/10. 11119//PAAN 2015. 2456899
Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates, 2020. URL http:////ppaaaeeeeeeeeeeeerrrrrr
16

---

Published in Transactions on Machine Learning Research (10/2025)
Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Erfani, and James Bailey. Normalized loss functions for deep learning with noisy labels, 2020. URL. https://axxii... http::///xxvvvvor/aaaae org/abs/2006 13554
Katerina Margatina, Giorgos Vernikos, Loic Barrault, and Nikolaos Aletras. Active learning by acquiring contrastive examples, 2021.
Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovee. Coresets for data-efficient training of machine learning models, 2020.
Saumyaranjan Mohanly, Chimata Anudeep, and Konda Reddy Mopuri. Noise-free loss gradients: A sur- prisingly effective baseline for coreset selection. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://opennreveet net tet//torum1dd00EP1118810
Nagarajan Natarajan, Inderjit S Dhillon, Pradcop K Ravikumar, and Ambuj Tewari. Learn- ing with noisy labels. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 26. Curran As- sociates, Inc., 2013. URL https://ppocete http://rwwednnnneeieiei......o http://///ppp...eee///////////// 18716666001112200051100004411113 Paper. pdf.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predic- tions for unrecognizable images. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 427-444-44 436, 2015. doi: 10. 1000///VR.2015.2296660
Dongmin Park, Seola Choi, Doyoung Kim, Hwanjun Song, and Jae-Gil Lee. Robust data pruning under label noise via maximizing re-labeling accuracy. In shérty-seventh Conference on Neural Information Processing Systems, 2023. URL https:// wtt::/orum/ddddiiOopolll
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In 2017 IEEE Conference Computer Vision and Pattern Recognation (CVPR), pp. 2233-2241, 2017. doi: 10.1109/CVPR.2017.22 2017.220....
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on data diet: Finding important examples early in training. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https:////ppereeeee wwt.ieewwwwee/rrrrm Dd-Uj7pF-D-YVT
Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. Deep learning on data diet: Finding important examples early in training, 2023.
Omead Pooladzandi, David Davini, and Baharan Mirzasoleiman. Adaptive second order coresets for data- efficient machine learning, 2022. URL https://arxx..org//b///000. 13887
Clayton Scott, Gilles Blanchard, and Gregory Handy. Classification with asymmetric label noise: Consistency and maximal denoising. In Shai Shalev-Shwarts and Ingo Steinwart (cds.), Proceedings of the 26th Annual Conference on Learning Theory, volume 30 of Proceedings of Machine Learning Research, pp. 489-511, Princeton, NJ, USA, 12-14 Jun 2013. PMLR. URL https://proceeddngg/ mlr mlr..press/v30//oottt3 html
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018. URL http:://ppearerrie..neffffoooore
Hialuk-RW
Burr Settles. Active Learning. Springer International Publishing, 2012. ISBN 9783031015601. doi: 10.1007/ 978-3-031-01560-1. URL
Yanyao Shen, Hyokun Yun, Zachary C. Lipton, Yakov Kronrod, and Animashree Anandkumar. Deep active learning for named entity recognition, 2018.
Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data 6:1-48, 2019. URL attp:::/app..emmttiiccchhh///// htt.:a/t.c......aooooooooaaaaaaa
17

---

Published in Transactions on Machine Learning Research (10/2025)
Hwanjun Song, Minseok Kim, and Jac-Gil Lee. SELFIE: Refurbishing unclean samples for robust deep learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 5907 5915. PMLR, 09-15 Jun 2019. URL httpps//prrceedding wttpeediiinnnnm..eesse.eeee wtt../eres//////og111/0 html
Hwanjun Song, Minscok Kim, Dongmin Park, Yooju Shin, and Jac-Git Lec.
Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 34: 8135 8153, 2020. URL
wttp:p/ss...mmaaccccccoaoooaoeoa (2: 220546541
Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with doep neural networks: A survey, 2022. URL httts: http:/o/////aaoooooooo h .8199
Ben Sorscher, Robert Gcirhos, Shashank Shekhar, Surya Ganguli, and Ari S. Morcos. Beyond neural scaling laws: beating power law scaling via data pruning- In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https:
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alex Alemi. Inception-v4, inception-resnel and the impact of residual connections on learning, 2016. URL
/tt////arii...rr/a//////////2 07261
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geof- frey J. Gordon. An empirical study of example forgetting during deep neural network learning, 2019a.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geof- frey J. Gordon. An empirical study of example forgetting during deep neural network learning, 2019b.
URL httts: http:///xxvvvv.g//e//// wttpi/i.rrrgaaa//11...... 05159
Amazon
Mechanical Turk.
Tutorial:
How
label thousands of im-
ages using the crowd
Nog.mturk.com.
https://blog htt////ll.mmuu//ee www.k.tu.k..mmmmm com/ [Accessed 10-
03-2025..
Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. In International Conference on Learning Representations, 2022. URL httpp/////ppeeeeeeee.ee/////rrra
Xiaobo Xia, Jiale Liu, Jun Yu, Xu Shen, Bo Han, and Tongliang Liu. Moderate coreset: A universal method of data selection for real-world data-efficient deep learning. In The Eleventh International Conference Learning Representations, 2023. URL https://
net het//formm1117777E00000
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In 2015 IEEE Conference on Computer Vision and Pattern Recognation (CVPR), pp. 2691 2699, 2015. doi: 10.1109/CVPR.C 7500//..........888555..
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. Unsupervised data augmentation for consistency training. In Proceedings of the 34th International Conference on Newral Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020a. Curran Associates Inc. ISBN 9781713829546
Qizhc Xie, Zihang Dai, Eduard Hovy, Minh- Than-Thanggaa Luong, and Quoc V. Lc. Unsupervised data augmentation for consistency training, 2020b. URL Ittpp://arxivv org/ abs/ 1904. 12848
Shuo Yang, Zeke Xie, Hanyu Peng, Min Xu, Mingming Sun, and Ping Li. Dataset pruning: Reducing training data by examining generalization influence. In The Eleventh International Conference on Learning Representations, 2023. URL https:// net httt:/ooooo//////444////0000000
Jiangehao Yao, Jiajie Wang, Ivor W. Tsang, Ya Zhang, Jun Sun, Chengqi Zhang, and Rui Zhang. Deep learning from noisy image labels with quality embedding. IEEE Transactions on Image Processing. 28(4): 1902- 2019. doi: TT10///T.011..7777777
18

---

Published in Transactions on Machine Learning Research (10/2025)
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W. Tsang, and Masashi Sugiyama. How does disagree- ment help generalization against label corruption?, 2019. URL
wwt.://...bbb......././ 04215
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization, 2017. URL btttp:://axxiv..rrg/bb/////
Tianyi Zhou, Shengjie Wang, and Jeff Bilmes. Robust curriculum learning: from clean label detection to noisy label self-correction. In International Conference on Learning Representations, 2021. URL https:
//openreview.o http:/oorrmmidddd//1a3330000
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. Learning transferable architectures for scalable image recognition. In 2018 IEEE/CVE Conference on Compuler Vision and Pallern Recognition, pp. 2018, doi: (0.1109/CVPR.2011 201...000000
19


Reviews:
[{"review_id": "W67g8Cg5AL", "reviewer": "Reviewer_mvgh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper addresses an important challenge in large-scale real-world datasets: annotation noise.\n\n2. The proposed method is notably simple, offering advantages in terms of ease of implementation.\n\nWeaknesses:\n\n1. The problem formulation and experimental setup are outdated in the era of large models. The authors claim that label noise is a critical issue in large-scale datasets, yet none of the tested datasets are sufficiently large to substantiate this claim. Evaluations on truly large-scale datasets are necessary to demonstrate real-world relevance.\n\n2. The practical significance is limited. Most contemporary large-scale datasets, such as LAION, no longer rely on labels but instead use text descriptions and are largely unsupervised. It is unclear how the proposed method would address noise in such settings.\n\n3. The experimental setup and learning paradigm lack real-world alignment. The authors argue that label noise impacts supervised learning, but the best ImageNet performances now come from fine-tuning large pre-trained models, not training from scratch. Fine-tuning is also more appropriate for coreset learning. \n\n4. The models used in the experiments are outdated. The authors should evaluate their method using state-of-the-art model architectures and benchmarks.\n\n5. The paper does not explore the possibility of leveraging large multimodal models to correct noisy annotations.\n\n6. Performance improvements are marginal under several noise ratios.\n\n7. Technical novelty is limited; the method is essentially a weighted adaptation of an existing approach.\n\n8. Most cited works are from 2022 or 2023, failing to account for recent paradigm shifts brought by large models. The conventional setup adopted has become less meaningful nowadays.\n\n9. Coreset selection is somewhat similar to the few high-quality examples used in few-shot learning or supervised fine-tuning of a pre-trained large foundation model. I would suggest this setting for this paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a improved version of gradient similarity based coreset selection method to deal with label noise. The idea is first to select the mislabeled but high similar concepts into the corset (as they improve generalization) and then use a sample-wise weighted confidence score to find the most suitable coreset samples. Experiments on typical datasets such as CIFAR-100N, ImageNet-1k, WebVision were conducted to verify the effectiveness of the proposed approach.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Evaluate the proposed method within the large model fine-tuning paradigm.\n\n2. Assess performance using state-of-the-art models on the SOTA benchmark leaderboards.\n\n3. Test the method on larger-scale and more contemporary datasets, as ImageNet-1k is now somewhat outdated.\n\n4. Compare against VLM-based annotation correction approaches, such as querying an open-source VLM for class labels and confidence scores.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "52DMoyrIph", "reviewer": "Reviewer_fLBQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The proposed method is simple and easy to implement. The method shows significantly improved performance when the coreset size is small.\n\nWeaknesses:\n1. The proposed method is a marginal modification of the previous method on coreset selection. Moreover, the intuition behind the modification is not very clear. The paper discusses that mislabeled samples are likely to be predicted with high confidence. However, the empirical evidence for that claim is insufficient. Ideally, we should observe some correlation between the prediction confidence and the mislabeled rate to make the claim more solid.\n\n2. More baselines could be added in the experiments. E.g., comparisons with submodular-based methods such as CRAIG or simply using submodular functions to construct the coreset. Also, it is useful to provide comparisons with other non-coreset noise-label learning methods as a reference. \n\n3. What if the noise has a particular systematic bias? For example, in the extreme case, all the representative samples are labeled correctly, and only the samples that are close to the boundaries are mislabeled. In such a case, as the method focuses on samples that are similar to most of their neighbors and are therefore representative, it may be the case that very few noisy labels are caught.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the problem of re-labeling for noise-label classification. Particularly, the paper proposes a coreset-based method, which first identifies a small coreset of crucial samples and then performs relabeling correction only on the small coreset. To construct the coreset, the paper builds upon a previous approach, which first identifies samples that are most similar to their neighbors and then weights them by their prediction confidence. Empirically, the proposed method can significantly outperform baseline coreset selection, especially for small coreset sizes.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address the weakness part.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xFqsb9sk8p", "reviewer": "Reviewer_2RBt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The paper addresses an important practical challenge of improving computational efficiency when training on noisy-labeled datasets.\n- The integration of confidence scores into gradient similarity is straightforward and easy to implement.\n- The method is thoroughly tested on multiple benchmark datasets with different re-labeling methods and various coreset baselines.\n- The method delivers noticeable training speedups at low coreset selection fractions, with manageable drops in accuracy.\n- The paper includes ablation studies, class-wise accuracy analysis, and sample visualizations to enhance result interpretability.\n\nWeaknesses\n- Limited Novelty (Key Issue): This work is not the first to combine coreset selection with re-labeling. The prior work Prune4ReL (Park et al., 2023) has already introduced and systematically studied this problem. The current paper represents an incremental improvement over this existing approach rather than a fundamental innovation.\n- Simplistic Confidence Definition: It is well established that softmax outputs tend to be overconfident, even when the model's predictions are incorrect (Guo et al., 2017; Nguyen et al., 2015). This limitation is particularly problematic in noisy label scenarios. The paper does not acknowledge or address this known issue, which may undermine the reliability of the proposed confidence-based selection.\n- Lack of Theoretical Support: The method lacks theoretical analysis, particularly regarding the choice of the key hyperparameter Φ (the gradient similarity threshold), which is selected empirically without justification.\n- Narrow Task Scope: The method is only evaluated on image classification tasks. Its applicability to other domains or tasks (e.g., object detection, NLP) is not addressed.\n- Lack of Diversity Consideration: The coreset selection focuses solely on representativeness but does not encourage sample diversity, which may negatively affect performance on imbalanced datasets.\n\nReferences\n\n[1] Guo C, Pleiss G, Sun Y, et al. On calibration of modern neural networks[C]//International conference on machine learning. PMLR, 2017: 1321-1330.\n\n[2] Nguyen A, Yosinski J, Clune J. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 427-436.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an improved coreset selection method aimed at enhancing the efficiency and effectiveness of re-labeling techniques for learning from noisy labeled datasets. Specifically, the authors build on the existing “Noise-free Loss Gradients” method by introducing a confidence score based on the model’s softmax outputs to weight gradient similarity and select a more representative subset for training. The proposed approach is evaluated on CIFAR-100N, WebVision, and ImageNet-1K, combined with two popular re-labeling methods (DivideMix and SOP+). The experimental results demonstrate that the method can improve classification accuracy while significantly reducing computational cost.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Explore or at least discuss the use of more robust uncertainty measures to mitigate the limitations of softmax confidence under label noise.\n- Provide theoretical justification or empirical heuristics for choosing Φ to improve reproducibility and robustness.\n- Consider analyzing or experimenting with other tasks beyond image classification to enhance the generality of the method.\n- Introduce mechanisms that encourage diversity in the selected coreset, particularly for imbalanced datasets.\n\n- How does your method fundamentally differ from Prune4ReL in terms of mechanism, confidence definition, and application scenario?\n- Why did you choose to use softmax confidence directly instead of more robust uncertainty estimation methods (e.g., entropy, MC Dropout, deep ensembles)?\n- Can you provide any theoretical insights or automated procedures to guide the selection of the threshold Φ?\n- Have you considered applying your method to tasks beyond image classification? Do you have any preliminary results or theoretical arguments supporting its generalizability?\n- Have you considered incorporating sample diversity constraints into the coreset selection to improve performance on class-imbalanced datasets?\n- Were all the baseline results in the comparative experiments reproduced by the authors? If so, could the authors clarify how the baseline results in Table 4 of this paper align almost exactly (with discrepancies smaller than 0.1) with the corresponding baseline results reported in Table 1 of the Prune4ReL paper? This raises questions about whether these results were independently reproduced or directly referenced, and clarification would improve the transparency and credibility of the experimental setup.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)