Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples
Authors: Suqin Yuan
Lei Feng
Bo Han
Tongliang Liu
Venue: NeurIPS 2025 poster ()
Keywords: label noise
learning with noisy labels
sample selection
Primary area: general_machine_learning

Abstract: Sample selection is a prevalent approach in learning with noisy labels, aiming to identify confident samples for training. Although existing sample selection methods have achieved decent results by reducing the noise rate of the selected subset, they often overlook that not all mislabeled examples harm the model's performance equally. In this paper, we demonstrate that mislabeled examples correctly predicted by the model early in the training process are particularly harmful to model performance. We refer to these examples as Mislabeled Easy Examples (MEEs). To address this, we propose Early Cutting, which introduces a recalibration step that employs the model's later training state to re-select the confident subset identified early in training, thereby avoiding misleading confidence from early learning and effectively filtering out MEEs. Experiments on the CIFAR, WebVision, and full ImageNet-1k datasets demonstrate that our method effectively improves sample selection and model performance by reducing MEEs.

Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples
Suqin Yuan
Lei Feng2* Bo Han Sydney Al Centre, The University of Sydney Southeast University Hong Kong Baptist University
Tongliang Liu'
Abstract
Sample selection is a prevalent approach in learning with noisy labels, aiming to identify confident samples for training. Although existing sample selection meth- ods have achieved decent results by reducing the noise rate of the selected subset, they often overlook that not all mislabeled examples harm the model's performance equally. In this paper, we demonstrate that mislabeled examples correctly predicted by the model early in the training process are particularly harmful to model per- formance. We refer to these examples as Mislabeled Easy Examples (MEEs). To address this, propose Early Cutting, which introduces recalibration step that employs the model's 's later training state to re-select the confident subset identified early in training, thereby avoiding misleading confidence from early learning and effectively filtering out MEEs. Experiments on the CIFAR, WebVision, and full ImageNet- Ik datasets demonstrate that our method effectively improves sample selection and model performance by reducing MEEs. Our implementation can be found https: / github.. com/ com/tmllr 2025 1eerrree_eereee IPS MEE.
Introduction
Deep Neural Networks (DNNs) have achieved remarkable success, while heavily relying on the availability of high-quality, accurately annotated data. In practice, collecting large-scale datasets with precise labels is challenging due to the high costs involved and the inherent subjectivity of manual annotation processes. Consequently, datasets often contain noisy labels, which can degrade the generalization performance of DNNs-a problem known as learning with noisy labels (LNL) (Natarajan et al., 2013). One prevalent approach address LNL. is sample selection, which aims to identify confident samples for training while discarding potentially mislabeled ones.
Sample selection methods can be categorized into two types: loss-based and dynamics-based. Loss- based methods rely on the assumption that clean samples tend to have smaller loss values than mislabeled samples (Han et al., 2018; Liu al., 2020; Xia et al., 2021; Li et al., 2024). In contrast, dynamics-baan based methods exploit the memorization effect of DNNs, which suggests that DNNs learn simple patterns first and then gradually fit the assigned label for each particular minority instance, including mislabeled samples (Liu et al., 2020; Zhang et al., 2021; Yuan et al., 2024). By analyzing the learning dynamics of DNNs, these methods aim identify clean samples that learned early and consistently throughout the training process (Yu et al., 2019; Xia et al., 2020a; Bai and Liu, 2021; Wei et al., 2022), considering them as confident samples for training. recent years, dynamics-based methods have gained attention due to their ability to select Clean Hard Examples (CHEs) challeninn clean samples that are difficult identify but crucial for achieving near-optimal generalization performance (Feldman and Zhang, 2020; Bai and Liu, 2021; Yuan et al., 2023).
"Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Conpartinn invemerilly Loarned Caan Masibeed Semples Cpech
Milabdol Har Saappi
84
Palaeedd Savera
CIFAR-10
30
100
25
30
75
100
Excmpia
Epochs
Epochs
ImageNet-11
Caac Sample
Gan laazçios
Accuscy
Jocura nouse
5002
CIFAR-100 100
96
Clas Saarpli
Dificut Select
Eoiy Belect
22003
75
100
23
50
75
100
Epoctts
Epochs
Learning
Learning with
Midiabeled Eng Evampprr
Misisbelid Hard Emaplle
(b)
Figure 1: (a) Test accuracy curves when the originally clean training subset is augmented with 4000 Mislabeled Easy Examples versus 4000 Mislabeled Hard Examples (see Section 2.1 for setup). Adding Mislabeled Easy Examples leads larger decrease in the model's generalization performance. (b) Histogram illustrating the distribution of ImageNet-1k examples with 40% symmetric label noise, showing the epoch at which each example first correctly predicted by the model during training. The horizontal axis represents the epoch when examples are first correctly predicted, and the vertical axis represents the number of examples predicted correctly at each epoch.
Although these sample selection methods have achieved decent performance by relying on early training stages to minimize noise in the selected subset and adopting advanced strategies to retain CHEs, they often overlook that not all mislabeled examples harm the model's performance equally, Specifically, even with a low noise rate the selected subset, the presence of certain mislabeled samples still significantly impair the model's generalization performance. As shown in Figure ((a), we demonstrate that mislabeled samples which are correctly predicted by the model early in the training process disproportionately degrade performance. We refer these easily learned and particularly harmful mislabeled samples as Mislabeled Easy Examples (MEEs). In our analysis (see Section 2.2), find that MEEs often closer to the centers of their mislabeled classes in the feature space of classifiers trained the early stages. This causes them to be easily and "reasonably" classified into the wrong classes during early training, thereby disrupting the model's early learning of simple patterns (Arpit et al., 2017). Consequently, these examples are learned earlier by the model and harm generalization performance more.
To address this issue, we propose a novel sample selection strategy called Early Cutting, which introduces a recalibration step using the model's state at later epoch to re-select the confident subset of samples identified during early learning. In this recalibration step, we identify samples that exhibit high loss yet predieted with high confidence and demonstrate low sensitivity to input perturbations- characctrstsaaatttsstttissssssss indicative of MÉEs. By further excluding these deceptive samples from the confident subset, we reduce MEEs negative impact on the model's generalization performance. Although this re-selection might result in the inadvertent removal of some clean samples, the impact is mitigated due to the nature of early-learned samples, which are abundant and often redundant representations of simple patterns. Removing portion of these samples has a smaller detrimental effect compared to the significant harm caused by retaining MEEs.
We conduct extensive experiments on CIFAR (Krizhevsky et al., 2009), Web Vision (Li et al., 2017), and full ImageNet-Ik 1k (Deng et al., 2009) datasets with different types and levels of label noise. The results demonstrate that our proposed method consistently outperforms state-of-the-ar sample selection methods across various computer vision tasks.
Our main contributions can be summarized as follows:
We discover that mislabeled samples correctly predicted by the model early in training dispro- portionately harm model's performance; we define these samples as Mislabeled Easy Examples (MEEs). MEEs are closer to the centers of their mislabeled classes in the feature space of models in early training stages, causing the model to easily learn incorrect patterns.
2. We introduce Early Cutting method, which recalibrates the confident data subset identified early in training by utilizing the model from later stages-a counterintuitive approach, given that later-stage models are typically regarded as less trustworthy.

---

Related Work. We briefly review the related work. Detailed reviewing is in Appendix A.
Sample Selection has been used in learning with noisy labels to improve the robustness of model training by prioritizing confident samples. An in-depth understanding of deep learning models, particularly their learning dynamics, has facilitated research in this area. Extensive studies on the Learning Dynamics of DNNs have revealed that difficult clean examples are typically learned in the later stages of training (Arpit et al., 2017; Yuan et al., 2024, 2025). This insight has led training- time metrics that quantify sample "hardness"" such as learning speed (Jiang et al., 2021). These metries inspire methods that leverage learning dynamics select clean samples (Zhou al., 2021; Maini et al., 2022). Various forms of Hard Label Noise have been studied, including asymmetric noise (Scott al., 2013), serranneeedeeddemen noise (Xia al., 2020b), natural noise (Wei al., 2021b), adversarially crafted labels (Zhang et al., 2024a), open-set noise (Wei et al., 202 and subclass-dominant noise (Bai et al., 2023). These noise types are designed from the perspective of the labels, aiming to simulate challenging real-world scenarios or malicious attacks. In contrast to prior studies that mainly focus on different types of label noise, our work offers a fresh perspective by re-examining sample selection methods that rely model's early learning stages. We demonstrate that some samples hidden among those considered "confident" are, in fact, the most harmful. This contributes new insights into effectively identifying and handling mislabeled data.
2 Our Observations
In this section, we investigate the varying effects that different mislabeled examples have on model's s generalization. In Section 2.1, we provide empirical evidence demonstrating that different mislabeled examples have varying impacts on the performance of model, with the mislabeled examples learned earlier by the model bring greater harm. In Section 2.2, we analyze the reasons why these examples are easily learned by the model and bring about greater harm.
2.1 Effects on Generalization from Mislabeled Examples Learned at Different Stages
Previous studies have shown that DNNs typically exhibit specific learning pattern: they tend to learn simple and clean patterns first and gradually memorize more complex or mislabeled examples later (Arpit et al., 2017; Toneva et al., 2018). Based on this, some sample selection methods (Liu et al., 2020; Bai and Liu, 2021) trust the samples learned early by the model, treating them as high- probability clean samples. In our study, distinguish the order in which the model learns different mislabeled examples, we refer to the definition in Yuan et al. (2023). Specifically, we consider that the model has learned a sample (x,, úi) time F. if it consistently predicts the given label fr for both epoch E; - 1 and E;, regardless of whether the label is correct. Formally, we define the learning time LT; of a sample (x,, (K) as:
LT = min Ex 10 F.
D. = T.),
where of denotes the model's predicted label for instance x, at epoch e. By tracking each sample's learning time LT;, we can analyze the order in which the model learns different samples and evaluate their impact on performance.
(a)lmpact on model performance from mislabeled examples learned at different stages, using CIFAR- 10.
Clean Nisiabeled
6000) Mizsbelied
Clann (8000 12000) Misaccted
(12010 20000) Nisiabeled
(15000 20000] Mistabeted
50
100
10
100
50 100
100
Epectts
Figure 2: Imppct of mislabeled samples learned at different stages on model generalization perfor- mance. Subfigure 2 shows the scenario in the CIFAR-10 10 dataset, which contains 20,000 mislabeled samples (40% destaneeddependet label noise) and 30,000 clean samples. We divided the 20,000 mislabeled samples into five groups based on the order in which an initial model learned
from
carliest to latest (ranging from (0 : 20, 000)). Each group was combined with the 30,000 clean samples, creating datasets with approximately 12% label noise (4, 000/34, 000). New models were then trained on these datasets. As shown by the decreasing test accuracy, models trained on datasets containing earlier-leamd leamed mislabeled samples (e.g., "Clean +(0 4000) Mislabeled") exhibited lower generalization performance. Subfigure 6 shows similar findings on CIEAR- 100.
3

---

(a) The speed at which pretrained models on CIFAR- 10 learn mislabeled examples from different stages.
Clean 4000) Medlabeled
Claan a (4000: 9200) Nolabeled
Clean 18000 12000) Malabeted
Ceat 122000 11000) Misubeled Clean (15002 20200] Nalabeled
103
100
3200
3200
Ecache
Eooche
Opeche
Opecha
Opeers
Figure 3: Comparison of how pretrained models learn mislabeled examples from different learning stages. Subfigure 3 shows results CIFAR-10 10 with 40% noise. We divided the mislabeled examples into five groups based on the order the initial model learned them, mixing each group with 30,000 clean examples form datasets with approximately 12% label noise (4000/344000. A model was pretrained on the 30,000 clean examples and then trained on these new noisy datasets. Reference lines indicate the number of epochs required for the pretrained model to learn different sets of 2,000 mislabeled examples. The results reveal that earlier- Icarned mislabeled examples are also learned more quickly by the robust model. Subfigure 7 shows similar findings on CIFAR-100.
To investigate how the learning order of mislabeled examples affects generalization, conducted experiments on CIFAR-10 and CIFAR- 100 with 40% instance-dependen label noise. First, we trained an initial model on the noisy dataset record the learning time LT; for each sample. Based on these times, we partitioned the 20,000 mislabeled examples into five sequential groups of 4,000, from earliest-leared to latest-learned. Each group was then combined with 30,000 clean examples form five distinct training datasets.
We then trained new models from scratch these datasets. As shown in Figure 2, the results are unambiguous: models trained with the earliest-leamed group of mislabeled examples exhibit significantly lower generalization performance than those trained with later-learned groups. This clearly indicates that mislabeled examples learned earlier by a model cause greater harm to its generalization. To validate this observation, we repeated the experiment using model pretrained on only the clean data. As shown in Figure 3, the model still learns the MEES from the "earliest" group much faster than those from later groups. This confirms that these samples inherently easy for the model to learn, regardless of the training starting point.
2.2 Mislabeled Easy Examples
In this subsection, we focus specifically on the mislabeled examples that the model learns during the early stages of training. Drawing inspiration from the concept of Clean Hard Examples, we formally define these particularly harmful mislabeled examples learned early by the model as Mislabeled Easy Examples (MEEs). This term indicates that although these samples are incorrectly labeled, they are casily learned by the model.
Notably, MEES are non-trivial because the early stages of model training are typically characterized by learning simple and correct patterns from clean samples (Arpit al., 2017; Toneva al., 2018), while the later stages are when the model starts to memorize mislabeled samples (Zhang et al., 2021; Yuan et al., 2024). Therefore, it is worthwhile counterintuitive way in which the model learns these mislabeled samples early in training to enhance our understanding of its learning process. To better understand the characteristics of MEES and their impact model generalization, we examine their positions in the model's 's feature space (the representation before the last fully-connected layer) and present some representative examples.
conduct an in-depth exploration of the
As shown in the Figure 4(a), we visualize the mislabeled examples that are correctly predicted by the carly-stage model using t-SNE (Van der Maaten and Hinton, 2008) in the feature space. For a detailed visualization of how this feature space and the associated distance ratios evolve at later training stages, please see Appendix
Further, to quantify the model's 's representations of mislabeled samples during early training, we compute the Euclidean distances from each mislabeled example learned by the early-stage model to the center of its true class and the center of its mislabeled class in the embedded feature space. We denote these two distances as dane and dmissaheeee. respectively. We then define the distance ratio r = dmistabeted / /dme-- If r < 1, the example closer to the mislabeled class center than to its true class center. As shown in the bottom row, MEEs cxhibit notably smaller

---

Mislabeled Easy Examcles CIFAR-10
#F2001
horer
Mrd
Mislabaled Easy Examples in CIFAR; 100
crarge
stange
1996
1000
ahan
Rountnic
(a)
(b)
Figure 4: 4(a) Visualization Mislabeled Easy Examples (MEEs) the feature space. Top row: t-SNE embeddings of CIFAR-10 training samples (20% instance-dependee label noise), colored by their given labels (left) and their true labels (middle). Bottom left: closer look at MEEs (red points) connected to their mislabeled class centers (black stars), demonstrating how these examples cluster in ambiguous regions that overlap with the mislabeled class. Bottom middle and right: comparisons of the distance ratio dmistabeeed duue for MEEs and other mislabeled samples, confirming that they are indeed closer to incorrect wrong labels than their true labels the learned feature space. 4(b) Representative MEEs. Each image is shown with its true label (blue) and the mislabeled label (red).
median distance ratio (0.830), with more than half (53.8%) of them having r < 1. In contrast, the remaining mislabeled samples (non-MEEs) have median ratio of 3.923, and only 5.4% are closer to the incorrect class,
Why MEEs are learned earlier and harm generalization more? Our analysis suggests that MEEs occupy regions in the feature space where their incorrect labels seem more reasonable to the model. During the early stages of training, instances of MEEs closely resemble their mislabeled classes in the feature space, the model learned them as if they were representative samples with simple patterns. Figure 4(b) presents representative MEEs. For instance, a CIFAR-10 image of airplane with dominant sea background is mislabeled as ship, and a CIFAR-100 image featuring a predominantly orange background is mislabeled as an orange. These examples illustrate how strong visual cues matching their given (incorrect) label classes-such as color, texture, or prominent features-can pull these samples closer the incorrect class in the feature space.
This phenomenon explains why MEEs are learned earlier: their misleading features align with the simple patterns of their given (incorrect) label classes that the model is tend to learn during the initial training stages. Thus, the early learning of MEEs has a disproportionately negative impact the model's generalization performance: since the model incorporates incorrect patterns associated with MEEs from the beginning, disrupts the initial formation of simple and accurate feature representations. The erroneous features learned from MEEs become intertwined with the representations of clean data, making it challenging for the model to disentangle the clean patterns. To further quantify this detrimental impact microscopic level, we conducted supplementary analysis using influence functions (Koh and Liang, 2017) to measure the precise impact of individual samples on model generalization. This analysis, detailed Appendix B, provides direct quantitative evidence that MEEs are substantially more harmful to the model's 's performance on clean data than other mislabeled examples.
Methodology
Based on the analysis above, MEES, mislabeled samples that the model learns easily during early training stages, can have disproportionately negative impact on model generalization performance. Previous methods (Liu et al., 2020; Bai and Liu, 2021) often rely trusting the model's early learning
5

---

stages or focusing on samples with small loss, are ineffective at filtering out MEEs due to their deceptive nature. To mitigate the influence of MEEs, we propose novel sample selection strategy called Early Cutting. Early Cutting leverages the model's state at a later training cinase-specffiallly at the early stopping epoch when the model begins to overfit-to re-evaluate and refine the subset of samples initially identified as confidently learned. The initial confident subset, Dr, is formed by selecting samples with small learning times LT; (as defined in Eq. (D)). This captures examples that are learned quickly, encompassing both genuinely clean, easy samples and, crucially, the MEEs.
From this initially selected confident subset D our objective samples that we characterize as MEEs in Section 2.2. These are samples that, when evaluated by the model for at the early stopping epoch t, exhibit predictions that differ from their given noisy labels ūt), yet are made with high confidence and possess stable gradients.
identify and subsequently remove
Formally, consider D = (x; D:) where E R represents input features, and 0 E ) denotes the corresponding observed labels from K classes. Let foe (x) be a model parameterized by 0' at the early stopping epoch I, generating class probabilities via the softmax function:
(1) (2) P = fox (x:) = Pi Pi
(K)
P
where (k) is the model's output probability for class k. The predicted label ğ and the prediction
confidence O are given by: ds- = arg maxk p. and a = (0.)) The cross-entropy loss for sample (x;, (.) is: L. = log p; (6))
While the selected subset D tends to retain high-quality set clean samples, it may still include MEES due to their deceptive nature. To address this issue, we leverage the model's parameters 0r at later training stage to identify and remove suspicious samples from D Specifically, we define a set of suspicious samples S within Dd based on the criteria of high loss and high confidence:
S f D 1, 8, G 7
where 8 and T are thresholds for the loss and confidence, respectively. The rationale is that a high loss L; indicates that the model's prediction at epoch t disagrees with the given label ÿr. and high confidence C; implies that the model is very certain about its (contradictory) prediction 9; * y;- Therefore, samples satisfying both conditions are likely be mislabeled, even if they were learned early. Our method remains robust even if the set S is empty, as the refinement step would simply be bypassed. Relying solely loss and confidence may not be sufficient, as some hard-to-learn samples may also exhibit high loss and high confidence due to their intrinsic difficulty. To further refine our selection, we introduce the concept of gradient stability. We compute the Euclidean norm of the gradient of the loss L; with respect to the input x;:
QL; 0x,"
Vx,Li
9. Vx Lill2
A small gradient norm 9i indicates that the loss L; is insensitive to small perturbations in Xi suggesting a strong (but potentially incorrect) association between the input features and the predicted label. MEEs tend to have low gradient norms because the model has confidently mislearned them, making the loss stable even under input perturbations. We refine S by selecting samples with high gradient stability (where is threshold):
S' S=
g.
(5)
Operational Definition (Mislabeled Easy Examples (MEEs)). Operationally, from the set of early- learned samples DD, we define the subset of Mislabeled Easy Examples (MEEs) as those samples that satisfy the conditions of high loss, high confidence, and low input gradient norm. Formally:
MEES = {i E D= (L, > 8) > 7) A (gi <))
(6)
This identification is practically implemented by selecting samples whose metrics fall into predefined percentiles derived from their distributions within D= Specifically, for all settings, we target the top 10% for loss, top 20% for confidence, and bottom 20% for gradient norm, which were determined on a validation set. Additionally, we set the early cutting rate to 1.5. Samples meeting these criteria are classified as MEES and subsequently removed from D=, yielding refined subset for further training. We formalize the complete Early Cutting procedure in Algorithm 1.

---

Algorithm 1 Iterative Sample Selection with Early Cutting
Require: Training data DD; Number of iterations Tame: Early cutting rate 7, Thresholds d, T
Ensure: Trained model parameters l"
I: for i= to Trate do
2: 1. Base Sample Selection:
3: Compute learning times LT; for all (x;; M/y D' using Eq. (1)
4: Select the initial carly-learned subset D based on smallest learning times.
5: 2. Early Cutting:
6:
Select candidate subset D from D' (e.g., the proportion with the earliest learning times).
7:
Compute loss Lj, confidence e and gradient norm 9j for all samples in D
8:
Identify the set of MEES in according to Definition I; MEEs
9:
{ E D (L, > 5) A (cg > T) A (9) < {
10: Create the refined subset by removing the identified MEEs: D ietined
De MEEEs.
11: Update the training data for the next iteration: D' +- D Piefined
12: end for
13: Final Training Phase: Train model from scratch on the final refined set Diae until convergence.
14: return Trained model parameters 8".
Notably, the proposed method operates on D=, the subset of samples learned quickly during the initial training phase. Such early-learned subsets are known to often contain significant redundancy, with multiple examples representing similar, dominant data patterns (Feldman, 2020; Feldman and Zhang. 2020; Yuan et al., 2024). This inherent redundancy contributes to the robustness our MEEs removal strategy. Firstly, it provides resilience against the inadvertent removal of small number of clean samples, as their informational content likely preserved by other remaining examples. Secondly, this characteristic makes the outcome less sensitive to the precise percentile thresholds used for MEEs selection. A sensitivity analysis of these thresholds is presented in Section 4.3, an ablation study is shown in Appendix D.8, and it transferability is shown in Appendix D.9.
4 Experiments
4.1
Preliminary Presentation of Effectiveness
We first provide empirical evidence to verify the effectiveness of Early Cutting. Using CIFAR-IO 10 with 40% various synthetic label noise and ResNet-18 as the backbone, compared our proposed Early Cutting with loss-based and dynamic-bate Cutring consistently achieved the highest test accuracy across all noise types. Although Early Cutting and the dynamic-based method selected training subsets with similar noise rates, our approach's 's better performance indicates that focusing filtering specific harmful mislabeled examples improves selection quality. The last row shows the number of additional samples filtered by Early Cutting and the high percentage of mislabeled samples among them, proving its effectiveness at identifying and removing noisy labels. As intuition, more challenging noise types result in more mislabeled samples being removed, leading larger performance gains. Detailed settings in Appendix D.
sample selection methods. Table shows that Early
Table 1: Training on 60% noisy training samples selected by each method. Test accuracy (noise rates in selected training samples).
Symmetric 40%
Asymmettic 40%
Pairflip 40%
Instance.
Lost-based Selection (Han et al,, 2018)
83.01% (10.44%)
83.79% (4.845)
84.16% (10. 88%)
82.87% (11. 17%)
Dynamic-bosed Selection (Yuan 2023)
89.39% (4.57%)
84.28% (3.37%)
84.71% (10.19%)
83. 13.12% (12.52%)
Dynamic- based + Early Cutting Selection
89.66% (4.94%)
84.85% (3.33%%
85.88% (9.52%)
84.31 % (12.06%)
Additional Samples Filtered by Farly Cutting
98 (56. 729))
191 (95.29%)
161 (45, 969))
300 (91.333)
4.2 Comparison with the Competitors
Competitors. We compare our approach with several methods: robust loss functions including GCE (Zhang and Sabuncu, 2018) and Student Loss (Zhang et al., 2024b); robust training methods, including Co-teaching (Han al., 2018) and CSGN (Lin al., 2024b); and sample selection methods,
7

---

Table 2: Test performance (mean:istd) of each approach using ResNet-18 18 on CIFAR-10.
Symmetric 20%
Symmetric 40%
Instance. 20%
Instonce. 40%
Ctoss-Entropy
86.64 + 0.18%
82.64 = 0.29%
87.62 0.09%
82.82 = 0.37%
GCE (Zhang and Sabuncu, 2018)
91.50 + 0.33%
87.02 0.16%
89.42 0.31%
83.10 0.29%
Co-teaching (Han et al., 2018)
89.13 + 0.38%
82.29 + 0.21%
89.42 0.22%
81.91 0.20%
Me-Momentumn (Bai and Liu, 2021)
92.76 1 0.15%
90.75 1 0.49%
91.87 1 0.22%
88,80 0.29%
Self-Filtering (Wei ct al., 2022)
92.88 + 0.22%
90.46 = 0.28%
92.35 = 0.13%
86.93 = 0.14%
VOG (Agarwal et al., 2022)
87.90 + 0.22%
84.37 0.21%
87.71 0.15%
82.52 0.13%
Late Stopping (Yuan al., 2023)
92.02 1: 0.17%
88.25 1.01%
91.65 0.26%
88.28 0.24%
Misdetect (Deng al., 2024)
92.20 1 0.38%
87.31 0.30%
88.44 0.51%
85.11 0,42%
RIM (Li et al., 2024)
93.11 + 0.29%
91.06 + 0.17%
93.13 + 0.05%
89.73 0.32%
Student Loss (Zhang et al., 2024b)
91.90 t: 0.37%
89.03 + 0.32%
89.99 + 0.50%
81,95 0,51
CSGN (Lin et al., 2024b)
90.09 1 0.32%
87.71 0.46%
0.07% 89.45
88,50
0,49%
Farly Cutting (Ours)
93.79 + 0.14%
91.80 0.18%
93.40
0.22%
90.78 + 0.31 %
Table Test performance (mean+ of each approach using ResNet-34 on CIFAR-100 100.
Symmetric 20%
Symmetric 40%
Instance. 20%
Instonce: 40%
Cross-Entropy
63.04 + 0.41%
51.81 0.33%
63.36 0.22%
$1.58 0.96%
GCE (Zhang and Sabuncu, 2018)
66.68 + 0.35%
59.42 0.19%
64.71 0.15%
55.49 0.34%
Co-teaching (Han et al., 2018)
66.72 + 0.26%
58.72 = 0.43%
66.45 0.28%
59.52 0.32%
Mc-Momentum (Bai and Liu, 2021)
71.94 + 0.27%
67.36 = 0.30%
72.47 = 0.39%
63.99 = 0.56%
Self Self-Fiitering (Wei et al., 2022)
70.18 + 0.39%
66.92 0.18%
69.52 0.38%
66.76 0.42%
VOG (Agarwal et al.. 2022)
66.78 + 0.21%
60.55 0.40%
66.81 0.23%
56.57 = 0.32%
Late Stopping (Yuan al., 2023)
71.09 + 0.71%
65.43 0.50%
70.32 0.06%
61.71 0.25%
Misdetect (Deng al., 2024)
73,90 + 0.34%
65.10 0.40%
70.45 0.14%
63.66 0.17%
RLM (Li et al,, 2024)
71.68 1: 0.32%
67.68 + 0.36%
68.26 1 0.37%
67.31 0.64%
Student Loss (Zhang et al., 2024b)
69.04 1 0.19%
64.21 0.49%
67.62 0.67%
56,24 0.24%
CSGN (Liin et al., 2024b)
69.89 + 0.22%
56.18
0.36%
71.97 0. 10%
65.43 0.52%
Early Cutting (Ours)
76.20 + 0.27% %
72.77 0.17%
75.03 = 0.23%
69.94 = 0.30%
Table 4: Test performance (td) of each approach using ResNet-18 and 34 on CIFAR-N.
10N Random
10N Random 2
10N Rondom 3
JON Worst
100N Fine
Cross-Entropy
86.16 + 0.14%
85.74 + 0.28%
85.91 + 0.14%
80.00 + 0.42%
54.53 + 0.13%
Late Stopping (Yuan et al., 2023)
89.71 : 0.73%
90.23 1 0.37%
90.49 1 0.31%
86.10 + 0.41%
57.32 1 0. 0.19%
RIM (Lá et al., 2024)
92.21 0.37%
92.27 0.31%
92.07 0.72%
86.25 0.24%
57.90 + 0.33%
Student Loss (Zhang el al., 2024b)
90.60 1 0.07%
90.44 1: 0.28%
90.44 1: 0.35%
86.16 -: 0.31%
58.55 1 0.53%
CSGN (Lin et al., 2024b)
89.14 + 0.23%
89.49 + 0.25%
89.25 + 0.31%
82.88 + 0.51%
58.13 + 0.49%
Early Cutting (Ours)
92.50 +: 0.14%
92.65
0.11% % 92.36 + 0.43%
87.43 +: 0.13%
66.52 1: 0.22%
Table 5: Test performance of each approach using ResNet-50 on large-scale naturalistic datasets.
Web Validation ILSVRC12 Validation
Full ImageNet- Ik (Sym. 40%)
Cross- Frtss-Fntropy
67.32%
63.84%
67.99%
Lute Stopping (Yuan al., 2023)
71.56%
68.32%
71.42%
RLM (Li et al., 2024)
72.28%
69.86%
68. 68.95%
Student Loss (Zhang et al., 2024h)
69.809)
67.62%
69.44%
CSGN (Lin et al., 2024b)
72.32%
69.52%
Early Cutting (Ours)
73.81
71.20%
73.28 %
including Me-Momentum (Bai and Liu, 2021), Self-Filterin (Wei et al., 2022), VOG (Agarwal et al., 2022), Late Stopping (Yuan et al., 2023), Misdetect (Deng et al., 2024) and RLM (Li et al., 2024).
Datasets and implementation. We conducted experiments several benchmark datasets to evaluate our proposed method compare with above competitors. For synthetic noise experiments, we used CIFAR-10 and CIFAR- 100 (Krizhevsky et al., 2009), adding symmettic and ristance-dependees label noise at rates of 20% and 40% following standard protocols (Bai and Liu, 2021; Yuan et al., 2023). We split 10% noisy trianing data for validation. For real- world noisy labels, we utilized CIFAR-N (Wei et al., 2021b), as well as the large-scale Web Vision dataset (Li et al., 2017). Following previous work (Lin et al., 2024b: Li et al., 2024), we used the first 50 classes of the WebVision dataset and validated on both the WebVision validation set and the ILSVRCI2 (Russakovsky et al., 2015) validation set. We further confirmed the scalability of our proposed method on the full ImageNee-IK (Deng et al., 2009) with 40% synthetic symmetric label noise. Training was performed using SGD (Robbins and Monro, 1951) with momentum (Rumelhart et al., 1986) of 0.9 and a weight decay (Krogh and Hertz, 1991) of 5 10-2 The initial learning rate was set to 0. and decayed using a cosine annealing schedule.
8

---

Table 6: Test accuracy comparison of different approaches using semi-supervised learning.
CIFAR-10
CIFAR-100
Methods
SSL
Symmetric 50%
Instance. 40%
Symmetric 50%
Instance. 40%
Early Cutting (Ours)
90.3%
90.7%
69.6%
69.9%
CORES (Cheng et al., 2020)
UDA
93.1 1%
92.2%
73.1%
71.9%
Divide-Mix (Li al., 2020)
MixMatch
94.6%
93.0%
74.6%
71.7%
ELR+ (Liu et al., 2020)
MixMatch
93.8%
92.2%
72.4%
72.6%
SFT+ (Wei et al., 2022)
MixMatch
94.9%
94.1%
75.2%
74.6%
RLM+ (Li et al., 2024)
MixMatch
95.1%
94.8%
72.9%
72.8%
Farly Cutring+ (Ours)
MixMatch
95.8%
95.5%
75.6%
75.4%
Models were trained (for the final iteration) for 300 epochs on the CIFAR datasets, for 200 epochs on Vision, and for 150 epochs on full ImageNet-I4 Jk. We re- re-implemented all competitor methods with consistent settings (unless otherwise specified). For all experiments reporting meantstd, the results are the average and standard deviation of three trials using different random seeds. Detailed settings are provided in Appendix D.
Discussions on experimental results. As shown in Tables 2, 3, 4, and 5, our proposed Early Cutting method consistently achieves outstanding performance across various datasets and noise conditions. On standard benchmarks like CIFAR. it attains the highest test accuracy regardless of the type of label noise-symmetric,, instance-dependent, or real-world-ann across different noise rates. Notably, it performs exceptionally well on CIFAR-1O0, which has larger number of classes, indicating strong robustness in handling label noise in fine-grained classification tasks. Without further fine- tuning of hyperparameters, Early Cutting also achieves significant performance improvements on large-scale datasets such as WebVision and full ImageNet-lk. This demonstrates the practicality and scalability of our method in handling challenging scenarios. By iteratively selecting confident samples and removing harmful mislabeled easy examples, our method helps the model learn from reliable data while avoiding overconfidence in carly-learned samples. Furthermore, to demonstrate the transferability of our method to different model architectures, we conducted additional experiments on the transformer-based TinyVil VIT (Vaswani et al., 2017: Wu et al., 2022), which are detailed in Appendix D.9. Appendix D.7 provides a training time evaluation, demonstrating that our proposed Early Cutting method, executed once per training round, incurs additional computational overhead of less than one percent of the total training duration. This is significantly lower than the base computational cost associated with selecting the initial confident subset D in each training round.
4.3 Further Analysis
Semi-supervised learning. To further evaluate the effectiveness of our Early Cutting method, we integrated with the MixMatch (Berthelot al., 2019) semi-supervised learning framework, resulting in Early Cutring+. We treat the confident samples obtained from sample selection as labeled data, and the samples removed from training in the fully supervised setting as unlabeled data. The confident set (labeled data) and the non-confident set (unlabeled data) are identified once and remain fixed throughout the subsequent MixMatch training. We compared Early Cutring+ with advanced SSL- based LNL methods, including CORES (Cheng et al., 2020), DivideMix (Li et al., 2020), and ELR+ (Liu et al., 2020); additionally, we compared with the latest sample selection methods integrating SSL, SFT+ (Wei et al., 2022) and RLM+ (Li et al., 2024). Some baseline results are taken from Wei et al. (2022). As shown Table 6, our Early Culling+ achieves the highest test accuracy on both CIFAR- 10 and CIFAR- 100 under 50% symmetric and 40% instance-dependent label noise, surpassing previous methods. These results underscore the capability of Early Cutting in selecting high-quality subsets of training samples.
Sensitivity analysis. We conducted sensitivity analyses to evaluate the robustness of our Early Cutting method. As shown in Figure 5, our method achieves optimal test accuracy on both CIFAR- 10 and CIFAR-100 with 40% symmetric label noise when using the same default thresholds (also used for Web Visvision and full ImageNet-1k) for identifying samples with large loss, high confidence, low gradient norm, and early cutting rate. Notably, even when the hyperparameters vary over a wide range, our method exhibits minimal sensitivity, with only slight (< 1%) performance degradation. Results indicate that our method is robust and effective across different datasets without requiring extensive hyperparameter tuning. A further ablation study is presented in Appendix D.8.

---

60.5
60.5
740
240
Eary cxain rale
Easy orang ra
52.0
860
730
730
61.5
29.6
796
725
01.0
Ling kos
High confiianee
01.0
720
Lagg koa
High confitance
720
Low gracien nom
Low gradent rom
0000 1,8n
00.0
710 Jp8n
710
(a) CIFAR-10 with 40% symmetric label noise
(b) CIFAR-100 with 40% symmetric label noise
Figure 5: Sensitivity analysis of hyperparameters on CIFAR-10 10 and CIFAR- 100 with 40% symmetric label noise. In each subfigure, the left plot shows test accuracy versus thresholds for the large loss, high confidence, and low gradient norm criteria, scaled by factors of 1 1. 2, and 4. The right plot 4: 2 shows test accuracy versus Early Cutting rate set to n/33, n 1/2 n². and n² where 2 1.
5 Conclusion
In this paper, uncovered oversight in existing methods for learning with noisy labels by demonstrating that not all mislabeled examples harm the model's performance equally. We identified specific subset termed Mislabeled Easy Examples (MEEs)-mislaablle) mislabeled samples that the model learns early and that significantly mislead the training process. To address this issue, we proposed Early Cutting, counter-intuitive sample selection strategy that recalibrates the confident subset by leveraging the model's ss later training state, which is typically considered unreliable, to effectively filter out MEEs. This work provides practical solution for learning with noisy labels and advances the understanding of how different mislabeled samples affect deep learning models.
Limitation. We acknowledge several limitations that suggest avenues for future work. First, our empirical validation is confined to visual classification tasks, and the method's applicability to other domains such as natural language processing or tabular data remains to be explored. Second, while our work provides strong empirical evidence for the detrimental impact of MEEs, this justification primarily observational; formal theoretical analysis explaining why these specific examples disproportionately harm generalization would further strengthen our claims. Third, as an iterative strategy, our method introduces computational overhead by requiring re-evaluation of samples and computation of metrics like input gradients at later training stages (see Appendix D.7 for detailed analysis). Finally, potential failure modes may arise extreme data scenarios. In datasets with severe label noise or significant class imbalance (e.g., long-tailed distributions), the distinction between MEES and clean hard examples from rare classes could become ambiguous. This poses a risk of inadvertently filtering out valuable samples from minority classes, which could potentially introduce fairness concerns by biasing the final model against underrepresented groups.
Acknowledgments
The authors would like to thank the anonymous reviewers for their insightful and constructive comments. The authors are also grateful to the Area Chairs for their diligent work. During the preparation of this paper, the authors utilized large language models to enhance the clarity of the writing and to generate code for training and visualization. TL is partially supported by the following Australian Research Council projects: FT220100318. DP220102121 LP220100527, LP220200949. BH was supported by RGC Young Collaborative Research Grant No. C2005-24Y and RGC General Research Fund No. 12200725.
References
Chirag Agarwal, Daniel D' D'souza, and Sara Hooker. Estimating example difficulty using variance of gradients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages (0368-10377) 2022.
Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et closer look at memorization in deep networks. In ICML, 2017.
10

---

Christina Back. Zico Kolter, and Aditi Raghunathan. Why is sam robust to label noise? arXiv preprint arXiv:2405.03666, 2024.
Yingbin Bai and Tongliang Liu. Me-momentum: Extracting hard confident examples from noisily labeled data. In CVPR, 2021.
Yingbin Bai, Zhongyi Han, Erkun Yang, Jun Yu, Bo Han, Dadong Wang, and Tongliang Liu. Subclass-dominant label noise: A counterexample for the success of early stopping. NeurIPS, 2023.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems, 32, 2019.
Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu, Learning with instance- dependent label noise: sample sieve approach. arXiv preprint arXiv:2010.02347, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
Yuhao Deng, Chengliang Chai, Lei Cao,, Nan Tang, Jiayi Wang, Ju Fan, Yuan, and Guoren Wang. Misdetect: Iterative mislabel detection using early loss. Association for Computing Machinery (ACM), 2024.
Erik Englesson and Hossein Azizpour. Robust classification via regression for learning with noisy labels. In The Twelfth International Conference on Learning Representations, 2024.
Vitaly Feldman. Does learning require memorization? a short tale about a long tail. In STOC, 2020.
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. NeurlPS, 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021.
Chen Gong, Yongliang Ding, Bo Han, Gang Niu, Jian Yang, Jane You, Dacheng Tao, and Masashi Sugiyama. Class-wise denoising for robust learning under label noise. IEEE Trunsactions on Puttern Analysis and Machine Intelligence, 45(3):2835-284 2023. doi: 10.1 10.1109/TPAMI.2022 3178690.
Bo Han, Quamming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. NeurlPS, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Ziming Hong, Zhenyi Wang, Li Shen, Yu Yao, Zhuo Huang, Shiming Chen, Chuanwu Yang, Mingming Gong, and Tongliang Liu. Improving non-transferable representation learning by harnessing content and style. In The twelfth international conference on learning representations, 2024.
Zhuo Huang, Chang Liu, Yinpeng Dong, Hang Su, Shibao Zheng, and Tongliang Liu. Machine vision therapy: Multimodal large language models can enhance visual robustness via denoising in-context learning. In Forty-first International Conference on Machine Learning, 2024.
Ziheng Jiang, Chiyuan Zhang, Kunal Talwar, and Michael C Mozer. Characterizing structural regularities of labeled data in overparameterizee models. In JCML, 2021.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference machine learning, pages 1885-1894 PMLR, 2017.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.

---

Anders Krogh and John Hertz. A simple weight decay can improve generalization. Advances in neural information processing systems, 4, 1991.
Fengpeng Li, Kemou Li, Jinyu Tian, and Jiantao Zhou. Regroup median loss for combating label noise. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13474-13482, 2024.
Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi- supervised learning, arXiv preprint arXivv2000.07399, 2020.
Muyang Li, Runze Wu, Haoyu Liu, Jun Yu, Xun Yang, Bo Han, and Tongliang Liu. Instant: Semi-supervised learning with instance-dependent thresholds. Advances in Neural Information Processing Systems, 36:2922-2938, 2023.
Wen Li, Limin Wang. Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data. arXiv preprint 178802178..02882,, 2017.
Runqi Lin, Chaojian Yu, Bo Han, and Tongliang Liu. On the over-memorization during natural, robust and catastrophic overfitting. In The Twelfth International Conference on Learning Representations, 2024a.
Yexiong Lin, Yu Yao, Xiaolong Shi, Mingming Gong, Xu Shen, Dong Xu, and Tongliang Liu, Cs-isolate: Extracting hard confident examples by content and style isolation. Advances Neural Information Processing Systems, 36:58556-58576 2023.
Yexiong Lin, Yu Yao, and Tongliang Liu. Learning the latent causal structure for modeling label noise. In The Thirty-eigtth Annual Conference on Neural Information Processing Systems, 2024b.
Sheng Liu, Jonathan Niles Weed, Narges Razavian, and Carlos Granda. Early-learning regularization prevents memorization of noisy labels. NeurlPS, 2020.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, In International Confer- ence on Learning Representations, 2019.
Yang Lu, Yiliang Zhang, Bo Han, Yiu-ming Cheung, and Hanzi Wang. Label-noise learning with intrinsically long-tailed data. In ICCV, 2023.
Pratyush Maini, Saurabh Garg, Zachary Chase Lipton, and Zico Kolter. Characterizing datapoints via second-split forgetting. In ICML 2022 Workshop on Spurious Correlations, Invariance and Stability, 2022.
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. Advances in neural information processing systems, 26, 2013.
Due Tam Nguyen, Chaithanya Kumar Mummadi, Thi Phuong Nhung Ngo. Thi Hoai Phuong Nguyen, Laura Beggel, and Thomas Brox. Self: Learning to filter noisy labels with self-ensembling. arXiv, 2019.
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. Sensitivity and generalization in neural networks: an empirical study. In International Conference on Learning Representations, 2018.
Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back procpprrpgg errors. nature, 323(668)) 1986.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 (3):211-255) 1-222, 2015. doi:
12

---

Clayton Scott, Gilles Blanchard, and Gregory Handy. Classification with asymmetric label noise: Consistency and maximal denoising. In Conference on learning theory, pages 489-511. PMLR, 2013.
Hwanjun Song, Minseok Kim, and Jae-Gil Lee, Selfie: Refurbishing unclean samples for robust deep learning. In ICMI., 2019a.
Hwanjun Song, Minseok Kim, Dongmin Park, and Jae-Gil Lee. How does early stopping help generalization against label noise? arXiv preprint arXiv:1911.08059 2019b.
Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Robust learning by self-transition for handling noisy labels. In Proceedings of the 27th ACM SIGKDD Conference Knowledge Discovery & Data Mining, pages 1490-1500, 2021.
Cheng Tan, Jun Xia, Lirong Wu, and Stan Z Li. Co-learning: Learning from noisy labels with self-supervision. In Proceedings of the 29th ACM International Conference on Multimedia, pages 1413, 2021.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey Gordon. An empirical study of example forgetting during deep neural network learning. In ICLR, 2018.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Xinshao Wang. Yang Hua, Elyor Kodirov, Sankha Subhra Mukherjee, David A Clifton, and Neil M Robertson. Proselfic: Progressive self label correction towards low-temperature entropy state. arXiv preprint arXhv:2207.001188 2022.
Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: joint training method with co-regularization. In CVPR, 2020.
Hongxin Wei, Lue Tao, Renchunzi Xie, and Bo An. Open-set label noise can improve robustness against inherent label noise. Advances in Neural Information Processing Systems, 2021a.
Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. arXiv, 2021b.
Qi Wei, Haoliang Sun, Xiankai Lu, and Yilong Yin. Self feff-filtering: A noise- sample selection for label noise with confidence penalization. In ECCV, 2022.
Kan Wu, Jinnian Zhang, Houwen Peng, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan. Tinyvit: Fast pretraining distillation for small vision transformers. In European conference on computer vision, pages 68 85. Springer, 2022.
Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong. Nannan Wang, Zongyuan Ge, and Yi Chang. Robust
Hindering the memorization of noisy labels. In ICLR, 2020a.
Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang. Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards -espedeed label noise. NeurlPS, 2020b.
Xiaobo Xia, Tongliang Liu, Bo Han, Mingming Gong, Jun Yu, Gang Niu, and Masashi Sugiyama. Sample selection with uncertainty of losses for learning with noisy labels. arXiv, 2021.
Xuanyu Yi, Kaihua Tang, Xian-Sheng Hua, Joo-Hwee Lim, and Hanwang Zhang. Identifying hard noise in long-tailed sample distribution. In ECCV, 2022.
13

---

Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In International conference on machine learning, pages 7164-7173. PMLR, 2019.
Suqin Yuan, Lei Feng, and Tongliang Liu. Late stopping: Avoiding confidently learning from mislabeled examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16079-16088, 2023.
Suqin Yuan, Lei Feng, and Tongliang Liu. Early stopping against label noise without validation data. In The Twelfth International Conference on Learning Representations, 2024.
Suqin Yuan, Runqi Lin, Lei Feng, Bo Han, and Tongliang Liu. Instance-dependent early stopping. In The Thirteenth International Conference on Learning Representations, 2025.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 2021.
Jingfeng Zhang, Bo Song, Haohan Wang, Bo Han, Tongliang Liu, Lei Liu, and Masashi Sugiyama. Badlabel: A robust perspective on evaluating and enhancing label-noise learning. IEEE transactions pattern analysis and machine intelligence, 2024a.
Shuo Zhang, Jian-Qing Li, Hamido Fujita, Yu-Wen Li, Deng-Bao Wang, Ting-Ting Zhu, Min- Ling Zhang, and Cheng-Yu Liu. Student loss: Towards the probability assumption in inaccurate supervision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4666):4460-44755 2024b. doi: 10.1 1..10///PMM12222:33555
Zhilu Zhang and Mert Sabuneu. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018
Tianyi Zhou, Shengjie Wang, and Jeff Bilmes. Robust curriculum learning: from clean label detection to noisy label self-correction. In ICLR, 2021.
14

---

Learning with Noisy Labels (LNL) has been an active research area recent years (Wang et al., 2022; Gong et al., 2023; Baek et al., 2024; Englesson and Azizpour, 2024; Huang et al., 2024), with numerous methods proposed mitigate the impact of label noise on deep neural networks (DNNs). Formally, let X denote the input space, and let ) {1, 2...., K} be the set of possible labels. Let Y be the random variable for the clean label and Y be the random variable for the observed noisy label, A Related Work
both taking values in V. Consider the clean data distribution P(X, Y), from which clean samples (x. y) are drawn. In practice, we often have access only to training dataset with potentially noisy labels:
D {(x,, 9t)
(7)
where x; E X and ü: E Y' are observed noisy labels. The aim is to learn a robust classifier f X - parameterized by f, which performs well on clean test data drawn from the distribution P(X, Y). The noise process is typically modeled using noise transition matrix T E k x defined as:
Ti P(Y j Y = i). for e
(8)
which represents the probability that a clean label y is flipped to a noisy label ÿ = j. The relationship between the clean and noisy label distributions can be expressed as:
P(Ý i x =x)
Σ kEY
The = | x).
(9)
In the context of deep learning, the classifier fx(x often trained by minimizing the empirical risk over the noisy dataset:
min 1 TI Σ e (fo(x.).
(10)
where (())) is loss function, such as the cross-entropy loss:
E {fo( 9) -log (580) (x))
(11)
and (x) denotes the predicted probability for class ÿ. However, due to label noise, directly minimizing this loss can lead to the model overfitting to noisy labels, degrading its performance on clean data. To address this issue, various strategies have been proposed. In the following discussion, we focus on heuristic methods, specifically sample selection techniques, which do not rely on the explicit estimation of T but instead incorporate strategies to mitigate the impact of noisy labels.
Sample selection strategies. Sample selection has been widely used in learning with noisy labels to improve the robustness of model training by prioritizing confident samples. An in-depth understanding of deep learning models, particularly their learning dynamics, has facilitated research in this area. Extensive studies on the learning dynamicy of DNNs have revealed that difficult clean examples are typically learned in the later stages of training (Arpit et al., 2017; Toneva et al., 2018; Lin et al., 2024a).
In general, sample selection methods assign a statistical characteristic to each sample and select subset of samples that fall below a certain threshold (Han et al., 2018). The selection indicator function s; is defined as:
1, if (ff iA) S
(12)
0, otherwise,
where 7 is a dynamically adjusted threshold. The training objective becomes:
1 EL (fo(()()))))))
min
(13)
A common approach is the small-loss trick, by focusing on low-loss samples, the model is less influenced by potentially mislabeled data. Methods like Co-teaching (Han et al., 2018), Co-teaching+ (Yu et al., 2019), JoCoR (Wei et al., 2020). and Co-learning (Tan et al., 2021) utilize two networks trained in parallel that teach each other using reliable samples. SELF (Nguyen et al., 2019) identifies clean samples by checking the consistency between network predictions and given labels, while
15

---

DivideMix (Li et al., 2020) employs wwo-oomoneet mixture model to separate the training data into clean and noisy groups. Moreover, ELR (Liu al., 2020) avoid overfitting to noisy labels by relying on early-learning.
Learning dynamics reaearch for sample selection. The intriguing generalization ability of modern DNNs has motivated extensive studies on their learning dynamics, which in turn has inspired a series of sample selection criteria using in (1. based on these dynamics. Studies have revealed that hard and mislabeled examples are typically learned during the later stages of training (Arpit et al., 2017; Song et al., 2019b, 2021; Maini et al., 2022; Li al., 2023; Lin al., 2024a; Hong et al., 2024). This empirical observation has led to the development of various training- -tmm metrics to quantify the "hardness" of examples, such learning speed (Maini et al., 2022; Jiang et al., 2021) and gradient variance. For instance, Agarwal et al. (2022) proposed Variance of Gradients (VoG) to estimate sample difficulty based on the temporal variability of gradient norms, while Novak et al. (2018) analyzed generalization through input-output Jacobian norms, connecting sensitivity in input space to learning robustness. These metrics have inspired LNL approaches that leverage learning dynamics to select clean samples. Methods like Self- Self-Filtering (Wei et al., 2022), FSLT & SSFT (Maini et al., 2022), SELFIE (Song et al., 2019a), and RoCL (Zhou et al., 2021) 1) adopt criteria to identify clean samples based on their learning dynamics. The success of learning dynamics-basel sample selection criteria in identifying high-confidence clean samples has driven researchers to further refine these strategies. By identifying larger subset of clean samples for model training, the generalization performance of the trained model can be improved. (Xia et al., 2021) discovered that using loss alone to select CHEs is suboptimal. RLM (Li et al., 2024) obtain robust loss estimation for noisy samples.
An advanced paradigm for sample selection involves a positive feedback loop: iteratively optimizing the classifier and updating the training set. Under this loop, the model's performance gradually improves, leading to better sample selection capabilities and, consequently. an enhanced ability to select clean hard examples. Me-Momentum (Bai and Liu, 2021) and Late Stopping (Yuan et al., 2023) employ similar positive feedback loops to iteratively update the model parameters and the training set, gradually improving the model's performance on noisy data.
Hard label noise. Various forms of hard label noise have been studied (Lin et al., 2023), including asymmetric noise (Scott et al., 2013), instance- deeundded noise (Xia et al., 2020b), natural noise (Wei et al., 2021b), adversarially crafted labels (Zhang et al., 2024a), open-set noise (Wei et al., 2021a), and subelass-dominatt noise (Bai et al., 2023). These noise types are designed from the perspective of the labels, aiming to simulate challenging real-world scenarios or malicious attacks. Recent work has also explored the impact of label noise in specific data distributions. For instance, H2E (Yi et al., 2022) and TABASCO (Lu et al., 2023) focus on the challenges posed by label noise in long-tailed distributions, where minority classes are more susceptible to mislabeling. NoiseCluster (Bai et al., 2023) introduces the concept of subelass-dominant label noise, where mislabeled examples dominate at least one subclass, leading to suboptimal classifier performance.
Our contributions. In contrast to prior studies that mainly focus on different types of label noise or sample selection based on learning dynamics, our work offers fresh perspective by re-examining sample selection methods that rely on a model's early learning stages. We demonstrate that some samples hidden among those considered "confident" are, in fact, the most harmful when mislabeled. Specifically, we systematically investigate the detrimental impact of Mislabeled Easy Examples mMilas-mislabelled samples that are correctly predicted by the model early in the training process. This insight challenges the conventional assumptions of existing methods, which often prioritize samples learned early in training as being clean. Our findings highlight the need for a more cautious approach when selecting samples based on early learning confidence. By adopting a refined sample selection criterion that accounts for the potential harm of MEEs, we can seamlessly integrate this approach with existing sample selection method (Yuan et al., 2023) to further boost it performance. Minimization (SAM)
Furthermore, our proposed method conceptually linking Sharpness- (Foret et al., 2021), but applied in the input space to identify stable yet mislearned examples.
16

---

B Quantitative Analysis of MEE Harm with Influence Functions
To provide more direct and quantitative measure of the harm caused by Mislabeled Easy Exam- ples (MEEs), we conducted an analysis using influence functions (Koh and Liang, 2017). This method allows us to calculate the impact score of different training samples on the model's per- formance (i.e., loss) on a clean, held-out validation set. A positive score indicates a harmful influence, while negative score indicates a beneficial one. This analysis, performed using the i ons library on CIFAR-10 10 with 40% instance-dependennt noise, parallels the experiment in Figure 2. We compared three distinct sample categories: (1) MEEs, defined as the first 4,000 mislabeled samples learned during training: (2) Clean Easy Examples, the first 4,000 clean samples learned; and (3) Mislabeled Hard Examples, the last 4,000 mislabeled samples learned. The results provide direct, quantitative proof of our central claim. While all mislabeled examples exhibited a harmful positive influence, the average harm of MEEs was significantly greater than that of Mislabeled Hard Examples, with a mean influence score of 4.96 versus 3.01 (and a median of 4.48 vs. 3.26). This demonstrates at microscopic level that MEES are far more detrimental to the model. In contrast, the Clean Easy Examples showed mean negative influence score of -2.22 (median -1.89), confirming their beneficial impact on generalization. This analysis further substantiates our claim in Section 2.2 regarding the unique and disproportionate harm caused by MEES.
pytorch_infl wennee
C Discussion on Broader Applicability
While the empirical validation in this paper is focused on visual classification, the core concepts of Mislabeled Easy Examples (MEEs) and Early Cutting are not inherently limited to this domain. We briefly discuss the potential translation of these idcas to other areas. In Natural Language Processing (NLP), MEE could manifest as text sample with strong but misleading keywords. For instance, sarcastic positive review such as, "Wow, can't believe how awful the service was," being mislabeled as "Positive". A model might quickly and confidently learn this incorrect association due to the presence of words like "Wow" and "can't 't believe", which often appear in positive contexts. The Early Cutting criteria could be conceptually adapted by, for example, measuring the gradient norm with respect to the input word embeddings to gauge stability. Similarly, in regression tasks, an MEE might be data point where specific feature has a strong, simple, and local linear relationship with the target value, but this relationship is globally spurious or an artifact of noise. The model would likely fit this deceptive pattern early in training. Our criteria could be conceptually translated by identifying samples that the later-stage model predicts with high confidence (c.g., in a value range far from the given label, thus incurring high loss) and stable input gradients. Though these examples are conceptual, they suggest that the underlying principle of identifying and removing confidently mislearned easy examples (MEEs) could be generalizable strategy for enhancing model robustness across diverse machine learning domains.
D Detailed Settings
D.1 Datasets
CO and CIFAR-100 (00000KK (Krizhevsky et al., 2009) are standard image classification datasets con- sisting of 32 x 32 color images. Both datasets were divided into 50,000 training images and 10,000 test images. CIFAR-N (Wei et al., 2021b) is version of CIFAR- 10 and CIFAR- 100 with real-world noisy labels collected from Amazon Mechanical Turk. These datasets simulate real-world scenarios where labels may be noisy due to human error. We used consistent 90%-10% data splits for training and validation across runs in all competitors.
Web Vision (Li et al., 2017) is large-scale dataset containing over 2.4 million web images crawled from the internet. It covers the same 1,000 classes as the ILSVRC12 ImageNet- - dataset (Deng et al., 2009) but includes noisy labels due to the automatic collection process. ILSVRC/2 ImageNet- K (Deng et al., 2009) is a large-scale dataset of natural images with 1,000 classes. We used to assess the scalability of our method on real-world data with synthetic noise.
17

---

D.2 Noise Settings
In preliminary presentation of our proposed method's effectiveness (Table 1), we tested four types of synthetic label noise. For Symmetric Noise, each label has fixed probability r of being uniformly flipped any other class. Asymmetric Noise flips labels to similar but incorrect classes, mimicking mistakes that might occur in real-world classification tasks. Puirflip Noise involves flipping labels to specific incorrect class in a pairwise manner. Instance-Dependen Noise (Xia et al., 2020b) is a more challenging setting where the prohability of label corruption depends on the instance features. reflects more realistic scenarios where difficult or ambiguous examples are more likely to be mislabeled.
Following prior practices (Bai and Liu, 2021; Yuan al., 2023), we primarily focused on Symmetric and Instance-Depaa Dependent noise types in our baseline comparisons (Table 2 and 3), they are the most common and challenging synthetic noise settings used to evaluate robustness methods. We experimented with noise rates of 20% and 40% to assess our method's 's performance under varying noise intensities. For the CIFAR-N task, we utilized the provided noisy labels.
D.3 Model Architectures
We employed variants of the ResNet architecture (He et al., 2016) in all our experiments, training each model from scratch. Specifically, we used R for CIFAR-1O 10, ResNer-34 for CIFAR-108 and ResNer-50 for WesVision and ImageNet-I4 IK datasets. This selection aligns with previous works and provides appropriate model capacity relative to each dataset.
D.4 Training Procedures and Hyperparameters
Training was performed using Stochastic Gradient Descent (SGD) with a momentum of 0.9 and weight decay of 5 x 10-1. The initial learning rate was set to 0.1 and decayed using cosine annealing schedule without restarts, decreasing
x 10-5 over the course of training. The number of training epochs was set to 300 for CIFAR, 200 for WebVision, and 150 for full ImageNet- K experiments. Batch sizes were set to 32 for CIFAR datasets and WebVision, and 256 for ImageNet-IK.
To enhance the robustness of our sample selection model, we also incorporated certain strategies from prior works (Lin et al., 2024b; Li al., 2024), training two networks and each network learn from the other's 's soft predictions and utilizing exponential moving averages to stabilize training. Weak data augmentation techniques were applied during training to improve generalization. These included random cropping with a padding of 4 pixels, random horizontal flipping, and normalization using the sataset-specific mean and standard deviation.
D.5 Sample Selection Mechanism
Building upon the Late Stopping strategy (Yuan et al., 2023), we iteratively select a confident subset D of training samples, progressively reducing mislabeled data and enhancing the model's focus on clean samples. We identify early-learned samples based on their learning times. For each sample (x,, ūs), we define its learning time LT, the earliest epoch when the model's prediction stabilizes:
LT: min
Ti
0.
F.
0. = =ū:
(14)
where if denotes the model's predicted label at epoch e.
To further address the issue of Mislabeled Easy Examples (MEEs), we introduce an Early Cutting step in the training loop. We first select candidates using an Early Cutting Rate 7 of 1.5, which corresponds to selecting the earliest R & of samples learned. Within these candidates, we remove samples that meet all three of the following criteria (detailed in Section 3). First, we consider samples with high loss, specifically those within the top 10% of loss values L,. Second, we look at samples with high prediction confidence, namely those within the top 20% of confidence scores G. Third, we identify samples with low gradient norms, that is, those within the bottom 20% of gradient norms 9:- By removing samples that satisfy all three conditions, we aim to eliminate MEES that the model has confidently mislearned early on.
The refined subset De then used for subsequent training. We repeat the sample selection process for total of Late rounds (set to 3), progressively improving data quality and model performance. The
18

---

proportion of DD retained in each round is calculated to achieve an overall retention rate equal to the complement of the noise rate after faee rounds. For example, with a noise rate of 40% (aiming to retain 60% of the data), the retention rate per round (60%) 1/3 84%.
D.6 Baselines and Competitors
We re-implemented these methods under the same experimental settings proposed method. When re-implementing CSGN (Lin et al., 2024b) using only supervised learning for Table 2, 3, 4, and 5. We used the A (Loshchilov and Hutter, 2019) optimizer and stepped decay learning rate schedule, as specified in the original code. Notably, CSGN (Lin et al., 2024b) cannot handle tasks with too many classes such ImageNet-Ik well.
D.7 Training Time and Computational Complexity
Computational Complexity. The additional computational overhead introduced by the Early Cutting step itself, when applied to subset of samples, involves three main operations:
1. Vectorized computation of the three metries (loss Li, confidence c,, and gradient norm g:): This is achieved in (n) time.
2. Identification of percentile thresholds: This requires sorting operations on the metrics, which takes (( log n) time.
3. Filtering the samples based on these thresholds: This is an (n) operation.
Thus, the dominant term for the Early Cutting step is O( log n) per application. Crucially, Early Cutting is executed only once training round (c.g., after certain number of epochs leading to model state fo rather than per epoch. This significantly amortizes its over the entire training process. For instance, in our CIFAR-10 experiments using ResNet-18 architecture, single training epoch typically required approximately 42.7 seconds. The cumulative overhead for all Early Cutting operations throughout the entire training (e.g., 3 rounds) was approximately 70.3 seconds. This represents less than 1% of the total computational cost for typical 200-epoch training process.
Taking more demanding scenario like CIFAR-100 with ResNet-34 as example, where the full Early Cutting method takes approximately 15 hours, the contribution of the Early Cutting specific steps remains proportionally small. The majority of the time is consumed by the base iterative sample selection and model retraining process. Table provides an illustrative breakdown.
Table 7: Illustrative runtime breakdown for "Early Cutting (Original)" on CIFAR 100.R-0000/esee--44
Component
Illustrative Runtime Contribution
Base sample selection & retraining (iterative) Early Cutting (Ours) MEEs filter (cumulative)
-15.2 hours
-0.1 hours
Total Training Time
15.3 hours
Overall Training Hours and Variants Early Cutting: While the Early Cutting step itself is efficient, the overall training duration also depends the number of iterative refinement rounds and the base training time per round inherited from the underlying iterative sample selection framework (e.g., based on Late Stopping). We explored faster variant of our Early Cutting framework, Early Cutting (Faster), which performed only one sample selection iteration (compared to three iterations in original design), to balance performance with computational budget. Table 8 provides comparative overview of total training hours and performance for various methods and our Early Cutting variants CIFAR- 100 with ResNet-34, tested single NVIDIA RTX 4090 GPU.
D.8 Ablation Study on Each Component in Early Cutting Method
To validate the contribution of each distinct component and the design choices within our Farly Cutting strategy, we conducted an ablation study. The Farly Cutting method identifies Mislabeled Easy Examples (MEEs) based their characteristics later training stage t: high loss L,, high
19

---

Table 8: Comparison of total training hours and test accuracy (mean:istd, CIFAR-100, ResNet-34, 40% Symmetric noise rate).
Method
Runtime (Hours)
Test Accuracy (9-
Me-Momentum (Bai and Liu, 2021)
-5
63.99 10.56
Late Stopping (Yuan et al., 2023)
-17
61.71 + 0.25
RLM et al., 2024)
-4
67.31 + 0.64
CSGN (Lin al., 2024b)
-9
65.43 + 0.52
Early Cutting (Faster- Ours)
-9
69.53 + 0.10
Early Cutting (Original Ours)
-15
69.94 + 0.30
prediction confidence Ci and low gradient norm 9:- Additionally, the 'early cutting rate' itself, which determines the initial pool of early-learned samples considered for MEE filtering, is key aspect.
performed the ablation study on a benchmark dataset (e.g., CIFAR-IO/I00 with a specific noise setting, for which the provided results are shown below, assumed to be for representative scenario like CIFAR-10 10 with 40% instance noise or similar, leading to the baseline 'No Early Cutting' accuracy of 18.8.122 13)). The results, detailed in Table 9, quantify the impact of removing each component not applying Early Cutting at all.
Table 9: Ablation study results demonstrating the contribution of each component in Early Cutting.
Method / Variant
Test Accuracy
Full Early Cutting
84.57%
Early cutting rate
84.12%
Loss values criterion
82.36%
Confidence scores criterion
84.10%
Gradient norms criterion
84.07%
No Early Cutting (Baseline using D only)
83.12%
The results demonstrate that:
The full Early Cutting method (84.57%) significantly outperforms the baseline where no MEE filtering is applied to the initially selected confident subset (8%..2% This highlights the substantial benefit of the recalibration and MEE removal process.
Removing the loss values criterion causes the most significant drop in performance (to 82.36%), underscoring its critical role in identifying samples where the model's later-stage understanding contradicts the noisy label.
Omitting the early cutting rate consideration (which might imply either considering all of DD for MEE filtering without pre-selection by earliness, or suboptimal rate) leads to noticeable decrease (84. 12%), suggesting that focusing the MEE search on the very earliest learned examples is beneficial.
Removing the confidence scores (84.10%) or gradient norms (84.07%) criteria also results in reduced accuracy, confirming their importance in refining the MEE identification by ensuring the model is certain and stable in its (incorrect) predictions for these MEEs.
These findings collectively validate that all components of the Early Cutting strategy synergistically contribute to its effectiveness in improving sample selection quality and model performance by precisely targeting and removing MEEs. The redundancy of clean, easy samples (learned early) also means that the accidental removal of few such samples during this stringent filtering has less detrimental impact compared to retaining harmful MEEs.
D.9 Transferability of Default Parameters and Method Robustness
Robustness to Threshold Variation. Our method primarily relies identifying proportions of samples based on their relative rankings for loss, confidence, and gradient norm. This percentile-baser approach naturally adapts to different data distributions without requiring absolute threshold values.
20

---

(a)lmpact model performance from mislabeled examples learned at different stages, using CIFAR -000.
Okin 10 40001 Misiaered
Claan 16000 80C01 Mishbeledd
Olaan e 19000 120059 Misacctoo Clean (12030: 160001 Misiabeied Clean 115000 200001 Mishaoeeed
Epochs
Epochs
Esok/r
Epccls
Epects
Figure 6: Impact of mislabeled samples learned at different stages model generalization perfor- mance.
(a) speed at which pretrained models CIFAR- 100 learn mislabeled examples from different stages.
@7 (020) Mitaceled
50CO) Mesooele
28000
Midateled
Cen 112000
10000)
(10020 20000) Menbeled
5000
1000
Eoocts
Esoche
Opedha
Opecrs
Ezache
Figure 7: Comparison of how pretrained models learn mislabeled examples from different learning stages.
As demonstrated in Figure 5 (in the main paper), varying these percentile thresholds by considerable factors (e.g., from 1/4 to times the default proportions) results less than a 1% change test accuracy across various datascts. This low sensitivity is partly because parameters by the theoretical characteristics of MEEs: high loss (reflecting incorrect predictions by the more mature model), high confidence (model's certainty in these incorrect predictions), and low gradient norm (stability these incorrect predictions). Furthermore, as clean samples learned early are often abundant and exhibit redundancy, the mistaken removal of small fraction of these due to slight variations in thresholds does not significantly impair generalization.
guided
Consistent Performance with Default Hyperparameters. As evidenced in Tables 2-5 (in the main paper), we applied consistent default hyperparameters for Early Cutting across wide range of experimental setups. This includes different datasets CIFAR-10, CIFAR- 100, WesVision, full ImageNet- 1k) and various noise conditions (symmetric, destane--depndentt real-world CIFAR-N). Despite this, Early Cutting consistently achieved Mlate-of-the-ar or highly competitive performance.
Further Validation on New Datasets and Architectures. To further substantiate the transferability and robustness of Early Cutting with its default settings, we conducted additional experiments on new datasets and with different model architectures, beyond those in the main paper. The results are presented in Table 10.
Table 10: Transferability of Early Cutting with default parameters to new datasets and architectures.
Dataset
Model
Method
Test Accuracy
CIFAR-10
Tiny VIT
Early Cutting (Ours)
75.42%
(Instance. 40%)
Late Stopping (Yuan et al., 2023)
72.96%
CE (Cross-Entropy)
69.31 %
Dishion-MNIST (Instance, 40%)
ResNet-18
Early Cutting (Ours)
94.1 %
Late Stopping (Yuan et al., 2023)
92.81 %
CE (Cross-Entropyy
90.87%
These results demonstrate that Early Cutting maintains its performance advantage even when applied the transformer-hasse Tiny-ViT architecture on CIFAR-10, and on different dataset like Fashion- MNIST with ResNet- 18 backbone, without any re-tuning of its core MEE identification parameters.
21

---

E Evolution of Feature Space and Distance Ratios
To supplement the analysis the main text (Figure 4()).. which shows the feature space and distance ratios at an early stage (epoch 10), we provide additional visualizations in Figure 8 for later stages of training. These figures illustrate the evolution of the feature space and, crucially, how the model's representation of MEEs and other mislabeled samples changes over time.
Laie
CaPoeN copertio
(h)
Figure 8: Evolution of the t-SNE feature space visualization on CIFAR-10 (20% instance -ependeddd label noise) (a) Epoch 40 and (b) Epoch 160. These figures complement Figure 4(a) (Epoch 10) from the main text.
22

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [NA].
Justification: NA
23

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper describes the experimental setup in detail. We will provide open source code after the potentially acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper describes the experimental setup in detail. We will provide open source code after the potentially acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https: /tte://nnppspee cc/
public/ /tclc//ggess//eee /ooeeSuodiiiseecccccco ssionPolicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NewrIPS code and data submission guidelines (https: / /nipss pouuliccguiideesseeeeeepppp Code Subeeuubmseee coomission onPolicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
25

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The paper describes the experimental setup in detail.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: The paper have reported error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
show in tables
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The experiments were fairly typical, and mentioned the servers ran them on when reporting the training times.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
26

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttt://neurrppp cc/ wwtppp/lcc/.ccccccaaaaoooooeaeaa
Answer: [Yes]
Justification: NA
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We introduced the social impact of learning with noisy labels in the introduc-
tion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
27

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does existing
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of the package should be provided. For popular datasets, papers senerwwi withhode com/ datasee asetasetio has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
28

---

Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer LLM policy (https: : /teee//nneeeee http///riss.c///offeeeeeeeeeeeee httc/Cooffeeenceeee 2025/ LLM) for what should or should not described.
29


Reviews:
[{"review_id": "YtPCvR51P6", "reviewer": "Reviewer_Sv4t", "summary": "The paper has proposed a re-selection way of the confident subset identified in the former iteration. To support this, the authors have tried to show some aspects of explaining why this operation is effective. Experimental tests have shown the performance of the selection way. But, in my view, the paper spent a considerable amount of space on meaningless analysis and definitions, resulting in the details of the core algorithm not being provided in the main text. Without so much analysis, the method is still intuitive and effective enough.", "questions": "See weaknesses.", "limitations": "Limitation is not clear.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. From the experiments, the proposed algorithm has achieved the SOTA performance.\n2. The paper is clear.\n\nWeaknesses\n1. Definition of MEE makes little sense. Its mathematical expression is not given.\n2. The authors employed around 3 pages to describe a kind of phenomenon that not all mislabeled examples harm the model’s performance equally, and then derived the MEE. But the phenomenon is intuitive. The redundant details did not uncover any significant insight of the phenomenon.\n3. In the Methodology section. What is the eventual method? What’s the specific objectives of the method? Why these important details are all in the appendix?", "was_revised": true, "final_justification": "I will keep my score", "rebuttal": "Dear Reviewer Sv4t and Area Chair,\n\nThank you for your thoughtful and constructive feedback on our manuscript. We have carefully revised the manuscript based on your suggestions and have detailed our revisions below.\n\n---\n\n>**1. On the Justification and Definition of Mislabeled Easy Examples (MEEs)**\n\nThank you for your feedback. We agree that the high-level notion that not all mislabeled examples are equally harmful can be intuitive. However, we would like to clarify that our core contribution is not merely to restate this intuition. Rather, our goal is to systematically quantify this phenomenon, providing it with a solid empirical and conceptual foundation.\n\nBased on quantitative experiments in Section 2, we extend this high-level notion to uncover a more specific and counter-intuitive finding: *the mislabeled examples learned earliest by the model, which we define as Mislabeled Easy Examples (MEEs), are particularly harmful to generalization.*\n\nThis key insight stands in direct opposition to a prevailing assumption in the *Learning with Noisy Labels* field: that early-learned examples are reliable and trustworthy, an assumption that forms the basis of several mainstream methods. Therefore, a detailed analysis in Section 2 is necessary to reveal this potential blind spot in current methodologies and to provide the fundamental motivation for our proposed Early Cutting strategy.\n\nTo address your valid concerns about clarity and conciseness, we have undertaken the following significant revisions:\n\n* **Streamlined Analysis:** We have streamlined the presentation of Section 2 to be more direct. We now focus on the most critical evidence supporting our core, counter-intuitive claim (e.g., Figures 1 and 2) and have significantly reduced the descriptive text.\n* **Formal Definition of MEEs:** In response to your request for a more rigorous formulation, we have introduced a precise operational definition in Section 3. Specifically, we have added a clear Definition 1 box, which is directly linked to and cross-referenced with Algorithm 1. As noted below, Algorithm 1 has been moved to the main text.\n\nWith these revisions, we aim to make our core conceptual contribution clearer, while conserving space in the main body for the methodological improvements described next.\n\n>**2. On the Clarity and Placement of the Methodology**\n\nWe fully agree with your assessment that core algorithmic details should not be relegated to the appendix. We have made the following revisions:\n\n* By streamlining Section 2, we have freed up space in the main body. This has allowed us to move key algorithmic details from Appendix B directly into *Section 3*. We have also included the pseudocode for *Algorithm 1* in the main body to provide a clear, step-by-step description of our final method.\n* In the revised Section 3, we now explicitly state the objectives of our proposed Early Cutting method and explain how the recalibration step is designed to identify and eliminate the MEEs identified in Section 2.\n\nThese changes make the methodology section more self-contained and ensure that readers can understand our core technical contribution without referring to the appendix.\n\n>**3. On the Missing Limitations Section**\n\nWe have added a *Section 6: Limitations* in the revised manuscript. In this section, we acknowledge the computational overhead of the training process and discuss potential fairness and performance concerns related to the method's application on datasets with class imbalance.\n\n---\n\nThank you once again for your valuable feedback.\n\nBest regards,\n\nAuthors"}, {"review_id": "B7k9VZSGqY", "reviewer": "Reviewer_5oqq", "summary": "Sample selection in learning with noisy labels typically identifies a confident subset of training\nexamples by reducing the noise rate within the selected data. However, such a strategy often overlooks\nthe fact that not all mislabeled examples are equally harmful to model performance. In contrast,\nthrough numerical experiments, this manuscript aims to demonstrate that predicted mislabeled\nexamples in the early stage of training, referred to as Mislabeled Easy Examples (MEEs), can be\nparticularly detrimental to performance. Building on this observation, the authors propose a method\ncalled Early Cutting to filter out MEEs. The method is evaluated on several benchmark datasets,\nincluding CIFAR, WebVision, and the full ImageNet-1k, to demonstrate its effectiveness.", "questions": "Below, I list some of my questions in the order of their appearance in the manuscript, rather than by significance. Please note that the list is not exhaustive, but is intended to illustrate key concerns.\n\n\\begin{itemize}\n    \\item  [1.]  Lines 39–43:\n\nThe statement ``As shown in Figure 1(a), we demonstrate that mislabeled samples which are correctly predicted by the model early in the training process disproportionately degrade performance\" lacks clarity. Specifically:\n\nA. There is no description of how Figure 1 was generated. As such, it is  unconvincing to support the authors’ claim.\n\nB. Clarify the meaning of ``mislabeled samples which are correctly predicted by the model...\".\n\n\nC. The phrase \"early in the training process\" needs clarification. How early is considered ``early\"? Does this vary across training algorithms or settings?\n\nD. The phrase ``disproportionately degrade performance\" is vague. Disproportionate to what? And whose performance is being degraded?\n\n\n\\item [2.] Lines 40-43:\n\n\nIn the statement ``In our analysis (see Section 2.2), we find that MEEs are often closer to the centers of their mislabeled classes in the feature space of classifiers trained in the early stages\", how are the ``centers” defined? In what sense are MEEs considered closer to these centers, e.g., in terms of Euclidean distance, cosine similarity, or other metric?\n\n\n\n\\item [3.] Lines 190-213:\n\n\n   The key idea is to introduce a set of suspicious samples ${\\cal S}$\n    in (3) and its refined version ${\\cal S}^\\prime$ in (5). While the rationale of using the scale of the cross-entropy loss, the prediction confidence, and  the gradient stability  seems reasonable, some fundamental issues are not addressed or fully discussed:\n\n   A. the predicted label $\\hat y_i$  is determined by the conditional probability $p_i^{(k)}$, which essentially requires the information on the clean label $y_i$ (BTW, there seems a typo in the definition of $p_i^{(k)}$: should $y$  be $y_i$?).\n   In the presence of label noise, the true label $y_i$ is typically unknown.\n     Then how do we determine $L_i$ and $c_i$\n   in order to construct ${\\cal S}$ and ${\\cal S}^\\prime$?\n\n\n\n\n   B. Assuming the issues in A is resolved, how do we  ensure that ${\\cal S}$ in (3) is not an empty   set? The constraints $L_i > \\delta$ and $c_i> \\tau$ can be equivalent written as\n   $$\n   p_i^{(y_i)} < \\exp(-\\delta) \\ \\text{and} \\ p_i^{(\\hat y_i)}> \\tau.\n   $$\n   To make ${\\cal S}$ mathematically meaningful and practically useful, appropriate constrains should be imposed on the thresholds $\\delta$ and $\\tau$. This important aspect seems to be neglected.\n\n    Although the author(s) state  \"Specifically, for all settings, we target the top 10\\% for loss, top 20\\% for confidence, and bottom 20\\% for gradient norm. These percentile-based selections are intentionally kept fixed across all experimental settings\n   to underscore the method’s general applicability and robustness, thereby obviating the need for dateset-specific hyperparameter tuning\", this treatment of the threshold values $\\delta$ and\n   $\\tau$ appears overly simplistic and does not adequately account for the diversity and  complexity of real-world data.\n\n\n\n\\item [4.] Lines 445 (in Appendix A):\n\n      A setting of instance-independent label noise is presented in (7). However, this key assumption does not appear to be clearly indicated in the main text. How\n        does this type of label noise relate to the construction of\n       ${\\cal S}$ and ${\\cal S}^\\prime$ in Section 3? Shouldn't these sets be constructed\n        based on the noisy labels and the cross-entropy loss defined in equation (10), rather than using $L_i$ as defined in Line 189?\n\n\\end{itemize}", "limitations": "They do not seem to make limitations clear. Addressing the questions or concerns above could help improve the clarity of the work.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n\n    1. This manuscript explores an important and often overlooked issue in sample selection strategies for learning with label noise.\n\n     2.  Specifically, it aims to distinguish the varying negative impacts of mislabeled examples on model performance, which is valuable.\n\n    3. The experimental results support the intended merit of the proposed method.\n\n\nWeaknesses:\n\n    1. The proposed methods is merely evaluated by numerical experiments, and there is no theoretical guarantee.\n\n    2. Some concepts or terms are introduced without sufficient motivation or justification regarding their rationale or effectiveness.\n\n    3.  Some notations are inconsistently defined or used. For example, the number of classes is denoted by $K$ in the main text (Lines 184–185), but by $C$ in \nAppendix A on Related Work.\n\n    4. Certain key statements lack clarity.\n\n    5. Repetitive descriptions seem to appear a bit too frequently throughout the manuscript. While repetition can help emphasize important points, its overuse may affect the overall conciseness.", "was_revised": true, "final_justification": "Dear AC:\n\nAll the rebuttals have been carefully reviewed. While I appreciate the authors' effort in trying to improve the work and some rebuttals help clarify certain comments,  critical concerns do not seem to be fully addressed. While the work may be of its own value, I do not feel it's up to the level of NeurIPS. As a result, I do not think I'm ready to raise the initial score. \n\nThank you", "rebuttal": "Dear Reviewer 5oqq and Area Chair,\n\nThank you for your detailed and constructive feedback. We appreciate your recognition of our work's motivation, that it is \"valuable to distinguish the varying negative impacts of mislabeled examples on model performance\", and your assessment that this is an \"important and often overlooked issue in sample selection strategies for learning with label noise\".\n\nYou have raised specific, valuable points regarding the paper's clarity and methodological details. In the following, we address your concerns and detail the significant revisions we have undertaken based on your suggestions.\n\n>**1. On Improving Clarity (in response to Weaknesses 2, 4 & Questions 1, 2)**\n\n**1.1. Clarification of Figure 1 (in response to Q1)**\n\nWe thank you for your questions regarding the clarity of Figure 1 and offer the following clarifications:\n\n* **Q1.A (Experimental Setup):** The experimental setup for Figure 1(a) is detailed in Section 2.1 (in the context of Figure 2). This experiment demonstrates the unique harm of MEEs by comparing the performance degradation when adding an equal number of MEEs versus Mislabeled Hard Examples to a clean dataset. In the revised manuscript, we have added an explicit reference to Section 2.1 when Figure 1 is first mentioned and have enhanced the figure caption with a concise summary of the setup.\n* **Q1.B (Definition):** The phrase \"mislabeled samples which are correctly predicted by the model\" refers to instances where a sample is inherently mislabeled, yet the model has learned to classify it according to its given (incorrect) noisy label.\n* **Q1.C (Early in Training):** The sentence immediately following the one you quoted explains this. We define the process of the model first learning these initial four thousand mislabeled samples as occurring \"early in the training process\".\n* **Q1.D (Disproportionate Impact):** The \"disproportionate\" impact is precisely what is illustrated in Figure 1(a). A training set contaminated with Mislabeled Easy Examples yields a model with significantly worse performance than one contaminated with the same number of Mislabeled Hard Examples, demonstrating that the negative impact of noisy labels is not uniform.\n\n**1.2. Definition of Key Concepts (in response to Q2)**\n\nRegarding the definition of \"class centers\" (Q2), we specified in both our analysis in **Section 2.2** and the caption of **Figure 4** that a \"center\" is the **mean embedding vector** of all samples belonging to that class, and distance is measured using the **Euclidean distance**. We also note that when this concept was first introduced in the introduction, we had already included a reference pointing to the detailed description via \"(see Section 2.2)\".\n\n**1.3. Notation and Prose (in response to W3, W5)**\n\nWe have conducted a thorough pass of the manuscript to unify all notation (e.g., consistently using $K$ for the number of classes), streamline the prose, and remove redundant descriptions.\n\nWe respectfully point out that while we strive for precision, our goal in the **Section 1 Introduction** is to provide a clear and concise overview of the phenomena we observe. For this reason, certain granular details are necessarily elaborated upon in their respective, dedicated sections.\n\n>**2. On Clarifying Methodological Details (in response to Weaknesses 1, 3, 5 & Questions 3, 4)**\n\n**2.1. On the Methodology (in response to Q3.A)**\n\nYou suggested that the calculation of the conditional probability $p_i^{(k)}$ requires the true label $y$. There may have been a misunderstanding here, which we are happy to clarify:\n1.  The term $p_i^{(k)}$ is a direct output of the model's forward pass and **requires no label information**. In the expression $Pr(y=k|x_i;\\theta^t)$, the symbol **$y$** represents a **random variable** for the class label, and **$y=k$** denotes the event that this random variable takes the value of class $k$.\n2.  The calculation of the loss $L_i$, the predicted label $\\hat{y}_i$, and the confidence $c_i$ do not depend on the unknown true label. They use either the **given (noisy) label $y_i$** (during backpropagation) or the model's own probabilistic output $p_i$.\n\nWhile we believe our notation is standard, we have revised this section (lines 184-190) in the manuscript to use more explicit notation (e.g., using **$Y$** to denote the random variable) to prevent any possible ambiguity.\n\n**2.2. On the Instance-Independent Noise Assumption (in response to Q4)**\n\nWe wish to clarify that our method **does not depend on the instance-independent label noise assumption**. The discussion of this assumption in Appendix A (Equation 7) is part of a standard protocol for describing how synthetic noisy datasets are generated; **it describes the data generation process**, not a prerequisite for our methodology. In Section 3, our method operates directly on the observed dataset $\\mathcal{D}^s$ with its given (noisy) labels $y_i$, irrespective of how the label noise was generated. Crucially, as shown in **Tables 2-5**, we have already validated our method's performance under various noise types, including synthetic symmetric, synthetic instance-dependent, and real-world label noise.\n\n**2.3. On Ensuring the Set $\\mathcal{S}$ is Non-Empty (in response to Q3.B)**\n\nWe agree with your suggestion that the definition should ensure the set $\\mathcal{S}$ is non-empty. We have incorporated a formal discussion of the constraints on the thresholds into the manuscript and updated Algorithm 1 accordingly. We would also like to humbly point out that in practice, our use of percentile-based thresholds effectively prevents the sets $\\mathcal{S}$ or $\\mathcal{S}'$ from being empty.\n\n**2.4. On Threshold Design (in response to Q3.B)**\n\nYou noted that using fixed percentile thresholds might be \"overly simple\". Our rationale for this design choice was to prioritize generality, robustness, and ease of use, thereby avoiding complex hyperparameters that require fine-tuning for each dataset. Our sensitivity analysis (Figure 5) demonstrates that the method's performance is highly stable across a wide range of threshold values, which empirically validates the robustness of this design.\n\n**2.5. On the Empirical Nature of the Work (in response to W1)**\n\nWe acknowledge that our work is primarily an empirically-driven study. We believe its main contributions are nonetheless significant: (1) identifying and defining a novel and impactful problem (the disproportionate harm of MEEs); (2) proposing a novel, effective, and practical solution (Early Cutting); and (3) validating this solution with extensive experiments on large-scale benchmarks, including full ImageNet-1k. We are confident that such empirical contributions hold significant value, particularly in an application-driven field like machine learning.\n\n---\n\nWe hope that these clarifications and our accompanying revisions will address your concerns and significantly improve your opinion of our paper. Thank you once again for your tremendous effort and valuable feedback.\n\nSincerely,\n\nThe Authors"}, {"review_id": "UPToaqmIiy", "reviewer": "Reviewer_bRY1", "summary": "This paper shows that not all incorrect labels are equally bad for training models. Mislabeled data learned early, which the authors call Mislabeled Easy Examples (MEEs), are especially harmful. The paper introduces a method named Early Cutting to fix this issue. This technique uses a more mature model from later in the training process to find and remove these MEEs from the initially selected data. Experiments on datasets like CIFAR, WebVision, and ImageNet confirm that this strategy successfully improves model performance by filtering out these specific harmful examples.", "questions": "1. Please use different notations for time ($T_i$), learning time ($LT_{i}$), and the noise matrix ($T$).\n\n2. Consider adding citations for fundamental methods like the SGD and AdamW optimizers you used.\n\n3. Please clarify the “Early Cutting Rate” hyperparameter and its selection in the main paper.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. This paper identifies a new problem: mislabeled examples learned early during training cause disproportionate harm to the model.\n\n2. This work proposes a counter-intuitive but effective method that uses a later model to correct the initial confident dataset.\n\n3. The method is thoroughly validated with strong results across various datasets, noise types, and strong competitor methods.\n\nWeaknesses:\n1. The use of similar notations like “$T_i$” on Line 98, “$LT_{i}$” in Eq. (1), and the noise transition matrix “$T$” on Line 444 can easily confuse readers.\n\n2. Citations for some important literature are missing, for example, the SGD and AdamW optimizers.", "was_revised": true, "final_justification": "Thank you for your response. It has addressed most of my concerns, and I will maintain my current rating.", "rebuttal": "Dear Reviewer bRY1 and Area Chair,\n\nWe sincerely appreciate the valuable suggestions, which have been instrumental in improving the quality of our manuscript.\nWe agree with all the points you raised and have revised the paper accordingly. Below, we detail the specific changes made:\n\n---\n\n>**On the Confusion of Notation:** \n\nWe thank you for pointing out the confusion with the overloading of the symbols. We have carefully revised the notation throughout the manuscript to ensure clarity. We now use $\\mathbf{E}$ to denote the number of training epochs.\n\n>**On Missing Citations:** \n\nWe appreciate your reminder to include citations for foundational methods. We have now added the appropriate references for SGD [1], AdamW [2], Weight Decay [3], and SGD with momentum [4]. These citations can be found in Section 4.2 (Datasets and implementation) and in Appendix B (Detailed Settings).\n\n>**On Clarification of a Hyperparameter:** \n\nThank you for pointing out the omission of the specific value for the \"Early Cutting Rate\" hyperparameter in the main paper. We have now specified in Section 3 (line 209) that the Early Cutting Rate was set to 1.5. In line with our treatment of other hyperparameters, we have also included a sensitivity analysis for the Early Cutting Rate in Section 4.3 and a corresponding ablation study in Appendix B.9.\n\n---\n\nThank you once again for your consideration of our work and for your constructive guidance. We are confident that these revisions have enhanced the readability of our paper.\n\nBest regards,\n\nAuthors\n\n---\n\nReferences:\n\n[1] A stochastic approximation method, The annals of mathematical statistics, 1951.\n\n[2] Decoupled Weight Decay Regularization, ICLR 2019.\n\n[3] A Simple Weight Decay Can Improve Generalization, NeurIPS 1991.\n\n[4] Learning representations by back-propagating errors, Nature, 1986."}, {"review_id": "GxKOTQfuZ1", "reviewer": "Reviewer_Dz13", "summary": "This paper introduces a sample selection strategy called Early Cutting, aimed at improving robustness against label noise by identifying and eliminating a specific group of samples termed Mislabeled Easy Examples (MEEs)—i.e., mislabeled data points that are learned early and confidently by the model. The method leverages the model state at a later training epoch to re-evaluate these \"early-learned confident samples\", applying a joint criterion of high loss, high confidence, and low gradient norm to isolate and remove MEEs. Experimental results on CIFAR-10, CIFAR-100, WebVision, and full ImageNet-1K demonstrate promising performance improvements over state-of-the-art methods.", "questions": "1. Can the authors clearly distinguish how MEEs differ from early-learned noisy samples in Late Stopping (Yuan et al., ICCV 2023)? Are they essentially the same, and if not, what distinguishes them?\n\n2. Why does the gradient norm reliably filter out MEEs rather than hard clean samples? Could the authors provide further evidence or ablations to justify this?\n\n3. How sensitive is the method to the percentile thresholds (e.g., top 10% loss, bottom 20% gradient)? While a small ablation is given, it could be expanded.\n\n4. Could the authors include comparisons against Me-Momentum and other dynamics-based selection methods targeting early-stage noise?\n\n5. In the introduction, the authors state:\n\"Specifically, even with a low noise rate in the selected subset, the presence of certain mislabeled samples can still significantly impair the model’s generalization performance.\"\nHowever, it is unclear what is meant by “low noise rate” in quantitative terms. Have the authors conducted any analysis to determine how low the noise rate needs to be for the impact on generalization to become negligible? In the experiments (e.g., Section 2.1), each subset contains 4,000 mislabeled samples among 34,000 total, which is still a relatively high proportion (~11.7%).\nWould the proposed conclusions still hold in more extreme settings—for instance, if the number of mislabeled examples was fewer than 1,000 among 30,000+ clean samples (i.e., <3%)? A discussion or additional experiment under such conditions would help validate the generality of the claim.\n\n6. When integrating the proposed Early Cutting method into the MixMatch semi-supervised learning framework (as described in Section 4.3), it is unclear whether the labeled and unlabeled subsets are fixed after the initial selection, or dynamically updated at each epoch during training.\nCould the authors clarify whether the confident (labeled) and unconfident (unlabeled) sets are re-evaluated iteratively, and if so, how often the update occurs? This detail is crucial for understanding the training dynamics and the consistency regularization effect.", "limitations": "The paper would benefit from a more thorough discussion of the limitations regarding the generality of its claims. In particular, the authors argue that even with a low noise rate, harmful mislabeled samples (MEEs) can significantly degrade generalization. However, no quantitative analysis is provided to determine how low the noise rate must be for the harmful effects to become negligible. For instance, if only 500 or 1000 mislabeled samples are mixed into 30,000+ clean examples (i.e., <3%), would the same degradation still occur?\nIn addition, the Early Cutting method may mistakenly eliminate a small number of clean samples—especially borderline or ambiguous cases—due to its reliance on high-loss, high-confidence, and low-gradient criteria. While the authors argue that such early-learned samples are redundant, this might not hold in low-data or long-tail scenarios where each clean instance carries significant value. A more balanced discussion of this trade-off would improve the paper's robustness.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \\\n1.The paper highlights an overlooked yet important issue: not all noisy labels are equally harmful, and some “easy” mislabeled examples can be particularly misleading.\\\n2.Experimental results are extensive and evaluated across diverse datasets and noise settings.\\\n3.The method is simple yet principled, with clearly defined criteria (loss, confidence, gradient norm).\\\n4.The authors provide visual and empirical evidence to support the existence and impact of MEEs (e.g., Figure 1, Figure 4).\\\n\nWeaknesses: \\\n1.Overstatement of Novelty / Lack of Clear Contribution Boundaries\\\nThe authors introduce the concept of Mislabeled Easy Examples (MEEs) and claim that these are particularly harmful mislabeled samples that are confidently learned by the model early in training. They state (e.g., Line 61):\n“We discover that mislabeled samples correctly predicted by the model early in training disproportionately harm model’s performance…”\nHowever, this phenomenon has been studied in prior works, including:\na. Me-Momentum (Bai & Liu, CVPR 2021), which proposes a momentum-based mechanism to identify memorized noisy samples early in training.\\\nb. Late Stopping (Yuan et al., ICCV 2023), which explicitly analyzes the impact of early-learned noisy examples and designs a strategy to mitigate their negative effect.\\\nAlthough these works are briefly acknowledged in Appendix A, they are not properly discussed or contrasted in the main text—particularly in Sections 2.2 and 3 where MEEs and the proposed Early Cutting strategy are introduced. The current manuscript presents the identification of MEEs as a novel discovery without clearly delineating the incremental contributions beyond these prior works.\nThe authors should explicitly cite and compare these related works in the main body and clearly articulate the difference between MEEs and previously defined early-learned noisy examples. If the contribution lies in improved detection or filtering of such examples (e.g., via gradient stability), that should be clearly emphasized as the novel component. Without this clarification, the claim of novelty appears overstated.\\\n2. Ambiguity in Method Motivation\\\nThe use of the gradient norm to distinguish MEEs from hard samples (Equation 5) lacks solid theoretical justification. Why does low gradient norm necessarily imply \"false patterns memorized\"? The connection is heuristic and may not generalize.\nSee Lines 200–204: “MEEs tend to have low gradient norms because the model has confidently mislearned them…”\nThis needs more analysis or ablation to support.\\\n3. Lack of Baseline Comparisons\\\nWhile experiments include many methods, the paper omits direct comparisons with methods that also target early-learned noisy samples—e.g., “Late Stopping” or “Me-Momentum”，which are representative state-of-the-art methods targeting similar early-stopping-based noisy label issues. This omission makes it difficult to convincingly demonstrate that the proposed method outperforms existing approaches in the same problem setting.\\\n4. Limited Novelty Beyond Gradient-Based Criterion\\\nIf we set aside the motivational premise—i.e., that early-learned mislabeled samples are especially harmful—since this has already been discussed in prior work (e.g., Me-Momentum, Late Stopping), then the core novelty of this paper is mainly the incorporation of the gradient norm as an additional filtering criterion. However, the use of combined loss and confidence thresholds for filtering mislabeled data is not new and has been explored in various existing methods. Therefore, the paper's overall innovation may appear limited unless the unique contribution of the gradient-based criterion is more clearly justified and empirically highlighted.", "was_revised": true, "final_justification": "Most of my concerns have been addressed, and I am willing to raise my score to 4 (Borderline Accept). I sincerely thank the reviewer for the detailed response, as well as the other reviewers, AC, PC, and the organizing committee for their hard work.", "rebuttal": "Dear Reviewer Dz13 and Area Chair,\n\nWe sincerely thank you for your valuable time in reviewing our manuscript. We have carefully studied all your questions and will address each point below.\n\n>**On the Paper's Core Contribution and Baseline Comparisons**\n\n**We would like to respectfully clarify a key point regarding the baseline comparisons.** The reviewer noted that our work was missing comparisons to Late Stopping [1] and Me-Momentum [2] (Weakness 3). We would like to respectfully point out that **both of these methods were included as core baselines in our main experimental results (Tables 2, 3)**. The results demonstrate that our proposed method significantly outperforms them across all tests.\n\n**This point is directly related to our core contribution, which we also wish to clarify.** The reviewer suggested that our motivation (that early-learned noisy samples are particularly harmful) has been discussed by Late Stopping [1] and Me-Momentum [2] (Weakness 1). We respectfully argue that **these paper are not discuss this motivation**.\n\nSpecifically, methods like Late Stopping [1] and Me-Momentum [2] are built on a core assumption: that models learn simple, clean samples first, and therefore, examples learned early in training are trustworthy. Our paper's central contribution is to demonstrate, through empirical analysis, that this assumption is risky and can even be detrimental. Our novelty is three-fold:\n\n* 1.  We are the first to systematically identify and define the concept of *Mislabeled Easy Examples* (MEEs): We show that while these samples are \"easily learned\" early in training, they cause disproportionate harm to the model's generalization ability (as shown in Fig. 1 and Fig. 2).\n* 2.  We propose a counter-intuitive solution (Early Cutting): Based on our insights into MEEs, our novel Early Cutting method introduces a counter-intuitive calibration mechanism. It uses a later-stage, overfitted model to correct and purify the confident set selected during the early stage. \n* 3.  We provide a precise identification tool: To enable Early Cutting, we designed the combined criteria of high loss, high confidence, and low gradient norm. This is not a simple heuristic but a standard specifically engineered to pinpoint MEEs within a late-stage model.\n\n**Our work is not an incremental improvement on existing methods but rather a significant revision to a mainstream methodology in learning with noisy labels, for which we provide a solution.** We would like to revise the Introduction and Related Work sections to more clearly emphasize this critical distinction and our contribution.\n\n>**2. Justification for Using Gradient Norm to Identify MEEs (Response to Weaknesses 2, 4)**\n\nYou raised an excellent question regarding why a low gradient norm can reliably help filter out MEEs. \n\n* Theoretical Motivation : When a model \"memorizes\" an incorrect pattern, it gives a wrong but high-confidence prediction for a sample (e.g., predicting an image of a plane labeled \"ship\" as \"ship\" with 99% probability). For this sample, the loss (relative to its incorrect label $y_i$) will be very high. But, for a small gradient norm indicates that the loss is insensitive to small perturbations in $\\mathbf{x}_i$, suggesting a strong (but potentially incorrect) association between the input features and the predicted label. MEEs tend to have low gradient norms because the model has confidently mislearned them, making the loss stable even under input perturbations. (Section 3, line 202-206)\n\n* Empirical Validation : To validate that the gradient norm is a crucial component, we provided empirical evidence in our ablation study (Appendix B.9, line 623). We showed that removing the gradient norm criterion from our selection process leads to a significant drop in final model performance.\n\n>**3. Details on MixMatch**\n\nThank you for pointing this out; this indeed requires clarification. In our current implementation, the confident set (labeled data) and un-confident set (unlabeled data) are determined once and remain fixed during the subsequent MixMatch training. We will add this explicit detail to Section 4.3 of the paper.\n\n\n>**4. Performance at Low Noise Rates**\n\n\nWe thank you for this interesting question about whether our experimental findings generalize to scenes with extremely low noise rates. Following your constructive suggestion, we have conducted additional experiments to test our hypothesis in very low noise rate.\n\nTo do this, we constructed two separate low-noise training sets for both CIFAR-10 and CIFAR-100. Each set contains the full clean dataset plus 2,000 mislabeled samples (~6% noise rate, which is much lower than the common low noise setting, 20% noise rate, in learning with noise labels). The difference lies in *which* mislabeled samples were chosen, based on their learning-time ranking from our original, higher-noise experiments (as detailed in Section 2).\nAll other experimental settings were kept identical to those used for Figure 2. The results, shown in the table below, confirm that even at this very low noise rate, the mislabeled samples learned earliest by the model are significantly more harmful to final generalization performance:\n\n| Test Accuracy | Clean+(0:2000] Mislabled (easy mislabeled) | Clean+(18000:20000] Mislabled (hard mislabeled) \n| :--- | :---: | :---: |\n| **CIFAR-10** | 85.99% | 88.09% | \n| **CIFAR-100** | 57.74% | 60.10% | *\n\n\n>**5. Discussion of Trade-offs and Limitations (Response to Limitations)**\n\nYou've raised an important point about the risk of our method removing a small number of clean samples (i.e., false positives). We agree this trade-off exists in any sample selection method. The critical question is whether the benefit of removing harmful mislabeled examples outweighs the cost of losing these false positives.\n\nOur Early Cutting method is designed to mitigate this risk through its strict, multi-part criterion. By engineering the method to require that a sample simultaneously meets three conditions (high loss, high confidence, and low gradient norm), we reduce the likelihood of removing hard clean samples. For instance, a challenging clean sample, such as from a long-tail class, might exhibit high loss. However, the model's confidence in its prediction would be low, and its gradient norm would be high, because a merely difficult sample does not confuse the model in the same way. Therefore, difficult long-tail samples typically do not fit our criteria for removal. Our experimental results confirm that this multi-part constraint significantly improves selection precision, ensuring the net effect is a substantial gain in performance.\n\nDespite these design choices, we agree with your sentiment that this trade-off becomes more critical in scenarios with extremely sparse data. We will add a *Section 6: Limitations* to the paper to discuss this trade-off, acknowledging that the method may require adjustments or further research for such specific use cases.\n\n---\n\nThank you once again for your thoughtful feedback.\n\nBest regards,\n\nThe Authors\n\n---\n\nReference:\n\n[1] Late Stopping: Avoiding Confidently Learning from Mislabeled Examples, ICCV 2023.\n\n[2] Me-Momentum: Extracting Hard Confident Examples from Noisily Labeled Data, ICCV 2021."}]

Author rebuttal:

Meta-review: The paper studies the hypothesis that mislabeled examples harm model performance to different degrees during training. The authors identify mislabeled easy examples which are correctly predicted by the model during early training are particularly harmful. They propose a method to address this. 

This paper has undergone an elaborate review process with a seven reviewers in total, three of which acted as emergency reviewers. Reviewers praised the core concept of the paper that not all mislabeled examples are equally harmful and the significance of this aspect, the experimental evaluation and the clear writing. 

On the other hand, reviewers Sv4t & Dz13 pointed out a lack of comparison to baselines but during the discussion period Dz13's concern was addressed (the desired baselines were already in the submitted manuscript). Reviewer 5oqq voiced concerns about a lack of theoretical guarantees but this issue was not seen as a strong enough reason by other reviewers to consider rejection. I therefore recommend acceptance of this paper.
Decision: Accept (poster)