TrainRef: Curating Data with Label Distribution and Minimal Reference for Accurate Prediction and Reliable Confidence
Authors: Murong Ma
Ruofan Liu
Yun Lin
Zhiyong Huang
Jin Song Dong
Venue: ICLR 2026 Poster (2026)
Keywords: Label misinformation
data curation
influence function
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Practical classification requires both high predictive accuracy and reliable confidence for human-AI collaboration. Given that a high-quality dataset is expensive and sometimes impossible, learning with noisy labels (LNL) is of great importance. The state-of-the-art works propose many denoising approaches by categorically correcting the label noise, i.e., change a label from one class to another. While effective in improving accuracy, they are less effective for learning reliable confidence. This happens especially when the number of classes grows, giving rise to more ambiguous samples. In addition, traditional approaches usually curate the training dataset (e.g., reweighting samples or correcting data labels) by intrinsically learning normalities from the noisy dataset. The curation performance can suffer when the noisy ratio is high enough to form a polluting normality.

In this work, we propose a training-time data-curation framework, TrainRef, to uniformly address predictive accuracy and confidence calibration by (1) an extrinsic small set of reference samples $D_{{ref}}$ to avoid normality pollution and (2) curate labels into a class distribution instead of a categorical class to handle sample ambiguity. Our insights lie in that the extrinsic information allows us to select more precise clean samples even when $|D_{{ref}}|$ equals to the number of classes (i.e., one sample per class). Technically, we design (1) a reference augmentation technique to select clean samples from the dataset based on $D_{{ref}}$; and (2) a model-dataset co-evolving technique for a near-perfect embedding space, which is used to vote on the class-distribution for the label of a noisy sample. Extensive experiments on CIFAR-100, Animal10N, and WebVision demonstrate that TrainRef outperform the state-of-the-art denoising techniques (DISC, L2B, and DivideMix) and model calibration techniques (label smoothing, Mixup, and temperature scaling). Furthermore, our user study shows that the model confidence trained by TrainRef well aligns with human intuition. More demonstration, proof, and experimental details are available at https://sites.google.com/view/train-ref.

Published as a conference paper at ICLR 2026
TRAINREF: CURATING DATA WITH LABEL DISTRIBU- TION AND MINIMAL REFERENCE FOR ACCURATE PRE- DICTION AND RELIABLE CONFIDENCE
Murong Ma'; Ruofan Liu', Yun Lin"; Zhiyong Huang', Jin Song Dong'
Schooo of Computing, National University of Singapore
School of Computer Science, Shanghai Jiao Tong University
ABSTRACT
Practical classification requires both high predictive accuracy and reliable confi- dence for human-Al Al collaboration. Given that a high-quality dataset is expensive and sometimes impossible, learning with noisy labels (LNL.) is of great importance. The state-of-the-ar works propose many denoising approaches by categorically correcting the label noise, i.e., change label from one class to another, which can suffer from normality pollution and class ambiguity. The normality pollution happens when the noise ratio gets higher, leading prediction inaccuracy, such approaches intrinsically learns normality from the noisy dataset. The class ambiguity happens when the number of classes increases, leading less reliable prediction confidence.
In this work, propose training-time data-curation framework, TrainRef, uniformly address prediction accuracy and confidence calibration by (1) an extrinsic small set of reference samples Dier to avoid normality pollution and (2) curate labels into a class distribution instead of a categorical class to handle sample ambiguity. Our insights lie in that the extrinsic information allows us select more precise clean samples even when [Derl equals to the number of classes (i.e., one sample per class). Technically, we design (1) a reference augmentation technique select clean samples from the dataset based on Drer and (2) motel-dataset co-evolving technique for a near- perfect embedding space, which is used to vote on the class- distribution for the label of a noisy sample. Extensive experiments on CIFAR-100, AnimalION, and WebVision demonstrate that TrainRef outperform the state-of- the-art denoising techniques (DISC, L2B, and DivideMix) and model calibration techniques (label smoothing. Mixup, and temperature scaling). Furthermore, our user study shows that the model confidence trained by TrainRef well aligns with human intuition. More demonstration, proof, and experimental details are available at ht https:://// wetss/////eeeeeeeeeeeeeeeee googl le cwm/viiaaeoooccc view/wewe wtw.ee/tttiieeeeeeeeeeeeee
INTRODUCTION
Practical classification application, such as medical diagnosis (Rosenbacke et al., 2024), autonomous driving (Delavari et al., 2025), and fraud detection (Perini & Davis, 2023), requires both accuracy and reliable contidence. Recent work by Kalai et al. (Kalai et al., 2025) shows that even highly capable LLMs tend produce overconfident false predictions (hallucinations), emphasizing the importance of calibration as a peer to accuracy. The confidence is useful for the model users to decide when adopt model decision (Corbière et al., 2019; Pan et al., 2020). Such model performance (both accuracy and confidence) usually requires high-quality datasets, however, which are usually expensive, sometimes impossible (AI, 2024; Forbes, 2024).
Therefore, learning with noisy labels (LNL) solutions emerge to address the challenge. The solutions evolve from label transition matrix (Hendrycks al., 2018: Patrini al., 2017), sample-reweighting techniques (Li et al., 2020; Sheng al., 2024; Liu et al., 2020), gravitating the pseudo-labeling
Part of this work was conducted during visit to SJTU. "oorrsponniig author: 1 lin_yune jtu. edu.cn en

---

Published as a conference paper at ICLR 2026
Label: See
Labet: Worm
Label: 170% Enake,
Label (80448co 40% Cloudd 10% [lann)
Label:
30% Warm)
Label: Cluud
Label: Worm
Figure I: Examples of ambiguous samples from CIFAR-100 where the ground-truth label is distribu- tional rather than categorical. (Left) Snake vs. Worm, where the label can be 70% Snake and 30% Worm. (Right) Sea vs. Cloud vs, Plain, where the label may be distributed as 50% Sea, 40% Cloud, and 10% Plain. Categorical labels for such cases encourage overconfident predictions, whereas distributional labels better capture inherent ambiguity.
techniques which curate the noisy data from one class to another via remi-supervised learning (Kim et al., 2021; Nishi et al., 2021; Lu et al., 2021; Zheltonozhskii et al., 2022; Li et al., 2023). While generally effective in improving predictive accuracy, the performance of such categorical curation (i.e., change one label with another) to learn reliable confidence limited, especially when the number of classes grows, leading to more ambiguous samples (see Figure 1). In practice, to classify a clinical note into an ICD code, there can be more than 10K classes. In addition, those solutions curate the labels by learning the label normalities from the intrinsic noisy dataset. When the noise ratio is high enough, the noisy samples can form the polluting normalities, leading to mis-curation. On the other hand, confidence calibration solutions include label smoothing (Müller al., 2019), mixup (Zhang, 2017), and temperature scaling (Hinton, 2015; Guo et al., 2017) which mitigates prediction overconfidence for more reliable class distributions. However, their performance limited in addressing the cassconiieeece caused by label noise.
In this work, we propose training-time data-curation solution, TrainRef, with the following features.
Distributional Curation: Different from categorical curation, we learn distributional curation during training, i.c., change label from one class a class distribution, for having high predictive accuracy and reliable confidence in a uniform manner.
Extrinsic Reference Set: Different from the state-of state-of-the-and the-trr-arss (Li et al., 2020; Sheng et al., 2024; Liu al., 2020; Li al., 2023) which curate the labels by learning the label normalities from the intrinsic noisy dataset, our approach introduces tiny trusted set of reference samples with the ground-truth label, Dreff as label normalities, to avoid normality pollution caused by high noise ratios. Our approach can be effective even when Drefl is as small the number of classes (one sample per class).
Given a trusted reference set Dier and a noisy dataset D. TrainRef adopts a three-stage training routine by co-evolving the model embedding space and the curated dataset. First, we obtaining label-agnostic encoder 0 by pre-training masked-image modelling (MIM) (Pathak et al., 2016; Peng et al., 2022) task on D. As a result, the embedding space of 0 is immune to noise. Next, we design reference augmentation technique to select clean samples D(D C D) based on Diss regarding the agreement between the samples in Drer and the samples in D through the influence functions (Koh & Liang, 2017; Pruthi al., 2020; ?) computed on Finally, we co-evolve the model 6* and the dataset DD by iteratively (1) fine-tuning a model 0* with learned clean dataset D and (2) curating and enhancing the dataset 2) by voting the label distribution for noisy samples with their neighbouring clean samples the embedding space of d
Our extensive experiments show that TrainRef significantly outperform the state-of-the-at denoising solutions (i.e., L2B (Zhou et al., 2024), DISC (Li al., 2023), LSL (Kim al., 2024)) by 1.82% to 8.19% across various benchmarks and the state-of of-the-art confidence calibration solutions (i.c., label smoothing (Müller al., 2019), mixup (Zhang, 2017), 7), and temperature scaling (Hinton, 2015;
2

---

Published as a conference paper at ICLR 2026
Guo et al., 2017)) by consistently achieving lower ECE indicating better calibration. Furthermore, in blind user study, participants agree with TrainRef's confidence estimates over previous SOTA (DISC mixup) in 75% of cases, corroborating its reliability in practice. More demonstration, proof, and experimental details are available at https:: //s tes. ee. wttt://///////////////////
2 PROBLEM STATEMENT
Given collected dataset D 5)),, where each sample consists of input Rd and one-hot categorical label ý = [/1 1/2 ..., ycl. Ye E {0, C' is the number of classes. We assume that the true label is a class distribution instead of one-hot class, that is, D {(x,y"), where y" yi. v2. ... Vel. Ve [0, 1), > Vo = 1. Thus, we define two types of label misinformation:
Categorical Noise: The ground-truth label y* is one-hot (e.g., a O-entropy distributional label) but arg max, ye 4 arg m Yo.
Distributional Noise: The ground-truth label y* is Dirichlet distribution over C classes, and y" # y. Intuitively, such noise can lead to an over- or under-confident model.
Our goal is to learn parameterized mapping fo(-) Rdd Rh that projects each input into embedding space for downstream tasks, We obtain our estimator 0 by minimizing the empirical risk over finite dataset with distributional labels (Fquation 1). Here, À [ denotes regularization term, such as weight decay, that penalizes large parameter values control the model's capacity.
i arg min
[F(So(x:), 81) + [|lfa[]]] (.)
Σ
N
In the limit as oo, well-behaved estimator å should converge to the true risk minimizer.
0" arg min E(xxxt)>D- [x(f((x)), vi)] Allfool
However, in practice, this asymptotic consistency break down when the dataset D contains label misinformation (i.e., both categorical and distributional noise).
Rationale If the optimal embedding fo (-) were available, and j go o f. where g(.) R RC the classification head. Then by Representer Point Theorem (Schölkopf et al., 2001), any model prediction at query sample X, can be expressed as linear combination of representative samples x, weighted by their similarity k(x,,, x;) (Bquation 3), where each coefficient as depends only on the representative sample Yt).
(xx))
Σ
ai k((x;, X1)
(XL.))
In TrainRef. (i) estimate the optimal embedding function f. and (ii) collect clean reference set with diverse class prototypes, Dref = {(Xre) Then given a sample ÿ), we can curate the label Equation 4.
3 (xx)
Σ (Ned (eedt- Erd
Yiet (x((x)))) fxxx))
[Dreel
Yeaf Here, set a4 [[ i.c. each clean label directly votes in proportion to its similarity with the query. We construct the curated dataset ò {xx,, y* (x:)) |x, 13} to replace D in the follow-up training processes, and we hope the Ó learnt from ò is closer to 0*. Practical Challenges. Applying Equation in practice requires overcoming two challenges:

---

Published as a conference paper at ICLR 2026
Reference Dr,f
Reference Augmentation
Augmented reference D
Masked Image Modelling Pretraining
Treining dataset
Pretrained model
Specialized training
Data
Updated model
Data curotion
Curated dataset D
Training
Updated model 6
Phase I:
Phase III:
General embedding construction
Specialized embedding space construction Embedding space refinement
Figure 2: Overview TrainRef: given a noisy training dataset D and a small reference dataset Dref= we (I) explore near-perfect embedding space and (2) augment the reference to learn function parameterized by 6 to minimize the empirical risk. The learned embedding space and the curated dataset are mutually influenced to converge.
Accurate embedding-space estimation. Even with heuristics like the memorization effect (Liu al., 2020) confidence- based selection (Li al., 2020; 2023), noisy or over-confident labels in D can skew the learned embedding space when minimizing the empirical risk (Eq. 1).
Reference-set diversity, Manually verified clean references are typically few. We therefore need methods for augmenting and measuring diversity that our reference pool contains sufficiently representative prototypes for reliable label voting.
Next, we introduce our solution TrainRef to address the above challenges.
3 APPROACH
Figure 2 illustrates the overall framework of TrainRef, which learns a deep classifier from a dataset D with both categorical and distributional noise, aided by small clean reference set Diet. The idea to construct a reliable embedding space S and an augmented reference set Di such that samples in Dr collectively provide accurate supervision for the rest of the dataset via neighborhood voting. The training procedure consists of three iterative phases:
Phase I: General Embedding Space Construction. In this stage, pre-train model fo from the training dataset D with loss functions independent from the labels. Specifically, choose MIM (Masked Image Modelling) (Pathak et al., 2016; Peng et al., 2022) as the pretraining task which aims recover patch in image sample from its neighboring patches. By this means, learn initial embedding space, allowing us to extract general semantically meaningful features from the samples.
Phase II: Special Embedding Space Construction. In the next stage, we augment the small reference set Dref by introducing subset of clean samples in D. We consider sample (x, y) in D as clean if (x, y) consistent with all the samples in Dref- In this work, we introduce influence function (Koh & Liang. 2017) to measure such consistency (see Section 3. 3.1). As result, we have an augmented reference set Diee which can be used to further retrain to Comparing to 0, the embedding space of 0 is more classification-releaar
Phase III: Embedding Space Refinement. In the final stage, we iteratively update the learned model 9. and its embedding space and the curated model DD. On one hand, we curate the dataset D to D based on the learned embedding space and set of discriminated reference samples. the other hand, the curated dataset D further used to update the model to 6r. The co-evolution process terminates once both the refined labels y and model parameters Ó converge (see Section 3.2).
Given the space limit, readers refer to the Appendix A for the dictionary of symbols.
4

---

Published as a conference paper at ICLR 2026
3.1 REFERENCE AUGMENTATION
Our insight lies in that noisy samples are likely to generate strong conflicting training signals with the ground-truth references, and clean samples generate aligning or negligible training signals.
Technically, given a pre-trained model fe from masked image modeling, we append fully connected classification head (o(.) I² EC parameterized by Ø, which is fine-tuned on Pref- The layer is used to measure the agreement between a target training sample S; = (x,, y:) E D and any reference samples Syef (Xref:-re] Dref. In this work, adopt Tracein (Pruthi et al., 2020), a practical influence function measuring how likely fitting a training sample s, is helpful or harmful to predict a reference sample Syef.
+ V
Volee
where {
( -lgs L(gs
fo(x)).
IF(s,, Stet)
(5)
V ViLill V
Cnt [igo (Xiet), (ret)
T Σ
Σ IF(s;, Dret) = T I.M 1=1 ScrM:
IF(s S,e)) where
M₁ = ((etet Diet Yiet = ý.}
(6)
As shown in Equation 6, the influence of training sample i is calculated the gradient alignment between this sample and all reference samples with the same label as ý;. The final influence score is averaged over T training checkpoints. Training samples with positive IF(s, Dref) larger than threshold F are used to construct a larger augmented reference set D Pat 6TF is tuned to be 0.8, and demonstrate its robustness with different settings in the Appendix F.3. The augmented reference set is further used update and é, collectively referred to Ó.
3.2 CURATION CERATION-TRAINING CO-EVOLUTION
We formalize co-evolution as an alternating optimization scheme closely analogous the Expecta- iinnMaaxmiization (EM) algorithm (Dempster et al., 1977), composed of the following two phases:
Curation (C-step): Holding the current embedding estimator fixed, we apply Equation to compute refined curated labels y*. In this phase, each sample is assigned "responsibility" weight over C' classes, analogous to the E-step in EM algorithm (Dempster et al., 1977), i.c., posterior mixture-component assignments.
Parameter update (P-step): Using the curated labels ÿ targets, update fe by empirical risk minimization. This update refines the embedding space, which in turn yields more accurate similarity estimation (xfre()))) f(x))) for the subsequent C-step, analogous to the M-step EM algorithm (Dempster et al., 1977), i.e., parameter re-estimation given fixed responsibilities.
By iterating these two phases, the embedding function and the curated labels co-adapt dynamically: improved embeddings produce more reliable aseudo-labels, and more accurate pseudo-labed guide sharper embeddings, thereby ensuring progressive convergence despite the presence of label noise.
C-step Curation Principle. We estimate refined labels y* for all samples in the noisy dataset D based on neighborhood voting from the augmented reference set Di Given a sample (x, y), the estimated label distribution ý is defined as:
f(x)= = Z Z(x)
Σ (oter
Yier 1 (Xxer E Dione (x)) Cosine (f(x),
where we use cosine similarity the kernel function, and 7(x) is normalization constant ensuring ý forms valid probability distribution: 7(x) E1(x.EDE (x)) Cosine(f(xx) (xxer)
To mitigate voting noise, we restrict the voting pool using an indicator function 1 L(xeef E Di (x)) that enforces two criteria: (i) semantic relevance to the query sample, defined as
5

---

Published as a conference paper at ICLR 2026
vxcc = {xxet, Dier Cosine(f(x)) f(xee)) 2 -}
(8)
where T is set to the 75th percentile of the cosine- distribution, and (ii) inter- diversity among selected references. For the latter, we construct a subset of samples (k is set to half the pool size) that covers the distribution of vote-
D Piace (x)
arg max
8CD (1)
To sharpen the refined label distribution and emphasize confident predictions, we apply temperature- controlled transformation following DivideMix (Li et al., 2020). Given & E (0, 1], the sharpened label is computed as y = y/// (1 (1)) with t 0.5 in all experiments.
P-step Optimization Objective. Given the current model parameters and curated dataset D (7) obtained from the C-step, we update 9 by minimizing the following objective:
D
1
arg min
Cen (go fo(x:). i!l0
(10)
D
0,6
Here, Lc denotes the cross-entropy loss between the model's prediction and the curated label, and |101| is &2 regularization term that discourages overfitting and promotes smoother parameter. We provide theoretical proof convergence on our website (TrainRef, 2025).
EXPERIMENT
evaluate our approach with the following research questions, each addressed with an experiment:
How effectively does TrainRef achieve predictive accuracy compared to state-of-the-art label denoising methods?
RQI (Predictive Accuracy):
RQ2 (Confidence Reliability): How effectively does TrainRef improve confidence calibration compared to kate-of-the-as calibration methods?
RQ3 (User study): In practical use, to what extent do humans agree with the confidence estimates produced by TrainRef?
RQ4 (Ablation study): What are the contributions of each component to the overall performance?
Implementation details and qualitative analyses are deferred to Appendix due to space limits.
4.1 RQI: PREDICTIVE ACCURACY
Synthetic Noisy Datasets. We evaluate TrainRef on CIFAR- (Krizhevsky et al., 2009) under three common noise types: (1) instance-dependen noise (IDN) (Xia et al., 2020), where each instance is assigned a noise rate from a truncated Gaussian distribution with class-level rates chosen randomly; (2) symmetric noise (Sym.) (Li et al., 2020), where labels are flipped uniformly random any other class; and (3) asymmetric noise (Asym.), where labels are flipped to semantically similar or neighboring classes
fixed rate. Following Li et al. (2023), set symmetric noise levels to E (20%, 50%, 80%] } and both asymmetric and instance-dependent noise to 40%.
Real-World Noisy Datasets. To assess Ref in practical settings, evaluate it on Web- Vision1.0 (Li et al., 2017) and Animal-10N (Song et al., 2019). WebVision .. contains 2.4M web-crawled images from Google and Flickr. Animal-1ON consists of noisy labels from five pairs of visually similar animal species. Both are challenging real-world benchmarks.
Results on CIFAR with synthetic noise. Table shows the generalization performance under various noise levels on the CIFAR-100 dataset. Overall, TrainRef consistently outperforms all the baselines, serving as a new state-of-the-art. Specificallyy, under severe noise setting (e.g., 80% symmetric), TrainRef surpasses L2B-C2D by over 10% and remains robust in challenging cases like asymmetric (40%) and instance deeenneed (40%) noise, outperforming DISC (Li et al., 2023) by over 3%. Figure provides training example curated by TrainRef with distributional label in the instance-dependent 40% noise setting on CIFAR- 100. The image belongs to the class "sea" but has been incorrectly labeled "plain". TrainRef assigns distributional label that balances between

---

Published as a conference paper at ICLR 2026
Table Comparison with SOTA methods on CIFAR-100 datasets with different types and levels of label noise. Mean + standard deviation is reported over runs. The results are primarily derived from (Li et al., 2023) or the original papers.
Noise type
Sym 20
Sym 50 %
Sym 80 %
Asym 40%
Irstt 40%
CE
55.17 0.12
32.40 0.16
7.70
40.63 0.26
43.17 0.15
Decoupling (Malach & Shalev-Shwartz, 2017)
52.75 0.11
27.59 = 0.16
7.38 1 0.09
39.12 = 0.08
Co-teaching (Han et al., 2018)
51.24 0.23
25.07 = 0.18
8.50 0.06
38.06 + 0.15
23.21 + 0.57
JointOptim (Tanaka al., 2018)
58.50 0.47
53.58 = 0.43
24.62 0.50
61.17 = 0.39
Co-teaching+ (Yu et al., 2019)
51.24 0.23
25.07 = 0.18
9.50 0.08
36,58 = 0.16
24.45 0.71
GCF (Zhang Sahuncu, 2018)
76.16
72.84 = 0.12
28.40 1:
+ 0.20
0.14
PENCIL (Yi & Wu, 2019)
37.12 0.12
0.33
40.63 0.26
JoCoR (Wei al., 2020)
54.70 0.08
26.45 0.13
7,35 0.05
37,09 0.09
23.95 0,44
DivideMix (1.i et al., 2020)
76.16
72.84
0.12
28.40 1 0.06
55.56 + 0.53
76.08 0.35
ELR (Liu et al., 2020)
0.11
58.10 0.17
28.40 = 0.06
46.08 + 0.20
ELR+ (Liu et al., 2020)
7.194
73.01 = 0.14
58.01 = 0.17
7.11
learning (Wei al., 2020)
0.14
58.10
0.17
41.77 1
51.50 0.24
DISC (Li et al., 2023)
78.75 0.13
75.21
0.15
57.61
76.50 0.15
78.44 0.19
L2B-C2D (Zhou et al., 2024)
79.67 0.14
78.23
= 0.16
69.66
= 0.19
78.22
0.14
79.43 = 0.17
Ours
85.44 = 0.21
82.07 0.17
77.85 0.35
79.67 0.22
82.33 0.16
Label Ditribbtion
863
Gloud
Training Sample Noray Lobet:Plain Label: San
Plath
Mountain
doud
plah
othera
Supporiens Rel. Sui
Figure 3: A noisy training sample (in the left) is curated by TrainRef to have class distribution where the probability over the class "sea", "cloud", "plain", "mountain", and etc (in the middle). In addition, the reference samples voting for such curation is shown in the right.
"sea" and "cloud", reflecting the inherent ambiguity in the image. In addition, show reference samples voting this label, explaining how TrainRef makes such a curation decision. More examples available at website (TrainRef, 2025).
Results on WehVision and Animal-10N. Tables and Tables 3 present the generalization perfor- mance of TrainRef with SOTA methods real-world noisy datasets, Web and Animal-ION. respectively. The results demonstrate that TrainRef improves the predictive accuracy by 2% - 44% on both benchmarks, highlighting its effectiveness in handling real-world label noise.
4.2 RQ2: CONFIDENCE RELIABILITY
Setup. We evaluate calibration on CIFAR-100 under three noise regimes: noise-free, symmetric 20%, and symmetric 80%. Baseline calibration methods include (i) Raw cross-entropy (CE) loss, (ii) Focal variants loss (Focal, AdaFocal, al., 2022; Tao et al., 2023)), (iii) Post-hoc calibration such Temperature Scaling (TS) (Guo et al., 2017), PTS (Tomani et al., 2022), Spline (Gupta et al., 2020), MnM (Zhang et al., 2020)), (iv) Denoising methods (DISC, L2B) with and without TS, and TrainRef with and without TS. We report accuracy, ECE (Nixon et al., 2019), and AdaECE (Mukhoti et al., 2020b). Extended results with additional metrics (ECFattbius F.CFxxweep (Roelofs al., 2022)). other CIFAR- noise settings, and two real-world datasets (Animal- 1ON, WebVisio are provided in the Appendix D.
(Mukhoti et al., 2020a; Ghosh
Results. Table 4 shows that TrainRef consistently achieves the best trade-off between accuracy and calibration across all noise settings. Focal variants losses amplify noise by up-weighting mislabeled "hard" samples, while post-hoc methods such as TS only rescale logits fax = softmax(22 2x: (Tx and cannot fix incorrect confidence rankings, yielding low ECE but poor accuracy. Denoising methods such as DISC and L2B improve accuracy through filtering or hard relabeling, but discard uncertain yet informative samples, thereby weakening calibration. By contrast, TrainRef utilizes soft labels
7

---

Published as a conference paper at ICLR 2026
Table 2: Prediction Accuracy on WebVision.
Table 3: Prediction Accuracy on Animal-1ON.
Accuracy (%)
Top-1
Top-5
Method
Accurncy (%)
F-correction (Patrini et al., 2017)
61.12
82.68
CE (Eaglesson Azizpour, 2021)
79.4 + 0.14
MentorNet (Jiang al., 2018)
63.00
81.40
GCE (7hang & Sabuneu, 2018)
1
Co-teaching (Han ol., 2018)
63.58
85.20
SEL (Song al., 2019)
81.8 + 0.09
ELR (Liu et al., 2020)
76.26
91.26
Mixup (Zhang. 2017)
82.7 + 0.03
DivideMix (1i et al., 2020)
77.32
91.64
Co Co-leamnng (Tan al., 2021)
83.0
EIRR+ (L et al., 2020)
77.78
91.68
PLC (Zhang et al., 2021)
83.4 0.43
GIS (Eaglesson & Azizpour, 2021)
77.99
90.62
Nosted Co-teaching (Chen et al., 202
84.1 + 0.10
(Zhao al., 2022)
93.64
GJS (Englesson Azizpour, 2021)
1:
DISC (1.i et al., 2023)
80.28
92.28
DISC (1 al., 2023)
87.1 + 0.15
LSL (Kim al., 2024)
81.40
93.00
LSL (Kim al., 2024)
89.1
Ours
82.28
95.14
Ours
96.90 0.24
(INlk Pretrained)
96.34
(INIk Pretrained)
0.15
Table 4: Comparison of confidence calibration performance across noise-free, symmetric 20% (Sym20), and symmetric 80% (Sym80) settings on CIFAR-100 100. Results report test accuracy (higher better) and calibration errors (ECE, AdaECE; lower is better). Grey-shaded rows indicate methods where temperature scaling (TS) is applied on top of the base method. Bold entries mark the best results under the same TS setting (either with TS or without TS).
Noteo-Frre
Sym30
Sym60
Mathed
Texl (16
(C
AdaECE (.)
Tou Ace (16
ECE (.)
AdaECE (3)
Tol. (9)
BCE
AdaECE (3)
(Baacinn))
0.1812
$1.76
0.033
0.879
0.0946
Rantne Method
Focal Losa (Malhali al. 2020a)
76.31
0.0864
0.0866
$2.16
0.1199
0.1988
16:26
DLIG5S
Ads local Loss (Chhsch et 2022)
76.55
11T11
51,60
0
RON1S
16,6
0.T15
Dual Rocal Loss (Tao at. 2023)
77,93
0U9944
47.32
0.1476
0.476
1695
0.1057
ELJOSS
CE Postboe
CB (Cuo el... 2017)
BJ295
51.76
0.0137
00038
D.0136
CB4PTS (Tenami 2013)
71.87
0.054
0,256
$1.76
0,065
0,008
16.36
0.014
0.0E3S
CB-Sppine (Gapta al. 20000)
71.87
01036
00131
51.76
0/242
0,028
15.35
0.004
CE+MaM (Zhong ctal., 2020)
71.87
80012
0.0201
$1.76
0.0177
0.022
16.38
0.0134
ELIORS
DISC 2023)
0.11
0.013
0.112:00011
0.118
0.011
D.114
0,016
57.61
0.20
0.12
0.013
IMSC-ITS
61.35 0:10
0.005
0.00
0,007
0.00
18.75 0.155
008T 0.005
D(MS 0.010
57.01
(179
0:00
0/00
01051 0.011
L2B 12hou 2024)
82.31 2
0.124
0.011
0,131
0,005
79.67 0.14
0,10 0,013
0.112 = 0,509
09.65
0.19
0,133
0,009
0,152 0.022
1:2++18
82:11 4 0.14
0.007
0.008
0.044
0.005
19.67
000 0.012
1/00 0.011
60
66
0.19
0.087
00015
00061 0.017
Ous
85.87 = 0.15
6.44
0,800
0,043 9.019
85.44
0.21
0,04 0,800
B.47
= 0.805
77.85
0.35
0,882
: 0.013
0.086 6.011
OurseTS
6,015
6,607
0.014
6.21
0005
0.000
0.01
0:005
77.85
and minimal reference set Dref to retain uncertainty and provide reliable supervision, resulting in stronger calibration and accuracy.
4.3 RQ3: USER STUDY
Setup. We collect confidence scores from the test set after training the models on different noise levels (i.e., 20%, 50%, and 80%) of CIFAR-100. Specifically, under noisy rate (e.g., 20%), ), we learn TrainRef and the baseline DISC. Then, we select 100 test samples (1) which are predicted to have low confidence by either TrainRef or DISC; (2) where TrainRef and DISC have large disagreement on their confidence We hire experts as participants, each with over 3 years of experience in model training and data labeling. Each participant is presented with two anonymous predictions (TrainRef or the baseline) and asked to choose predicted confidence to agree with.
Results. Table 5 shows that across all noise settings, participants consistently prefer the predictions from TrainRef by a significant margin. As the noise rate increases, the preference for TrainRef becomes even more pronounced. Notably, an 80% noise rate, in over 75% of cases, participants consider TrainRef's 's predictions to be more reliable, showing its advantage to produce semantically meaningful and robust classifications under extreme noise conditions. More examples and results are available at our website (TrainRef, 2025).
4.4 RQ4: ABLATION STUDY
Effectiveness of Initial Reference Set Size. Table 6 shows the effect of varying the initial reference set size [Dres| on the noise rate of the augmented set Di and test accuracy. Results are reported on CIFAR-100 with 80% symmetric noise and on two real-world datasets (Animals- 10N and Web Vision), for which only test accuracy is available due to the lack of clean labels. TrainRef consistently maintains a low noise rate (< 5%) in the augmented reference set and high test accuracy across varying
8

---

Published as a conference paper at ICLR 2026
Table User Study Evaluation of Prediction Reliability: At each noise rate, the participants choose
to agree with the predicted confidence of the test sample by either TrainRef or DISC.
Sym. Noise Rate
TrainRef (%)
DISC (%)
Total (%)
20%
62%
38%
100%
50%
74%
26%
100%
80%
78%
22%
100%
Table 6: Noise Rate and Sample Counts Initial Reference Set Size
Table 7: Effectiveness of Label Distribution on
Calibration (CIFAR 100 Inst. 40%)
Datasel Noise
Metric
k=1
k=5
k=100
Method
Hard
Sott Relabel
NNL
ECE
Test Ace (%)
CIFAR-100 100 (Sym 80%)
CleanNum
6069
6352
6389
6409
Mis
140
110
103
101
Ours
0,710
0.055
81.07
NRC
0.0230
0.0173
0.0161
0,0157
Ours-TC
0.707
0.032
77.85
77.91
Ours
0,756
0.065
81.77
TC
WebVision
Test Ace
82.21
82.28
82.27
82.36
Ours
0
0.046
82.33
Animals-10N
Test Ace
90.75
90.90
90.97
90.88
Ours-TC
0.669
0.017
reference sizes, remaining effective even with just one reference sample per class (DDrefl = 1). In our balance accessibility and performance. These results demonstrate TrainRef's ability to efficiently identify clean samples under extreme noise and minimal supervision. Effectiveness of Label Distribution We evaluate TrainRef on CIFAR- -100 with 40% instance- dependent noise, focusing on the effect of label distribution. Specifically, we replace soft labels with one-hot labels and apply temperature scaling in RQ2. Lower ECE (Expected Calibration Error) and NNL (Negative Log-liken log-likelihood) indicate better calibration. As shown in Table 7, TrainRef achieves the lowest ECE with and without temperature scaling, outperforming ablations. While one-hot labels (hard relabel) yield similar test accuracy, they degrade calibration by sharpening decision boundaries and ignoring ambiguous samples.
experiments, we use [Drey]
Additional ablations Phase III iteration counts (Appendix F.2), the role of influence-ber based reference augmentation (A (Appendix F.1), as well as analyses of computational cost (Appendix B), backbone fairness (Appendix F:5) and limitations (Appendix H) provided in the Appendix.
RELATED WORK
Learning with Noisy Labels (& Data Curation) Early LNL methods (Hendrycks et al., 2018; Patrini et al., 2017) assume class-conditional noise modeled by a label transition matrix. However, theoretical work (Chen et al., 2021a; Xia et al., 2020) shows that real-world noise is largely instance- dependent, making transition matrix estimation both inaccurate and computationally expensive. To handle instance-specifie noise, prior work proposes re- re-eeighting or filtering noisy samples using loss, confidence, or multi-view signals (Han et al., 2018: Yu al., 2019; Li al., 2020; Zhu et al., 2021; Kim et al., 2024). Semi-supervised learning (SSL) approaches (Sohn et al., 2020; Li et al., 2023) assign pseudo-labels to noisy samples, often relying on augmentations (Nishi et al., 2021; Cubuk et al., 2020). However, these heuristics degrade under high noise, mislabeling ambiguous samples and harming generalization (Das Sanghavi, 2023). Meta-learning methods (Wu et al., 2021; Li et al., 2019) use clean references for guidance but incur high computational cost due to bi-level optimization. Confidence Calibration. Confidence calibration aims align predicted probabilities with true correctness likelihoods. Classical post-hoe methods include Temperature Scaling (TS) (Guo et al., 2017), parameterized transformations such as PTS (Tomani al.. 2022), spline-based mappings (Gupta et al.. 2020), and Mix-n-Me Matahh (MnM) (Zhang et al., 2020). These methods adjust output probabilities after training, but they cannot fix mis-ordered confidence rankings learned under noise. Train-time calibration has also been explored: focal-style losses (Mukhoti et al., 2020a; Ghosh et al., 2022; Tao et al., 2023) emphasize hard samples but risk amplifying label noise, while denoising approaches such DISC (Li et al., 2023) and L2B (Zhou et al., 2024) combine label correction with calibration. Evaluation metrics like ECE (Nixon et al., 2019), AdaECE (Mukhoti et al., 2020a), and recent unbiased estimators such as ECEaebias and ECEs (Roelofs al., 2022) provide multiple perspectives on calibration quality. Our work builds upon these foundations by integrating calibration with noise-robust training via reference-guided distributional curation.

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
We propose TrainRef, a training-time data curation framework that unifies label denoising and confidence calibration. Through Curation-Training Co-evolution, TrainRef refines the embedding space, maintains a diverse reference set, and assigns reliable soft labels. Extensive experiments show it outperforms state-of-the-art methods, improving accuracy and confidence calibration, with qualitative studies confirming its reliability for real-world noisy-label scenarios. In the future, we will deliver a library of TrainRef for the community and generalize the technique on generative models.
ACKNOWLEDGMENTS
We would like to thank Koh Pang Wei from the University of Washington for his insightful discussions and inspiration. This research is supported in part by the National Natural Science Foundation of China (62572300), the Ministry of Education, Singapore 0004), the National Research Foundation, Singapore and the Cyber Security Agency under its National Cybersecurity R&D Programme (CCRP2S-P4-TAICCN)) DSO National Laboratories under the AI Singapore Programme (AISG Award No: Security Agency of Singapore under its National Cybersecurity R&D Programme and CyberSG R&D Cyber Research Programme Office. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore, Cyber Security Agency of Singapore as well as CyberSG R&D Programme Office, Singapore.
100--T2EP2112-00017 MOET32020-
(800-1BB, and the Cyber
REFERENCES
CVAT AI. How much does cost to outsource annotation to data labeling ser- vice? service, 2024.
Pengfei Chen, Junjie Ye, Guangyong Chen, Jingwei Zhao, and Pheng-Ann Heng. Beyond class- conditional assumption: A primary attempt to combat instance-dependent label noise. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11422-11450, 2021a
Yingyi Chen, Shen, Shell Xu Hu, and Johan AK Suykens. Boosting co-teaching with compression regularization for label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 2688-2692, 2021b.
Charles Corbière, Nicolas THOME, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. Ad- dressing failure prediction by learning model confidence. In H. Wallach, H. Larochelle, d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neu- ral Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL http///poooeeees/eee/eeeeeeeesee h neurips httppps.cccppreeee cctpaper_caaiieceeccee www.ffflesseeeeeee httpppaesfilleeppprr///////////e pttppppppe/////leee///// 757f843a 2ETE843al11 69cc (1199cc677064995300111 Pappparr.pppp
A. Beygelzimer,
https:
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 702-703, 2020.
Rudrajit Das and Sujay Sanghavi. Understanding self-distillation in the presence of label noise, In International Conference on Machine Learning, pp. 7102-7140. PMLR, 2023.
Elahe Delavari, Aws Khalil, and Jaerock Kwon. Caril: Confidence-aware regression in imitation learning for autonomous driving, 2025. URL https:://// http://axxvvvooooee abs 2503.50000 00783.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological), 39(1): 1-22, 1977.
Erik Englesson and Hossein Azizpour. Generalized jensen-shannon divergence loss for learning with noisy labels. Advances in Neural Information Processing Systems, 14:30284-30297 2021.
Forbes. Andrew ng launches
campaign for data-centric ai. wwwpchhhe---ammaignnnnn
centric-ai/, 2024.
10

---

Published as a conference paper at ICLR 2026
Arindam Ghosh, Thomas Schaaf, and Matthew R. Gormley. Adafocal: Calibration-awan adaptive focal loss. In Advances in Neural Information Processing Systems, 2022.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu, and Richard Hartley. Calibration of neural networks using splines. arXiv preprint arXiv:2006. /2800, 2020.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.
Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks on labels corrupted by severe noise. Advances in neural information processing systems, 31, 2018
Geoffrey Hinton. Distilling the knowledge in neural network. arXiv preprint arXiv: 1503.02531, 2015.
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data- driven curriculum for very deep neural networks on corrupted labels. International conference machine learning, pp. 2304-2313, 13. PMLR, 2018.
Adam Tauman Kalai, Ofir Nachum, Santosh Vempala, and Edwin Zhang. Why language models hallucinate. arXiv preprint arXiv:2509.04664 2025.
Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. Unicon: Combating label noise through uniform selection and contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9676-9686, 2022.
Suyeon Kim, Dongha Lee, Seong Kang, Sukang Chae, Sanghwan Jang, and Hwanjo Yu. Learning discriminative dynamics with label corruption for noisy label detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22477-22487, 2024.
Youngdong Kim, Juseung Yun, Hyounguk Shon, and Junmo Kim. Joint negative and positive learning for noisy labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9442-9442999 9451, 2021.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning. pp. 1895-1894. PMLR, 2017.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Jichang Li, Guanbin Li, Feng Liu, and Yizhou Yu. Neighborhood collective estimation for noisy label identification and correction. In European Conference on Computer Vision, pp. 128 145. Springer, 2022.
Junnan Li, Yongkang Wong, Qi Zhao, and Mohan S Kankanhalli. Learning to learn from noisy labeled data. In Proceedings of the IEEE/CVF conference compuler vision and pattern recognition, pp. 5051-5059, 2019.
Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi- supervised learning, arXiv preprint arXiv:2002.07394 2020.
Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data. arXiv preprint arXiv: 02862, 2017.
Yifan Li, Hu Han, Shiguang Shan, and Xilin Chen. Dise: Learning from noisy labels via dynamic instance-specific selection and correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24070-24079 2023.

---

Published as a conference paper at ICLR 2026
Sheng Liu, Jonathan Niles- Weed, Narges Razavian, and Carlos Fernandez-Granda. Early-learning regularization prevents memorization of noisy labels. Advances neural information processing systems, 33:20331-20344 2020.
Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over- parameterization. In International Conference on Machine Learning. pp. 14153-14172. PMLR. 2022.
Yangdi Lu and Wenbo He. Selc: self-ensemble label correction improves learning with noisy labels. arXiv preprint 2xXv:2205.00156, 2022.
Yangdi Lu, Yang Bo, and Wenbo He. Co-matching: Combating noisy labels by augmentation anchoring. arXiv preprint arXiv:2103. 12814, 2021.
Eran Malach and Shai Shalev-Shwartz. Decoupling" when to update" from" how to update". Advances in neural information processing systems, 30, 2017.
Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H. S. Torr, and Puneet K. Dokamia. Calibrating deep neural networks using focal loss. In Advances in Neural Information Processing Systems, pp. 15744-15755, 2020a.
Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H.S. Torr, and Puneet K. Dokania. Calibrating deep neural networks using focal loss. In Advances in Neural Information Processing Systems, 2020b. URL https: http:///pooe///eeeee/eeee httpstt:ppp/roceeiiiiiiimm
file/ #ile/aeb7b30eff d02 (eF.2221110000033330000 ngs. neuri i ps. cc ccc/paper/2000
Rafael Müller, Simon Korblith, and Geoffrey E. Hinton. When does label smoothing help? Advances in neural information processing systems, 32, 2019.
Kento Nishi, Yi Ding, Alex Rich, and Tobias Hollerer. Augmentation strategies for learning with noisy labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8022-8031 2021.
Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 38-41, 2019.
Feiyang Pan, Jia He, Dandan Tu, and Qing He. Trust the model when it is confident: Masked madel-based actor-critic. Advances in neural information processing systems, 33:10537-10546, 2020.
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2536-2544, 2016.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1944-1952, 2017.
Zhiliang Peng. Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei. Beit v2: Masked image modeling with vector-quantized visual tokenizers. arXiv preprint arXiv:2208.08 06366, 2022.
Lorenzo Perini and Jesse Davis. Unsupervised anomaly detection with rejection. In A. Oh. T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neu- ral Information Processing Systems, volume 36, pp. 19673-69691 Curran Associates, Inc., 2023. URL https: https::/prroceedingg neurips cc/paper____ceeeeeeeeeeeeeee fww.ffllesaaae pappr/2023///////////// orrer .pf.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 19920-19930, 2020.
12

---

Published as a conference paper at ICLR 2026
Rebecca Roelofs, Nicholas Cain, Jonathon Shlens, and Michael C. Mozer. Mitigating bias in calibration error estimation. Proceedings of the 25th Interationall Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research. PMLR. 2022.
Rikard Rosenbacke, Asa Melhus, and David Stuckler. False conflict and false confirmation errors are crucial components of ai accuracy in medical decision making. Nature Communications, 15(1): 6896, 2024.
Bernhard Schölkopf, Ralf Herbrich, and Alex Smola. A generalized representer theorem. In International conference on computational learning theory, pp. 416 426.. Springer, 2001.
Mengmeng Sheng. Zeren Sun, Tao Chen, Shuchao Pang. Yucheng Wang, and Yazhou Yao. Foster adaptivity and balance in learning with noisy labels. In European Conference on Computer Vision, pp. 217-231355 Springer, 2024.
Xiaoshuang Shi, Zhenhua Guo, Kang Li, Yun Liang, and Xiaofeng Zhu. Self-paced resistance learning against overfitting noisy labels. Pattern Recognition, 134:109080 2023.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Do- gus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi uuii-suprriised learning with consistency and confidence. Advances neural information processing systems, 13:596-608, 2020.
Hwanjun Song, Minseok Kim, and Jae-Gil Lee. Selfie: Refurbishing unclean samples for robust deep learning. In International conference on machine learning, pp. 5907-5915. PMLR, 2019.
Zeren Sun, Huafeng Liu, Qiong Wang, Tianfei Zhou, Qi Wu, and Zhenmin Tang. Co-ldl: co- training-based label distribution learning method for tackling label noise. IEEE Transactions on 2021.
Multimedia,
Cheng Tan, Jun Xia, Lirong Wu, and Stan Li. Co-learning: Learning from noisy labels with self-supervision. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 1405-1413. 2021.
Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa. Joint optimization framework for learning with noisy labels. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5552-5560, 2018.
Linwei Tao, Minjing Dong, and Chang Xu. Dual focal loss for calibration. Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research. PMLR, 2023.
Christian Tomani, Daniel Cremers, and Florian Buettner. Parameterized temperature scaling for boosting the expressive power in post-hoc uncertainty calibration. In European conference on computer vision, pp. 555-569. Springer, 2022.
TrainRef. Trainref, 2025.
Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. Combating noisy labels by agreement: A joint training method with co-regularization. In Proceedings of the IEEÉ/CVF conference on computer vision and pattern recognition, pp. 13726-13735, 2020.
Yichen Wu, Jun Shu, Qi Xie, Qian Zhao, and Deyu Meng. Learning to purify noisy labels via meta soft label corrector. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 10388-10396, 2021.
Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. Advances in Neural Information Processing Systems, 33:7597-7610, 2020.
Yazhou Yao, Zeren Sun, Chuanyi Zhang, Fumin Shen, Qi Wu, Jian Zhang, and Zhenmin Tang. Jo-sre: A contrastive approach for combating noisy labels. In Proceedings of the JEEE/CVF conference computer vision and pattern recognition, pp. 5192-5201, 2021.
13

---

Published as a conference paper at ICLR 2026
Kun Yi and Jianxin Wu. Probabilistic end-to-end noise correction for learning with noisy labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7017-7025, 2019.
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor Tsang, and Masashi Sugiyama. How does disagreement help generalization against label corruption? In International conference on machine learning, pp. 7164-7173. PMLR, 2019.
Hongyi Zhang. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.094122 2017.
Jize Zhang, Bhavya Kailkhura, and T Yong-Jin Han. Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning. In International conference machine learning, pp. 11117-11122... PMLR, 2020.
Yikai Zhang, Songzhu Zheng, Pengxiang Wu, Mayank Goswami, and Chao Chen. Learning with feature-dependent label noise: A progressive approach. arXiv preprint arXiv:2103.07756 2021.
Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018.
Ganlong Zhao, Guanbin Li, Yipeng Qin, Feng Liu, and Yizhou Yu. Centrality and consistency: two- stage clean samples identification for learning with instance- deaunddee noisy labels. In European Conference Computer Vision, pp. 21-37. Springer, 2022.
Evgenii Zheltonozhskii, Chaim Baskin, Avi Mendelson, Alex M Bronstein, and Or Litany. Contrast divide: Self-supervised pre-training for learning with noisy labels. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1657-1667, 2022.
Xiong Zhou, Xianming Liu, Deming Zhai, Junjun Jiang, and Xiangyang Asymmetric loss functions for noise-tolerant learning: Theory and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):8094-8109 2023.
Yuyin Zhou, Xianhang Li, Fengze Liu, Qingyue Wei, Xuxi Chen, Lequan Yu, Cihang Xie, Matthew Lungren, and Lei Xing. L2b: Learning to bootstrap robust models for combating label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23523-23553 2024.
Zhaowei Zhu, Tongliang Liu, and Yang Liu. A second-order approach to learning with instance- dependent label noise. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10-13-11 2021.
14


Reviews:
[{"review_id": "wHkp3Z89lr", "reviewer": "Reviewer_3eSK", "summary": "The paper proposes a training-time data curation approach. This method enables learning under label noise by using a small trusted reference set to avoid learning from contaminated label norms, while also utilising soft distributional labelling instead of the hard categorical labelling used in previous studies. It presents a three-phase method consisting of: (1) self-supervised pre-training using MIM to obtain a noise-robust embedding; (2) augmentation of the small reference set to create a larger, high-precision set; and (3) iterative co-evolution, in which the model and the curated soft labels are updated alternately. Experimental results demonstrate state-of-the-art performance on CIFAR-100, WebVision and Animal-10N, with high predictive accuracy and improved confidence calibration. The paper also includes ablation studies and a qualitative study involving human experts.", "questions": "1. The pipeline relies on MIM to create a noise-robust embedding. How important is this choice? Would other self-supervised methods produce similar results?\n\n2. How were the initial reference images selected for each dataset?\n\n3.How were the 100 test samples selected in the user study?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written and easy to follow. The method is clearly explained and the experimental part is described in quite a lot of detail.  \n\n2. The proposed method is well motivated and intuitive. It properly addresses the limitations of previous methods: miscalibration and ambiguous instances arising from hard re-labelling and noisy-set denoising only. \n\n3. The reported experimental results are strong. The improvements in CIFAR100, WebVision and Animal-10N in terms of both accuracy and calibration are substantial in both low- and high-noise environments. The experimentation is extensive. \n\n4. The paper also presents the results of a qualitative study involving data labelling experts. In this user study, the experts consistently preferred the predictions from TrainRef, demonstrating the method's effectiveness.", "weaknesses": "1. A major limitation of this paper is that it does not explain how to select the initial reference set. In real-world settings, selecting an initial reference (e.g. one example per class) can introduce bias. Therefore, it is important to explain how to select the reference set and the extent to which this selection introduces additional bias.\n\n2. While the authors do provide a computational time comparison in the appendix, I assume that, as the dataset size increases, the computational overhead of TrainRef will also increase significantly. This needs to be clarified, and if possible, demonstrated empirically.\n\n3. The approach relies entirely on the Masked Image Modelling (MIM) architecture. It may not generalise to other architectures. \n\n4. Lastly, while the qualitative study in Section 4.3 is helpful, using only five human testers is insufficient for making any solid claims. It is unclear whether the qualitative results are biased. Therefore, it would be beneficial for the paper to include a brief explanation of how it was ensured that the qualitative results were not biased due to the small number of human experts involved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kwDMl4InIO", "reviewer": "Reviewer_6MHt", "summary": "The paper propose a method to curate data label into soft-format using a clean reference dataset. The reference dataset is built with small number of examples and augmented using label-free features. The pipeline looks promising and novel and the results show improvement compared with previous method on different dataset and type of noise. Some analysis and ablations can be conducted to further improve the completeness。", "questions": "1. Section 2 claims two types of noise, Categorical Noise and Distributional Noise. Is there analysis on how good the proposed method is at dealing with each type? Like the after the curation, which type of noise will be more reduced.\n2. Section 2 also claims the challenge of Reference-set diversity. How to measure the diversity? Is the number of samples a good way of measuring? How the diversity of Reference-set affect the final results and how to set the threshold of influence score to select clean data? Actually the influence score will select similar examples as the initial reference dataset, so the diversity may not change much.\n3. What is the necessity of phase two? Is it a cold start for the EM optimization in phase three? What is the comparison of between with and without phase two\n4. How does the initial noise level and threshold of influence score affect the convergence of phase three.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The method utilize a reference set for data curation, and label-free embeddings to select data in reference set. Also the final phase conduct  a iteratively optimizing between model and label. The pipeline sounds novel and promising.\nThe motivation is clearly illustrated and easy to follow.\nThe cured label is in soft format, which is good for analysis and interpretability.\nThe experiments results look good compared with previous method and also work on different datasets.", "weaknesses": "Some analysis and ablations can be conducted for better understanding of this method\nSee questions part below for details", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "55e6yTbhjf", "reviewer": "Reviewer_GRDX", "summary": "This paper introduces TrainRef, a training-time data-curation framework for learning with noisy labels that aims to improve both predictive accuracy and confidence calibration. The method assumes access to a small external reference set of clean samples (as small as one sample per class). TrainRef uses reference augmentation to identify and expand a clean subset from the noisy dataset and employs a model–dataset co-evolution strategy to construct an embedding space used to assign distributional labels (class probability vectors) rather than categorical labels, thereby addressing sample ambiguity. Experiments on several research benchmarks report competitive or state-of-the-art results against LNL methods and improved calibration relative to label smoothing, Mixup, and temperature scaling. A small user study is reported to support the human interpretability of model confidence.", "questions": "- Iteratively expanding the “clean” pool may amplify early mistakes or biases in the data. Could the authors provide observations on these line of research? \n- What mechanisms prevent confirmation bias when expanding the clean set?\n- Can you elaborate on why “one sample per class” in the reference set suffices to avoid normality pollution and enable accurate clean-sample selection in high-class settings?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The problem is clearly framed, and the pipeline has a cohesive design. The paper targets accuracy and calibration in LNL and aligns the design of TrainRef to those twin objectives.\n- Distributional labels for ambiguity: Replacing categorical relabeling with class distributions is a principled way to mitigate class ambiguity and improve calibration.\n- Results are strong across multiple datasets; comparisons include both LNL and calibration baselines.\n- The alternating “reference expansion - model training” scheme is simple enough to integrate with common backbones.", "weaknesses": "- The concept of \"normality pollution\" is introduced in the abstract and briefly mentioned in the introduction. Since this is an important motivation for the paper, this phenomenon should be further discussed and evaluated.\n- The paper does not deeply study how the quality, coverage, or size of the reference set​ affects selection precision, convergence, or final performance. Robustness to partially wrong or class-imbalanced​ is not shown. This includes the intuition that different (more or less prototypical) samples in the reference set will affect model performance and convergence.\n- Several compared LNL baselines are not designed to use an external clean set. To isolate TrainRef’s contribution, ablations that (a) remove the reference set​, and (b) enable a comparable “oracle clean set” for baselines would clarify fairness.\n- The computational overhead of reference augmentation and model–dataset co-evolution (e.g., nearest-neighbor searches, embedding updates, voting) is not fully quantified; behavior with many classes and very high noise rates remains unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)