LANE: Label-Aware Noise Elimination for Fine-Grained Text Classification
Authors: Tiberiu Sosea
Cornelia Caragea
Venue: ICLR 2026 Poster (2026)
Keywords: label noise
fine-grained classification
label relationships
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: In this paper, we propose Label-Aware Noise Elimination (LANE), a new approach to learning with noisy labels. At its core, LANE introduces a new metric---label-aware margin---aimed at quantifying the degree of noise of each training example (or quality thereof). LANE leverages the semantic relations between classes and monitors the training dynamics of the model on each training example to dynamically lower the weight of training examples that are perceived to have noisy labels. We test the effectiveness of LANE on multiple text classification tasks and benchmark our approach on a wide variety of datasets with various numbers of classes and amounts of label noise. LANE considerably outperforms strong baselines on all datasets and settings, obtaining significant improvements ranging from an average improvement of 2.88% in F1 on manually annotated datasets to a considerable average improvement of 4.75% F1 on datasets with high level of injected label noise. We carry out a comprehensive analysis of LANE and identify the key components that lead to its success.

Published as a conference paper at ICLR 2026
LANE: LEARNING WITH NOISY LABELS USING LABEL-AWARD MARGINS
Tiberiu Sosea
Cornelia Caragea
Computer Science, University of Illinois Chicago
tsosea26uic.edi cormeliabuic.en
ABSTRACT
In this paper, we propose Label-Aware Noise Elimination (LANE), a new ap- proach to learning with noisy labels. At its core, LANE introduces a new metric- label-aware margin-aimmed at quantifying the degree of noise of each training ex- ample (or quality thereof). LANE leverages the semantic relations between classes and monitors the training dynamics of the model each training example to dy- namically lower the weight of training examples that are perceived to have noisy labels. We test the effectiveness of LANE multiple text classification tasks and benchmark our approach on a wide variety of datasets with various numbers of classes and amounts of label noise. LANE considerably outperforms strong baselines on all datasets and settings, obtaining significant improvements rang- ing from average improvement of 2. 2.88% in Fl on manually annotated datasets to considerable average improvement of 4.75% F1 on datasets with high level of injected label noise. We carry out a comprehensive analysis of LANE and identify the key components that lead to its success. We release our code at: hhttss://itthin http:/thc.ccoomaaaaa//a//a tww.e///eeeeeee/ Labe -wware-Noo Noi se- weliimmnniiin iiiiiitttaiiiiiiiii ion.
INTRODUCTION
Supervised deep learning models are ubiquitous in many applications, but their success depends on the quality the training data. Many existing datasets are annotated by humans on crowdsourcing platforms (Demszky et al. (2020) or by automatic approaches such as distant (or weak) supervision (Mintz et al.) 2009 Wang et al. 2012 Abdul-MMaged & Ungar 2017). and, while weak supervision inherently introduces unwanted mislabeled samples, humans nn matter how careful, are also prone making labeling errors, especially on tasks that involve distinguishing between a large number of closely confusable overlapping classes, e.g., emotion detection (Demszky et al.) (0200) or topic classification (Lewis et al.) 2004). The mislabeled training samples are particularly harmful when learning large overparameterized neural networks, since these networks achieve training error on any dataset, and have very poor generalization capabilities (Zhang et al.] 2016).
Pleiss et al. 2020 proposed the Area Under the Margin (AUM) metric identify mislabeled samples from a training set. The AUM for a sample measures the difference between the logit corresponding to its assigned label and the largest logit among all non-assigned labels averaged across the training epochs. The assigned label is the label assigned by either humans, weak supervision, even Large Language Models. The AUM for a mislabeled sample is expected to be low (likely negative) since the model-through generalization-tenn its (hidden) true class, and hence, the largest logit (among all logits) does not correspond to the assigned (wrong) label, but to the (hidden) true label. Pleiss et al.) (2020) subsequently removed samples with low AUM from the training set using fixed AUM threshold. Similarly, Han et al.) (2018);[Li et al. (2000) proposed the small-loss trick that uses the loss value in the convergence to identify mislabeled samples as those with a large-loss and removed the large-loss samples from the training set. However, through either the fixed AUM threshold or large loss elimination, hard but valuable clean samples are unnecessarily removed from the training set. More recently, Zhang et al.) (2024) proposed to use all training samples, each with different weight estimated using sample weighting mechanism called Hyperspherical Margin Weighting (HMW). That is, HMW assigns a weight each sample according the Integrated Area Margin (IAM) extension of the AUM
to predict the sample in

---

Published as a conference paper at ICLR 2026
metric that contrasts the logit of the assigned label with the two largest other logits. However, neither AUM nor IAM captures any semantic similarities that inherently exist between labels. For example, in emotion detection, "anger" is semantically more similar to "fear" than it is to "joy", and hence, a sample with the true (hidden) label "anger" but with assigned label "fear" should be penalized less than the same sample having the assigned label "joy".
To this end, we introduce Label-Awar Noise Elimination (LANE), a new approach to learning with noisy labels that specifically captures semantic relations between labels. In our approach, we retain all training samples, but we weight them differently based on the model's behavior on each sample. Thus, similar to Zhang et al. (2024). our model has access to much larger diversity of samples during training, including the hard but clean ones. In weighting the samples, we estimate the degree of "noisiness" of the assigned labels by introducing label-aware margins averaged across training iterations that capture inter-class semantic similarities. Our label-aware margins extend the concept of margins (Pleiss et al. 2020) by adaptively weighting samples when the assigned label does not consistently match the model's predicted label (over the training iterations). Note that the model's predicted label likely corresponds to the true (hidden) label when that label is consistently predicted by the model over the training iterations because of the ability of the model to generalize from other training samples that belong to the same label. LANE leverages AUM and jointly trains two networks to identify mislabeled samples and consequently assign per sample weight that accounts for the semantic similarity between labels so that an assigned mislabel receives a lower weight when it is more distant from the true (hidden) label and a higher weight when it is close the true label. We learn the inter-class semantic similarities using a label-aware supervised contrastive loss, trained jointly with a cross-entropy loss, better distinguish between easily confusable labels.
We evaluate the effectiveness of LANE on ten datasets: Empathetic Dialogues (Rashkin et al., 2019). GoEmotions (Demszky et al.) 2020), ISEAR (Scherer & Wallbott, 1994). CancerEMO (Sosea & Caragea 2020), RCVI (Lewis et al. 20044, SciHTC (Sadat & Carageal (022), SST-5 (Socher et al., 2013a), Amazon Review (McAuley Leskovee 2013). Yelp Review (Asghar 2016), and Yahoo Answer (Chang et al.) 2008). Using these datasets, we show that LANE works well on wide range of tasks and domains (emotion and general text classification; social networks, dialogues, and personal experiences). In all our experiments, automatically scaling down the weight of identified noisy samples from the training set shows great potential, improving the average performance on our original datasets by 2.88% F1 over AUM and by 2.32% over HMW. On noisy datasets, method boosts the performance by an average 3.37% F1 over AUM and 3.4% over HMW. The performance gap even more pronounced in extreme scenarios; for instance, 40% label noise, LANE significantly outperforms AUM and HMW by 4.75% and 4.01% % F1, respectively.
We summarize our contributions follows: 1) We introduce LANE, new approach that allows models to learn under label noise from a large diversity of samples and, at the same time, leverages inter-class semantic similarities to automatically identify and minimize the harmful effects of noisy samples; 2) We evaluate the effectiveness of our approach on ten text classification datasets from different tasks and domains and show improvements in performance compared with strong baselines and prior works; 3) We carry out a comprehensive analysis and ablation study of LANE to validate the effectiveness of our proposed method.
2 RELATED WORK
Noise-robustness Robustness noisy labels is addressed through robust loss functions, label cor-
rection, and regularization. Robust loss functions are designed to mitigate noise and reduce overfit-
ting (Liu & Tao 2015 Goldberger Ben- Reuven 2016 Lin et al., 2017 Ren et al. 2018 Saxena
ct al. 2019 Wang et al. 2019 Liu & Guo 2020 Englesson & Azizpour 2021 Li ct al., 202 Bai
lt al. 2022 Gao et al. 2023] Garg et al. [023]] Wei et a 2023a blc). For example, Goldberger &
Ren-Reuven (2016) propose to add a noise layer in the neural network architecture, whose param- eters can be learned for an accurate label estimation. Saxena et al. (2019) introduce a curriculum- learning approach that uses learnable data parameters to rank the importance of examples in the learning process. These parameters are then leveraged to decide the data to use at different training stages. [Wang et al. (2019) introduce Symmetric cross entropy Learning (SL), method that ad- dresses the issue of both under-learning of "hard" classes and overfitting of "casy" classes. Focal loss (Lin et al. 2017) incorporates a soft weighting scheme that puts emphasis harder samples.
2

---

Published as a conference paper at ICLR 2026
Liu & Guo 2020)
the other hand propose to alter the loss function to make it more robust under label noise and introduce Peer Loss Functions, which evaluate predictions on both the samples at hand, as well carefully automatically constructed peer samples. In our work,
also alter the loss and introduce a weighted cross-entropy loss where sample's weight reflects its quality or level of noise that is learned jointly using label-aware supervised contrastive loss. Label correction approaches gradually update the assigned labels based on the model's predictions (Tanaka et al.,
2018 Yi & Wu/ 2019; Li et al. 2022 Xu et al., 2025). For example, Xu et al. (2025) propose label correction method that uses normalized Jensen Shannon divergence to dynamically interpo- late between noisy labels and model predictions. Regularization approaches aim to prevent training data memorization and include early-learning regularization (Liu et al.) 2020), mixup (Zhang et al.
2018 label smoothing (Wei et al. 2022), and representation regularizer (Cheng et al. 2023).
Contrastive learning Supervised contrastive learning brings the latent representations of input samples closer together if they belong to the same class (positives) and further apart they belong to different classes (negatives). Gunee et al. (2020) use a supervised contrastive loss to improve fine-tuning performance of pre-trained language models in few-shot scenarios. Khosla et al. (2020) introduce variation of the traditional contrastive loss which aims to produce more samples in the
positive set by using data augmentation to generate more positive samples. |Suresh & Ong (2021)
argue that not all negative samples are equal and propose label-aware contrastive loss to infer the relations between classes and weight samples differently. [iu et al. (2025) propose a contrastive framework construct a reliable negative set of each sample by filtering out inappropriate negative pairs. In contrast, we propose label-aware margins that adaptively weight samples based on noise levels and inter-class semantic similarity, learned via a label-aware supervised contrastive loss.
Sample selection Pleiss et al. (2020) use Area Under the Margin (AUM) to monitor the behavior of the model on each sample and identify low-AUM samples as mislabeled samples. [Zhang et al. (2024) propose Hyperspherical Margin Weighting (HMW) to assign per sample weight based on Integrated Area Margin (IAM). However, neither AUM nor IAM captures the semantic relation be- tween the assigned (wrong) label and the true (hidden) label. Pan et al. (2025) identify correctly labeled yet hard-to-learn samples using the confidence gap between the annotated label and the re- maining labels during training to separate clean and mislabeled samples. Swayamdipta et al. (2020) separate training data into three regions, easy-to-learn, ambiguous, and hard-to-learn (potentially mislabeled) and use each region separately (by removing the other two regions) to understand its benefits to learning and generalization. DISC (Li et al.] 2023) uses an instance-specifi dynamic thresholding to block access to specific training samples based the momentum of each sample memorization strength. Co-teaching (Han et al., (2118) uses two networks to combate noisy labels, with each network extracting samples with small loss and feeding them to its peer network for fur- ther training. DivideMix (Li et al. (020)) divides the training data into labeled set with clean samples and an unlabeled set with noisy samples and trains the model in a semi- superiieee fashion, maintaining two diverged networks where each network uses the dataset division from the other net- work. Unicon (Karim et al. (202)) considers potentially noisy labeled data as unlabeled samples in a leverages semi-supervised learning framework. We compare LANE with many of the above works.
3 PROPOSED APPROACH
Here, we first provide background on Area Under the Margin (AUM) introduced by Pleiss et al.) 2020) (33.T) and then present Label-A ware Noise Elimination (LANE), our new approach that leverages AUM to improve model robustness from noisy labels ( 3.2).
3.1
BACKGROUND
The margin M (Pleiss al. 2020, Bartlett et al. 2017 Elsayed al., 2018 Jiang et al. 2018 of sample x with assigned label at a traiiig epoch i is defined as follows:
( (1)
(x (x(())
= 20 (x) mark-=y* (x)
(1)
where =1/ (x) is the logit corresponding to assigned label y. and mav (1) xx is the largest other logit corresponding to label A: (from among all non-assigned labels). The margin measures how
3

---

Published as a conference paper at ICLR 2026
Algorithm LANE: Label-Aware Noise Elimination
1: Input: Training data {(x,, y: )=)) classifier network auxiliary network II, total epochs T.
2: Initialize: For each example (xx, vi), initialize ALM v)) = 0.
3: for epoch ( = ................................ do
for each batch For each x, E B, compute logits (1) (x,) using and class weights (probabilities) Uxy. using for all labels k. for all (x., v4) in batch B do
6:
7:
M (x,, 34) < Sys (x,) max 2i (xx) (Equation 1) LM (x., +
(1)
(0)
8:
"xx,; M ( y:) if ( (x., y1)
< 0 else (x., -
(x.. where argmax (x))))
9:
ALM (x;; - LM (x),
10:
for
11:
Nº < [(x., w:) E ALM (x: w:) < 0}
12:
Compute u4 and of according to Equations 5 and 6. for all (x,, 34) in batch B do
13:
14:
(x, U1) f-
15:
if (x,, v.) E No and ( (x,, Ut)
Mh then (ALM (x. )-
16:
X Ui) & exp
20
17:
end if for
18:
ex exp((12 (h)) Fue "xy mopp(12
19:
CLSCL = Σ B8 H(II(x:), + (Px,
EpErA, log
20:
LOCK + E
-i= NoE(x.N)) ys) H((((x),,
21:
Minimize L LOOR + Crsser
22:
end for
23:
end for
different the assigned label is compared to a model's belief in a label at some epoch. negative margin likely implies incorrect prediction, whereas a positive margin implies correct prediction. The label quality (or noise) of sample x is measured by averaging the margins of x across all training epochs T, i.e., the Area Under the Margin (AUM) (Pleiss et al. 2020), defined as follows:
T
AUM(X, y)
Σ
0 (x.
Pleiss et al. (2020) first identify mislabeled samples by learning a threshold of separation between the AUMs clean and erroneous samples through new artificial class that mimics the training dynamics of mislabeled data and then remove all samples that fall under this threshold.
3.2 OUR PROPOSAL: LABEL- AWARE NOISE ELIMINATION
While the AUM metric is effective for identifying noisy data, it has two key weaknesses: 1) treats all label errors equally, ignoring the semantic relations between classes, and 2) it relies a hard threshold to completely remove samples, which can discard valuable but difficult clean samples. To address these issues, we introduce Label-Aware Noise Elimination (LANE). Instead of removing samples, LANE retains all training samples and assigns a per sample dynamic weight. This is achieved by jointly training two networks assess not only the likelihood of mislabel but also its semantic severity, ensuring that hard-but-clean samples are preserved while the impact of noisy labels is minimized. The core of this mechanism is redefinition of the traditional margin. We call this new metric the Label-aware Margin (LM).. Algorithm presents the learning of LANE.
Label-aware Margin (LM) LM operates within LANE's two-network architecture, where main classifier network 8 and an auxiliary network II are trained jointly. The LM rescales the standard margin M cllallulated from the logits of the main classifier 8-now using semantic similarity weights produced by the auxiliary network II. This rescaling is applied specifically when the margin negative, which strong indicator of mislabel. We adjust the margin dynamically follows:
4

---

Published as a conference paper at ICLR 2026
TIXT
SDN
J0Y
FER
ANG
LOGIT
SSP
D60
TRS
ANT
dostors do sny options for him.. have found much and support this
1.1
-067
1.1
00.5
-1.5
Table I:
Comparison of Margin (M) and Label-aware Margin (LM) for two examples. The assigned label (fear) is shown in red bold and the model predicted label for each example is shown in blue bold. For both examples, observe that M is -0.6 (i.e., 1.2 1.8). In the first example, LM is rescaled slightly since the assigned emotion fear semantically close the emotion corresponding to the largest other logit (i.e., anger). contrast, we observe that the second example, the assigned emotion fear is semantically distant from the emotion corresponding to the largest other logit which is trust, and hence, LM becomes much smaller.
( v) if ( (x. y) < 0 (1) where j =/ (x)
Wx.j
L (x,y)
(3)
(x,
otherwise
where Wx.jj is the weight obtained using the auxiliary network II, which produces higher values if the (potentially wrong) assigned label y of x is semantically close to the (hidden) likely true label predicted by the model, and lower values otherwise (i.e., if the potentially wrong assigned label is semantically distant from the model prediction). Note that we scale the margins only the margins negative, since these are the potentially problematic samples that may be overly ambiguous or mislabeled. To showcase the difference between our proposed label-aware margin LM and the vanilla margin M, we present in Table [[ two examples from an emotion dataset alongside the logits produced by the model as well as the margin M and label-aware margin LM. Both of these examples have the assigned label the fear emotion-wild mislabeled. However, although the margin of both examples is the same M = -0.6, we notice that the assigned label fear is semantically close to the label corresponding to the largest other logit (i.e., anger)- the model prediction in the first example, whereas in the second example, it is semantically distant from the label corresponding to the largest other logit (i.e., trust)-the model prediction. We emphasize that our LM captures this semantic difference between labels. Specifically, we observe that the LM of the first example, where the prediction and the assigned label are semantically close, ie., anger and fear, is larger than the LM of the second example where the prediction and the assigned label are semantically distant, i.e., trust and fear.
X1 can be viewed as ambiguous. X2 is clearly
Average Label-aware Margin (ALM) At an arbitrary iteration t we average the LMs across the training iterations, from the beginning up until the current iteration t and obtain the Average Label- aware Margin (ALM) as follows: (1) (x, y) = E LM (x,
Mitigating the harmful effect of mislabeled examples We propose a weighted cross entropy loss during training and assign higher weights for high-ALM samples and lower weights otherwise. Let N = {x; ( (x,, y/) < 0} be the set of samples at iteration that have negative ALMs and ) be the distribution of their ALMs. At I, we scale down the loss samples from N whose ALM is below the mean of the ALM distribution. We assume that samples with ALM above the mean hard but clean and we do not reduce their importance (same as for positive ALM). Specifically, we dynamically a truncated Gaussian distribution of mean 14 and variance o, at iter- ation t on all samples with ALM under the mean and assign weight for each sample x; as follows:
exp (ALM (x,,)))-1)3 20 and ALM' (x:, < Pt
if E N
(x.(X.)))))
(4)
otherwise
During training, we estimate the mean be and variance o, using the model's historical predictions:
#4 =
1
Σ
A (x5,8:)
(5)
N
5

---

Published as a conference paper at ICLR 2026
E (ALM (x,, MM)
ot
N
(XL.)))E
Intuitively, a low weight for an example indicates that the example produced an ALM that is con- sistently below the mean of the negative ALM distribution. we have shown, such examples are potentially mislabeled and may hurt generalization. Thus, at each training iteration / we simply rescale the cross entropy loss, assigning lower weight to potentially mislabeled examples:
B
LECE = Σ (x v:) (0(x,)),,
(7)
where 0xx,) is the class distribution of model on example x,, [B] is the batch size, and H is the cross-entropy. To better distinguish between easily confusable classes, we extend the supervised contrastive loss by Gunel et al. (2020) and propose label-aware supervised contrastive loss:
[BI
[BI
Wx, exp(h -19))
CLSCL = Σ (II() yi)
log
(8)
Px.
Σ. Es Wx... v.. exp (h h2)
Pc Ps:
where B is the current batch, Px. the set of positives for example x; (i.e., in the context of supervised contrastive learning the positives all examples that belong to the same class x; and augmentations (Khosla et al. (020 he, the embedding of x; produced by our classifier 0. Wx.0x; and Wx..ya represent the soft-assignment of sample x; to its assigned label //. and to simply utilize
the non-assigned label y, where y/+ 3/., To obtain these soft-assignments projection layer on top of 11 followed by softmax, which produces the weights Wx.,y.-
The final loss in LANE is the sum of the weighted cross entropy loss and the contrastive loss:
L= L= + CLSCL
Appendix A shows the architecture of LANE.
EXPERIMENTS
4.1 DATASETS
We the following datasets in experiments:
Empathetic Dialogues (Empath) (Rashkin)
et al. 2019). 2. GoEmotions (GoEmo) (Demszky et al. 2020). 3. ISEAR (Scherer & Wallbott
1994
4. CancerEMO (CEmo) (Sosea & Caragea [0200), , 5. RCV1 (Lewis et al. | 2004), Sci-
HTC (Sadat & Caragea 2022 7. SST-5 (Socher et al. 201 8. Amazon Review (Amazon R)
McAuley Leskovee 2013). 9. Yelp Review (Yelp) (Asghar 2016). 10. Yahoo Answer (Yahoo)
Chang et al. 2008).000))0 We provide details of these datasets in Appendix
4.2 EXPERIMENTAL SETUP
We evaluate the effectiveness of LANE on the above datasets under two label noise setups: 1) Original datasets, where the label noise comes from annotation errors in the dataset collection process; and 2) 20% noise, where we randomly shuffle the labels of 20% of the training data (an additional setup of 40% random noise shown in Appendix C). We use the HuggingFace Transformers (Wolf et al. 2020) library for our BERT implementation. Both 0 and IT are BERT base uncased models. The datasets we consider make their train/ validationn/es splits available, hence, we use the provided splits in our experiments. Similar to Khosla et al. (2020), to expand the positive set of examples in the contrastive loss, we augment our data using synonym replacement (Kolomiyets) et al. 2011), SwitchOut (Wang et al., 2018), and backtranslation (Tideemann & Thottingal (0200). For all datasets we follow the evaluation metrics used the works introducing the datasets. The initial batch size is set to 32, hence the total batch size (i.e., including augmentations) is 256. In

---

Published as a conference paper at ICLR 2026
Dataset
Empath
GoEmo
ISEAR
CEmo
RCVI
BASE
58.5 t 1.2
63.6 t: 1.2
71.5 0.6
76.8 + 0.8
56.8 + 0.8
E2L
07.6 + 0.8
63.2 1.2
0.7
#.9
54.3 = 1.1
H2L
58.9
0.77
72.0 0.6
1
55.8 1.4
AMG
59.0 0.6
64.8
0.3
76.1
52.3
58.1
63.8
72.2
0.7
55.7 1.3
58.4
0.8
71. 1.2
7
56.7 2.2
AUM
58.4 0.6
63.1
71.8 + 0.8
76.0
56.3 + 0.6
LCL
69.1 1.0
64.8 0.7
72.4 0.3
76.5
57.9 0.6
SCL
58,9 1 0.7
12.81111 1.1
71.5 1 0.9
76.210.6
56.9 1 1.7
DISC
59.4 0.9
63.2 1.4
72.3 1 1.3
76.4
56.5 1.4
UNICON
58.4 0.7
63.1 0.9
72.5 1.1
76.6 + 1.3
56.9 + 1.1
57.6
1.6
70.4
77.1
56.7
78.2
59.3
DATASET
SciHTC
SST-5
Amazon R
Yelp
Yahoo
BASE
32,5 1 1.75
56.3 1 0.6
67.5 1 0.6
65.9 1 0.6
75.4 0.6
E2I.
31.6 1.5
55.7 1.1
0.9
62.8 2.3
70.4
H2L
32.2
56.6
67.9 0.8
62.3 + 1.7
74.
30.6 1.1
55.1 1.3
67.4
65.1
72.3
32.81.55 1 1.5
54.1
65.8 1.7
65.1
74.6 1
3 1.4
55.7 1.1
6.14
65.8 1.8
74.8 + 1.6
AUM
31.2 + 2.63
56.4
66.4 0.6
08.1 0.6
72.9 0.6
0.6
LCL
33.1 = 1.42
1.42
57.6 0.9
68.2
06.8
76.8 0.6
SCL
32.7 32.7 1.1
56.8 1.5
67.8 1.3
66.11177
75.3 1.1
DISC
32.8 3
56.7 1.3
67.8 2.4
66.1
75.1
UNICON
32.7
56.5
1.6
67.5
67.9
77.
31.6 1. 1.4
57 1.1
67.4 2.2
08.1117
77
34.1 - 0.87
58.9 0.4
69.7
1 0.6
69.2 =
78.4
Table 2: Results LANE on the original datasets. The results are averaged across five runs and standard deviations are provided. Best results are shown in bold blue and second best are underlined.
terms of computational overhead, the auxiliary network LANE increases training compute by ap- proximately 1.8x compared to standard BERT fine-tuning run. Despite this, the model maintains a similar convergence rate in terms of epochs and remains compatible with a single consumer GPU, This positions LANE computationally efficient alternative to the resource-intensive process of utilizing Large Language Models for noisy label correction. down the importance of examples during training if their ALM is below threshold that set as the ALM mean of examples with negative ALMs (Eq, 5). We also experimented with different ALM thresholds such but observed worse performance than using the (see Appendix D).
our training setup, we only scale
4.3 BASELINE MODELS
We use BERT (Devlin et al. 2019) base uncased model in all experiments (denoted by BASE).
We compare LANE against methods that use training dynamics to assess the data quality, as well as approaches focused on exploiting the relationships between classes and approaches aimed at learning under label noise: Data Cartography (E2L, H2L, AMG) (Swayamdipta et al. 2020). Noise Layer (NSE) (Goldberger & Ren-Reuven 2016), Peer Loss Function (PLF) (Liu & Guo 2020 Area Under the Margin (AUM) (Pleiss et al.) 2020). Supervised Contrastive Learning
(SCL) Gunel et al. (2020), Label-aware Contrastive Learning (LCL)Suresh
Suresh & Ong (2021). DISC
(Li et al.] 2023). UNICON (Karim et al. 2022),, and Hyperspherical Margin Weighting (HMW)
Zhang al. 2024). We provide more details into these baselines in Appendix H.1
5
RESULTS
Results on Original Datasets We show the results on the original datasets in Table 2 We make following observations. LANE outperforms baselines in all setups. We observe improve- ments of 1.1% weighted Fl CancerEmo, 1.4% weighted Fl on RCVI, 1.5% accuracy on Amazon Review and 1.1% accuracy on Yahoo over the best performing baseline. Notably, over the base BERT model, Yahoo. We note that LCL, which leverages inter-class relations through the label-aware contrastive learning loss is the best performing baseline in out of the 10 datasets. Since LANE utilizes similar
2.9% weighted F1 improvement GoEmotions and 3.0% improvement

---

Published as a conference paper at ICLR 2026
Dataset
Empath
GoEmo
ISEAR
CEmo
RCVI
11.6 3.4
21.5
37.6 3.0
46.7
44.4
E2L
10.3 0.8
22.6
37.1 1.110.77
47.5
4 1.5
H2L
10.6
21.8 0.7
37.3 0.6
47.9
46.8 2.4
AMG
1..412
0.6
36,9 0.5
48.4
15.9
NSE
10.2
15.6 1.1
36.4 0.8
4..2+
1..9
PLF
12.5 12.5
16.7 16
36.8 1.3
5..1
48.2
AUM
0.6
23.5+
3..6
4..8
47.6
10.4
21.4
37.3+
46.4+
4
10.8
22.1
38.3
16.6
47.2
DISC
1..
22.5 12.1..7
40.5 0.5
50.3
47.1 2.2
UNICON
1.
21.9 1 1.2
39.5 1 0.9
42.3
49.2 1 2.3
HMW
12.4
22.0 + 1.5
38.1 2.1
50.7 2.2
48.2 + 1.8
LANE
15.9 1.3
24.3
1.2
40.4 = 0.8
52.5 0.9
49.4 = 2.1
DATASET
SeiHTC
SST-5
Amazon R
Yelp
Yahoo
BASE
24.5 4.6
3.7
61.5 1 1.5
60.7 1 1.3
64.8 1 1.7
E2L
24.1
2.7
60.7 2.4
62.3 + 2.9
64.9 3.1
H21.
2:
48.7
60.9 + 2.3
62.6 2.1
65.7 1.8
AMG
2.
1.5
61.3244
62.9 2.3
66.5
NSE
2.3
50.4
61.7
63.5
1.7
11.
62.1
64.7
68.1 1.3
AUM
2.2
+ 2.5
62.4 1.7
63.3 + 1.4
65.9 + 2.4
LCL
3.9
1.7
61.7 1 2.4
63.1 1 3.1
65.9 3.0 3.0
SCL
24.1 3.4
51.5 3.2
62.3 = 3.5
13.7+3.9
66.8 1
DISC
2.1
51.7 2.6
62.1 12112.7
63.2 2.5
67.3
UNICON
28.9 08.9
50.8 00.8331
61.5 3.7
62.3 3.9
64.2 3.7
28.7
51.3 1.8
61.2 1.1
62.5 1.9
66.3
2.97
65.2
68.9
Table 3: Performance of LANE in the 20% noise setting. The reported results are averaged across five runs and standard deviations are provided. Best results are shown in bold blue and second best are underlined.
inter-class relations during training, we postulate improvements over LCL arise from correctly identifying mislabeled ambiguous examples and eliminating their harmful effect during training.
Results on 20% % Noise Datasets The results obtained on the 20% noise (20N) datasets where 20% of the labels are intentionally flipped are shown Table We observe that this setup significantly more challenging for the model. For instance, on Empathetic Dialogues, the weighted FI of the BASE model drops from 58.5% on the original dataset 11.6% on the 20N dataset, with a similar trend on all the other datasets. However, even in this more challenging setup, LANE still outperforms the majority of baselines all setups. For example, on SST-5, LANE outperforms AUM in accuracy by 2.7%, DISC by 1.4%, UNICON by 2.3%, and SCL by 1.6%. The improvements over the base model are larger, with an average performance increase of 4.11%.
ANALYSIS
Ablation Study For our ablation, we design version LANE that uses averaged margins instead of ALMs so that the semantic relations are not incorporated into the model. We achieve this by replacing the ALM term with AUM in Equations 4, 5, and and denote this method by LANE SonE-sin sim This approach utilizes all training samples with per-sample AUM weight. Second, we design version of LANE that does not use weighted cross entropy in Equation i.e., X We denote this method by LANE alm. Third, we compare LANE against the vanilla AUM approach, which removes examples from the training set that have low AUMs.
We show the results on both the original and 20% noise (20N) datasets in Table 4 We observe that LANE consistently outperforms LANE eim LANEEalm, and AUM in all settings. The improve- ments are particularly noticeable the more challenging 20N setup. For instance, on the RCV1 dataset, which has large number of classes, LANE improves the micro Fl score to 49.4%. boost of 4.2% over LANE sim. 3.2% over LANE afm and 1.8% over AUM. The trend also holds on the original datasets. On SciHTC, LANE achieves F1 score of 34.1%. outperforming AUM by 2.9% and LANE sim by 1.7%. These results show that our proposed Average Label-awart Margin and the semantics-aware contrastive loss play important role in the success of LANE.
8

---

Published as a conference paper at ICLR 2026
Original Dataset
20% Noise
DATASET:
Empath
SciHTTC
Amazon R
RCV1
Empath
SciFTC
Amazon R
RCV1
LANE-
1
32.4p.s
66.8o.s
57.3p.s
14.7:
28.50.8
61.2p.s
45.2a.s
LANE-atm
59.1e.
32.11.2
68.21
47.91.
13.80.
29.31.2
61.312
46.21.4
AUM
58.20.7
31.2a.7
66.11.4
56.322
14.50.a
27.44.2
62.41.
47 G2.z
LANB
60.81
34.12.31
69.72.1
59.32.3
15.915
30.52.41
63.12
49.41.
Table 4: Ablation study: comparison between LANE, LANE sim -amm datasets using original data and 20% noisc. Best results are shown in bold blue and second best are underlined. Subscripts denote standard deviation.
LANE olmm and vanilla AUM on the
DATASET:
Amazon R
Yelp
AG News
Yahoo
DIVIDEMIX
60.66 3.1
64.66 + 2.5
89.05 + 1.1
69.17
CO-TEACHING
60.62 + 3.7
63.65 + 2.7
89.04
68.69 + 3.7
LANE
69.7 + 1.3 69.2 + 1.2
90.01 0.8
78.4 0.8
Table 5: Performance of LANE compared Dual Network Approaches.
Comparison with 2-Network Approaches To further contextualize LANE's performance, compare it against other popular two-network architectures designed to handle label noise: Co- teaching HHn et al. (2018) and DivideMix Li et al. (2020), presented in detail in Appendix E.2 We evaluated these approaches against LANE on four benchmark datasets, with the results detailed in Table 5 The analysis reveals that LANE consistently and significantly outperforms both Co- teaching and Divide.Mix across all tested datasets. On the sentiment classification tasks, LANE achieves accuracy 69.7% on Amazon Review and 69. 5% on Yclp Review, surpassing the next- best method, DivideMix, by substantial margins of over 9 and 4.5 percentage points, respectively. This performance gap is even more pronounced on the Yahoo dataset, where LANE's 78.4% accu- racy represents greater than 9-point improvement over DivideMix. Even on the AG News dataset, where all models perform well, LANE still establishes a new state-of-the-ar result with 90.01 % ac- curacy. These results strongly suggest that LANE's 's use of label-aware margins for dynamic sample weighting is a very effective strategy for mitigating label noise.
Understanding LANE's Weight Distribution A core hypothesis behind LANE is its ability to differentiate between clean and noisy labels by assigning lower weights to samples it perceives as incorrectly labeled. To validate this, we analyze the distribution of weights assigned by LANE to both correctly and incorrectly labeled examples. This experiment is conducted on the RCV1, Amazon Review, and SciHTC datasets, each injected with 20% label noise. Figure | illustrates these distributions. The results provide clear and compelling evidence supporting our hypothesis.
Across all three datasets, the weight distribution for incorrectly labeled examples (shown in red) heavily skewed towards the left, with a distinct peak around weight O. For instance, in the RCV1 dataset, over 40% of all incorrect samples are assigned a weight in the [O, 0.1] interval. This demon- strates that LANE is highly effective at identifying noisy samples and drastically reducing their impact on the model's training process by assigning them near-zero weights. Conversely, the weight distribution for correctly labeled examples (shown in blue) is skewed towards the right, with the majority of weights concentrated in the higher ranges (approximately 0.4 to 0.9). This indicates
0.1
Incurrect
Incorrect
Incorrect
Proportion
Correct
Correct
Correct
0.2
0
0.2
0.4
0.6 0.8
0.2 0.4 0.6 0.8
0
0.2 0.4 0.6 0.8
Weight
Weight
Weight
(a) RCVI
(b) Amazon Review
(c) SciHTC
Figure 1: Distribution of Correct and Incorrect Examples by Weight

---

Published as a conference paper at ICLR 2026
Method
Emputh
GoFmo
ISEAR
RCVI
SciTTC
SST-5
(BERT)
1.2
1.2
71.5
5:.8+:0.8
5
56.3
67.5
65.0
75.4 4:06
LANE (BERT)
00.8
08.5
74.3
78.2 0.7
50.3
34.1 0.8
58.0
09.7
09.2
78.4
(RoBEKTal
16.1
61221.4
0
07
08.1
56.5
67.9 0.5
06.1
73.7
LANE (RoRERTa)
61.2++077
60.8 1.2
74.5 0.9
78.3+ 2.1
50.1
34.0
0.8
0..0++00.
70.1 +.0.7
60.8
78.9
Table 6: Performance of LANE using different backhone architectures (BERT vs. RoBERTa).
that the model preserves the valuable signal from clean data by assigning these samples high impor- lance. Therefore, the clear separation between the two distributions validates the core mechanism of LANE. The label-aware margin effectively serves as a reliable proxy for label correctness, allowing the model to dynamically filter out noise and prioritize learning from clean, high-quality examples.
Note that while Figure T Tillustrates the continuous weight distribution across these samples, LANE still enforces the ALM thresholding mechanism during actual training (Equation 4 ples with an ALM above the dynamic threshold are not penalized and simply retain a full weight of 1.
therefore, sam-
In our primary experiments, we selected BERT Devlin et al. (2019) as the backbone architecture to ensure fair and direct comparison with an extensive suite of es- tablished baselines (e.g., AUM, DivideMix, UNICON), which predominantly utilize BERT as the standard in the noise learning literature. However, LANE is fundamentally model-agnostic. The core mechanism, the Label-Aware Margin (LM), relies exclusively on the logits and latent embed- dings produced by the network. Because these are standard outputs for any deep learning classifier, LANE can be seamlessly integrated with other architectures, including RoBERTa [Lu et al.] (2019), DeBERTa He et al. (2020), even vision models like ResNet He al. (2016). To empirically val- idate this generalizabilityy we conducted supplementary experiments replacing the BERT backbone LANE integrated with RoBERTa performs robustly, match- ing or exceeding the performance gains observed with BERT. This confirms that LANE's dynamic sample weighting is an sechtecturallindeeenddde strategy for mitigating label noise.
Applicability Beyond BERT
with RoBERTa. As shown in Table 6
Additional Analyses We conduct several supplementary evaluations in the Appendix to further validate the robustness and adaptability of LANE across various scenarios. Performance Under Extreme Noise (40% % ): In Appendix [ we evaluate LANE under severe noise setting where 40% of the training labels are randomly shuffled, demonstrating that LANE maintains its effectiveness and significantly outperforms baselines like AUM and DISC. ALM Threshold Analysis: We an- alyze the impact of dynamically learning the Gaussian re-weighting parameters versus using fixed ALM thresholds (μ- 0 and μ 11, with a = 1) in Appendix D confirming that dynamically inferring the threshold yields the best performance. Performance Against LLMs: We compare LANE's noise robustness against few-shot Large Language Models (ChatGPT and Llama-2 13B) under 20% label noise setup in Appendix F where LANE consistently outperforms the evaluated LLMMs across the majority of datasets. Semantic-Aware Label Noise: To simulate realistic human annotation errors, we evaluate LANE on datasets where noise is injected based on inter-class se- mantic similarities rather than uniform randomness in Appendix G showing that LANE effectively mitigates this ambiguity- parison: Finally, we benchmark LANE against a recent state-of-the--ar interpolation- based noise correction approach by Xu et al. (025) in Appendix [H] highlighting that LANE outperforms this recent method on four out of the five evaluated datasets.
label noise better than existing baselines. Noise Correction Com-
7 CONCLUSION
In this work, we introduced LANE, new approach that boosts the capabilities of deep learning models when learning under increased label noise. LANE leverages the inter-class semantic sim- ilarities and utilizes training dynamics to boost the performance in fine-grained text classification. We tested LANE on ten fine-grained text classification datasets where it obtained improvements in performance over strong baselines and prior works. In the future, we plan to extend approach to other domains and data types, e.g., image classification and the legal domain. We make our code available to further research in this area.
Acknowledgement This research is supported in part by the NSF IIS award 2107518. Any opin- ions, findings, and conclusions expressed here are those of the authors and do not necessarily reflect the views of NSF We thank our anonymous reviewers for their constructive feedback, which helped improve the quality of our paper.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Muhammad Abdul-Mageed and Lyle Ungar. Emonet: Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 718-728, 2017.
Nabiha Asghar. Yelp dataset challenge: Review rating prediction. arXiv preprint 2016.
Yingbin Bai, Erkun Yang, Zhaoqing Wang. Yuxuan Du, Bo Han, Cheng Deng, Dadong Wang, and Tongliang Liu. Rsa: reducing semantic shift from aggressive augmentations for self-supervised learning. Advances Neural Information Processing Systems, 35:21128-21144 2022.
Peter L. Bartlett, Dylan J. Foster, and Matus Telgarsky. Spectrally-norrmaiized margin bounds for neural networks. CoRR, 08498
2017. URL htt../t..//aa.....or////s//////
Ming- Wei Chang, Lev-Arie Ratinov, Dan Roth, and Vivek Srikumar. Importance of semantic repre- sentation: Dataless classification. In Aaai, volume 2, pp. 830-835, 2008
Hao Cheng, Zhaowei Zhu, Xing Sun, and Yang Liu. Mitigating memorization of noisy labels via regularization between representations. In The Eleventh International Conference on Learning Representations, 2023. /tLLhttts///ppeereeeee
n F
Dorottya Demszky, Dana Movshovitz-Atiias Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Su- jith Ravi. GoEmotions: A dataset of fine-gra one-grained emotions. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4040-4054, Online, July 2020. Association for Computational Linguistics. doi: 10. 111666//112222.llmmmamiii a00l-aammma n.372
URL https://acl http:///caaatthhhooooo ogy org/
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers). pp. 4171-4186, Minncapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 18155//11/N19-1423 URL https:
7/ 7/www. wwwwwb.www......... www.oog//tttttto httpb///gggaaatoooggollll------
Gamaleldin Fathy Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, and Samy Bengio. Large margin deep networks for classification. 2018. URL https: http::://axxvvvrr.gooo /tt.tttts////rrvoorrr////dd pttpa/awwww..r/////////..
05598. Fd
Erik Englesson and Hossein Azizpour. Generalized
divergence loss for learning with noisy labels. In M. Ranzato. A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. www.ccppppppppeeeeeeee
Cur- files/
Associates, Inc., 2021. WRUU/tttp::/prrcceeiiit www.iiiinnnnnnnunsnooeee.o....ee wttporriipsccppppppeerpppp optp:r//0///11fffeoo000saaaaaeee
Jiahui Gao, Renjie Pi, LIN Yong, Hang Xu, Jiacheng Ye, Zhiyong Wu, WEIZHONG ZHANG, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. Self-guided noise-free data generation for effi- cient zero-shot learning. In The Eleventh International Conference on Learning Representations, 2023. URL URt //pp:///peevvviieeee 2023.URR//tttts:///oeerrrr
www.:nettffoummmem for.m? d=h50p htte://orrmmmmhhh0p000000
net.
1o6
Arpit Garg, Cuong Nguyen, Rafael Felix, Thanh-Toan Do, and Gustavo Carneiro. Instance- dependent noisy label learning via graphical modelling. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2288-2293 January 2023.
Jacob Goldberger and Ehud Ben-Reuven. Training deep neural-networks using a noise adaptation layer. 2016.
Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. Supervised contrastive learning for pre-trained language model fine-tuning, arXiv preprint arXiv:2011.0142 2020.
11

---

Published as a conference paper at ICLR 2026
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL
ngs neuripsrrps.ee www.eris...pppppeeeeeeeee wtt.cccppppppeieee files/ (BBe:///OO77LAASSSSSSSSSsT555550 Pap.esspppppppppppp.....
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding -enaidnngenhheeedd bert with disentangled attention. arXiv preprint Stt::2000003655:0 2020.
Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions, 2018. URL PRLLLtttss//axx...oor//////////
00113
Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rahnavard, Ajmal Mian, and Mubarak Shah. Unicon: Combating label noise through uniform selection and contrastive learning. Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp- 1076-9686, June 2022.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33: 33:18661-18673 2020.
Oleksandr Kolomiyets, Steven Bethard, and for textual temporal analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers Volume 2, HLT '11, pp. 271-276, USA, 2011. Association for Computational Linguistics. ISBN 9781932432886.
Moens. Modellportabilityy experiments
David D Lewis, Yiming Yang, Tony Russell-Rose, and Fan Li. Rev1: A new benchmark collection for text categorization research. Journal of machine learning research, 5(Apr):361-397, 2004.
Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi-
supervised learning. In International Conference on Learning Representations, 2020.
Shikun Li, Xiaobo Xia, Hansong Zhang. Yibing Zhan, Shiming Ge, and Tongliang Liu. Estimating noise transition matrix with label correlations for noisy multi-label learning. Advances in Neural Information Processing Systems, 1124484--21999 2022.
Xuefeng Li, Tongliang Liu, Bo Han, Gang Niu, and Masashi Sugiyama. Provably end-to-end label- noise learning without anchor points. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 6403-6413, PMLR, 18-24 Jul 2021. URL https: http:////ccccee........
mlr. www.ppressvvll ctteess/v19//12211 htmll
Yifan Li, Hu Han, Shiguang Shan, and Xilin Chen. Disc: Learning from noisy labels via dynamic instance-speciff selection and correction. In Proceedings of the IEEE/CVF Conference Com- puter Vision and Pattern Recognition, pp. 24070-24079, 2023.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp- 2980-2988, 2017.
Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez-Granda. Early-leariing regularization prevents memorization of noisy labels. Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. JEEE Transactions on pattern analysis and machine intelligence, 38(3):447-411 2015.
12

---

Published as a conference paper at ICLR 2026
Xiaoyu Liu, Beitong Zhou, Zuogong Yue, and Cheng Cheng. Piremix: Combating noisy labels with pseudo-label relaxed contrastive representation learning. In 2025 IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pp. 6517-6527. IEEE, 2025.
Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. In International Conference on Machine Learning, pp. 6226-6236. PMLR, 2020.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 1907.19907.11690 2019.
Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimen- sions with review text. In Proceedings of the 7th ACM conference on Recommender systems, pp. 165-172, 2013.
Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. Distant supervision for relation ex- traction without labeled data. In Keh-Yih Su, Jian Su, Janyce Wiebe, and Haizhou Li (eds.), Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th In- ternational Joint Conference on Natural Language Processing of the AFNLP, pp. 1003-1011, Suntec, Singapore, August 2009. Association for Computational Linguisties. URL https: //aclanthologys hwwpchoology.orrrPP---111
Weiran Pan, Wei Wei, Feida Zhu, and Yong Deng. Enhanced sample selection with confidence tracking: Identifying correctly labeled yet hard-to-learn samples in noisy data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 19795-19803, 2025.
Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kilian Q Weinberger. Identifying mislabeled data using the area under the margin ranking. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Bal- can, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 77044-17056 Curran Associates, Inc., 2020. URL https: /ttppe///prccce..ggggg neurips. pdff Robert Plutchik. general psychoevolutionnay theory of emotion. In Theories of emotion, pp. 3-33. Elsevier, 1980.
cc/ ftt2g///000011112660002222111162
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau, Towards empathetic open- domain conversation models: a new benchmark and dataset. In ACL, 2019.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International conference on machine learning, pp. 4334 4343. PMLR, 2018.
Mobashir Sadat and Cornelia Caragea. Hierarchical multi-label classification of scientific doc- uments. Yoav Goldberg. Zornitsa Kozareva, and Yue Zhang (eds.). Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 8923-8937, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguis- tics. doi: 10.
URL
emn 1ec:1pp-mmin 610
Shreyas Saxena, Oncel Tuzel, and Dennis DeCoste. Data parameters: A new family of param- eters for learning differentiable curriculum. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc E. Fox, and Garnett (eds.), Advances in Neural Information Processing Sys- tems, volume 32. Curran Associates, Inc., 2019. URL httpp:::/prccceedingg//////////t tcc/paper/200 22019 9 htffil////
meurips.
Klaus R Scherer and Harald G Wallbott. Evidence for universality and cultural variation of dif- ferential emotion response patterning. Journal of personality and social psychology, 66(2):310, 1994.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, October 2013a. Association for Compu- tational Linguisties. URL htl.htttss///acaatttie ogy.
13

---

Published as a conference paper at ICLR 2026
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language pro- cessing, pp. 1631-1642, 2013b.
Tiberiu Sosea and Cornelia Caragea. Canceremo: A dataset for fine ame-grrined emotion detection. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 8892-8904, 2020.
Varsha Suresh and Desmond Ong. Not all negatives equal: Label-aware contrastive loss for fine-grained text classification. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pp. 4381 4381-4394, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguisties. doi: 10. 10.18653/v122021 .em.pp-mmmmllpa......
359. URL httpp:://aclanttoooooooeo/ wwtp:/og.....g./// /2021 www.p- miillp-main main. 359
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pp. 9275-9293, Online, November 2020. Association for Computa- tional Linguistics. doi: 10. 10.18653/v11200000 220.:///111222000mmmmmaaaaaaaa 746. URL htt.tttpp////mm /tllhttpp:::/allatthllo.. orr/2022 emnlp-malla..meemmmm n. 746
Daiki Tanaka, Daiki Ikami, Go Yamashita, and Hiroshi Ono. Joint optimization framework for learning with noisy labels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.
June 2018
Jörg Tiedemann and Santhosh Thottingal. OPUS-MT Building open translation services for the World. In Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT), Lisbon, Portugal, 2020.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint arXiv: 1397,, 2023.
Wenbo Wang, Lu Chen, Krishnaprasad Thirunarayan, and Amit P Sheth. Harnessing twitter" big data" for automatic emotion identification. In 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing, pp. 587-592. IEEE, 2012.
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. SwitchOut: efficient data aug- mentation algorithm for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 856-861, Brussels, Belgium, October- November 2018. Association for Computational Linguistics. doi: 10.18653/11/D18-111 URL
htttss://aclanee httpp:///claattoooooy..og///1/1. www.ooyyy..o/o///.1..00
Yisen Wang, Xingjun Ma. Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 322-330, 2019.
Hongxin Wei, Huiping Zhuang, Renchunzi Xic, Lei Feng. Gang Niu, Bo An, and Yixuan Li. Mit- igating memorization of noisy labels by clipping the model prediction. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. (66868-36886 PMLR, 23-29 Jul 2023a. URL https: fttps://ppoceeddings ml www..rppeessvveee
wtt.eaa..ngg......epesss//002 wei23e. wttp://wweeeeeememmme....
Jiaheng Wei, Hangyu Liu, Tongliang Liu, Gang Niu, Masashi Sugiyama, and Yang Liu. To smooth or not? When label smoothing meets noisy labels. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th Inter- national Conference Research, pp. 2 PMLR, 17-23 Jul 2022. URL https: http:/sss//ppcceedngs...m mmr. press/ vl 62 we 22b. html
Machine Learning, volume 162 of Proceedings of Machine Learning
14

---

Published as a conference paper at ICLR 2026
Jiaheng Wei, Zhaowei Zhu, Tianyi Luo. Ehsan Amid, Abhishek Kumar, and Yang Liu. To aggregate or not? learning with separate noisy labels. In Proceedings the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, KDD '23, pp. 2523-2535, New York, NY, USA, 2023b. Association for Computing Machinery. ISBN 9798400701030. doi: 10.1145/3580305. 3599522. htt.tttss://do. org/ 1 0 11....1...00 000111155558300 35995222
Qi Wei, Lei Feng, Haoliang Sun, Ren Wang, Chenhui Guo. and Yilong Yin. Fine-grainen classifi- cation with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1651-11660, June 2023c.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jemnite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-ar natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. Association for Computational Linguistics, doi: 1t..8653//1/2220.corm http://2200.mmmmmmmmmooo... URL emnlp demos.
aclweb. org wttp://rraanoooooo////ooooo
Yuanzhuo Xu, Xiaoguang Niu, Jie Yang. Ruiyi Su, Jian Zhang, Shubo Liu, and Steve Drew. Revis- iting interpolation for noisy label correction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 21833-21841, 2025.
Kun Yi and Jianxin Wu. Probabilistic end-to-end noise correction for learning with noisy labels. In 2019 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 7010- 7018, 2019. doi: 211109///R..201.0000
Chiyuan Zhang, Samy Bengio. Moritz Hardt. Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 2016.
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empiri- cal risk minimization. In 6th International Conference Learning Representations, ICLR 2018, Vancouver; BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL http:///ppppeeeeeeeee/epeeeeee.p net worum?id-ridds
Shuo Zhang, Yuwen Li, Zhongyu Wang, Jianqing Li, and Chengyu Liu. Learning with noisy labels using hyperspherical margin weighting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 16848-16856, 2024.
15

---

Published as a conference paper at ICLR 2026
LANE: label-Aware Noise Elimination for Learning With Noisy Labels
Dataset Input Panel
2. Model Overview
3. Outpurt Panel
Margle Coleulation
Main Classifier Network e
Logis per
Mazignn Mic 7) 2,00 man0000000
Auxiliary Network (TranstomeBBBRf)
Class remanttc Weughts
(tsugg +0
Classification Ustaset
Noley Text
Transfome/BBEr)
Label-Aware Morgle (LM)
Maxil (v fionniT
Improved Model Robustness Accuracy
Labutt
TThe
Jein Train Entropyy commottiv
lecer klive -rigerr
gleeed wiwe
Juset Trining (Cress Entrary astiva Lowa)
Tabe?
c. Averuge Lubel Label-Aware Margin (ALM) Loott Mi dornge
Gains over hasselines
TTUM baa 71
Lakel
?
Auxiliary Network TI (Transfomer
Weighted booss-Entrrpy Lens
Label Awar Supervized Contrastive Less
Joint duoi- Aenlnnetwork archiaccture
Truef))
LCe AUM -t lono gits yoi
hegpeeee cample colcioonn
Haadoout XTTP Impertance molyy toiieeniigiedd
Semantic simllrity ay-e margin
Wiightd Crozerropyy Libooooorre Superiied Conrractie Loss
Upditto
Legend and Callouts
smilariing
No Iabel
Connect Whell
Classiffer
Auziliary
Semantic similanity
KK
Margin
Weighted
curve
Figure 2: Architecture of LANE.
A ARCHITECTURE
Figure D shows the architecture of our LANE approach.
B DATASETS
We evaluate LANE on: 1. Empathetic Dialogues (Rashkin et al. 2019). a dataset composed of conversations between speaker and listener annotated with 32 emotions. We consider solely the first turn of the conversation in our experiments, resulting in 22, 000 total examples. 2. GoEmotions (Demszky et al. 2020), sentence-level dataset created using Reddit comments and contains more than 58, 000 sentences annotated with 27 emotions. 3. ISEAR (International Survey on Emotion Antecedents and Reactions) (Scherer & Wallboti 1994). dataset of 700 personal experiences annotated with 7 emotions. 4. CancerEMO (Sosea & Caragea 2020). a dataset of 8, 500 examples collected from a cancer forum annotated at sentence level with the basic Plutchik-8 (Plutchik] 1980) emotions. 5. RCV1 (Lewis et al. 2004), a large scale dataset composed of news stories labeled with a total of 105 different topics. 6. SciHTTC (Sadat Carageal 2022). dataset from 186, 160 scientific papers, annotated with 80 topics. 7. SST-5 (Socher et al.) (013b)), dataset composed of 11, 855 sentences from movie reviews, annotated with five sentiment labels: negative, somewhat negative, neutral, somewhat positive, and positive. 8. Amazon Review (McAuley & Leskovec 2013), a sentiment classification dataset composed of 600, 000 training and 130, 000 test Amazon reviews annotated with 5 sentiment classes. 9. Yelp Review (Asghar (016). sentiment classification dataset with 130, 000 training and 10, 000 test samples annotated with the same 5 classes, and 10. Yahoo Answer (Chang et al., 2008), topic classification dataset with 10 topic classes, composed of 140, 000 training and 6, 000 test samples.
C DATASETS WITH 40% LABEL NOISE
We show in model did not convege. We notice that LANE stays effective across the ten datasets, and we observe that AUM yields poor results on this dataset with very high amounts of noise, indicating that it may not work in high-noise setups. For example, AUM outperforms DISC by an average of 1.5% on 20N across the datasets whereas DISC outperforms AUM on 40N by a significant 2. 2.9%. Critically, LANE outperforms both DISC and AUM on 40N by average of 2.8% and 4.75%, respectively.
on the 40% noise (4ON) datasets. Results marked with indicate that the
16

---

Published as a conference paper at ICLR 2026
Dataset
Empathhetc Dialogues (wfi)
Golmotions (wTI))
ISEAR (wFl)
CancerEmoo (xFI)
RCVI (wFI)
BASE
E21
H2L.
AMG
NSE
31.4 1.7
AUM
10.1 0.6
17.5 # 1.3
27.8
11.8 0.9
82.5 1.3
SCL
DISC
1 1.7
19.6 1 0.7
31.4 33.1 0.9
47.6.1
31.7
UNICON
13.7 +
17.4 = 1.2
16.5
34.6
HMW
11.2 3.5
18.6 + 3.2
3.7
41.5
31.9 244
LANE
0.6
DATASET
SciIICC (MFI)
SST-5 (Acz)
Amazon Review (Acc)
Yelp (Ace)
Yahoo (Acc)
BASE
6:2
AMG
18
2.
PLF
17.2111
51.4111
52.6
12.7
LCL
DISC
18.5 1 2.3
43.8
1.8
52.9
1.9
53.8
2.3
44.7
2.1
UNICON
19.6
13.
1
1.6
55.2
4 1.3
53.9
14.7
2.1
HMW
17.111
13
1.7
53.1
2.3
53.7 1.7
12.8
LANB
20.5 1.5
45.7 1.3
56.8
2.2
56.2 + 2.3
46.3
2.5
Table 7: Performance of LANE the the benchmark datasets under 40% label noise. The reported results are averaged across runs standard deviations are provided. results are shown bold blue and second best are underlined. Results marked with indicate that the model did not converge.
D ANALYSIS THE ALM THRESHOLD
In our main experiments, we automatically learned the Gaussian re- re-weighting parameters from the data, as described in the main paper. Specifically, during training, we estimate the mean jr and variance using the model's historical predictions. We also experimented with using fixed, hard- coded values for the threshold, setting μ to and -1, with a = 1. When fixed threshold used, weighthhh sample if its Average Label-aware Margin (ALM) is less than the threshold (e.g., ALM < 0); otherwise, the sample's 's weight remains 1, per our weighting function (Equation 4
We present the results of these experiments on the Yelp and Yahoo datasets in Table 8 The results show that the best performance is achieved when the parameters are inferred dynamically from the data's historical predictions, validating the approach used in paper.
Approach / 0
-1
Inferred from Data
Yelp
64.4
64.1
65.2
Yahoo
68.1
65.1
68.9
Table 8: Performance comparison (Accuracy) the Yelp and Yahoo datasets using different ALM thresholds (u). The "Inferred from Data" column uses the dynamic method from Equations and the main paper.
E BASELINES
E. SINGLE NETWORK APPROACHES
Data Cartography Following (Swayamdipta et al. 2020), we identify three types of training examples: easy-to-learn (E2L), hard-to-learn (H2L), and ambiguous (AMG) and analyze the importance of each type to the training process by removing the other two types.
Noise Layer Following (Goldberger & Ben-Reuver 2016), introduce noise layer to the BERT model which we train for correct label estimation. We denote this model by NSE our experiments.
17

---

Published as a conference paper at ICLR 2026
DATASET:
Empathetic Dialogues (wFI)
GoEmotions (mFI)
ISEAR (ACC)
CancrEMO (mF1)
RCVI (mF1)
CHATGPT
12.8 1: 3.1
21.4 2.5
37.3 1: 1.1
48.9 1.9
42.9
LLAMA-2
10.9 = 3.7
20.4 =
35.4 = 1.6
50.2 1.7
39.7 1.8
LANE
15.9 1: 1.3
24.3 1: 1.2
40.4 1: 0.8
52.5 1 0.9
49.4 1:
DATASET:
SciHTC (MF1)
(Acc)
Amazon Review (Acc)
Yelp (Acc)
Yahoo (Acc)
CHATGPT
28.3 +: 5.0
49.6 0.6
62.6 +: 0.9
64.5 + 0.9
64.9 0.9
LLAMA-2
16.1 # 6.2
54.2 0.4
61.3 2.3
62.3 1.4
61.1 2.3
1.6
2.3
68.9 2.5
Table 9: Performance of LANE compared with LLMs. Best results shown in bold blue and second best are underlined.
Peer Loss Function We also compare method against Peer Loss Function (PLF) (Liu & Guo. 2020). a method that alters the training loss function to account for label noise.
Area Under the Margin We consider the AUM method (Pleiss al. 2020 baselines. This method computes Area Under the Margin metric for each training example and eliminates low-AUM examples that are potentially noisy, using fixed threshhold for elimination.
of
Contrastive Learning We compare LANE to the label-aware supervised contrastive learning
(LCL) method proposed by Suresh & Ong (2021 and the traditional supervised contrastive learning
(SCL) KKosla et al. 2020).
proposes instance-specific dynamic thresholding mechanism that blocks access to specific training examples based on the momentum of each instance's 's memorization strength. Additionally, DISC proposes to correct the labels of potentially noisy examples.
DISC (Li al. 2023
UNICON (Karim et al. 2022) leverages semi-supervised learning (SSL) to mitigate the harmful effects of noisy labels by considering the potentially noisy labeled data as unlabeled examples in an SSL algorithm. UNICON also proposes new selection mechanism for these unlabeled examples during training.
Hyperspherical Margin Weighting (HMW) Zhang et al. 2024 is sample weighting strategy that improves learning with noisy labels by using a novel metric called the Integrated Area Margin (IAM). To better distinguish clean but hard-to-learn examples from mislabeled ones, the IAM metric constructed by combining two distinct margin-based signals: the established AUM ranking Pleiss et al. (2020) and newly proposed Top-K Under the Margin (TKUM) ranking.
E.2 DUAL NETWORK APPROACHES
Co-teaching Han al. (2018) take peer-teaching approach and simultaneously trains two deep neural networks. each mini-batch, each network identifies samples it believes have a small loss (and are therefore likely to be correctly labeled) and feeds these "clean" samples to its peer network for subsequent training. This cross-training helps the models avoid overfitting to noisy labels that one network might have memorized.
DivideMix Li et al. (2020) reframes learning with noisy labels as a semi-supervised learning problem. It also maintains two diverged networks and at the start of each epoch, it uses Gaussian Mixture Model on the per-sample loss distributions to dynamically divide the training data into a labeled set of likely clean samples and unlabeled set of likely noisy samples. Each network then trains on the dataset division provided by the other, enhancing robustness.
F PERFORMANCE AGAINST LLMS
We test our approach against few-shot large language models: ChatGPT and Llama-2 13B Toovrron ct al. 2023) to compare the robustess to label noise of LANE with that of popular LLMs in 20% noise setup and show results in Table 9 For all datasets except SciHTC we fit large number of examples in the prompt and set the number of few-shot examples to 100. We use only 10 few-shot examples for SciHTC since the examples (i.e., paper abstracts) much longer and exceed the text window. Similar to the original 20% noise setup. 20% the few-shot examples are purposefully mislabeled. To account for the variance produced by the particular fhw-shot examples selected,
18

---

Published as a conference paper at ICLR 2026
ChatGPT 10 times with different few-shot examples in the prompt and report average values. Similarly, we run Llama-2 20 times with different few-shot examples. We observe that LANE out- performs the LLMs on all datasets except SST-5. Notably, LANE improves upon Llama-2 by 15.4% on SciHTC and by 18.7% on RCV1 and improves the performance over ChatGPT by 3.1% accuracy on ISEAR and 6.5% micro F1 on RCV1. Among the L.Mss, ChatGPT obtains the best results, out- performing 2 especially in complex tasks such as RCVI and SciHTC. Concretely, ChatGPT obtains 28.3% macro Fl on SciHTC, 13.2% improvement over Llama-2.
G SEMANTIC-AWARI LABEL NOISE GENERATION DETAILS
In fine- fine-grained classification, annotation errors are rarely uniformly random; human annotators are more likely to confuse semantically similar classes. To evaluate LANE under a more realistic am- biguity setting, we designed Semantic-Aware Noise setup. Instead of flipping labels randomly, biased the label-flipping probability so that instances much more likely to be mislabeled as a semantically similar class rather than a completely unrelated one. The semantic noise datasets are generated described below. We evaluated LANE against the best-performing baselines on datasets with well-defined class similarities. As shown in Table LANE consistently outperforms all baselines in this challenging scenario, confirming that the Lwarl-Aware Margin effectively miti- -ringeeety-riieennno
gates realistic, ambiguity driven label noise.
Method
Empath
GoEmo
Yelp
CEmo
SST-5
BASE
50.3
57.2
64.2
73.4
$3,3
AMG
52.7
60.3
66.2
74,8
54.1
AUM
51.2
61.1
66.5
74.1
52.7
LCL
53.6
65.4
73.9
S3.2
DISC
52.4
60.2
64.3
74.2
53.6
LANE
54.2
61.4
67.4
75.1
54.3
Table 10: Performance comparison under Semantic-Aware Noise.
Semantic-aware label noise generation: In standard noisy label benchmarks, synthetic noise is typically introduced by flipping a clean label to any other available class with uniform probability. However, to account for the realistic scenario of how human annotation errors occur (i.e., being heavily biased toward semantically similar classes), instead of a uniform distribution, we define the lubel-flippig probability using an exponential decay function parameterized by a semantic distance metric, d(y, k). between the true label y and candidate noisy label k. The definition of the semantic distance d(y, k)) depends on the inherent structure of the dataset's 's label space:
Sentiment Datasets (SST-5, Yelp): For datasets with ordinal scale (e.g., highly negative to highly positive), the distance defined as the absolute difference between their discrete scalar values. For example, 5-point scale, the distance between 1-star review (y 1) and 2-star review (k: 2) is cd(1,2) 2) = 1, whereas the distance to a 5-star review is d(1,5) = 4.
Emotion Datasets (Empathetic Dialogues, Gołmotions, CancerEMOO): For emotion datasets, map the target classes onto Plutchik's 's wheel of emotions. We define the semantic distance d(y, kt) as the shortest path distance (i.e.. the number of adjacent steps) between emotion y and emotion & along the wheel. For instance, the distance between Joy and Trust is I, while the distance between Joy and its polar opposite, Sadness, is 4. For secondary emotions larger taxonomies (e.g., GoEmotions), distances are computed based on their primary emotion roots.
Probability Formulation: Given target overall noise rate p 1) (c.g., p = 0.2 in our primary experiments), the probability of an instance retaining its true label remains P(ğ v) p.
For any incorrect candidate class i * y, the probability of the label being flipped by normalizing the exponential decay of the semantic distance across all possible incorrect classes:
calculated
exp(-y d(y, k))
P(ğ k = p. Ejjy exp( -y d(y.j))
(10)
19

---

Published as a conference paper at ICLR 2026
where y > 0 is scaling parameter that controls the concentration of the semantic bias. A higher restricts the injected noise almost exclusively to the most immediate neighboring classes, simulating highly nuanced human annotator ambiguity. In experiments, set y 1.0 ensure strong bias toward semantic neighbors while still allowing a non-zero probability for distant classes.
H NOISE CORRECTION COMPARISON
Furthermore, we compare LANE against recent advancements in noisy label correction, specifically the interpolation-basee approach by Xu et (2025). As shown in Table [1 LANE outperforms this recent state-of-the-art method on 4 out of 5 evaluated datasets, demonstrating its strong capability in mitigating the effects of label noise.
Method
Emph
GoEmo
RCV1
SST-5
Yelp
LANE
60.8
66.5
59.3
58.9
69.2
et al. 2025
60.2
66.4
58.7
59.2
68.5
Table 11: Performance comparison of LANE against the recent approach by Xu et al.
2025)
20


Reviews:
[{"review_id": "3VE2SUu3C1", "reviewer": "Reviewer_xwV9", "summary": "This paper proposes Label-Aware Noise Elimination (LANE), a method that introduces a novel metric called label-aware margin. By incorporating inter-class semantic similarities and model training dynamics, LANE quantifies the label noise level of each training sample. It dynamically down-weights suspected noisy samples, thereby mitigating the harmful effects of noisy labels while retaining all training samples (including hard but clean ones). Experiments across ten text classification datasets demonstrate LANE's effectiveness under varying noise levels.", "questions": "1. The experiments use BERT as the base model. Is LANE compatible with other architectures and large language models?\n2. Why wasn't PLF compared in the 20% noise setting experiments?\n3. The complexity of this work is not low. Why not include a main framework diagram? This would make the overall workflow and contributions of the work clearer.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clear Motivation: The paper effectively identifies two key limitations in existing methods: (1) valuable samples below a low AUM threshold are unnecessarily removed from the training set, and (2) AUM computation treats labels independently, ignoring semantic similarities between them. The authors propose LANE to address these specific shortcomings.\n2. Clear Methodology: The theoretical analysis and step-by-step description of the proposed method are clear and well-presented.\n3. Extensive Experiments: The comparative experiments are comprehensive, covering numerous datasets spanning most text classification tasks. The proposed LANE method achieves near-universal improvements over baselines.", "weaknesses": "1. Several works on noisy label learning emerged in 2025. As a submission to a 2026 conference, the lack of comparison with new methods such as [1], [2], and [3] detracts from the overall novelty of the work.\n2. The approach requires training two BERT models for text classification, which is inefficient. Furthermore, while LLM-related experiments are included, the LLMs rely solely on context. The work could be strengthened by incorporating parameter-efficient fine-tuning techniques like LoRA or integrating noisy label learning methods specifically designed for LLMs, such as [3], to enhance persuasiveness.\n3. The Related Work section is disorganized. Noisy label learning encompasses various approaches, but the authors resort to simple listing. This issue is also reflected in the main experiments (Table 2), which lack intuitive grouping. It is recommended to group and present comparisons by methodology type.\n4. This paper lacks the reproducibility statement required by ICLR.\n\nReferences:\n\n[1] Pan et al., Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data, AAAI, 2025.\n\n[2] Xu et al., Revisiting Interpolation for Noisy Label Correction, AAAI 2025\n\n[3] Ye et al., Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement, SIGKDD 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KYZkgYowl9", "reviewer": "Reviewer_6TRi", "summary": "This paper introduces Label-Aware Noise Elimination (LANE), a novel approach to enhance deep learning model robustness against label noise in fine-grained text classification—a task where classes are semantically similar (e.g., distinguishing between subtypes of news articles or product reviews), making label noise particularly detrimental and common. LANE’s core design integrates two complementary signals to identify and downweight noisy training examples: (1) semantic relations between classes and (2) model training dynamics .", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Fine-grained text classification is ubiquitous in real-world applications (e.g., e-commerce product categorization) but uniquely vulnerable to label noise—human annotators often confuse semantically similar classes.\n2. LANE’s fusion of dynamic training dynamics (capturing how the model learns an example over time) and static class semantics (capturing inherent class ambiguities) is innovative.\n3.  The reported F1 improvements (2.4–4.5%) are meaningful for fine-grained tasks", "weaknesses": "1. it is not clear how the supervised contrastive loss helps in Eq (8)? Is there any intuitive illustration?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BJJVeeqQEc", "reviewer": "Reviewer_5euU", "summary": "LANE provides a method for learning with label noise. The authors suggest that label noise could arise from error-prone labelling processes or just genuinely ambiguous labels and they must be weighed differently. The key idea is to assign high weights to hard but clean examples that have high semantic similarity of class labels and down-weigh truly mislabelled examples.", "questions": "* Have the authors considered the case of class imbalance? \n* How much compute power is required to learn the weights?\n* Could the authors provide analysis of the learned weights in relation to semantic similarity?\n* How would LANE perform for increased label ambiguity and not just random noise?\n* Edit suggestions for better readability\n1. Algorithm 1 - lines 170-173 could be written more legibly \n2. Font too small for all tables\n3. Line 245 typo ‘mislabeled’", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Addresses an important and novel aspect of handling noisy labels - distinguishing between ambiguous or similar labels and erroneous ones\n* The method identifies and retains hard but clean examples with higher weight, taking into consideration the semantic similarity of label names\n* Extensive empirical analysis has been done comparing LANE with other existing methods", "weaknesses": "* Requires additional network training to learn the weights - a computational overhead\n* The label-aware supervise contrastive loss could be explained a bit more intuitively, right now, it requires readers to have a strong prior understanding of contrastive learning\n* The method is motivated to handle ambiguous or semantically close label noise, but the experiments only consider 20% random noise settings. \n* L95 to L99: piling up such a larger number of references without explaining the individual contributions is not helpful at all", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LkY8KBFjPQ", "reviewer": "Reviewer_B3Cs", "summary": "This paper introduces LANE, a training framework designed to enhance model robustness under noisy labels. LANE dynamically adjusts the contribution of training examples by reducing the weight of samples likely to be mislabeled. To identify such samples, it leverages training and semantic relationships. Extensive experiments across multiple fine-grained text classification tasks demonstrate consistent gains, showcasing LANE’s effectiveness in handling label noise.", "questions": "See weakness above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The reviewer notes the following strengths:\n- The paper clearly articulates the limitations of prior methods and justifies the need for LANE.\n- The proposed LANE methodology is intuitive and technically sound.\n- The authors provide extensive experiments on multipole fine-grained text classification datasets that demonstrate consistent and meaningful improvements over multiple baselines.\n- The authors also includes a large set of ablations alongside thoughtful exploration of weight distributions that help provide insight into the underlying methodology.", "weaknesses": "From the reviewer's perspective, LANE relies heavily on the foundational capabilities of the underlying model. Therefore additional evaluations on language models beyond BERT would add to the real-world applicability of LANE. But in general, the reviewer finds the overall methodology to be sane and showcase valuable improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)