Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance
Authors: Song Wang
Zhen Tan
Ruocheng Guo
Jundong Li
Venue: EMNLP 2023 Findings (2023)
Keywords: Pretrained Language Models
Large-scale Language Models
Learning from Noisy Labels

Abstract: Adopting a two-stage paradigm of pretraining followed by fine-tuning, Pretrained Language Models (PLMs) have achieved substantial advancements in the field of natural language processing. However, in real-world scenarios, data labels are often noisy due to the complex annotation process, making it essential to develop strategies for fine-tuning PLMs with such noisy labels. To this end, we introduce an innovative approach for fine-tuning PLMs using noisy labels, which incorporates the guidance of Large Language Models (LLMs) like ChatGPT. This guidance assists in accurately distinguishing between clean and noisy samples and provides supplementary information beyond the noisy labels, thereby boosting the learning process during fine-tuning PLMs. Extensive experiments on synthetic and real-world noisy datasets further demonstrate the superior advantages of our framework over the state-of-the-art baselines.

Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance
Song Wang University of Virginia sw3wvvvrrginia. edu
Zhen Tan
Arizona State University
ztan36easu. edu
Ruocheng Guo
Jundong Li University of Virginia jundong@virginia.. edu
ByteDance Research ruocheng, wuwegggeeggooyyyyeeeeoooo com
Abstract
0.8
Adopting two-stage paradigm of pretraining followed by fine-tuning, Pretrained Language Models (PLMs) have achieved substantial ad- vancements in the field natural language pro- cessing. However, in real-world scenarios, data labels are often noisy due to the complex anno- tation process, making it essential develop strategies for fine-tuning PLMs with such noisy labels. this end, we introduce an innovative approach for fine-tuning PLMs using noisy la- bels, which incorporates the guidance of Large Language Models (LL.Ms) like ChatGPT. This guidance assists in accurately distinguishing between clean and noisy samples and provides supplementary information beyond the noisy la- bels, thereby boosting the learning process dur- ing fine-tuning PLMs. Extensive experiments synthetic and real-world noisy datasets fur- ther demonstrate the superiority of our frame- work over the state-of-the-as baselines.
Confidence
0.6
0.4
Clean-20Ng Noisy 20Ng
Clan-AGNews
0.2
Noiey-AGNews
5 Epoch
8
Figure 1: The average confidence of clean and noisy samples during fine-tuning BERT on datasets 20Ng and AGNews with 20% noisy labels.
Prior research (Arpit et al., 2017; Cheng et al., 2021; Zhang et al., 2021c; Wang et al., 2023a) has shown that PLMs are prone to overfitting and gener- ally deliver subpar performance when fine-tuned on datasets containing label noise. Since fine-tuning involves incorporating supervision information enhance performance on downstream tasks, the presence of noisy labels will mislead the training process and significantly impede the efficacy of PLMs (Zhu et al., 2022). Therefore, there is an immediate need to design effective algorithms for fine-tuning PL.Ms in the presence of noisy labels.
1 Introduction
In recent years, the development of language mod- cls has significantly expanded the applications within the field of natural language processing (NLP). Fine-tuning Pretrained Language Models (PLMs) like BERT (Devlin et al., 201 8) for specific downstream tasks has become an essential step in weal-world implementations (Alt et al., 2020; Wang et al., 2021). In general, achieving signifi- cant performance gains fine-tuning PLMs neces- sitates the availability of task-specific data for fine- tuning (Zhou and Chen, 2021; Wang et al., 2022). However, obtaining high-quality labeled datasets for this purpose poses significant challenges due to the expensive, complex, and labor-intensive na- ture of the annotation process (Yu al., 2019; Bac al., 2022). For example, large-scale datasets, often derived from web-crawling (Li et al., 2017; Song et al., 2019) or nowddsourcing (Yan al., 2014; Williams et al., 2017: Sakaguchi et al., 2021), 1), frequently suffer from the presence of noisy labels.
In the context of learning from noisy labels, an intuitive approach is separate clean samples from the noisy ones in the training set for model train- ing (Han et al., 2018). For the remaining noisy samples, prevalent strategy is to pseudo-label them based on model predictions to reduce the ad- verse impact of noise (Berthelot et al., 2019; Sohn et al., 2020). However, it remains challenging when applying this paradigm to PLMs. This is because PLMs, equipped with prior knowledge encoded in large size of parameters, tend easily memo- rize noisy samples early on during the fine-tuning process. As illustrated in Figure 1, PL.Ms exhibit similar prediction confidences for both clean and noisy samples, which will result in two challenges

---

when utilizing existing methods: (1) Existing ap- proaches often rely on confidences generated by models trained on potentially noisy samples (Li at., 2020; Karim et al., 2022). However, as shown in Figure 1, it is challenging to accurately distin- guish clean and noisy samples solely based on con- fidences. (2) Existing methods are susceptible to erroncous information during pseudo-labeling, as they directly utilize model predictions as pseudo- labels, which can be detrimental when the predic- tions are inaccurate. PLMs, being capable of easily remembering noisy samples, further exacerbate the risk of capturing and amplifying erroneous infor- mation during pseudo-labeling, For example, in the 20Ng dataset (Lang, 1995), when an "autos" sam- ple is assigned a wrong label "hardware". PLMs easily memorize the erroneous information and thus cannot infer the correct labels in predictions, resulting in incorrect pseudo-labeling.
We conduct extensive experiments on synthetic and real-world noisy datasets and demonstrate the superiority our framework in fine-tuning PLMs with noisy labels.
2 Related Work
2.1 Learning from Noisy Labels
Various strategies for learning from noisy labels have been proposed, falling primarily into three categories. The first category is Sample Selection methods, which typically employ loss or confi- dences to identify reliable samples for model op- timization. These methods generally necessitate a predefined threshold (Li et al., 2021; Karim et al., 2022) or prior knowledge concerning the noise label rate (Han et al., 2018; Yu et al., 2019) to choose the instances. The second category, dubbed Label Transition methods, aims to learn (Tanaka et al., 2018) or generate pseudo-labels (Zhang al.. 2021c; Sohn et al., 2020) to replace the original noisy labels. Lastly, Regularization methods de- sign robust loss functions (Liu et al., 2020; Eng- lesson and Azizpour, 2021) or regularization tech- niques (Zhang and Sabuncu, 2018) that can effee- tively utilize all samples to enhance the model ro- bustness against label noise. Nevertheless, these methods generally do not consider the scenario of fine-tuning pretrained model with noisy labels.
To overcome these challenges, we propose a novel framework, named LAFT, that harnesses the power of Large LAnguage Models (LLMs) for Fine-Tuning PLMs. We leverage LLMs trained on extensive corpora and fine-tuned with human instructions, such GPT4 (OpenAI, 2023), to ob- tain external guidance in the form of confidence values. Our approach addresses the first challenge of separating reliable samples by utilizing L.MM- generated confidences for each class of all training samples. By comparing these confidences with the assigned labels (i.e., the given noisy labels) of train- ing samples, we categorize them into three disjoint subsets: the Easy Clean (EC) Set, the Hard Clean (HC) Set, and the True Noisy (TN) Set. Regarding the second challenge, we propose novel method incorporate LLM-generated confidence scores as robust supervision information for all samples to ensure that PLMs learn useful information from them. As LL.--generted confidences are not af- fected by label noise, they can provide potentially relevant labels that are useful even if they are not entirely accurate. In summary, our contributions are follows:
2.2 Confidence-Guided Sample Separation
To separate noisy samples, existing works lever- age the loss or confidences that provide insights into the model's prediction behavior as training proceeds (Yu et al., 2019; Karim et al., 2022). In the context of learning from noisy labels, the key concept is to leverage these dynamics as criteria for identifying and separating noisy samples. Several works propose identify samples with lower train- ing loss as the clean subset (Li et al., 2020; Han et al., 2018; Jiang et al., 2018: Zhao et al., 2022; Wang et al., 2023b), however, they are generally simplistic and inflexible, resulting in the selection of only easy samples. To address this limitation, alternative approaches have been proposed to effec- tively utilize the loss confidences during training, as demonstrated in (Zhang et al., 2021a) and (Nishi et al., 2021). In contrast existing methods that only rely on confidences generated by PLMs, we leverage LLMs as external guidance that provides more precise separations for fine-tuning PLMs.
We are the first to explore the potential of lever- aging supervision information (i.e., confidence scores) generated by LLMs to tackle the noisy label problem in fine-tuning PLMs.
We propose novel framework LAFT that can effectively separate clean and noisy samples and leamn from noisy labels, based on the LLMs- generated confidences.

---

3
Problem Definition
Noisy Labeled Data
this work, we study the problem of fine-tuning PLMs for text classification with noisy labels. For- mally, consider a noisy training dataset contain- ing samples: Drr = {(1 D:), i 1,2...., n.. where 2; is an input sample and V1 denotes the assigned label of z;. Note that T/ is potentially comuppedd, and we denote the true label of x; as Vi. which is inaccessible during fine-tuning. Specifi- cally, we aim to fine-tune PLMs with samples in Der to achieve satisfactory prediction performance on test samples, while utilizing LLMs as external guidance. Notably, the LLMs remain fixed in our framework, which means can also use black- box LLMs. In practice, we implement PLMs for text classification by attaching an additional classi- fier, which will take the output of the PLM as input and generate class probabilitics for classification.
LLM Label y Assigned Label yi
Coarse-graing Separation
Lg Easy Clean Set
Disagreed
LLM Confidence p
Fine- geaiggrained Separation
PI.M Confidence p
Hard Clean Set
True Noisy Set
Lgg
The detailed process of our framework LAFT. We perform two steps of separation to divide all training samples into three subsets with different degrees of label noise: Easy Clean Set E, Hard Clean Set H. and True Noisy Set M. We further propose three different losses to effectively learn from them: Le, Lx. and Lx-
Figure 2:
4 Methodology
To deal with this issue, we propose the utilization of external guidance, in the form of confidences generated by LLMs. Before delving into the specifics of our framework, we provide a formal definition of confidences. Denote the output of the final layer (i.e., the classifier) in PLMs for sample x; as z(x;) E R.,, where N is the number of classes. The confidence of x, for the j-th class Cj be represented as follows:
shown in Fig. 1.
The overall framework of LAFT is illustrated in Fig. 2. In partieular, we propose to divide all train- ing samples into three subsets based on the accor- dance among --M-geenrrttdd confidences, PLM- generated confidences, and the assigned labels of samples. In particular, we query an LLM to pro- vide confidences for each training sample, spanning all classes. Combining confidences obtained from both LLMs and PLMs, we perform two steps of sep- aration to segregate the entire training set into three subsets: Easy Clean (EC) Set, Hard Clean (HC) Set, and True Noisy (TN) Set. Each of these sub- sets, displaying unique behaviors, is then subjected to specific noise-robust fine-tuning strategies.
exp (2(cçaa)))
plegi 24)
(I)
Ek= exp (z(cH)=4))
where z(cy; E R is the j-th value in zxx)). No- tably, the confidences are obtained from z(x;) after a softmax function and thus sum up to one.
4.1 Confidences
Although LLMM-generatd confidences can pro- vide external guidance, they are not completely accurate. Thus, We conduct two-step sample sep- aration process based on both LLM-generated and PLM-ggeerated confidences, with the second step providing a more granular distinction.
Existing studies establish correlation between confidences and the degree which deep models memorize specific samples during training (Arpit al., 2017; Karim al., 2022). It has been ob- served that as the model's memorization of a partic- ular sample strengthens, the model tends to assign higher confidence for this sample (Li et al., 2023). Therefore, these methods generally employ confi- dences to distinguish between clean and noisy sam- ples based on the assumption that the model cannot easily memorize noisy samples (Pleiss al., 2020; Swayamdipta et al., 2020). However, applying these strategies to fine-tuning PLMM suboptimal, as PLMS also present high confidence for noisy samples even in the early stage of fine-tuning, as
4.2 Coarse-Grained Separation
For the first step of separation, we aim to select sam- ples that are easy to be identified as clean data with guidance from LLMs. Thus, we perform Coarse- grained Separation, utilizing confidences gener- ated by LLMs with the raw text data included in the prompt. Here we provide an example of the prompt for querying the LLM to obtain the confi- dence value for each class:

---

Classify the following content: {imput teat)- Select the label from {Class 1}. (Class 2), .... (Class N) and output a confidence value for each of them.
be categorized into D. Therefore, although we can learn from samples in & directly with their LLM- generated label y/s it is still challenging to learn from samples in D, which are only partially clean. Therefore, we further propose to separate the sam- ples in the disagreed set 'D into two subsets: the Hard Clean (HC) Set H. and the True Noisy (TN) Set N, referred to fine-grained separation.
We denote the LLM-ggnerated confidence for sam- ple I; regarding class Cj as pley; zi)), where oj E C {c1, C2,.... ........c.............. the label obtained by LLM can represented as
) .,,...,,c,,, and the class set. Then
The intuition is that within the disagreed set D, the LLM -.g-enerrteed labels can be incorrect for spe- cific hard samples with correct assigned labels, re- ferred to as Hard Clean (HC) samples. Specifically, the ideal separation for 'D is as follows:
= argmax x.). j=1.2.....N
(2)
To perform coarse-grained separation, we first pro- vide an assumption as the justification:
7 D n (xg T. v4),
(5)
Assumption 1. Samples whose assigned labels are the same ALM-generaae labels (i.e., Di) can be considered almost clean.
N = D n (Z Mi + v.)
where y; is the true label of x4. Note that although this ideal separation can be completely precise for separating noisy samples in D, it is infeasible in practice, as the true labels are unknown. Therefore, to precisely separate the true noisy samples in DD we propose two thresholds for LLM-generated and PLM PT--eeeeerteed confidences, respectively.
Note that Assumption I is empirically verified in Table 5 Sec. 5.5. The underlying intuition is that the probability of an LLM-generated label being identical to the assigned label and inconsis- tent with the ground-truth label is significantly low and thus negligible. In concrete, this assumption allows to segregate the training samples into two distinct subsets based on the concordance between the LLM-ggnerated label yJ and the assigned (po- tentially noisy) label De z;, More specifically. we define the resulting two subsets, the Easy Clean (EC) Set & and the Disagreed Set D, as follows:
In order to achieve more robust LLM-generated confidences distributed over the label space C, we adopt M different augmentations for each input sample to encourage input diversity while keeping the semantics immutable. Denote the augmented samples of x; as um((z;) where m = 2,,,.... M,, and M is the number of augmentations. We can obtain the M LLM-generated confidences P(cy Um (z:) for regarding the j-th class ej. where 1,2, ..., M and j 1, .,, ,....,,N.. N. We aggregate the L.L--generated confidences via the M augmentations follows:
E = {a,, 0. J. D {at, T. + ū.).
It is naturally satisfied that EUD Dr and E D 0, where D,, is the training set. Since the samples in E are already clean, we can directly fine- tune PLMs using LLM-generated labels Di based on the cross-entropy loss as follows:
M
P.(cH = M
Σ (m)))) m=1
(6)
E
N
LE =
Mis log p(c;;a));
(4)
3
where Fm (x;) is the input example after applying the m-th augmentation. As LLM-generated confi- dences remain fixed during fine-tuning PLMs, we adopt a fixed threshold for fine-grained separa- tion based on Pa (cj:z;)
where (1)y:)) is the PLM generatee confidence for z; regarding the j-th class Cj. Here ūij if
π Cy. and J. = 0, otherwise.
On the other hand, however, PL.Ms-enerre confidences for cach sample will change the fine- tuning proceeds, which results in subpar separation performance if we adopt fixed threshold select high- high-confidence samples as clean ones. Intuitively, the confidences should be lower for HC samples at the beginning of fine-tuning men-ttning as the model cannot
4.3 Fine-Grained Separation
It is noteworthy that, however, the samples in 'D are not completely noisy. This because the LLM-ggnnrated labels are not perfectly correct, as shown in Table 5. Thus, the samples that are clean but incorrectly classified by LLMs will still

---

easily fit these hard samples within several epochs. Nonetheless, the noisy samples are relatively casier to achieve higher confidence, and thus their confi- dences will easily achieve higher at the beginning. Therefore, we propose an adaptive threshold F(t) that will increase as fine-tuning proceeds while not reaching an excessively high value:
weight adjustment that increases when the sum of LLM confidence Pa(c;; and PLM confidence p(cy m;) is larger. Intuitively, if the LLM and PLM are with high confidence regarding a specific class, then the information in their confidence can be use- ful as they are more likely to be correct. Therefore, we define dig as follows:
F(I) = F - exp(-Xt),
Dis = max (Pa xx) + p(eci - -a)(t,,),
(11)
where À and F are hyper-parameters that control the value of threshold ftt) as fine-tuning proceeds. denotes the current number of fine-tuning epochs. Combining the two thresholds, we can perform fine- aue-ggained separation by selecting the HC set H as follows:
where a > 1 is a hyper-parameter that controls the threshold (+ for Φi.- As such information can still be inaccurate, we subtract it by the threshold (+) to control its magnitude, such that Φi,) also acts as an adaptive loss weight for these samples.
4.5
Learning from the True Noisy (TN) Set
H- = Dn fa;, Vel ma {}
(8)
After the fine-grained separation, most samples the True Noisy (TN) Set should be identified as noisy ones. Nevertheless, it is still challenging for PLMs to use their output as pseudo-labels for fine-tuning, the prediction errors will accumu- late and affect subsequent pscudo- labeling results. Fortunately, the confidences generated by LLMs can provide additional guidance to identify the po- tential labels for samples in the TN set. We first provide Remark to justify the effectiveness of using LLM-generated labels for optimization.
n frg Vil max ((; < 7(t)),
eEC
where i and F(t) are the thresholds for LLM- generated confidences (Pale):;)) and PLMs- generated confidences (f(c;; æ;))), respectively. In this manner, the process of separating HC and TN samples, i.e., fine- grained separation, can benefit from both the LLMs and PLMs. Then the remain- ing samples are categorized into the True Noisy (TN) set N:
= = D (H. H.
(9)
Remark 1. LLM- -LM-generated labels on the True Noisy Set preserve the same accuracy as that on the whole dataset, as LLMs are not affected by noise.
4.4 Learning from the Hard Clean (HC) Set
Now we have divided the disagreed set 'D into H W. Recall that ideally, samples in Ht are hard correct. Thus, for these samples, we can directly utilize their assigned labels ÿ as training labels. Nevertheless, since the fine-graine -raiind separation of H and A cannot perfect, the samples in H may still be noisy. As both LLMs and PLMs fail to provide a confident prediction for samples in H. we propose to prioritize the assigned label while also incorporating additional information from LLMS and PLMs. Specifically, we employ weighted loss based on cross-entropy. The weight is enlarged if the summed confidence of the LLM and the PLM is high. We define the loss as follows:
Remark 1, as empirically verified in Sec. 5.5, demonstrates that even when we categorize most noisy samples into True Noisy Set. the LLM- generated labels can still provide decent guidance without sacrificing accuracy. Given the confidences provided by LLMs, we employ a loss that can en- able the PLMs to benefit from them. Specifically,
W
N
1
CN
Palcn x;) log Pley;
W
W
1 W
max ((c; log max ((c,,,)) 8(x))
CEC
aEC
(12)
M
N
L#
1
(Jh. + (ir)) log a(eg
Here in the first term, we leverage the confidences generated by LLMs learn from potentially cor- rect labels. This is because although LLMS can- not completely predict the correct labels, the con- fidences still preserve the potentially useful infor- mation in other incorrect but relevant labels. Such
H
(10)
where p(cy: m44 the PLM -LM--geerated confidence for Di regarding the j-th class cj. Here Ti,j = 1
if T = Cj. and Ui = 0. otherwise. Oijj acts as a

---

benefits cannot be provided by pseudo-labeling. which tends to output a definitive label. For the second term, we utilize the model predictions to exploit useful information from PLMs. The intu- ition is that, with the relevant label information provided by LLMs, the PLMs can learn accurate label information from the noisy samples in TN set. Consequently, if the model output tends to be confident specific samples, we utilize the prediction to further enhance the learning from Thus, we further set threshold for this term, i.e., 8(z)). defined as follows:
Hausa (Hedderich et al., 2020). More details about these datasets are provided in Appendix B.
Baselines. We compare our framework to state- of-the-art baselines for learning from noisy labels. In particular, we compare to (1) Base (Devlin et al., 2018) that performs fine-tuning with standard cross-entropy loss; (2) Regularization Methods: Mixup (Zhang et al., 2018) and GCE (Zhang and
Sabuncu, 2018); (3) Sample-selection Methods:
Co-teaching (Han et al., 2018), Co-teaching+ (Yu et al., 2019), JoCoR (Wei et al., 2020), CR (Zhou and Chen. 2021), and NPC (Bae et al., 2022). Ad- ditional details are provided in Appendix C.
1, if max ((G.)) (7(), aEC
Implementation Details. We use BERT (Devlin et al., 2018) as the text encoder for all datasets except Hausa, for which we mBERT. The clas- sifier is implemented and randomly initialized the beginning, while both the encoder and the classifier will be updated via gradient descent during fine-tuning. All ex- periments are evaluated on a clean test set, and the average accuracy along with the standard de- viation over ten runs is reported for each dataset. We provide more details about the implementa- tion in Appendix D, and our code is provided at http://github..om/SSnr W.SWW/FF.................
8(z;) (o, otherwise,
(13)
where F(t) is computed by Eq. (7). To reduce the effect of confirmation bias, multiply F(t) by 9, a pyper-paramcter that controls the final adaptive threshold, i.e., (7(().
fully-connected layer
4.6 Fine-tuning Objective
After we separate all training samples into E, H. and N, we can combine the three individual losses for them for PLM fine-tuning. Our final fine-tuning objective can be represented as follows:
L -- L- LE + AuCH + ANCAN
(14)
5.2 Comparison on Synthetic Datasets
where d and Aw are hyper-parameters that con- trol the importance of L. and LN. respectively.
In this subsection, compare framework with other baselines on synthetic datasets 20Ng and AGNews, considering different noise types and ratios. Specifically, for Symmetric Noise (SN), we conduct experiments on three different noise rates: 20%, 40%, and 60%. For the other two types of noise, i.c., Asymmetric Noise (AN) and Instance- Dependent Noise (IDN), we adopt two noise rates: 20% and 10%. We present the results in Table 1, 2. and 3. The key observations drawn from the out- comes are follows: (1) Regardless of the noise type, our LAFT framework persistently surpasses other state-of-the-arr baselines, thus showcasing its effectiveness in fine-tuning PLMs with noisy labels. (2) LAFT's performance improvement over other baselines is slightly more pronounced on the 20Ng dataset compared to AGNews. This is attributable to 20Ng containing greater num- ber of classes (N 20) as opposed to AGNews (N = 4). Consequently, our strategy of utilizing LLM L..M-generate confidences can capitalize on the similar labels within 20Ng for PLM fine-tuning, even if LLM predictions are not entirely accurate.
5 Experiments
5.1 Experimental Settings
Datasets. To evaluate the performance of our framework, we first conduct experiments on two synthetic-noise datasets: 20Ng (Lang, 1995) and AGNews (Li and Roth, 2002: Zhang et al., 2015). Following existing works on learning from noisy labels (Patrini et al., 2017: Yao et al., 2020; Zhuang al., 2023), we adopt three types of synthetic la- bel noise: (1) Symmetric Noise (SN) uniformly changes labels to other classes (Han et al., 2018; Xia et al., 2021). (2) Asymmetric Noise (ASN) changes labels to other similar classes (Tanaka et al., 2018; Bae et al., 2022). (3) Instance- Dependent Noise (IDN) changes labels based probability in proportion the specific sample fea- tures (Cheng et al., 2021; Yao et al., 2021). More- over, we further conduct experiments on three real- world datasets with noisy labels: SemEval (Zhou et al., 2020), TREC (Awasthi et al., 2020), and

---

Table The overall performance of various models on synthetic noisy datasets 20Ng and AGNews with Symmetric Noise (SN), where accuracy and standard deviation reported in %, and the best results are in bold.
Dataset
20Ng
AGNews
Ratio
20%
40%
60%
20%
40%
60%
Base
79.15 = 0.29
71.74 + 0.07
61.87 + 1.58
81.77 + 0.33
78.33
0.47
73.07 + 1.49
Mixup
79.10 = 0.21
70.58 + 0.22
62.48 2.04
81.25 + 0.02
78.24 1: 0.77
73.12 + 1.58
GCE
79.56 = 0.39
70.46 0.42
62.61 1..0+1100
82.98 + 0.20
78.46 18.46:100 0.20
72.75 0.27
Co-teaching
76.68 = 0.86
0.62
61.39 3.15
80.99 + 0.88
7.00
2
Co-teaching+
78.78 = 0.96
71.33 2.31
62.90+1.33
80.95 + 0.86
77.98
72.73 + 0.98
JoCoR
80.11 = 0.42
73.52 4 0.88
1.22
83.60 + 0.54
79.45
75.45 1 0.30
CR
81.80 = 0.48
74.16 + 0.13
64.22 + 1.25
84.09 + 0.39
75.06 + 1.01
NPC
80.38 = 0.35
72.93 + 0.23
64.38 + 0.38
84.28 + 0.47
80.26 1.39
73.81 + 0.18
LAFT (Ours)
82.04 0.11
76.93 0.60
70.79 1.98
90.86 + 0.02
88.61 0.25
80.79 + 2.32
Table 2: The overall performance on 20Ng and AGNews Asymmetric Noise (AN).
Table 3: The overall performance on 20Ng and AGNews with Instance- Deannnee Noise (IDN).
Dataset
20Ng
AGNsws
Dataset
20Ng
AGNews
AN Ratio 20%
40%
20%
40%
IDN Ratio
20%
40%
20%
40%
Base
75.34
59.65
82.24
76.35
Base
78.23
70.28
85.40
78.66
Mixup
75.84
62.19
82.12
77.45
Mixup
78.21
70.45
84.71
79.22
GCE
76.41
62.37
83.19
77.
GCE
79.99
71.74
87.11
80.49
Co-teaching
77.49
64.55
84.15
79.16
Co-teaching
77.43
70.76
84.09
78.43
Co-teaching+
77.98
63.4
86.12
81.02
Co-teaching+
79.89
71.52
88.21
82.30
JoCoR
81.27
70.99
87.6 87.63
82.79
JoCoR
81.83
74.04
88.
82.66
CR
81.08
67.22
88.58
83.07
CR
83.04
75.51
90.72
84.38
NPC
79.03
65.54
86.83
80.21
NPC
81.84
74.45
89.90
82.40
LAFT (Ours)
83.70
81.97
91.95
90.12
LAFT (Ours)
83.61
80.49
91.94
89.34
(3) Certain regularization methods, such as Mixup and GCE, exhibit subpar performance when ap- plied to datasets with a higher noise ratio of 60%. This is because these methods rely on all training samples for PLM fine-tuning, making them more susceptible to strong label noises.
lyze the resulting impact performance. We conduct experiments with the following variants: (1) LAFT',C performs coarse-graines separation without LLMs and thus only separates the Easy Clean Set based on PLM predictions. (2) LAFT F performs fine-grained separation without LLMs, which means when separating Hard Clean Set, only PLM-generated confidences are used. (3) LAFT removes proposed loss L. for True Noisy Set and replaces it with pseudo-labeling based on the most confident PLM prediction. From the results presented Figure we observe that LAFT out- performs all variants, which verifies the effective- ness of these designs in LAFT. Specifically, re- moving the learning loss LN leads to significant performance degradation, indicating that such de- sign effectively alleviate the adverse impact of noisy labels. Moreover, without the fine-grained separation strategy, the performance deteriorates rapidly when the noise ratio larger, indicating the importance of fine- fine-grained separation in the presence of higher noise ratios.
5.3 Comparison on Real-world Datasets
In this subsection, conduct experiments on real-world datasets: SemEval (Zhou et al., 2020), TREC' (Awasthi et al., 2020), and Hausa (Hed- derich et al., 2020). From the results Table 4, we observe that LAFT outperforms other baseline methods on all three datasets. Moreover, the results of LAFT are noticeably competitive on TREC and Hausa with relatively higher noise ratios, which further demonstrates the strong robustness and gen- eralization ability of LAFT to label noise.
5.4 Ablation Study
this subsection, we systematically remove spe- cific components from our framework and ana-

---

Table The overall performance various models on three real-world noisy datasets.
Table 5: The -LM-generated prediction accuracy on the training set 20Ng and AGNews in the ideal separation and real separation (shown in gray) during fine-tuning.
Dataset
SemEval
TREC
Hausa
Noise Ratio
16.00%
38.56%
50.37%
Dataset
EC Set
HC Set
TN Set
Overall
Base
70.61
67.42
47.80
99.46%
0%
76.47%
75.34%
20Ng
Mixup
73.41
68.44
47.66
99.46%
3.58%
69.44%
75.34%
GCE
71.91
67.86
48.12
99.03%
0%
79.35%
81.41 4
AGNews
Co-teaching
72.13
68.19
47.56
99.03%
2.05%
76.07%
81.41 4
Co-teaching+
72.24
69.83
48.25
JoCoR
70.11
66.62
46.48
CR
72.94
68.90
48.34
Test Accuracy (%)
NPC
71.22
67.84
47.48
60
LAFT (Ours)
73.56
72.34
51.71
Ours-SN
40
CR-SN
Ours AN
90
LAFTIC LAFTF
LAFT
20
Ours-IDN
LAFT
CR-IDN
Test Accuracy (%)
85
0.1
0.2
0.3
0.4 0.5 0.6 0.7
0.8
0.9
Noise Ratio
80
Figure 4: The results of our framework and the best baseline CR on 20Ng with different noise ratios for three types of noise: SN, AN, and IDN.
75
70
S-0.2
S-0.4
A-0.2 A-044 ID-0.2 ID-0,4
directly used to replace PLMs for the classifica- tion task when the noise ratio not extremely high. Nevertheless, our strategy can effectively leverage the LLM-generated confidences, despite their lack of complete correctness, to enhance the fine-tuning performance of PLMs with noisy labels.
Figure 3: Ablation study of our framework on 20Ng. S-0.2 and S-0.4 (A, ID) refer to the noise ratios of 20% and 40% for SN (AN, or IDN), respectively.
5.5 Evaluation of LLMs--enerated Labels
5.6 Results with Different Noise Ratios
In this subsection, evaluate the effectiveness of LLMs regarding the quality of generated confi- dence, presented in Table 5. Specifically, we report the accuracy of LLM --M-eeeeetted prediction on the training set in 20Ng and AGNews. The statistics are from experiments on 20Ng and AGNews with 20% Symmetric Noise (SN). From the results, observe that: (1) For Easy Clean Set, the LLM- generated labels exhibit remarkably high degree of correctness. This empirical finding provides the justification for Assumption 1, which allows for leveraging LLMs to perform coarse- -aarsneg sepa- ration in our framework. (2) For the TN set, LLM- generated labels are not entirely correct, while still preserving similar accuracy compared to that on all samples. This result empirically verifies Re- mark and justifies our strategy of utilizing LLM- generated confidences for learning from the TN set, (3) Considering the overall result, L.M--generted labels generally exhibit lower accuracy than PLM- based baselines. That being said, LLMs cannot be
Figure 4 presents the evaluation of our proposed framework across a spectrum of noise ratios on 20Ng, ranging from 0.1 to 0.9, while considering three distinct types of noise: SN, AN, and IDN, From the results, several significant observations are discovered: (1) With increasing noise ratios, all bascline methods consistently show detcrioration in performance. This decline can be tied to the amplified difficulty of label noise during PLM fine- tuning. (2) Despite the escalating noise ratios, framework maintains relatively more robust per- formance. This is attributed to our sample separa- tion strategy which adeptly discerns clean samples within the training set, thus alleviating the adverse effects of label noise. (3) Our framework demon- strates a slower performance decrease with AN noise compared to other types of noise. This can be credited to the adaptive nature of our approach that employs L.MM-generated confidences to effectively learn from similar labels for each sample.

---

6 Conclusion
the effectiveness of our approach in more domain- specific contexts where the nature of the noise could be different. (5) Computational Costs: Fi- nally, our approach entails using LLMs which can be computationally expensive and could thus pose challenge when applied to large datasets or in resource-constraing environments. In summary, future research could focus overcoming these limitations and exploring the adaptability of our proposed framework to other noise types, higher noise ratios, more complex noise patterns, as well as different task domains.
This paper delves into the issue of fine-tuning Pre- trained Language Models (PLMs) with noisy labels for text classification tasks. We address the detri- mental effects of label noise by harnessing external guidance from Large Language Models (LLMs) in the form of confidences. Our approach entails two-step separation strategy that accurately segre- gates all training samples into three distinct subsets, each treated with innovative noise-resilient strate- gies. In summary, our framework adeptly fine- tunes PLMs in the face of noisy samples, requiring minimal external guidance from LLMs.
Ethics Statement
Acknowledgements
This research adheres to the principles of ethical conduct in artificial intelligence research. The de- velopment and utilization of our work are carried out with full recognition of the potential implica- tions. While our method improves the performance of Pretrained Language Models (PLMs) with noisy labels in text classification tasks, acknowledge that it could be potentially used in contexts that may involve ethical concerns, such disinforma- tion or manipulation of opinion on social platforms. Our experiments are performed on five publicly available datasets, namely 20Ng, AGNews, Se- mEval, TREC, and Hausa. The datasets are already anonymized and do not contain any personally iden- tifiable information. Moreover, we have complied with all guidelines and standards for the of these datasets. Furthermore, the use of Large Lan- guage Models (LLMMs) in our work is subject to the terms and conditions put forth by the creators and hosts these models. We only emply them for the purpose of enbancing the fine-tuning performance of PLMS and do not exploit them for any inappro- priate or unauthorized purposes. While our work focuses improving the robustness of PLMs label noise in data, recognize the broader soci- etal concerns related to the potential misuse of our framework. We advocate for the responsible use of our research findings and encourage continued dis- course around the ethical use of machine learning and artificial intelligence technologies in real-world applications. Finally, it is crucial to note that the development of technologies and methodologies in machine learning should be done in conjunction with ongoing considerations about their ethical im- plications, potential misuse, and societal impacts. our responsibility to foster an environment of ethical vigilance AI research and development.
Song Wang and Jundong Li are supported by the National Science Foundation under grants (ISS-2066844, IIS-2144209, IIS-2223769, CNS2154962, and BCS-2228534), the Common- wealth Cyber Initiative awards (VV-1Q23-007, HV- 2Q23-003, and VV-1Q24-011), the JP Morgan Chase Faculty Research Award, the Cisco Faculty Research Award, the Jefferson Lab subcontract, and the UVA 4-VA collaborative research grant.
Limitation
Despite the promising results, our work presents several limitations that we must acknowledge: (1) Dependence on Large Language Models (LLMs)): Our framework leverages the guidance from LLMs in the form of confidences. This implies that the quality of these confidences and hence the effec- tiveness of our model are closely tied to the per- formance of the LLM. Unsatisfactory LLM perfor- mance could thus directly impact our framework's efficacy. (2) Noise Types and Ratios: The experi- ments in this paper mainly focus on three types of noise: Symmetric Noise (SN), Asymmetric Noise (AN), and Instance-Dependen Noise (IDN), with noise ratios up to 60%. However, there may be other noise types or higher noise ratios prac- tice, and it remains to be investigated how well our framework performs under these circumstances. (3) Limitations in Sample Separation Strategies: The efficacy of our framework relies on accurately di- viding the training samples into clean and noisy subsets. If this distinction not clear becomes more complex, our current approach may encounter difficulties. (4) DSmain-Specifie Text Classifica- tion: While the experiments are performed on spe- cific text classification tasks, we do not investigate

---

References
Bohan Li, Yutai Hou, and Wanxiang Che. 2022. Data augmentation approaches in natural language pro- cessing: A survey, Al Open.
Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020. Tacred revisited: A thorough evalua- tion of the tacred relation extraction task. ACL.
Junnan Li, Richard Socher, and Steven CH Hoi. 2020. Dividemix: Learning with noisy labels as semi- supervised learning. In ICLR.
Devansh Arpit, Stanistaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kan- wal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. 2017. A closer look at memo- rization in deep networks. In ICML.
Junnan Li, Caiming Xiong. and Steven CH Hoi. 2021. Learning from noisy data with robust representation learning. In ICCV.
Abhiject Awasthi, Sabyasachi Ghosh, Rasna Goyal, and Sunita Sarawagi. 2020. Learning from rules general- izing labeled exemplars. In ICLR.
Wen Li. Limin Wang. Wei Li, Eirikur Agustsson, and Luc Van Gool. 2017. Webvision database: Visual learning and understanding from web data. arXiv preprint arXiv: 1 700.00262...22
HeeSun Bac, Seungjae Shin, Byeonghu Na. JoonHo Jang. Kyungwoo Song, and I1-Chul Moon. 2022. From noisy prediction to true label: Noisy predic- tion calibration via generative model. In ICML.
Xin Li and Dan Roth. 2002. Learning question classi- fiers. In COLING.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Paperno, Avital Oliver, and Colin A Raf- fel. 2019. Mixmatch: A holistic approach to semi- supervised learning. NeurIPS.
Yifan Li, Hu Han, Shiguang Shan, and Xilin Chen. 2023. Disc: Learning from noisy labels via dynamic instance-specific selection and correction. In CVPR.
Sheng Liu, Jonathan Niles- Weed. Narges Razavian, and Carlos Fernandez-Granda. ularization prevents memorization of noisy labels. NeurIPS.
Hao Cheng. Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. 2021. Learning with instance- dependent label noise: A sample sieve approach. In ICLR.
Granda. 2020. Early-learning reg-
Kento Nishi, Yi Ding, Alex Rich, and Tobias Hollerer. 2021. Augmentation strategies for learning with noisy labels. CVPR.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. Xrv preprint arXiv:1810.04800.
OpenAL. 2023. Gpt-4 technical report.
Erik Englesson and Hossein Azizpour. 2021. Gener- alized shasnn-shanoon divergence loss for learning with noisy labels. NeurIPS.
Giorgio Patrini, Alessandro Rozza, Aditya Kr- ishna Menon, Richard Nock, and Lizhen Qu. 2017. Making deep neural networks robust to label noise: A loss correction approach. In CVPR.
Bo Han, Quamming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels. NeurIPS.
Geoff Pleiss, Tianyi Zhang, Ethan Elenberg, and Kil- ian Q Weinberger. 2020. Identifying mislabeled data using the area under the margin ranking. NeurIPS.
Michael A Hedderich, David Adelani, Dawei Zhu, Jesujoba Alabi, Udia Markus, and Dietrich Klakow. 2020. Transfer learning and distant supervision for multilingual transformer models: A study on african languages. In EMNLP.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavat- ula, and Yejin Choi. 2021. Winogrande: An adver- sarial winograd schema challenge at scale. Commu- nications of the ACM.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. NeurIPS.
Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. 2018. Mentoret: Learning data- driven curriculum for very deep neural networks corrupted labels. In ICMI..
Nazmul Karim, Mamshad Nayeem Rizve, Nazanin Rah- navard, Ajmal Mian, and Mubarak Shah. 2022. Uni- con: Combating label noise through uniform selec- tion and contrastive learning. In CVPR.
Hwanjun Song, Minseok Kim, and Jae- Gil Lee. 2019. Selfie: Refurbishing unclean samples for robust deep learning, In ICML.
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. 2020. Dataset cartography: Mapping and diagnosing datasets with training dynamics. EMNLP.
Ken Lang. 1995. Newsweeder: Learning to filter net- news. In Machine Learning Proceedings.

---

Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2018. Joint optimization frame- work for learning with noisy labels. In CVPR.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In ICLR.
Song Wang, Xingbo Fu, Kaize Ding, Chen Chen, Huiyuan Chen, and Jundong Li. 2023a. Federated few-shot learning. In SIGKDD.
Jieyu Zhang, Yue Yu, Yinghao Li. Yujing Wang, Yam- ing Yang. Mao Yang. and Alexander Ratner. 2021b. WRENCH: A comprehensive benchmark for weak supervision. In NeurIPS Datasets and Benchmarks Track.
Song Wang, Jing Ma, Lu Cheng, and Jundong Li. 2023b. Fair sho- learning with auxiliary sets. In ECAI.
Yaqing Wang. Song Wang, Yanyan Li, and Dejing Dou. 2022. Recognizing medical search query intent by few-shot learning. In SIGIR.
Xiang Zhang. Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. NIPS.
Yaqing Wang. Song Wang, Quanming Yao, and Dejing Dou. 2021. Hierarchical heterogeneous graph rep- resentation learning for short text classification. In EMNLP.
Yivan Zhang, Gang Niu, and Masashi Sugiyama. 2021c, Learning noise transition matrix from only noisy la- bels via total variation regularization. In ICML.
Zhilu Zhang and Mert Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. Neur!PS.
Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. 2020. Combating noisy labels by agreement: A joint training method with co-regularization. In CVPR.
Ganlong Zhao, Guanbin Li, Yipeng Qin, Feng Liu, and
Jason Wei and Kai Zou. 2019. Eda: Easy data augmen- tation techniques for boosting performance on text classification tasks. In EMNLP-JJCNLP.
Yizhou Yu. 2022. Centrality and consistency: two- stage clean samples identification for learning with instance-dependent noisy labels. In ECCV.
Adina Williams, Nikita Nangia, and Samuel R Bow- man. 2017. A broad-coverage challenge corpus for sentence understanding through inference, arXiv preprint arXiv: 704.004426.
Wenxuan Zhou and Muhao Chen, 2021. Learning from noisy labels for entity-centric information extraction. In EMNLP.
Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nan- nan Wang. Zongyuan Ge, and Yi Chang. 2021. Ro- bust learnnne. Hindering the memorization of noisy labels. In ICLR.
Wenxuan Zhou, Hongtao Lin, Bill Yuchen Lin. Ziqi Wang, Junyi Du, Leonardo Neves, and Xiang Ren. 2020. Nero: A neural rule grounding framework for label-efficient relation extraction. The WebConf.
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Lu- ong, and Quoe Le. 2019. Unsupervised data aug- mentation for consistency training. arXiv preprint arXiv: 12848.
Dawei Zhu, Michael Hedderich, Fangzhou Zhai, David Adelani, and Dietrich Klakow. 2022. Is bert robust to label noise? a study on learning with noisy labels in text classification. In Insights from Negative Results in NLP.
Yan Yan, Rómer Rosales, Glenn Fung, Ramanathan Subramanian, and Jennifer Dy. 2014. Learning from multiple annotators with varying expertise. Machine Learning.
Yuchen Zhuang, Yue Yu, Lingkai Kong, Xiang Chen, and Chao Zhang. 2023. Dygen: Learning from noisy labels via dynamics-enhanced generative modeling. arXiv preprint arXiv:2305. 19395.
Yu Yao, Tongliang Liu, Mingming Gong, Bo Han, Gang Niu, and Kun Zhang. 2021. Instance-dependent label- noise learning under structural causal model. In NeurIPS.
A Details of Three Subsets
In this section, we further provide details about the three subsets of samples in our framework.
Yu Yao, Tongliang Liu, Bo Han, Mingming Gong, Jiankang Deng, Gang Niu, and Masashi Sugiyama. 2020. Dual t: Reducing estimation error for transi- tion matrix in label-noise learning. NewrIPS.
1. Easy Clean (EC) Set. this subset, the labels predicted by LLMs are the same as the assigned labels. Therefore, based on Assumption 1, we can infer that the labels of these samples are al- most clean, which allows to directly them for model training. Note that small portion of samples in this category can still be noisy when the LLM predictions happen to be the same as noisy labels. However, we empirically verify in Sec. 5.5 that this proportion is small.
Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Tvor Tsang, and Masashi Sugiyama. 2019. How does disagreement help generalization against label cor- ruption? In ICML.
Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu. Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. 2021a. Flexmatch: Boosting semi- supervised learning with curriculum pseudo labeling. In NewrIPS.

---

2. Hard Clean (HC) Set. In this (ideal) category, the labels predicted by LLMs deviate from the assigned labels, while the true labels are ac- cordance with the assigned labels. In this case, we are acknowledged that these samples are also clean. However, due to the semantic difficulties of these samples, the LLMs cannot easily pre- dict the correct labels of them. Nevertheless, since the assigned labels are correct as true la- bels, we can use assigned labels for model train- ing on these samples. It is noteworthy that in practice, it is infeasible to achieve a perfect sep- aration of this subset. Therefore, in our frame- work, we propose to leverage LLM-generated and - confidences to improve the separation performance.
Table 6: The detailed statistics the five datasets used in experiments.
Dataset
# Training
# Validation
Test
# Class
20Ng
9,051
2,263
7,532
20
AGNews
40,000
7,600
7,600
4
SemEval
1,749
200
692
9
TREC
4,965
500
500
6
Hausa
2,045
290
582
5
Hausa, the second most spoken indigenous lan- guage in Africa, with 40 million native speakers. For this dataset, gazetteers are used for automatic labeling, which results in feature-dependent label noise. Here we provide the detailed statistics of these datasets in Table 6.
3. True Noisy (TN) Set. In this (ideal) category, the labels predicted by LLMs and the true la- bels are both different from the assigned labels. In other words, these samples are all noisy as their assigned labels are different from true la- bels. Since they are noisy, the labels predicted by LLMs are also likely to be different from the assigned labels. As these samples are noisy, we cannot directly use their labels for training. However, since we narrow down the potential range of noisy samples to this category, we can resort specific techniques to learn from these noisy samples. In our framework, we utilize LLM-generated confidences as additional su- pervision information to effectively learn from these noisy samples.
C Baselines
In this section, we provide the details of the base- lines used in our experiments.
Base (Devlin et al., 2018) is the BERT base model fine-tuned based the standard cross- entropy loss. Note that for dataset Hausa, we utilize the multilingual BERT model.
Mixup (Zhang et al., 2018) is a semi-supervised approach that performs a linear interpolation be- tween clean and noisy samples.
GCE (Zhang and Sabuncu, 2018) denotes Gener- alized Cross-Entropy loss, which can be regarded as general loss that combines mean absolute error (MAR) and cross-entropy (CE).
B Datasets
Co-teaching (Han al., 2018) trains two differ- models and selects small-loss samples feed them into each other for optimization.
In this section, we introduce the details of the datasets used in our experiments. In particular, 20Ng (Lang, 1995) and AGNews (Li and Roth, 2002: Zhang al., 2015) are news topic classifi- cation datasets that are prevalently used for text classification tasks. We manually inject different types of noise (SN, AN, and IDN) into these two datasets for evaluation in our experiments. For real-world datasets, we utilize SemEval (Zhou et al., 2020), TREC (Awasthi et al., 2020), and Hausa (Hedderich et al., 2020). Specifically, Se-
Co-teaching+ (Yu et al., 2019) is an updated ver- sion of Co-teaching that explicitly ensures the difference between the two models.
JoCoR (Wei et al., 2020) trains two models and selects samples based on the sum of loss from the two models.
CR (Zhou and Chen, 2021) also trains multiple models with regularization strategy based on soft target.
mEval is relation extraction dataset, and we fol-
low the process introduced in (Zhou al., 2020) obtain noisy labels. TREC a question classifi- cation dataset the weak supervision benchmark WRENCH (Zhang et al., 2021b). Morcover, Hausa is a text classification dataset in the language of
NPC (Bae et al., 2022) utilizes generative model to estimate the transition matrix from noisy predictions to the ground-trute -ooun labels samples and uses it to correct noisy labels.

---

D Implementation Details
this section, we introduce the implementation de- tails for our experiments. We run the experiments on single 48GB Nvidia GeForce RTX A6000 GPU. The experiments are run 10 times to achieve the average accuracy and the standard deviation.
For the augmentations used in our framework, following existing works (Xie et al., 2019; Wei and Zou, 2019; Li et al., 2022), we adopt four typi- cal text augmentations: Back Translation, Random Insertion, Random Deletion, and Random Swap. We set the threshold F for LLM-generated confi- dences as 0.8. We set the hyper-parametie for the adaptive threshold F(t) for PLM-ggenerate confidences 0.7 and 0.8, respectively. We set a as 1.5 and 3 0.9, For the loss weight, set Xx Aw
and
We the model fine-tuning learning rate as 10 with batch size of 32. The maximum length of texts is set as 256. The number training epochs set as 100 with early stopping based on the performance of the validation set. We optimize the model using the Adam (Kingma and Ba, 2015) optimization strategy. For the LLM, utilize ChatGPT based GPT4.
In addition, we provide more detailed package requirements listed below.
Python == 3.9.12
torch = 2.0.1
transformers 4.29.2
huggingface-hub- = 0.15.1
keras 2.12.0
numpy = 1.23.5
scipy 1.5.3
cuda- == 11.6
networkx = 2.5.1
scikit-leam = 0.24.1


Reviews:
[{"review_id": "3ToYuVO0xB", "reviewer": "Reviewer_br2H", "summary": "", "questions": "A: In Figure 4, LAFT with 0.9 ratio noise from AN or IDN obtains nearly 60% accuracy. Why the accuracy under these settings do not have a significant degradation?\n\nB: What happens when the noise ratio becomes 0, but the framework doesn't know this prior ratio while keeping the assumption that there are noises in the dataset?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer br2H,  \n\nThanks for your insightful feedback and suggestions. Here we would like to address your questions as follows:\n\n&nbsp;\n> W1: As shown in the Limitation section, the performance of LAFT is highly correlated to the quality of the LLMs, which limits the framework's potential.\n\nA1: Thank you for emphasizing the relationship between the performance of LAFT and the quality of LLMs, as highlighted in our Limitation section. While we acknowledge this dependence, we also view it as a **potential strength**. As our framework LAFT is designed to flexibly leverage various (black box) LLMs, higher-quality LLMs can provide more comprehensive knowledge and thus further promote performance, thus an advantage of our framework. There also exists a considerable number of works that harness the power of LLMs on specific tasks for smaller models [1,2,3].    \n\n[1] Tamkin, Alex, et al. \"Active learning helps pretrained models learn the intended task.\" In NeurIPS 2022.  \n[2] Bansal, Parikshit, and Amit Sharma. \"Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost.\" arXiv preprint arXiv:2306.15766 (2023).  \n[3] Thirunavukarasu, Arun James, et al. \"Large language models in medicine.\" Nature Medicine (2023).  \n\n  \n  \n\n&nbsp;\n\n\n> W2: The assumption, as well as the remark, are too strong. 20Ng and AGNews are well-known public datasets that can be regarded as \"very clean\" datasets. Therefore, the LLM's accuracy in the EC (easy clean) category should be intuitively high. The accuracy in the EC category under the truly-noised dataset will be more convincing. Meanwhile, the percentage of the three categories (EC, HC, and TN) is missed.\n\nA2: Thank you for highlighting your concerns regarding our assumptions and the clarity of our remarks on the datasets. We believe that the assumption is achievable, as the LLM’s accuracy in the EC (easy clean) set is **not directly related** to the difficulty of a dataset. Particularly, for each sample misclassified by the LLM within the EC set, we can infer that its LLM-generated label happens to be the same as the noisy assigned label, which is only relatively probable when the number of classes is small and the dataset is difficult. Therefore, the assumption **can be satisfied** in most cases. To verify this, we report the EC accuracy in truly-noised datasets (SemEval, TREC, and Hausa).\n\n  \n\nAcc| Easy Clean (EC) | Hard Clean (HC)| True Noisy (TN)| Overall\n|----------|-------------|------------|-----|------| \n SemEval| 96.94% | 0.53% |57.69%| 63.35% \n TREC| 97.20%| 1.67% |47.45% |68.76%\nHausa| 95.65%| 2.41% | 46.12% | 50.17%\n\n  \n\nThe results show that the LLM accuracy is still **high in most cases**. Nevertheless, the accuracy is relatively low on Hausa, as this dataset is significantly more difficult and also with only five classes. Note that according to the definition of the HC (hard clean) set, the LLM accuracy should be relatively low on these samples. Thus, in our framework, we do not directly use LLM-generated confidences, but instead use PLM-generated confidences for fine-tuning.\n\n  \n  \n\nRegarding the percentages of the three categories in experiments, we present the detailed statistics of datasets with 20% Symmetric Noise as follows. For brevity, we only report datasets 20Ng and AGNews here, and will include all detailed statistics in the appendix in the revised version.\n\n  \n\n| 20Ng| Easy Clean (EC) | Hard Clean (HC)| True Noisy (TN)|  \n|:----------:|:-------------:|:-------------:|:------:|\n| Groundtruth| 60.91% (5,513)|19.79% (1,791) |19.30% (1,747)|\n| Percentage | 60.91% (5,513) | 18.80% (1,702) | 20.29% (1,836) |\n\n\n|AGNews|Easy Clean (EC) | Hard Clean (HC)| True Noisy (TN)|  \n|:----------:|:-------------:|:-------------:|:------:|\n| Groundtruth| 65.31% (26,124)|13.60% (5,440) |21.09% (8,436) |\n| Percentage | 65.31% (26,124) | 13.04% (5,217) | 21.64% (8,659) |\n\n  \nNote that the ideal separation of the EC set is always identical to the separation in practice, as this process does not require access to the true labels.\n\nFrom the statistics, we can observe that the TN set ratio is very close to the real noise ratio (20%), which indicates that our framework can **precisely separate noisy samples**. The ratios of EC and HC sets are also close to the real ratios, denoting that a more precise separation helps learning from noisy labels.\n\n&nbsp;\n\n\n> Q1: In Figure 4, LAFT with 0.9 ratio noise from AN or IDN obtains nearly 60% accuracy. Why the accuracy under these settings do not have a significant degradation?\n\nA1: Thank you for pointing out the performance of LAFT under high noise ratios. We achieve this by leveraging the confidences provided by LLMs. As a result, the performance also benefits from the inherent property of LLMs, which do not require training and are thus more **robust to noisy data**. On the other hand, existing baselines appear to struggle with large noisy ratios as they need to be trained on selected clean data, which can be hard to achieve when a large noisy ratio is present.\n\n&nbsp;\n\n\n> Q2: What happens when the noise ratio becomes 0, but the framework doesn't know this prior ratio while keeping the assumption that there are noises in the dataset?\n\nA2: Thank you for posing an insightful question about the framework's behavior in the absence of any noise. Since our work is proposed for noisy scenarios, we do not include results on totally clean datasets, as in existing works for learning from noisy labels [1,2,3].\n\nWhen the noise ratio becomes 0, instead of separating a large TN set (as the number of identified noisy samples is small), our framework still separates EC and HC sets, because these two sets exist even in totally clean data. With samples in these two sets, our designed loss in Eq. (10) can still help fine-tune the PLM with the guidance from LLM. As such, our framework **retains its efficacy** when on datasets without any noise.\n\n[1] Liu, Sheng, et al. \"Early-learning regularization prevents memorization of noisy labels.\" In NeurIPS 2020.  \n[2] Englesson, Erik, and Hossein Azizpour. \"Generalized jensen-shannon divergence loss for learning with noisy labels.\" In NeurIPS 2021.  \n[3] Zhang, Yivan, Gang Niu, and Masashi Sugiyama. \"Learning noise transition matrix from only noisy labels via total variation regularization.\" In ICML, 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea of utilizing LLMs to help improve PLMs fine-tuning is worth exploring.\n2. This paper is well-written and can be easy to read.\n3. The work is solid, and the experiments are exhaustive. Details are provided so that this work can be followed easily.", "reasons_to_reject": "1. As shown in the Limitation section, the performance of LAFT is highly correlated to the quality of the LLMs, which limits the framework's potential.\n2. The assumption, as well as the remark, are too strong. 20Ng and AGNews are well-known public datasets that can be regarded as \"very clean\" datasets. Therefore, the LLM's accuracy in the EC (easy clean) category should be intuitively high. The accuracy in the EC category under the truly-noised dataset will be more convincing. Meanwhile, the percentage of the three categories (EC, HC, and TN) is missed.", "questions_to_authors": "A: In Figure 4, LAFT with 0.9 ratio noise from AN or IDN obtains nearly 60% accuracy. Why the accuracy under these settings do not have a significant degradation?\n\nB: What happens when the noise ratio becomes 0, but the framework doesn't know this prior ratio while keeping the assumption that there are noises in the dataset?", "paper_topic_and_main_contributions": "To address the noise label problem in text classification, this paper proposes LAFT, a pre-trained language model fine-tuning method with guidance from larger language models. LAFT separates the training examples into three parts (easy clean, hard clean, and true noisy) and utilizes three objectives to learn from different categories of examples. This paper conducts exhaustive experiments on synthetic and real-world noisy datasets to demonstrate the effectiveness of LAFT.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4UDIzYIwih", "reviewer": "Reviewer_No3t", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer No3t:  \n\nThanks for your valuable feedback and suggestions on the missing reference! We would like to address your questions as follows:\n\n  &nbsp;\n  \n> **W1: A basic concept throughout the paper is undefined: confidence of both pre-trained model and external LLM are undefined.**\n  \nA1: Thank you for highlighting your concern regarding the definition of confidence for both pre-trained language models (PLM) and LLMs. We wish to clarify that we have **indeed defined** the confidence of PLMs on **Line 219-224** in **Eq. (1)**, and the confidence of LLMs on **Line 243-245** in our paper. We really appreciate your feedback and acknowledge that the presentation could be clearer. To address this issue, we will insert a formal definition of \"confidence\" in the revised version to enhance clarity. We value your suggestion and will ensure that the concept is presented more clearly in our revision. The following definition is obtained based on Line 219-224.\n\n\n***Definition**. The confidence score denotes the certainty when a model classifies a sample, regarding a specific class. Generally, the confidences are represented by the classification probabilities. Denote the output of the final layer (i.e., the classifier) in PLMs for sample $x_i$ as $\\mathbf{z}(x_i)\\in\\mathbb{R}^{N}$, where $N$ is the number of classes.\nThe confidence of $x_i$ for the $j$-th class $c_j$ can be represented as follows:*\n$$\n\\widetilde{p}(c_j; x_i)=\\frac{\\exp\\left(z(c_j; x_i)\\right)}{\\sum_{k=1}^N \\exp\\left(z(c_k; x_i)\\right)},\n$$\n*where $z(c_j;x_i)\\in\\mathbb{R}$ is the $j$-th value in $\\mathbf{z}(x_i)$. Notably, the confidences are obtained from $\\mathbf{z}(x_i)$ after a softmax function and thus sum up to one.*\n\n  &nbsp;\n  \n> **W2: There is also the possibility of information leaking from the LLM into the pre-trained LM, making it unclear to what extent the approach proposed is not in fact a form of distillation.**\n\nA2: Thank you for raising the concern about potential information leakage and the similarity between our framework and knowledge distillation.\n\nAlthough our work is related to knowledge distillation, we wish to clarify that our framework **differs from** traditional knowledge distillation from two unique perspectives.\n\n\n1. Rather than having the PLMs directly mimic the labels and probabilities produced by LLMs [1,2,3], we utilize LLMs as **external guidance** for fine-tuning PLMs. This guidance, in the form of predictions and confidences from LLMs, aids the **sample separation** process in our pipeline. Additionally, this information from LLMs is also crucial in enabling PLMs to effectively learn from noisy data.\n\n2. In contrast to knowledge distillation, in our framework, PLMs fine-tuned under the guidance of LLMs can outperform solely using LLMs (demonstrated in Sec. 5.5). The benefits come from the sample selection process, instead of directly learning knowledge from LLMs.\n\n  \n\nTo better illustrate the difference between our work and knowledge distillation, we conduct experiments using traditional knowledge distillation based on predictions and confidences from LLMs. The results in accuracy are presented as follows (20% Symmetric Noise for 20Ng and AGNews):\n\n|Model (Accuracy)| 20Ng| AGNews|SemEval|TREC|Hausa|\n|:----------:|:-------------:|:-------------:|:-------------:|:-------------:|:------:|\n|Base|79.15%|81.77%|70.61%|67.42%|47.8%|\n|ChatGPT| 75.34%| 81.41%| 63.35%|68.76%| 50.17%|\n|KD w/ ChatGPT Confidences| 79.33%| 85.61%| 67.68%| 68.94%| 48.06%|\n|KD w/ ChatGPT Predictions| 77.93%| 83.32%| 67.09%| 67.11%| 46.80%|\n|LAFT (Ours)|**82.04%**|**90.86%**| **73.56%**| **72.34%**|**51.71%**|\n\n  \n\nFrom the results, we can observe that directly distilling knowledge from LLMs **cannot outperform our framework**. Moreover, when the performance of LLM is inferior (e.g., on SemEval), the KD methods are even less competitive than the Base method.\n\n  \n\nSince LLMs are not fine-tuned on this specific dataset, their prior knowledge may not provide useful information regarding the classification. Nevertheless, in our framework, we can effectively leverage the confidence produced by LLMs to separate clean and noisy data for fine-tuning PLMs. With our designed loss to learn from separated data, our framework can achieve outperforming results.\n\n  \n\n[1] Magister, Lucie Charlotte, et al. \"Teaching small language models to reason.\" arXiv preprint arXiv:2212.08410 (2022).  \n[2] Fu, Yao, et al. \"Specializing Smaller Language Models towards Multi-Step Reasoning.\" arXiv preprint arXiv:2301.12726 (2023).  \n[3] Hsieh, Cheng-Yu, et al. \"Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes.\" arXiv preprint arXiv:2305.02301 (2023).  \n\n  \n  &nbsp;\n  \n\n> **W3: Usually fine-tuning data is scarce and critical for good performance on the domain/task of interest, so making sure it is of high quality (not noisy) is a top priority.**\n\n  \n\nA3: Thanks for your insightful opinion. We agree on the importance of clean data used for fine-tuning the pre-trained language models (PLMs), especially in scenarios where domain knowledge is necessary. However, for many downstream tasks, the labels can be noisy due to the labor-intensive and time-consuming labeling process [1,2].\n\nIn fact, this is **the motivation** of our framework for learning from noisy labels. We aim to maximally leverage information from both clean and noisy data, and thus propose to **separate clean data** under the guidance of LLMs.\n\n\n\nTo verify that our framework can indeed separate clean data, we present the percentages of the Easy Clean (EC) set and the LLM accuracy on it. We show the detailed statistics on datasets 20Ng and AGNews with 20% Symmetric Noise as follows:\n\nThe **LLM accuracy** on each set:\n| Accuracy| Easy Clean (EC) | Hard Clean (HC)| True Noisy (TN)| Overall|  \n|:----------:|:-------------:|:-------------:|:------:|:------:|\n| 20Ng| 99.46%|3.58%| 69.44% | 75.34%|\n|AGNews|99.03% |2.05%| 76.07% |81.41%|  \n  \nThe **percentage** of each set:\n| Percentage|  Easy Clean (EC) | Hard Clean (HC)| True Noisy (TN)|  Overall|\n|:----------:|:-------------:|:-------------:|:------:|:------:|\n| 20Ng| 60.91% (5,513) | 18.80% (1,702) | 20.29% (1,836) | 100% (9,051)|\n| AGNews| 65.31% (26,124) | 13.04% (5,217) | 21.64% (8,659) | 100% (40,000)|\n\n  \n\nFrom the results we can observe that, our framework can effectively separate an Easy Clean set with relatively sufficient data (larger than 60% of all data). The LLM accuracy is also high on this set, indicating that the data is **mostly clean**.\n\nTo evaluate the **quality of the separated EC set**, we additionally report the result when fine-tuning PLMs with only the EC set.\n|Method (Accuracy)| 20Ng| AGNews|\n|:----------:|:-------------:|:------:|\n|ChatGPT| 75.34%|81.41%|\n|Fine-tune w/ All (Base) |79.15%|81.77%|\n|Fine-tune w/ EC |81.04%|83.49%|\n|LAFT (Ours) |**82.04%**|**90.86%**|\n\nWe can observe that the performance of fine-tuning with only the EC set **surpasses the Base method** and the LLM. The results demonstrate that the EC set separated by our framework indeed **contains useful and important information** for classification. Nevertheless, only using the EC set cannot outperform our framework LAFT, which demonstrates the importance of also learning from noisy data with our strategy.\n\n[1] Zhou, Wenxuan, and Muhao Chen. \"Learning from Noisy Labels for Entity-Centric Information Extraction.\" In EMNLP 2021.  \n[2] Wei, Hongxin, et al. \"Combating noisy labels by agreement: A joint training method with co-regularization.\" In CVPR 2020.  \n\n  &nbsp;\n  \n\n> **W4: One important reference that is clearly relevant is label smoothing.**\n\n  \n  \n\nA4: Thank you for your valuable suggestion regarding the significant reference to label smoothing. We will incorporate the relevant works of label smoothing [1,2,3,4] in the revised version of our paper.\n\n  \n\nWe would also like to clarify the difference between our framework and label smoothing from two perspectives.\n\n1.  **Objectives**. Label smoothing aims to deal with overfitting and lack of generalizability. In contrast, our work proposes to **separate** a clean set from all noisy data for learning from noisy labels.  \n    \n2.  **Strategies**. Label smoothing is achieved by regularizing the classification probability. However, our work harnesses the predictions and confidences provided by **LLMs** to effectively separate clean data and also help fine-tune PLMs with noisy data.\n    \n\n  \n\n[1] Szegedy, Christian, et al. \"Rethinking the inception architecture for computer vision.\" In CVPR 2016.  \n[2] Müller, Rafael, Simon Kornblith, and Geoffrey E. Hinton. \"When does label smoothing help?.\" In NeurIPS 2019.  \n[3] Lukasik, Michal, et al. \"Does label smoothing mitigate label noise?.\" In ICML 2020.  \n[4] Wei, Jiaheng, et al. \"To Smooth or Not? When Label Smoothing Meets Noisy Labels.\" In ICML 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Mitigating noise in data is of general interest and any significant contribution is likely to have wide impact.\n ", "reasons_to_reject": "A basic concept throughout the paper is undefined: confidence of both pre-trained model and external LLM are undefined.\n\nThere is also the possibility of information leaking from the LLM into the pre-trained LM, making it unclear to what extent the approach proposed is not in fact a form of distillation.\n\nUsually fine-tuning data is scarce and critical for good performance on the domain/task of interest, so making sure it is of high quality (not noisy) is a top priority.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper tackles a relevant problem, namely mitigating the effects of noisy training data for fine-tuning pre-trained language models.\n\nIt proposes to do so by employing an external large language model.\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "superior advantages -> advantages"}, "extra_text": {"Missing_References": "One important reference that is clearly relevant is label smoothing, as introduced in:\n@misc{szegedy2015rethinking,\n      title={Rethinking the Inception Architecture for Computer Vision}, \n      author={Christian Szegedy and Vincent Vanhoucke and Sergey Ioffe and Jonathon Shlens and Zbigniew Wojna},\n      year={2015},\n      eprint={1512.00567},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n\nand studied in:\n\n@misc{müller2020does,\n      title={When Does Label Smoothing Help?}, \n      author={Rafael Müller and Simon Kornblith and Geoffrey Hinton},\n      year={2020},\n      eprint={1906.02629},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG}\n}"}}, {"review_id": "tojwwda0SS", "reviewer": "Reviewer_Qwqa", "summary": "", "questions": "1. In Section 4.3, from line 301, the author directly jumped from the motivations to the detailed methods. Why is the threshold set for the highest value of the max confidence score? Is that based on the intuition that LLMs perform well in identifying true noisy labels? \n\n2. What is the overall efficiency of the methods of separating clean samples from the training dataset, compared with other methods in noise-robustness? \n\n3. In section 4.5, is it possible to use negative sampling as a way to process a true noisy set?\n\n4. When using a more powerful model to filter the data for a smaller model like LLMs to filter clean data for BERT-based PLM. What do you think the best performance of the PLM will be (close to the LLM or even better, or others) and why? Even if not fine-tuning LLMs, do you consider it a fair comparison between your approach and other PLM-based approach, when adopting LLM as a filter? ", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer Qwqa:\n\nThanks for your insightful feedback and suggestions. Here we would like to address your questions as follows:\n\n  &nbsp;\n  \n\n> W1: The study is based on the intuitive approach of separating clean samples from the training set for model optimization can benefit the PLM. From a large perspective, there is no discussion on the efficiency or benefits of this approach compared with other approaches for noisy labels. The user also fails to extend his idea to other areas in NLP with noisy data labeling or other LLM-based models\n\n\n> Q2: What is the overall efficiency of the methods of separating clean samples from the training dataset, compared with other methods in noise-robustness?\n\n  \n  \n\nA1: Thank you for highlighting the intuition and potential comparisons of our framework. We apologize for not comparing the efficiency of our framework with other baselines. We will include a comprehensive discussion in our Limitation section.\n\n  \n\nWe would also like to briefly discuss the efficiency of our framework here. The main time consumption results from the process of querying an LLM for each sample. Nevertheless, we observe that with the guidance of LLMs, the fine-tuning of PLMs can be more efficient. Moreover, with the separated data, our framework only trains **one** network, in contrast to prevalent baselines that train **multiple** networks (Co-teaching, Co-teaching+, JoCoR, CR, and NPC). The guidance from the LLMs ensures a **faster convergence** for fine-tuning PLMs, as shown in the results below.\n\n  \n \nHere we include the results of running time until convergence on 20Ng and AGNews (20% SN) as follows, trained on a single 48GB Nvidia GeForce RTX A6000 GPU.\n\n  \n\n|Model| 20Ng| AGNews|\n|:----------:|:-------------:|------:|\n|Co-teaching| 431.21s| 993.48s|\n|Co-teaching+| 598.63s| 1706.1s|\n|JoCoR| 1078.39s| 2357.55s|\n|CR| 1380.39s| 3510.82s|\n|NPC| 304.18s| 1068.11s|\n|LAFT (Ours)| **265.09s**| **768.19s**|\n\n  \n  \n\nFrom the results, we observe that our framework **surpasses other baselines** in terms of running time. This is primarily due to that we only fine-tune one network instead of multiple ones in baselines. It is noteworthy that another time consumption is about querying the LLMs. Nevertheless, as we utilize black box LLMs, it is feasible to pre-process the queries via provided LLM APIs (nearly 1 second for each sample via ChatGPT). Such a process does not require computational resources and can be achieved in parallel with fine-tuning.\n\n  \n  \n\nRegarding the extension of our idea to other areas, thanks for your valuable suggestion! While our work mainly focuses on text classification, which is a fundamental and crucial problem in NLP, we recognize the potential of our method in various NLP tasks with noisy data. In future work, we will also consider applying our idea in other NLP tasks as well as incorporating it into other methods based on LLMs. To precisely summarize our work, we would like to revise our title to “Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance for Text Classification”.\n\nRegarding generalizability, as our framework utilizes **black-box** LLMs, we believe it can be easily generalized to a variety of LLM-based methods, e.g., in-context learning and reasoning.\n\n  \n  \n  &nbsp;\n  \n\n> W2: Even though the methods with LLMs are proposed, the LLMs play only the role of a more powerful classification model, which can also be replaced with other methods. There lacks an in-depth discussion on the motivation for using LLMs and how it benefits the overall process.\n\n  \n\nA2: Thank you for highlighting the role of LLMs in our work. We would like to clarify that the significance of our work lies in the proposed methodology to separate clean and noisy data and also learn from them under external guidance from LLMs. We leverage LLMs as external guidance because they bear prior knowledge that can provide useful information in both separation and learning (with our designed loss).\n\nWe agree that other models (e.g., PLMs) can also generate confidences. However, as illustrated in our Sec. 4.1, they tend to output high confidences for noisy samples and thus will harm the performance, due to the incorrect separation. That being said, the model will treat potentially noisy samples as clean data and thus learn incorrect information. To verify the superiortiy of using LLMs instead of PLMs, we conducted an additional set of experiments, where **the confidences** are generated by various PLMs using the classification probabilities. We report the results on datasets 20Ng and ANNews with 20% Symmetric Noise (SN).\n\n  \n\n|Model (Accuracy)| 20Ng| AGNews|\n|:----------:|:-------------:|:-------------:|\n|LAFT w/ BERT| 78.60%| 82.31%|\n|LAFT w/ RoBERTa| 80.05%| 85.09%|\n|LAFT w/ GPT-2| 77.92%| 84.17%|\n|LAFT w/ ChatGPT (Ours)|**82.04%**|**90.86%**|\n\n \nFrom the results, we can observe that using LLMs can outperform the methods with PLMs. This is because the LLMs provide more precise confidences and are also more robust to noise. We appreciate your feedback and will ensure that our revised version offers a more comprehensive understanding of the motivation and benefits of incorporating LLMs.\n\n  \n  \n  &nbsp;\n  \n\n> Q1: In Section 4.3, from line 301, the author directly jumped from the motivations to the detailed methods. Why is the threshold set for the highest value of the max confidence score? Is that based on the intuition that LLMs perform well in identifying true noisy labels?\n\n  \n  \n\nA1: Thank you for pointing out the confusion. We set the threshold for the highest confidence for two reasons:\n\n \n\n1. The strategy of selecting samples based on the highest confidence is **widely adopted** by existing methods for learning from noisy labels [1,2,3]. This is because the highest confidence reflects the model’s certainty on a specific sample.\nA high confidence usually indicates that a sample can be easily classified, suggesting that it is more likely to be clean. Such an approach is not only intuitive but is also empirically validated as effective in multiple works.\n\n\n2. The highest confidence is **consistent** across datasets. Given that the confidences for all classes of a sample sum up to 1, the range of the highest confidence value is bound in $[1/N,1]$, where $N$ is the number of classes. This consistent behavior makes the threshold more robust and generalizable to various datasets.\n\n  \n\n[1] Zhang, Bowen, et al. \"Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling.\" In NeurIPS 2021.  \n[2] Xie, Qizhe, et al. \"Unsupervised data augmentation for consistency training.\" In NeurIPS 2020.  \n[3] Nishi, Kento, et al. \"Augmentation strategies for learning with noisy labels.\" In CVPR 2021.  \n\n  \n  \n  \n  \n  \n  \n  &nbsp;\n  \n  \n  \n\n> Q3: In section 4.5, is it possible to use negative sampling as a way to process a true noisy set?\n\n  \n\nA3: We appreciate your constructive suggestion regarding the potential use of negative sampling. It is indeed an intriguing idea to leverage noisy data. In our framework, samples in the TN set are mostly noisy, making them potential candidates for negative samples in contrastive learning. However, such a strategy **cannot fully utilize the information** provided by LLMs, because it cannot utilize the LLM-generated confidences in other classes. In contrast, our designed loss in Sec. 4.5 can help PLMs learn from other potentially correct labels based on LLM-generated confidences. We value your feedback and will incorporate this discussion in our revised version.\n\n  \n  \n  \n  &nbsp;\n  \n\n> Q4: When using a more powerful model to filter the data for a smaller model like LLMs to filter clean data for BERT-based PLM. What do you think the best performance of the PLM will be (close to the LLM or even better, or others) and why? Even if not fine-tuning LLMs, do you consider it a fair comparison between your approach and other PLM-based approach, when adopting LLM as a filter?\n\n  \n  \n\nA4: Thank you for your thoughtful query regarding the comparison between our framework and other PLM-based baselines.\n\n\nWe believe (and also observe) that the performance of PLMs fine-tuned on LLM-filtered data can surpass the performance of **solely using LLMs** (Table 5). For example, LLMs achieve 75.34% on 20Ng (irrelevant to noise ratios as LLMs are not fine-tuned), while our method can achieve 76.93% even with 40% noisy data for fine-tuning. The results demonstrate that the filtered data can significantly promote the fine-tuning performance of PLMs.\n\n  \n  \n\nIn terms of the comparison between our framework and other PLM-based methods, we believe it is justifiable. This is because we do not directly transfer knowledge, but use the LLM's predictions to guide the training of the PLM. Such a strategy enables us to leverage **black-box LLMs** for fine-tuning PLMs without significant computational costs. Moreover, our framework does not need to use LLMs during inference, which means **the practicality** is the same as other PLM-based methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Overall, the method is written concisely and clearly and is easy to read and understand. \n\nThe methods of incorporating LLMs as a tool for the benefits of PLM are also a popular and promising direction. \n\nThe user uses experiments and data to support their motivation of these challenges, and propose solutions accordingly, which is effective and accurate. ", "reasons_to_reject": "The study is based on the intuitive approach of separating clean samples from the training set for model optimization can benefit the PLM. From a large perspective, there is no discussion on the efficiency or benefits of this approach compared with other approaches for noisy labels. The user also fails to extend his idea to other areas in NLP with noisy data labelling or other LLM-based models. \n\nEven though the methods with LLMs are proposed, the LLMs play only the role of a more powerful classification model, which can also be replaced with other methods. There lacks an in-depth discussion on the motivation for using LLMs and how it benefits the overall process. ", "questions_to_authors": "1. In Section 4.3, from line 301, the author directly jumped from the motivations to the detailed methods. Why is the threshold set for the highest value of the max confidence score? Is that based on the intuition that LLMs perform well in identifying true noisy labels? \n\n2. What is the overall efficiency of the methods of separating clean samples from the training dataset, compared with other methods in noise-robustness? \n\n3. In section 4.5, is it possible to use negative sampling as a way to process a true noisy set?\n\n4. When using a more powerful model to filter the data for a smaller model like LLMs to filter clean data for BERT-based PLM. What do you think the best performance of the PLM will be (close to the LLM or even better, or others) and why? Even if not fine-tuning LLMs, do you consider it a fair comparison between your approach and other PLM-based approach, when adopting LLM as a filter? ", "paper_topic_and_main_contributions": "The paper focus on the study of fine-tuning PLM with noisy labels, where they provide a method to use LLM as external guidance to filter the clean data and noisy data. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Is noise robust equals to cleaning all the noise and training with clean data? Some discussion is here: https://arxiv.org/pdf/2202.12024.pdf", "Typos_Grammar_Style_and_Presentation_Improvements": "Figure 4 is not mentioned in the paragraph of section 5.4. It can be analysed more, for example, why LAF\\C is dropping with noise ratio increasing? "}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses the problem of noisy training data for fine-tuning pre-trained language models and proposes a method that leverages external large language models. In particular, the paper separates the training examples into three parts (easy clean, hard clean, and true noisy) and utilizes different objectives to learn from those different categories.

Pros / Strengths:
- The authors conduct exhaustive experiments on both synthetic and real-world noisy datasets.
- The paper is well written and the method is well described.
- Mitigating noise in data is of general interest and any significant contribution is likely to have wide impact.

Cons / Weaknesses:
- The method assumes the existance of high-quality large language models.
- The relationship of the method (and empirical comparisons) to other areas in NLP with noisy data labeling or other LLM-based methods is missing.

Action items for improving the paper:
- The comparisons with knowledge distillation as presented during the author rebuttal phase are an interesting baseline and should be added to the paper to contrast the work against this line of work.
- The related work section should be extended, i.a., with work on label smoothing
Decision: Accept-Findings