Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLM-Powered Assistance (2024)

Abstract: Learning from noisy labels (LNL) is a challenge that arises in many real-world scenarios where collected training data can contain incorrect or corrupted labels. Most existing solutions identify noisy labels and adopt active learning to query human experts on them for denoising. In the era of large language models (LLMs), although we can reduce the human effort to improve these methods, their performances are still subject to accurately separating the clean and noisy samples from noisy data. In this paper, we propose an innovative collaborative learning framework NoiseAL based on active learning to combine LLMs and small models (SMs) for learning from noisy labels. During collaborative training, we first adopt two SMs to form a co-prediction network and propose a dynamic-enhanced threshold strategy to divide the noisy data into different subsets, then select the clean and noisy samples from these subsets to feed the active annotator LLMs to rectify noisy samples. Finally, we employ different optimization objectives to conquer subsets with different degrees of label noises. Extensive experiments on synthetic and real-world noise datasets further demonstrate the superiority of our framework over state-of-the-art baselines. The code is available at https://github.com/byuan186/NoiseAL.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10977–11011 August 11-16, 2024 ©2024 Association for Computational Linguistics
Learning from noisy labels (LNL) is a challenge that arises in many real-world scenarios where collected training data can contain incorrect or corrupted labels. Most existing solutions identify noisy labels and adopt active learning to query human experts on them for denoising. In the era of large language models (LLMs), although we can reduce the human effort to improve these methods, their performances are still subject to accurately separating the clean and noisy samples from noisy data. In this paper, we propose an innovative collaborative learning framework NoiseAL based on active learning to combine LLMs and small models (SMs) for learning from noisy labels. During collaborative training, we first adopt two SMs to form a co-prediction network and propose a dynamic-enhanced threshold strategy to divide the noisy data into different subsets, then select the clean and noisy samples from these subsets to feed the active annotator LLMs to rectify noisy samples. Finally, we employ different optimization objectives to conquer subsets with different degrees of label noises. Extensive experiments on synthetic and real-world noise datasets further demonstrate the superiority of our framework over state-of-the-art baselines. The code is available at https://github.com/byuan186/NoiseAL.

heading: 1 Introduction
text: The core of deep learning models’ success lies in the precision of large amounts of annotated data, which can be a complex and labor-intensive process. In practical scenarios, some researchers often collect datasets by web-crawling (Li et al., 2017) or crowd-sourcing (Yan et al., 2014) to reduce the burden of annotation. However, the obtained datasets frequently suffer from the presence of noisy labels, which will mislead the learning patterns and
*Corresponding Author
subsequently result in incorrect predictions. Existing LNL methods include designing robust loss functions (Zhang et al., 2018; Ma et al., 2020), estimating noise transition matrices (Patrini et al., 2017; Zhang et al., 2021c), etc. Among them, active learning (Zhao et al., 2011; Younesian et al., 2021) is a popular solution that provides more accurate labels for noisy samples by querying experts, but it still requires human effort in the annotation.
Recently, large language models (LLMs), such as ChatGPT, have exhibited strong zero-shot learning ability, avoiding human costs for text annotation tasks. Some studies indicate that zero-shot ChatGPT classifications outperform crowd workers in some domains (Shu et al., 2019; Gilardi et al., 2023). Despite the promise, further studies (Bang et al., 2023; Xiao et al., 2023) observe that LLMs tend to underperform compared to BERT supervised on complex datasets. This observation is further supported by our own empirical studies (Table 1). Inspiringly, LLMs emerge with the in-context learning (ICL) ability to learn from a few labeled
10977
samples for boosted performance. However, ICL is sensitive to the input prompt, where acquiring ground-truth labeling of the input demonstrations is important for good ICL performance (Mavromatis et al., 2023). So, we study the following problem: Given a noisy dataset, how can we accurately identify noisy samples for active querying LLMs and successfully select clean samples as ICL examples?
Before the era of LLMs, the traditional methods (Han et al., 2018; Shu et al., 2019; Qiao et al., 2022) to divide the noisy data depending on the loss value. The key idea is to set a fixed threshold for loss value, such that the clean samples are associated with a smaller mean loss value and noisy ones with bigger values. However, a recent study (Gao et al., 2023) argues that these methods fail to select clean samples under some synthetic noisy scenarios. Our empirical studies verify this opinion and further observe the mean value of samples with clean labels and noisy labels gradually decreases during training (see Appendix A). So it is not ideal to set a fixed threshold, particularly when we consider SMs’(e.g., BERT, BiLSTM) inherent memory effect that the memorization strength of samples increases during training (See Figure 4 (a-c)). Besides, some memory effect-based work indicates that the confirmation bias (Li et al., 2020) makes SM memorize itself mistakes and hard to discern noisy labels by itself (Xiao et al., 2023). While previous methods could partition datasets, they often struggle to accurately select clean samples in different LNL scenarios, thus failing to construct high-quality ICL for querying LLMs.
To overcome the above limitations of traditional selection methods, we present a novel collaborative learning paradigm, NoiseAL, that utilizes active learning to marry LLMs with SMs by filtering noisy data with the help of SMs and distilling the related knowledge from the LLMs. Specifically, we first adopt two SMs to form a co-prediction network, focusing on different predictive capabilities and producing multiple predictions for each text to mitigate confirmation bias. Our intuition is that the presence of noisy labels may make it difficult for the two SMs to arrive at a consensus on the outputs. In this scenario, one SM may start fitting the noise before another, leading to divergent copredictions. With two prediction results, we then propose a dynamic-enhanced threshold strategy following memory effect to select clean and noisy samples to LLMs. To integrate SMs and LLMs as a whole, we design a collaborative training frame-
work where SMs operate as filters to divide the noisy dataset into different subsets and the LLM acts as an active annotator to correct noisy samples from subsets. During collaborative training, SMs can learn the knowledge of LLMs to boost their performance, while LLMs can also benefit from the divided clean samples to boost their ICL ability. As shown in Figure 1, NoiseAL achieves competitive results compared with its counterparts supervised on the ground truth label under some scenarios. Overall, our main contributions are:
• Based on the memory effect, we innovatively utilize a co-prediction network, combined with a dynamic-enhanced threshold strategy, to select clean samples and noisy samples from noisy data.
• We propose a novel collaborative learning framework termed NoiseAL, which employs SMs as filters to segment noisy data and LLMs as active annotators without any human effort.
• We conduct experiments on diverse text classification datasets under varied noise conditions, revealing the superiority of our proposed NoiseAL against current baselines.

heading: 2 Related Work

heading: 2.1 Memory Effect of Small Models (SMs)
text: Carlini et al. (2019) demonstrated that BiLSTM models are able to consistently memorize examples during the very first phase of training. Tänzer et al. (2021) show that BERT forgets examples at a slower rate than BiLSTM and other non-pretrained models. Inspired by the above work, we believe that BiLSTM (weak model) starts fitting the noise before BERT (strong model) and adopt these two SMs to compose a co-prediction network producing different predictions, which helps to mitigate confirmation bias and separate noisy data. Moreover, Li et al. (2023) also observes the memorization strength of SMs for given labels towards individual samples improves during training, which resembles our observations and inspires us to provide a dynamic-enhanced threshold strategy.

heading: 2.2 Learning with Noisy Labels
text: Previous LNL studies can be categorized into three groups: (1) Sample Selection approaches (Han et al., 2018; Shu et al., 2019; Qiao et al., 2022) based on loss values that rely on the assumption that clean samples tend to have a smaller mean loss value. These methods require manually setting
a fixed threshold for the loss value, which makes their performance questionable in some scenarios. Based on our observation that the loss distributions dynamically evolve during training, we propose a dynamic-enhanced threshold strategy to improve these methods. (2) Label Correction methods aim to correct the noisy labels, which adopt noise transition matrix estimation (Sohn et al., 2020; Zhang et al., 2021c) or AL. While traditional AL relies on expensive expert labeling, we explore the rich knowledge of LLMs to generate labels for noisy samples without human effort. (3) Regularization methods (Wang et al., 2019; Liu et al., 2020; Northcutt et al., 2021b; Zhuang et al., 2023) enhance model resistance to label noise by crafting robust loss functions or adopting regularized strategies. Although DyGen (Zhuang et al., 2023) also considers dynamic training, it uses dynamic patterns in the embedding space, which is different from ours.

heading: 3 Background
text: Let D={(xi, yi)}Ni=1 denote the training data with noisy labels, where x is the text, y is its label, and N is the data size. Denote fθ(x) ∈ RK as the output of the model f with a linear layer (i.e., the classifier), where θ is its parameters and K is the number of classes. The confidence of x for each label k ∈ [1,K] can be represented as follows: p(k;x) = e
fθ(k;x)∑K k=1 e fθ(k;x) . Generally, as the SM’s
memory strength regarding a text increases, the prediction confidence also correspondingly rises. Based on this association, we quantify the SM’s memorization strength through a confidence metric. Thus, we regard p(k;x) as the value of memorization strength. For the text classification, if a model memorizes a text, its confidences p(k;x) of k exceed a certain threshold or reach the maximum.

heading: 4 Method
text: In this section, we introduce our proposed framework NoiseAL which explores the opportunity for active learning to address LNL problems in the LLMs era. Although LLMs can generate new labels for noisy samples directly, we still need SMs to help separate noisy data. In each training loop, we alternate the following steps: (1) Training a coprediction network consisting of two SMs to divide the noisy data into different subsets. (2) Selecting clean and noisy samples from subsets, the noisy samples are corrected by active querying LLMs, and the clean samples are used to prompt ICL. Figure 3 shows the framework of NoiseAL. And its overall pipeline is shown in Algorithm 1. Following that, we will provide a detailed explanation of our proposed NoiseAL framework.

heading: 4.1 Dynamic-Enhanced Threshold Strategy
text: For clean samples and noisy samples, we observed that their mean loss value decreases and loss distribution gradually tends to be consistent during training, as shown in Figure 2. The observed phenomena could potentially be attributed to the memory effect of SMs. Specifically, in the early epochs, noisy samples tend to have a higher loss and can be selected by setting a fixed threshold. However, as training continues, these noisy samples will be memorized gradually, and their loss accordingly decreases. In this situation, it is inappropriate to set a fixed threshold. So, we propose a novel dynamic-enhanced threshold strategy to provide both dynamic and fixed thresholds.
Dynamic Threshold. To capture the increasing memorization strength, we introduce a dynamic threshold τ(t) for each sample x: τ(t) = λp(t) + (1− λ)τ(t− 1), τ(0) = 0, where p(t) = max(p(k;x)), p(t) is the maximum confidence of the current training epoch t, λ is a hyperparameter controlling threshold stability. The idea of τ(t) is that the threshold for determining whether a model memorizes a sample should also increase accordingly with the increase of historical confidence. However, the confidence may be unstable when SM begins to overfit noise samples. Hence, we utilize the momentum p(t) attained by each sample across all prior epochs as the threshold value.
Fixed Threshold. Preliminary studies indicate that the loss distributions of both clean and noisy samples during training appear to follow two Gaussian distributions (Qiao et al., 2022). Exploiting
these findings, we compute the loss L (Section. 4.4) on the training samples and fit L to a twocomponent Gaussian Mixture Model (GMM) (Li et al., 2020) using the Expectation-Maximization algorithm. For sample x, let l(x) represent its loss, and o(x) = p(g|l(x)) represent its clean probability o(x), where g is the Gaussian component with a smaller mean value (smaller loss). Then, we set a fixed threshold ϕ for o(x) to distinguish whether x belongs to clean or noisy.

heading: 4.2 Dynamic-Enhanced Selection
text: We combine dynamic thresholds τ(t) and fixed thresholds ϕ to present our dynamic selection.
Co-prediction Network. Recent research found that training a model using the data divided by itself could lead to confirmation bias (Li et al., 2020), as noisy samples would keep having lower losses due to the model overfitting their labels. To solve this problem, we propose a co-prediction network, in which two models diverge from each other due to different (random) parameter initialization, sequence lengths, and training text features. Being diverged offers the two models diverse evidence (confidence) to memorize samples, making the whole network more robust than a single model.
We first use the diverse confidence of the coprediction network and dynamic threshold τ(t) to divide the noisy data into two subsets (see Figure 3), i.e., a consistency set C and a discrepancy set I which are obtained according to the consistency and discrepancy of confidences. Next, we finely group these two sets into three subsets according to the fixed threshold ϕ, i.e., a clean set R obtained through selecting clean samples from consistency set, a hard set H obtained through selecting clean
Algorithm 1 Pipeline of NoiseAL Input: D={(xi, yi)}Ni=1, yi ∈ [1,K] ; small mod-
els Ss and Sw; Large Language Model L 1: Set epoch = 1; 2: while epoch ≤ 6 do 3: Get two confidence Ps and Pw from Ss and Sw for all i = 1, . . . , N ; 4: Calculate dynamic threshold τ(t); 5: Divide the noisy dataset into Consistency
set C and Discrepancy set I based on τ(t), Pw and Ps;
6: Calculate fixed threshold ϕ; 7: Based on the ϕ, further divide the C and I
into Clean set R, Hard set H , and Purified set P ;
8: Using L to generate new labels for each sample from P ; 9: Train Ss and Sw with a combination of different loss functions based on the different subsets (R,H,P ) as Eq.(4) and Eq.(6)
10: epoch = epoch + 1 11: end while
samples from discrepancy set, and a purified set P obtained through selecting noisy samples from consistency set followed by LLMs to query. As a result, these subsets are utilized to train the co-prediction network improving its performance. Meanwhile, the co-prediction network with better performance also prompts the precise partitioning of subsets.
Selection. For a text xi given label yi, the confidence from the weak and strong model can be denoted as pw(y;x) and ps(y;x), respectively. If both confidences in co-prediction network are higher than τ(t), we put xi into the consistency set
C:
C = {(xi, yi)|pw(yi;xi) > τw(t)}∩ {(xi, yi)|ps(yi;xi) > τs(t)} .
(1)
That is to say, if a sample can be consistently memorized by two models with high confidence, we regard it as a consistent sample. Besides, if only one model’s confidence surpass τ(t), we can include it in the discrepancy set I:
I = {(xi, yi)|pw(yi;xi) > τw(t)}∪ {(xi, yi)|ps(yi;xi) > τs(t)} − C. (2)
However, strong and weak models eventually memorize the noisy label, which means both models may memorize a sample with noisy labels during training, we need to distinguish further whether the sample in the consistent set C may be clean or noisy. To achieve it, we apply ϕ for C to distinguish the clean samples from noisy samples. Specifically, for a sample in consistency set C, if its clean probability is higher than ϕ, we put it into the clean set R: R = {(xi, yi)|o(xi) ≥ ϕ} ∩ C.
In short, if a sample is consistently memorized by two models and meanwhile has a high clean probability, we regard it as a clean sample. On the contrary, if a consistent sample’s clean probability is lower than ϕ, we put it into P: P = {(xi, yi)|o(xi) < ϕ} ∩ C. That is to say, if a sample that is consistently memorized by two models has a low clean probability, this sample is highly likely to be a noisy sample.
Qiao et al. (2022) found that pretrained language models (PLMs) BERT memorize noisy samples at a dramatically lower rate compared to BiLSTM in noisy scenarios. When BiLSTM starts to memorize the noisy samples, its confidence for clean samples will decrease, but BERT still maintains high confidence for these clean samples. Thus, we think some samples in the remaining discrepancy set I may be clean and have the potential to improve our method’s generalization ability. We put them into the hard set H: H = {(xi, yi)|o(xi) ≥ ϕ} ∩ I.

heading: 4.3 Active Querying by LLMs
text: In this part, we leverage the strong ICL ability of LLMs to generate new labels for noisy samples. In particular, the core challenge lies in constructing a proper prompt containing demonstration samples. To perform active querying, we first provide an assumption as the justification:
Assumption. Samples in R and P can be considered almost clean and noisy respectively.
Note that the above Assumption is empirically verified in Table 2. The underlying intuition is: that the noisy ratio in R is significantly low and thus negligible, while the noisy ratio in P is significantly high and needs to be corrected by querying LLMs.
Demonstration Construction. We devise a feature-aware example sampling strategy. Given an input xi ∈ P , we can obtain its text feature f(xi) and calculate the cosine similarity of the text feature between xi and other training text xj ∈ R, then sample top-K nearest neighbors to form demonstration examples:
E = argTopN j∈{1,2,...,|R|}
f(xi) T f(xj)
∥f(xi)∥∥f(xj)∥ , (3)
where E is an index set of top-K similar samples in R. Note that text features of datasets are computed and stored beforehand, allowing efficient sampling.
Querying. Our LLMs prompt consists of the following three components: (1) Task description, which describes the task. (2) Demonstration, which consists of a sequence of samples from R. (3) Input, which is the test text from P to classify. Our initial investigation indicates that although this prompt design can already yield reasonable results, the prediction is unstable. Specifically, we observe variations in the results when we randomly rearrange the order of the demonstrations. This indicates that the model faces difficulty comprehending the user’s viewpoint accurately. To address this instability, we incorporate Chain-of-Thought (COT) reasoning into our setting. The details of prompts are provided in Appendix J. By prompting LLMs to generate high-quality labels for noisy samples, the high noise rate of P is greatly reduced.

heading: 4.4 Training Co-prediction Network (SMs)
text: Learning From the clean Set R. For the R, we directly utilize the cross-entropy loss for the co-prediction network (SMs):
LR = − 1 N
NR∑
i=1
logps(yi;xi) + logpw(yi;xi) (4)
where NR denotes the size of R, N denotes the size of the entire dataset.
Learning From the Purified Set P . After generating the labels by LLMs for samples in P , the number of noisy samples in P is greatly reduced. However, for LLMs, even the powerful GPT-4, cannot generate right labels for every sample in P . Our
solution comes from rethinking the way of learning from the P: can we design an objective such that our model can be optimized with access to the P with a lower noise ratio? To this end, we resort to a family of noise-robust loss functions (ℓrobust) (Ghosh et al., 2017; Zhang and Sabuncu, 2018). Some loss functions has the following property:
K∑
k=1
ℓrobust(fθ(x), k) = C,∀x, f (5)
where x is the input, and C is a constant. Previous research has shown that these loss functions have a consistent minimum under label noise. This means that the minimizer (θ∗) of ℓrobust is the same when most of the training samples are correctly labeled, no matter if the training data is noisy or clean. The theoretical proof will be given in Appendix E. This robust property lets us optimize the model given a dataset with a lower noise ratio. Specifically, we utilize the reversed cross-entropy loss for sample (xi, yi) in P:
LP = − 1 N
NP∑
i=1
K∑
k=1
(ps(k;xi) + pw(k;xi))log q(k|xi),
(6) where q(k|x) is the ground-truth distribution over labels, NP denotes the size of P . Remark. The reversed cross-entropy loss satisfies Property 5 and is theoretically noise-tolerant. The proof of Remark is given in Appendix F. Learning From the Hard Set H. Since the sample in hard set H cannot be memorized by two models at the same time, if we directly use crossentropy for supervision, the model is prone to accumulate errors. Inspired by EmbMix (Qiao et al., 2022), which is an effective regularization technique that applies the [CLS] embedding encoded by PLMs in noise-robust training for text classification. Based on the [CLS] embedding encoded by PLMs, we also apply interpolations to it. To be specific, we randomly choose two samples (xi, yi), (xj , yj) and the mixed sample (e′i, y ′ i) can be defined as e′i = λ ′ei + (1 − λ′)ej , y′i = λ′yi + (1− λ′)yj , ei =PLMs(xi), ej =PLMs(xj), λ ∼ Beta(α, α), λ′ = max(λ, 1 − λ). Then, we perform EmbMix on H to obtain the mixed set H̃: H̃ = {(e′i, y′i)|(xi, yi), (xj , yj) ∈ H} . For H̃, we compute the loss: LH = − 1N ∑NH̃ i=1 logps(y ′ i; e ′ i), where NH̃ is the size of H̃. Finally, our overall training objective L can be calculated by: L = LR + LP + LH.

heading: 5 Experiments

heading: 5.1 Experimental Settings
text: Datasets. We first experiment with five datasets: 20ng (Lang, 1995), SST-2 (Socher et al., 2013), Trec (Li and Roth, 2002), AGNews (Gulli, 2005), and IMDB (Maas et al., 2011). Three different types of synthetic label noise are generated and injected into these datasets following the setups of existing LNL works (Qiao et al., 2022; Zhu et al., 2022): (1) Symmetric Noise (S) flips labels uniformly to other classes (van Rooyen et al., 2015) (2) Asymmetric Noise (A) flips their labels to the corresponding class according to the asymmetric noise transition matrix. (Chen et al., 2019; Zhu et al., 2022) (3) Instance-dependent Noise (I) flips origin labels to the class with the highest prediction probability (other models as the feature extractor) among other classes (Algan and Ulusoy, 2020). Then, we conduct experiments on datasets with (4) Real-world Noise: TREC (Awasthi et al., 2020), ChemProt (Krallinger et al., 2017), SemEval (Zhou et al., 2020), and so on. Details are in Appendix B.
Baselines. We compare NoiseAL with the most relevant LNL baselines as follows: (1) Basic models without Noise-handling (Devlin et al., 2019); (2) Regularization Technology, including SCE (Wang et al., 2019), ELR (Liu et al., 2020), CL (Northcutt et al., 2021b), DyGen (Zhuang et al., 2023); (3) Sample Selection, including Co-Teaching (Han et al., 2018), SelfMix (Qiao et al., 2022), LAFT (Wang et al., 2023). See Appendix C for details. For these baselines, we perform their public code (except LAFT) to implement them.
LLMs and Prompts We use GPT-3.5-Tubor0613 API (e.g., ChatGPT), and run the generation 5 times with a temperature of 0.5 to produce different reasoning paths and predictions. Then we use majority voting to get the final prediction results.
The implementation details are in Appendix D.

heading: 5.2 Main Results
text: Table 1 and Table 3 show the main results for three synthetic and real-world noisy datasets ( See Appendix G for more datasets’ results). From these results, we have the following observations: (1) NoiseAL significantly outperforms all baselines on synthetic datasets with varying noise types and ratios. (2) Instance-dependent Noise (I) is featuredependent noise, which is more challenging than other synthetic noise. The results in Table 1 show that NoiseAL still outperforms other methods un-
der this label noise setting, which shows NoiseAL
has stronger robustness and generalization ability compared with others. (3) We also provide the base model’s performance on the ground truth data (upper bounds) and the performance of ChatGPT (zero-shot). On simple datasets with fewer label categories, ChatGPT performs better than certain baselines, yet our method still maintains a certain advantage. For upper bounds, the results of NoiseAL are closest to it compared to others.

heading: 5.3 Ablation Studies
text: To evaluate the contribution of each component in our NoiseAL, we conduct ablation studies on the Trec dataset (refer to Table 4). More ablation studies on other datasets are shown in Appendix H.
Co-prediction Network. Co-prediction Network (CN) provides different memorization strengths to fit datasets, which can alleviate the overfitting of noisy labels and confirmation bias during training. The performance of NoiseAL will decrease greatly when we remove the CN. These results indicate that utilizing a co-prediction network is crucial, particularly in scenarios with substantial label noise. The details for the selection of strong
and weak models are in Appendix H.2. Dynamic-enhanced Selection. Dynamicenhanced Selection (DS) utilizes the distinct hints provided by the dynamic-enhanced threshold strategy to partition the noisy dataset. If we remove DS and only rely on a fixed threshold strategy for selection, the performance of NoiseAL will decrease by a large margin. From the overall results, we further found that the performance degradation is the most significant if DS or CN are removed, which proves these two modules contribute the most to performance improvement.
Different Subsets. To better exploit the useful information in noisy data, we divide noisy data into R, P , and H. The results in Table 4 demonstrate that these subsets are all important for improving the performance of our NoiseAL. Failure to employ them leads to a decline in the results. For P , we feed its noisy samples to LLMs for query and generate new labels, and then we adopt the reversed cross-entropy loss to learn from them. More ablation studies in Appendix H.1.

heading: 5.4 Analysis
text: Effect of In-Context Examples. We explore the effect of different numbers N of examples during the process of ICL on the Trec datasets. Moreover, we conduct comparison experiments by replacing the ICL with much simpler approaches such as Knn. As shown in Figure 5, our approach performs better than KNN over a wide range of N from 1 to 20, which verifies the effectiveness of NoisyAL. In our
experiments, we adopt N=5 for fair comparisons. Effect of Collaborative Training. From a more nuanced perspective, we report the statistical distribution of the different subsets during training. After warming up the model for 2 epochs, the SMs begin to divide the noisy data according to the previously generated confidences and loss distributions. From the Table 2, we find the number of samples with correct labels in purified sets P has greatly increased after querying LLMs, which verifies the effectiveness of querying LLMs for label denoising. During training, we observe that: (1) for R and H, the number of samples with correct labels in these two subsets is gradually increasing; (2) for P , the ratio of correct labels provided by LLMs is also gradually increasing. This observation proves our collaborative training can enable SMs and LLMs to promote and improve their performance mutually.

heading: 5.5 Capability of Distinguishing Noisy Samples
text: For clean samples and noisy samples, we demonstrate their confidence distributions of the base model (Figure 4(a-c)) and strong model (Figure 4(e-g)) on Trec under 40% asymmetric noise. During training, the confidence generated by the strong model is getting more polarized while the base model has already overfitted the wrong labels. Furthermore, we demonstrate the loss distributions of the strong model (Figure 4(d,h)). During training, the loss output by the strong model is also getting more polarized while the loss of the base model (as
shown in Figure 2) is gradually decreasing (loss distributions of clean and noisy samples are becoming consistent). That is to say, the base model eventually memorizes the noisy samples. These fine-grained experiments and visualization show that NoiseAL can better isolate noisy samples and clean samples, which can help the strong model better perform fine-tuning on clean samples and avoid memorizing (overfitting) noisy samples.

heading: 5.6 Comparison with Human Annotations
text: We discuss human and LLM annotations from two perspectives: expense and label quality. In terms of the expense of human annotations and LLM annotations, human annotation tasks can be conducted by crowd-workers on platforms such as MTurk and trained annotators, such as research assistants. However, LLM annotations are generally less expensive than human annotations when dealing with large datasets (For instance, AGNews, which has 127600 samples; and IMDB, which has 50000 samples). This is primarily because LLMs can generate annotations quickly and at scale, without the need for extensive human labor. Moreover, the per-annotation cost of ChatGPT is less than $0.003—about twenty times cheaper than MTurk. But, this cost advantage must be weighed against the potential decrease in label quality. While human annotations can potentially provide a cleaner set of labels, the costs associated can be significantly higher, especially for large-scale datasets. Human annotation requires time, financial resources for compensating annotators, and sometimes additional rounds of validation to ensure quality. However, the benefits include higher accuracy and potentially fewer errors, which can be crucial for certain tasks.
Since our goal is to reduce the noise rate in the original noisy subset P , that is, to generate a subset with a lower noise rate rather than to produce com-
pletely clean data. Therefore, the LLM annotations could be considered a more cost-effective approach, even though they might introduce some noise. For these noises, we utilize the robust loss function to learn from them and, to some extent, mitigate the negative impact of noises. Moreover, some previous work shows that the zero-shot accuracy of ChatGPT exceeds that of crowd-workers for some tasks (Gilardi et al., 2023; Törnberg, 2023).
Overall, in terms of cost and accuracy, as well as our goals, we believe that LLM annotation has more advantages than human annotation.

heading: 5.7 Case Analyse for Cost Efficiency
text: When it comes to leveraging pretrained knowledge, it’s also essential to analyze cost efficiency (Aguda et al., 2024). In this paper, we incur costs by using ChatGPT. Thus, we provide a single case to calculate how many dollars need to be expensed by multiplying the total number of consumed tokens with the ChatGPT model price ($0.003 per thousand tokens). Specifically, for the Trec dataset with 20% symmetric label noise (20%S), all samples (Task descriptions+Demonstration+Inputs) to query ChatGPT were tokenized into 19652 tokens, and ChatGPT generated 6338 tokens, resulting in a total of 25990 tokens. Since we run each case using 5 random seeds and report the average performance, the final tokens are 25990*5=129950 tokens, so we need to spend $0.38985.

heading: 6 Conclusion
text: In this paper, we focus on solving a challenging text classification problem with noisy labels. We propose a novel framework called NoisyAL that introduces active learning to combine the nonpretrained model (BiLSTM), PLMs (BERT), and LLMs for learning from noisy labels. The key idea of NoisyAL is to separate noisy datasets by SMs (BiLSTM and BERT) and distill related knowledge from LLMs with a collaborative framework, where SMs act as filters to select clean and noisy samples for LLMs and the LLM is employed as an active annotator. The results of our experiments demonstrate the versatility of our method, as it significantly improves the accuracy of benchmark datasets with both synthetic and real-world noise. We hope that our work can inspire people’s interest in developing new active denoising algorithms by collaboratively utilizing LLMs and SMs.
Limitations
We proposed a collaborative framework NoiseAL to handle label noise for multi-class text classification. Although the proposed approach outperforms the baselines by a large margin, there is still much room to improve. One promising direction is to generalize the NoiseAL to multi-label classification or hierarchical text classification problems, which needs to consider the dependence relation and hierarchical structure of labels. To capture such label information and obtain more fruitful label features, we could introduce a powerful label encoder, for example, by applying GCN, GAT, or Graphormer in NoiseAL. However, when we introduce these complex network structures, the computational cost will increase accordingly, which is also our main obstacle to pursuing this front. We plan to conduct such experiments in the future when we have access to better computing resources.
Ethics Statement
This paper will not pose ethical problems or negative social consequences. The datasets used in our paper are all publicly available and are widely adopted by researchers to evaluate models. We do not introduce extra bias or potential bias in the data compared to vanilla/other methods.

heading: Acknowledgments
text: This work is supported by the China Knowledge Centre for Engineering Sciences and Technology (CKCEST-2022-1-7).

heading: A Analysis for Loss-Value-Based Method
text: Recent studies (Zhang et al., 2021a) have illuminated that during SMs (such as non-pretrained neural network models and pretrained language models) training, the loss distributions of clean and noisy samples typically adhere to two Gaussian Distributions. Notably, clean samples tend to have a smaller mean loss value, as shown in Figure 2. While some research based on SM (e.g. Bert), such as SelfMix, has shown promise in leveraging this insight to separate noisy datasets(Qiao et al., 2022), a recent study (Gao et al., 2023) indicates that these fixed-loss-value-based methods fail to help select clean samples in some scenarios. We further empirically analyze why the popular fixed-loss-valuebased methods, such as SelfMix, do not perform well in noisy scenarios (Qiao et al., 2022). For this experiment, we generate two types of synthetic label noise with different ratios (20% asymmetric, 40% asymmetric, 20% symmetric, 40% symmetric), and inject them into SST-2 datasets. As shown in Figure 8, the loss values of the correctly labeled data and the mislabelled data still coincided together during the training process. Then, we observe that the mean loss value gradually decreases during training, as shown in Figure 9.
From the above observation, it can be inferred that the adoption of a uniform fixed threshold throughout training is less than ideal, particularly when we consider SMs’ inherent memorization effect that the memorization strength of samples increases during training (See Figure 4 (a-c)). Our stance is that an intricate interplay exists between memory effects and loss distributions, and that these loss distributions should dynamically evolve during training. Therefore, it is not appropriate to set a fixed loss threshold.
A.1 Comparison of Dividing Dataset Capabilities
From a more nuanced perspective, we assess the capability of SelfMix and our NoiseAL to divide noisy datasets. Specifically, we report the statistical distribution of the different subsets on the Trec dataset under 20% asymmetric label noise (Table 2). According to the results in Table 2, it was observed that NoiseAL outperforms SelfMix in accurately selecting clean samples with correct labels. This ability of NoiseAL is particularly beneficial in constructing demonstrations for LLMs by utilizing clean subsets. Moreover, the subsets divided
by SelfMix are coarse-grained sets, indicating that SelfMix fails to separate clean and noisy samples accurately. Therefore, these fixed-loss-value-based methods are not applicable in our scenario.

heading: B Dataset Details
text: In this work, we select 20ng, SST-2, Trec, AGNews and IMDB for experiments on synthetic noise datasets. For these datasets, we explain the details of synthetic noise generation processes in the following:
Asymmetric noise (Asym) Asymmetric noise attempts to simulate the incorrect classification of classes. Modeling such noise can be achieved by flipping the labels of the samples according to a pre-defined noise level ε ∈ [0, 1) (Zhu et al., 2022):
pflip(ŷ = j|y = i) = {
1− ε, i = j ε, i ̸= j .
Due to these noise generation processes are feature independent (i.e. p(·|y = i, x) = p(·|y = i)), we describe them by an asymmetric noise transition matrix, which can be used to generate noisy labels.
Symmetric noise (Sym) Modeling such noise can be achieved by uniformly flipping the labels of the samples to other classes according to a predefined noise level ε ∈ [0, 1) (van Rooyen et al., 2015):
pflip(ŷ = j|y = i) = { 1− ε, i = j ε
K−1 , i ̸= j ,
where K is the number of classes. Instance-dependent noise (IDN) We follow the noise generation process in existing literature (Bae et al., 2022; Qiao et al., 2022) for IDN generation in our experiments. The Trec dataset comprises only 5452 training samples and exhibits a significant class imbalance. Consequently, when considering a high noise ratio, there’s a possibility that the count of clean samples might be lower than that of generated noisy samples within the long-tailed class. This circumstance renders the classification task meaningless. As a result, we exclusively generate IDN on other datasets expect Trec. The detailed algorithm of IDN noisy label generation is summarized in Algorithm 2.
Regarding the real-world noise, we follow the work in DyGen (Zhuang et al., 2023) to carry out extensive experiments on three real-world noisy datasets: TREC, ChemProt, and SemEval. Specifically, TREC is a question classification
with 6 classes in the weak supervision benchmark (Zhang et al., 2021b); ChemProt is a chemicalprotein interaction dataset with 10 classes; and SemEval is a relation extraction dataset with 9 classes. For these three datasets, we use the pre-defined heuristic rules from prior work DyGen (Zhuang et al., 2023) as weak supervision to obtain noisy labels.
Then, we conduct experiments on “real data”. Specifically, a work (Northcutt et al., 2021a) identifies label errors in the test sets of 10 of the most commonly-used computer vision, natural language, and audio datasets. Further, we viewed 1 the test set errors across the natural language datasets (20news and IMDB datasets) and reproduced the label errors of IMDB datasets and 20news dataset (Mitchell, 1999) by their public code 2. For these label errors, we remove them from the test set and place them in the original training set, constructing a dataset that naturally contains noise (real-noisy datasets). The noisy labels in real-noisy datasets are human-validated via crowdsourcing, which is different from those generated or label-flipped label errors. Strictly speaking, the label errors in realnoisy datasets are the noise of the real world, without any flips and rules. We compare our method NoiseAL with other baselines on real-noisy 20news dataset and IMDB dataset. From the results in Table 7, we found that there isn’t a significant difference in performance among the various methods, which is due to the low proportion of noise (the label errors ratio in 20new test sets is 1.09% and the label errors ratio in IMDB test sets is 3.90%). However, our method still outperforms the others, demonstrating its effectiveness in scenarios with real noise.
Table 6 introduces detailed statistics about all
1https://labelerrors.com 2https://github.com/cleanlab/label-errors
datasets used in our experiments.

heading: C Baselines Details
text: We compare with the most relevant state-of-theart baselines on learning with noisy labels, including: (1) BERT/Base (Devlin et al., 2019) We train the BERT (base model) model fine-tuned only with standard cross-entropy loss without noise-handling; (2) Co-Teaching (Han et al., 2018) concurrently develop two deep neural networks, and allow them to instruct one another using each mini-batch; (3) SCE (Wang et al., 2019) propose Reverse Cross Entropy to boost Cross Entropy symmetrically with a noise-robust learning; (4) ELR (Liu et al., 2020) designs a regularization term that steers the model implicitly forget the false labels; (5) CL (Northcutt et al., 2021b) employ confidence learning to quantify ontological class overlap and moderately increase model accuracy by cleaning data prior to training; (6) SelfMix (Qiao et al., 2022) separates samples via GMM and leverages semi-supervised learning to handle label noise; (7) DyGen (Zhuang et al., 2023) uses the variational auto-encoding framework to infer the posterior distributions of true labels from noisy labels to improve noisy label predictions; (8) LAFT (Wang et al., 2023) also segregates all training samples into different subsets by generating confidences for each sample of training
Algorithm 2 Instance Dependent Noise Generation Input: Clean samples (xi, yi)ni=1, yi ∈ [1,K] ; Noisy ratio τ ; 1: Train an LSTM classifier f ; 2: Get output from an LSTM classifier fxi ∈ RK for all i = 1, . . . , n; 3: Set Nnoisy = 0; 4: while Nnoisy < n× τ do 5: Randomly choose a sample xi, argmax(softmax(fxi)) ̸= yi; 6: set its noisy label ȳi = argmax(softmax(fxi)); 7: Nnoisy = Nnoisy + 1; 8: end while
datasets, which is a way that introduces the external guidance from LLMs. Although the segregation method based on confidence is similar to ours, LAFT ignores the inaccurate of LLM-generated confidences. Compared to LAFT, our methods only utilize the LLM on one subset, which can reduce the cost of LLM expenses. Then, we apply the noise-robust loss functions on LLM-generated labels, which can avoid the additional biases introduced by inaccurate results from LLMs. So our method is more efficient and effective than LAFT.
D Implementation Details
All experiments are evaluated using accuracy on a clean test set, and the reported test performance is selected according to the performance on a clean development set. This applies to both NoiseAL and all baselines. We report the average performance as well as standard deviations using 5 random seeds. We implement our framework with Python 3.7, PyTorch 1.13, and HuggingFace, and train our framework on Nvidia RTX 3090 and Nvidia A100 GPU. In addition, we use Adam (Kingma and Ba, 2015) as an optimizer. In the main experiments, we choose the BERT as the backbone model for NoiseAL and all baseline methods. Further, we verify the generalization of NoiseAL across different PLMs in Appendix I.
Table 8 shows the detailed hyperparameter con-
figuration. The selection of hyperparameters, especially λ, is not trivial. In our experiment, for the small dataset (trec and TREC), we select λ from [0.95, 0.96, . . . , 0.99] for strong model and select λ from [0.1, 0.2, . . . , 0.9] for the weak model; for other datasets, we select λ from [0.95, 0.96, . . . , 0.99] for strong model and select λ from [0.5, 0.6, . . . , 0.9] for weak model.
To see how λ affects the final performance of NoiseAL. The experimental results of adjusting the value of parameter λ for strong and weak models are shown in Figure 6 (Trec datasets) and Figure 7 (AGNews datasets). From these results, we find that the appropriate values λ are not entirely consistent for different datasets. This is because the data distribution of datasets of different types is complex and inconsistent. Hence, we should comprehensively consider various situations and carefully tune the value of λ according to actual datasets. Based on our experience with parameter tuning during experiments, we have the following suggestions:
Determine the range for parameter adjustment: Establishing an approximate range for parameter adjustments based on the size of the dataset. Specifically, in our experiment, for the small dataset (trec and TREC), we select λ from [0.95, 0.96, . . . , 0.99] for BERT and select λ from [0.1, 0.2, . . . , 0.9] for BiLSTM; for other datasets,
we select λ from [0.95, 0.96, . . . , 0.99] for BERT and select λ from [0.5, 0.6, . . . , 0.9] for BiLSTM.
Determine the direction for parameter adjustment: After determining the range of parameter adjustments, we typically set the parameters to start experimenting from the median value of the range interval. Then, for datasets with a smaller sample size, we typically adjust the parameters in the direction of decrease, while for datasets with a larger sample size, we usually adjust the parameters in the direction of increase.
Taking BERT as an example, we first determine the adjustment range for the parameter λ to be [0.95, 0.96, 0.97, 0.98, 0.99], and then we start experimenting with parameter λ at 0.97 and make adjustments from there. For datasets with a smaller sample size (such as SST-2 and Trec), we adjust the parameter λ in the direction of [0.97, 0.96, 0.95]; for datasets with a larger sample size (such as AgNews, 20ng and ChemProt), we adjust the parameter λ in the direction of [0.97, 0.98, 0.99].

heading: E Theoretical Analysis
text: E.1 Risk Minimization problem for losses
Generally, for a dataset D = {(xi, yi)}Ni=1 with data size N , given any loss function L and classifier fθ, we define the loss on D:
L(fθ,D) = ED[L(fθ(x), y)] = Ex,y[L(fθ(x), y)]. (7)
Under the risk minimization problem for losses, our object is to learn a classifier f , which is a global minimizer of D depending on the loss function L. That is to say, we want to obtain the optimal parameters θ∗ of f with loss function L over dataset D, i.e., θ∗ = argmin
θ Lrobust(fθ,D).
E.2 Noise Robustness of Loss Functions
Let Dclean = {(xi, yi)} represent the clean training dataset, and Dnoisy = {(xi, ŷi)} represent the noisy training dataset with noise rate ε, where
ŷi = { yi, 1− ε others, ε .
Previous work (Ghosh et al., 2017; Zhang and Sabuncu, 2018; Xu et al., 2019; Gao et al., 2023) on noise-robust loss functions has shown that the loss function satisfying formula 5 is a robust loss Lrobust, which has the below noise-tolerant property (Gao et al., 2023):
argmin θ Lrobust(fθ,Dclean) = argmin θ Lrobust(fθ,Dnoisy). (8)
E.3 Proof For Noise-tolerant Property
We include the aforementioned work here to ensure comprehensiveness. More precisely, we consider three scenarios of label noise: asymmetric noise, symmetric noise, and instance-dependent noise as described in the following.
Symmetric Noise. In a multi-class classification task with K classes, given a loss function Lrobust satisfying property 5. Then Lrobust is noisetolerant under symmetric label noise if noise rate ε < 1− 1K , the proof as follows:
Lrobust(fθ,Dnoisy) = Ex,ŷ[Lrobust(fθ(x), ŷ)] = ExEy|xEŷ|y,x[Lrobust(fθ(x), ŷ)] = ExEy|x[(1− ε)Lrobust(fθ(x), y)+
ε K − 1 ∑
j ̸=y Lrobust(fθ(x), j)]
= ExEy|x[(1− ε+ ε K − 1 − ε K − 1)
Lrobust(fθ(x), y) + ε K − 1 ∑
j ̸=y Lrobust(fθ(x), j)]
= Ex,y[(1− ε+ ε K − 1 − ε K − 1)
Lrobust(fθ(x), y) + ε K − 1 ∑
j ̸=y Lrobust(fθ(x), j)]
= Ex,y[ K − 1−Kε K − 1 Lrobust(fθ(x), y)] + εC
K − 1 =
K − 1−Kε K − 1 Lrobust(fθ,Dclean) + εC
K − 1 (9)
where C is a constant due to the property 5. Suppose θ∗ is the optimal parameter of f over the clean dataset Dclean, then for any θ:
Lrobust(fθ∗ ,Dnoisy)− Lrobust(fθ,Dnoisy)
= K − 1−Kε
K − 1 (Lrobust(fθ∗,Dclean)− Lrobust(fθ,Dclean))
≤ 0. (10)
Thus, when ε < 1 − 1K , θ∗ is also the optimal parameter of f over the noisy dataset Dnoisy.
Asymmetric Noise. For a loss function Lrobust satisfying property 5. Then Lrobust is noisetolerant under asymmetric label noise if noise rate ε < 12 , the proof as follows:
Lrobust(fθ,Dnoisy) = Ex,ŷ[Lrobust(fθ(x), ŷ)] = ExEy|xEŷ|y,x[Lrobust(fθ(x), ŷ)] = ExEy|x[(1− ε)Lrobust(fθ(x), y)+ ε ∑
j ̸=y Lrobust(fθ(x), j)]
= ExEy|x[(1− ε+ ε− ε Lrobust(fθ(x), y) + ε ∑
j ̸=y Lrobust(fθ(x), j)]
= Ex,y[(1− ε+ ε− ε) Lrobust(fθ(x), y) + ε ∑
j ̸=y Lrobust(fθ(x), j)]
= Ex,y[(1− 2ε)Lrobust(fθ(x), y)] + εC = (1− 2ε)Lrobust(fθ,Dclean) + εC (11)
where C is a constant due to the property 5. Suppose θ∗ is the optimal parameter of f over the clean dataset Dclean, then for any θ:
Lrobust(fθ∗ ,Dnoisy)− Lrobust(fθ,Dnoisy) = (1− 2ε) (Lrobust(fθ∗,Dclean)− Lrobust(fθ,Dclean))
≤ 0. (12)
Thus, when ε < 12 , θ ∗ is also the optimal parameter of f over the noisy dataset Dnoisy. Instance-dependent Noise. For a loss function Lrobust satisfying property 5 and 0 ≤ Lrobust(fθ(x), i) ≤ CK−1 ,∀i ∈ [K]. Suppose θ∗ is the optimal parameter of f over the clean dataset Dclean and Lrobust(fθ∗ ,Dclean) = 0. Then Lrobust is noise-tolerant under instance-dependent noise label if noise rate εj < 1 − εij ,∀j ̸= i,∀i, j ∈ [K], εij represents the probability of class i mislabeled into class j. For instance-dependent noise, we have:
Lrobust(fθ,Dnoisy) = Ex,ŷ[Lrobust(fθ(x), ŷ)] = ExEy|xEŷ|y,x[Lrobust(fθ(x), ŷ)] = ExEy|x[(1− εy)Lrobust(fθ(x), y)+∑
j ̸=y εyjLrobust(fθ(x), j)]
= ExEy|x[(1− εy)(C − ∑
j ̸=y Lrobust(fθ(x), j)])
+ ∑
j ̸=y εyjLrobust(fθ(x), j)]
= Ex,y[(1− εy)(C − ∑
j ̸=y Lrobust(fθ(x), j)])
+ ∑
j ̸=y εyjLrobust(fθ(x), j)]
= Ex,y(C(1− εy)− (1− εy) ∑
j ̸=y Lrobust(fθ(x), j)
+ ∑
j ̸=y εyjLrobust(fθ(x), j))
= CEx,y(1− εy) − Ex,y ∑
j ̸=y (1− εy − εyj)Lrobust(fθ(x), j)
(13)
where C is a constant due to the property 5. Suppose θ† is the optimal parameter of f over the noisy dataset Dnoisy and θ† = argmin
θ Lrobust(fθ,Dnoisy), then:
Lrobust(fθ† ,Dnoisy)− Lrobust(fθ∗ ,Dnoisy) = Ex,y ∑
j ̸=y (1− εy − εyj)(Lrobust(fθ∗(x), j)
− Lrobust(fθ†(x), j)) ≤ 0. (14)
Since we are given Lrobust(fθ∗ ,Dclean) = 0, we have Lrobust(fθ∗(x), y) = 0. Considering the conditions stated before, we can get that Lrobust(fθ∗(x), i) = CK−1 ,∀i ̸= y. If we assume that 1− εy − εyj > 0, in order the Eq. 14 to hold, we must have Lrobust(fθ†(x), i) = CK−1 ,∀i ̸= y, which implies Lrobust(fθ†(x), y) = 0 due to the symmetric property of Lrobust. Thus, when εj < 1− εij , θ† is also the optimal parameter of f over the clean dataset Dclean.

heading: F Proof For Reversed Cross-entropy Loss
text: Theorem. The reversed cross-entropy loss function satisfies formula 5 and has the noisy-tolerant property 8.
Proof. For the input x and its label y, the predicted probability of x for each label k ∈ [1,K] can be represented as p(k;x) = e
fθ(k;x)∑K k=1 e fθ(k;x) . q(k|x)
is the ground-truth distribution over labels, and∑K k=1 q(k|x) = 1. If the ground-truth label is y, then q(y|x) = 1 and q(k|x) = 0 for all k ̸= y. Based on it, we can obtain the reversed crossentropy loss function Lrce:
Lrce(fθ(x), y) = − K∑
k=1
p(k;x) log q(k|x)
= −p(y;x) log q(y|x)− K∑
k ̸=y p(k;x) log q(k|x)
= − K∑
k ̸=y p(k;x) log q(k|x)
= − K∑
k ̸=y p(k;x) log(0). (15)
We approximate the log(0) as a constant A, then∑K k=1 Lrce(fθ(x), y) = −(K − 1)A, which satisfies formula 5 and C=−(K − 1)A.

heading: G More detailed Results
text: We report the detailed performance (accuracy with standard deviation %) on Trec ( refer to Table 9), AGNews ( refer to Table 10), IMDB (refer to Table 11), SST-2 ( refer to Table 12), 20ng (refer to Table 13).

heading: H More Ablation Experiments
text: To evaluate the contribution of each component in our NoiseAL, we conduct ablation studies on all datasets: Trec (refer to Table 17), IMDB (refer to Table 21), SST-2 (refer to Table 18), AGNews (refer to Table 19), 20ng ( refer to Table 20).
H.1 Effect of Robust Loss Function Due to LLMs being unable to generate correct labels for each sample in the subset P , we utilize the reversed cross-entropy loss functions to better learn from P with a certain noise ratio. We conduct an ablation experiment (refer to Table 22) to verify the effectiveness of this robust loss function by replacing it with cross-entropy loss functions.
H.2 Further Analysis for Co-prediction Network
In our paper, we design a co-prediction network consisting of a strong and weak model. For a strong model, pretrained language models (PLMs) might be a better choice since the whole training process can be divided into two stages, the wrong labels do not corrupt the pre-training process, which makes PLMs more robust again label noise (Qiao et al., 2022) than other traditional networks (such
as BiLSTM, and Text-CNN). Therefore, we select BERT as the strong model and BiLSTM as the weak model. These two models have different fitting speeds for label noise, which can provide different predictions to avoid overfitting.
To evaluate the impact of strong model or weak model selection on performance, we conduct other approaches to reduce overfitting. To be specific, we first utilize two BERT with different dropout ratios to construct the co-prediction Network, then we select the Text-CNN as the weak model. From the
experimental results in Table 15 (the Trec dataset) and Table 16 (the SST-2 dataset), we have the following observations: (1) the co-prediction Network consisting of BERT (dropout:0.1) and BERT (dropout:0.5) achieves the best result under 20% asymmetric label noise; (2) within the same model architecture (BERT+BERT), the larger the gap in dropout rates (0.1 and 0.9), the better the network performance; (3) eventually, the combination of different model network (BERT+LSTM) outperforms the use of a single model type (BERT+BERT). With these observations in mind, we believe that combining models with more distinct architectures can provide more diversified information to prevent confirmation bias and noise overfitting. On the whole, our selection of strong and weak models is optimal. H.3 Further Analysis for Prompt contents In Table 14, we ablate the prompt contents in the default settings by: (a) removing the Task description (r.m. a); (b) removing the Chain-of-Thought
(COT) (r.m. b); (c) removing the Demonstration (r.m. c); (d) replacing the feature-aware example sampling with random example sampling (r.p. d). The results yield the subsequent observations: Firstly, the performance of our NoiseAL is heavily influenced by demonstrations. This is due to the fact that they possess crucial information that enables ChatGPT to comprehend our tasks. Secondly, the COT is necessary for ChatGPT to activate its capability to adapt to our task, and enhance the accuracy of answer generation. Thirdly, the task prompt is of less importance, indicating that ChatGPT is capable of understanding the task directly from the demonstration. Finally, the feature-aware example sampling strategy is important to the performance of our NoiseAL, especially under a high noise ratio.

heading: I Additional Results of More PLMs
text: To verify the effectiveness of our proposed NoiseAL on other PLMs, we perform experiments on RoBERTa (refer to Table 23, Table 24, Table 25 and Table 26). All experiment results show the improvement brought by NoiseAL is significant.

heading: J Prompts Details
text: J.1 Prompts Structure
Our LLMs prompt consists of the following three components:
(1) Task description, which describes the task. For different classification tasks, e.g., question classification, sentiment classification, topic classification, etc, the descriptions are different.
Trec: You are a text classifier and your task is to classify a given text according to candidate categories. The true category must be one of the candidate categories.
SST-2: You are a sentiment classifier and your task is to classify a given text according to candidate sentiment. Your answer can be either positive or negative.
IMDB: You are a Sentiment classifier and your task is to classify a given text according to candidate labels. Your answer can be either positive or negative.
Agnews: You are a text classifier and your task is to classify a given text according to candidate topics. Your answer must be exactly one of [’World’, ’Sports’, ’Business’, ’Science/Technology’].
Chemprot: You are a text classifier and your task is to classify a given text according to candidate labels. Your answer must be exactly one of [’Part of’,’Regulator’,’Upregulator’,’Downregulator’,’Agonist’,’Antagonist’,’Modulator’, ’Cofactor’,’Substrate/Product’,’NOT’].
Semeval: You are a text classifier and your task is to classify a given
text according to candidate labels. Your answer must be exactly one of [’Cause-Effect’,’Component-Whole’, ’Content-Container’,’Entity-Destination’, ’Entity-Origin’,’Instrument-Agency’,’Member-Collection’,’Message-Topic’,’ProductProducer’]. 20ng: You are a text classifier and your task is to classify a given news according to candidate categories. The true category must be one of the candidate categories.
(2) Demonstration, which consists of a sequence of annotated samples and is only needed for the few-shot learning setup. Two functions of demon-
stration are as follows: (a) it gives the LLM evidence to refer to when making decisions, which will greatly improve performance; and (b) it establishes an output format that the LLM’s outputs must adhere to, allowing the output—which is natural language—to be readily converted into labels (Sun et al., 2023). For the demonstration sampling, there are some common methods including:
Random Sampling Sampling k examples at random from the training dataset.
kNN Sampling An inherent drawback of random sampling is the absence of assurance that the chosen samples possess semantic relevance to the input sequence. An effective approach is to employ kNN search to select samples that closely resemble the test input. Given a test input xi ∈ P , we can obtain its text feature f(xi) and calculate the cosine similarity of the text feature between xi and other training text xj ∈ R, then sample top-K nearest neighbors to form in-context examples:
E = argTopN j∈{1,2,...,|R|}
f(xi) T f(xj)
∥f(xi)∥∥f(xj)∥ , (16)
where E is an index set of the top-K similar samples in clean set R. Note that the text features of datasets can be computed and stored beforehand, allowing efficient sampling.
(3) Input, which is the test text to classify.
(4) Chain-of-Thought (COT), our initial investigation indicates that although this prompt design can already yield reasonable results, the prediction is unstable. Specifically, we observe variations in the results when we randomly rearrange the order of the demonstrations. This indicates that the model faces difficulty comprehending the user’s viewpoint accurately. To address this instability, we incorporate Chain-of-Thought (COT) reasoning into our setting.
J.2 Prompts Cases Here, we present the prompts we design for the LLMs, including SST-2 (Table 27), 20ng (Table 28), Trec (Table 29), AGNews (Table 30), IMDB (Table 31). Besides, we also present the good case (Table 32, 33) and bad case (Table 34) generated by LLMs for some test examples under one-shot settings, respectively. Given the example “i will be .”, there is a sentence with the sentiment "positive", but this example is classified as "neutral" (as shown in Table 34). In this case, we can observe that LLMs sometimes can not accurately identify the category, which indicates that the label generated by LLMs may be semantically reasonable but without being fully aligned with the annotations in the dataset.
IMDB
Task description:
You are a Sentiment classifier and your task is to classify a given text according to candidate labels. Your answer can be either positive or negative.
Demonstration:
Text: I read some previous comments stating that this movie loses steam towards the end of the movie and also that it has a similar ending to Cape Fear. I completely disagree. I’m going to give a simple review for the normal moviegoers out there. I thought the casting was perfect. I thought this was one of Ed Harris’s best performances. What an evil psychopath!!! I have a lot of respect for his acting after viewing his performance i?
Candidate label: positive, negative
Label: Positive
Inputs:
Text: Woah! Is one thing I can say about this movie. Personally I’m one of those people who loves cats so that would have been a big down side to the movie, but I loved it how cats from every were got their revenge at the end.I liked the movie, but I have to admit it was because I found Brian Krause who played Charles Brady very very irresistible, I guess lust got the better of me while watching this movie.There was one thing that I?
Candidate label: positive, negative

heading: Future Work
text: We plan to conduct such experiments in the future when we have access to better computing resources