Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLM s-Powered Assistance (2024)

Abstract: Learning from noisy labels (LNL) is a challenge that arises in many real-world scenarios where collected training data can contain incorrect or corrupted labels. Most existing solutions identify noisy labels and adopt active learning to query human experts on them for denoising. In the era of large language models (LLMs), although we can reduce the human effort to improve these methods, their performances are still subject to accurately separating the clean and noisy samples from noisy data. In this paper, we propose an innovative collaborative learning framework NoiseAL based on active learning to combine LLMs and small models (SMs) for learning from noisy labels. During collaborative training, we first adopt two SMs to form a co-prediction network and propose a dynamic-enhanced threshold strategy to divide the noisy data into different subsets, then select the clean and noisy samples from these subsets to feed the active annotator LLMs to rectify noisy samples. Finally, we employ different optimization objectives to conquer subsets with different degrees of label noises. Extensive experiments on synthetic and real-world noise datasets further demonstrate the superiority of our framework over state-of-the-art baselines.

## introduction the core of deep learning models' success lies in the precision of large amounts of annotated data, which can be a complex and labor-intensive process. in practical scenarios, some researchers often collect datasets by web-crawling @xcite or crowd-sourcing @xcite to reduce the burden of annotation. however, the obtained datasets frequently suffer from the presence of noisy labels, which will mislead the learning patterns and subsequently result in incorrect predictions. existing lnl methods include designing robust loss functions @xcite , estimating noise transition matrices @xcite , etc. among them, active learning @xcite is a popular solution that provides more accurate labels for noisy samples by querying experts, but it still requires human effort in the annotation. recently, large language models (llms), such as chatgpt, have exhibited strong zero-shot learning ability, avoiding human costs for text annotation tasks. some studies indicate that zero-shot chatgpt classifications outperform crowd workers in some domains @xcite . despite the promise, further studies @xcite observe that llms tend to underperform compared to bert supervised on complex datasets. this observation is further supported by our own empirical studies ( before the era of llms, the traditional methods @xcite @xcite to divide the noisy data depending on the loss value. the key idea is to set a fixed threshold for loss value, such that the clean samples are associated with a smaller mean loss value and noisy ones with bigger values. however, a recent study @xcite argues that these methods fail to select clean samples under some synthetic noisy scenarios. our empirical studies verify this opinion and further observe the mean value of samples with clean labels and noisy labels gradually decreases during training (see appendix a). so it is not ideal to set a fixed threshold, particularly when we consider sms'(e.g., bert, bilstm) inherent memory effect that the memorization strength of samples increases during training (see figure 4 (a-c )). besides, some memory effect-based work indicates that the confirmation bias @xcite ) makes sm memorize itself mistakes and hard to discern noisy labels by itself @xcite . while previous methods could partition datasets, they often struggle to accurately select clean samples in different lnl scenarios, thus failing to construct high-quality icl for querying llms. to overcome the above limitations of traditional selection methods, we present a novel collaborative learning paradigm, noiseal, that utilizes active learning to marry llms with sms by filtering noisy data with the help of sms and distilling the related knowledge from the llms. specifically, we first adopt two sms to form a co-prediction network, focusing on different predictive capabilities and producing multiple predictions for each text to mitigate confirmation bias. our intuition is that the presence of noisy labels may make it difficult for the two sms to arrive at a consensus on the outputs. in this scenario, one sm may start fitting the noise before another, leading to divergent copredictions. with two prediction results, we then propose a dynamic-enhanced threshold strategy following memory effect to select clean and noisy samples to llms. to integrate sms and llms as a whole, we design a collaborative training frame-work where sms operate as filters to divide the noisy dataset into different subsets and the llm acts as an active annotator to correct noisy samples from subsets. during collaborative training, sms can learn the knowledge of llms to boost their performance, while llms can also benefit from the divided clean samples to boost their icl ability. as shown in figure 1 , noiseal achieves competitive results compared with its counterparts supervised on the ground truth label under some scenarios. overall, our main contributions are: ## background let @xmath0, y i )} n @xmath1, where x is the text, y is its label, and n is the data size. denote f θ (x) ∈ r k as the output of the model f with a linear layer (i.e., the classifier), where θ is its parameters and k is the number of classes. the confidence of x for each label @xmath2, k] can be represented as follows: (k;x) . generally, as the sm's memory strength regarding a text increases, the prediction confidence also correspondingly rises. based on this association, we quantify the sm's memorization strength through a confidence metric. thus, we regard p(k; x) as the value of memorization strength. for the text classification, if a model memorizes a text, its confidences p(k; x) of k exceed a certain threshold or reach the maximum. ## method in this section, we introduce our proposed framework noiseal which explores the opportunity for active learning to address lnl problems in the llms era. although llms can generate new labels for noisy samples directly, we still need sms to help separate noisy data. in each training loop, we alternate the following steps: (1) training a coprediction network consisting of two sms to divide the noisy data into different subsets. (2) selecting clean and noisy samples from subsets, the noisy samples are corrected by active querying llms, and the clean samples are used to prompt icl. figure 3 shows the framework of noiseal. and its overall pipeline is shown in algorithm 1. following that, we will provide a detailed explanation of our proposed noiseal framework. ## conclusion in this paper, we focus on solving a challenging text classification problem with noisy labels. we propose a novel framework called noisyal that introduces active learning to combine the nonpretrained model (bilstm), plms (bert), and llms for learning from noisy labels. the key idea of noisyal is to separate noisy datasets by sms (bilstm and bert) and distill related knowledge from llms with a collaborative framework, where sms act as filters to select clean and noisy samples for llms and the llm is employed as an active annotator. the results of our experiments demonstrate the versatility of our method, as it significantly improves the accuracy of benchmark datasets with both synthetic and real-world noise. we hope that our work can inspire people's interest in developing new active denoising algorithms by collaboratively utilizing llms and sms.