Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance (2023)

Abstract: Adopting a two-stage paradigm of pretraining followed by fine-tuning, Pretrained Language Models (PLMs) have achieved substantial advancements in the field of natural language processing. However, in real-world scenarios, data labels are often noisy due to the complex annotation process, making it essential to develop strategies for fine-tuning PLMs with such noisy labels. To this end, we introduce an innovative approach for fine-tuning PLMs using noisy labels, which incorporates the guidance of Large Language Models (LLMs) like ChatGPT. This guidance assists in accurately distinguishing between clean and noisy samples and provides supplementary information beyond the noisy labels, thereby boosting the learning process during fine-tuning PLMs. Extensive experiments on synthetic and real-world noisy datasets further demonstrate the superior advantages of our framework over the state-of-the-art baselines.

## introduction in recent years, the development of language models has significantly expanded the applications within the field of natural language processing (nlp). fine-tuning pretrained language models (plms) like bert @xcite for specific downstream tasks has become an essential step in real-world implementations @xcite . in general, achieving significant performance gains in fine-tuning plms necessitates the availability of task-specific data for finetuning @xcite . however, obtaining high-quality labeled datasets for this purpose poses significant challenges due to the expensive, complex, and labor-intensive nature of the annotation process @xcite . for example, large-scale datasets, often derived from web-crawling @xcite or crowd-sourcing @xcite @xcite , frequently suffer from the presence of noisy labels. in the context of learning from noisy labels, an intuitive approach is to separate clean samples from the noisy ones in the training set for model training @xcite . for the remaining noisy samples, a prevalent strategy is to pseudo-label them based on model predictions to reduce the adverse impact of noise @xcite . however, it remains challenging when applying this paradigm to plms. this is because plms, equipped with prior knowledge encoded in a large size of parameters, tend to easily memorize noisy samples early on during the fine-tuning process. as illustrated in figure 1 , plms exhibit similar prediction confidences for both clean and noisy samples, which will result in two challenges when utilizing existing methods: (1) existing approaches often rely on confidences generated by models trained on potentially noisy samples @xcite . however, as shown in figure 1 , it is challenging to accurately distinguish clean and noisy samples solely based on confidences. (2) existing methods are susceptible to erroneous information during pseudo-labeling, as they directly utilize model predictions as pseudolabels, which can be detrimental when the predictions are inaccurate. plms, being capable of easily remembering noisy samples, further exacerbate the risk of capturing and amplifying erroneous information during pseudo-labeling. for example, in the 20ng dataset @xcite , when an "autos" sample is assigned a wrong label "hardware", plms can easily memorize the erroneous information and thus cannot infer the correct labels in predictions, resulting in incorrect pseudo-labeling. to overcome these challenges, we propose a novel framework, named laft, that harnesses the power of large language models (llms) for fine-tuning plms. we leverage llms trained on extensive corpora and fine-tuned with human instructions, such as gpt4 @xcite , to obtain external guidance in the form of confidence values. our approach addresses the first challenge of separating reliable samples by utilizing llmgenerated confidences for each class of all training samples. by comparing these confidences with the assigned labels (i.e., the given noisy labels) of training samples, we categorize them into three disjoint subsets: the easy clean (ec) set, the hard clean (hc) set, and the true noisy (tn) set. regarding the second challenge, we propose a novel method to incorporate llm-generated confidence scores as robust supervision information for all samples to ensure that plms learn useful information from them. as llm-generated confidences are not affected by label noise, they can provide potentially relevant labels that are useful even if they are not entirely accurate. in summary, our contributions are as follows: ## problem definition in this work, we study the problem of fine-tuning plms for text classification with noisy labels. formally, consider a noisy training dataset containing n samples: d @xmath0, y i ), @xmath1, 2, . . . , n}, where x i is an input sample and y i denotes the assigned label of x i . note that y i is potentially corrupted, and we denote the true label of x i as y i , which is inaccessible during fine-tuning. specifically, we aim to fine-tune plms with samples in d tr to achieve satisfactory prediction performance on test samples, while utilizing llms as external guidance. notably, the llms remain fixed in our framework, which means we can also use blackbox llms. in practice, we implement plms for text classification by attaching an additional classifier, which will take the output of the plm as input and generate class probabilities for classification. ## methodology the overall framework of laft is illustrated in fig. 2 . in particular, we propose to divide all training samples into three subsets based on the accordance among llm-generated confidences, plmgenerated confidences, and the assigned labels of samples. in particular, we query an llm to provide confidences for each training sample, spanning all classes. combining confidences obtained from both llms and plms, we perform two steps of separation to segregate the entire training set into three subsets: easy clean (ec) set, hard clean (hc) set, and true noisy (tn) set. each of these subsets, displaying unique behaviors, is then subjected to specific noise-robust fine-tuning strategies. ## conclusion this paper delves into the issue of fine-tuning pretrained language models (plms) with noisy labels for text classification tasks. we address the detrimental effects of label noise by harnessing external guidance from large language models (llms) in the form of confidences. our approach entails a two-step separation strategy that accurately segregates all training samples into three distinct subsets, each treated with innovative noise-resilient strategies. in summary, our framework adeptly finetunes plms in the face of noisy samples, requiring minimal external guidance from llms.