Meta Self-Refinement for Robust Learning with Weak Supervision (2023)

Abstract: Training deep neural networks (DNNs) under weak supervision has attracted increasing research attention as it can significantly reduce the annotation cost. However, labels from weak supervision can be noisy, and the high capacity of DNNs enables them to easily overfit the label noise, resulting in poor generalization. Recent methods leverage self-training to build noise-resistant models, in which a teacher trained under weak supervision is used to provide highly confident labels for teaching the students. Nevertheless, the teacher derived from such frameworks may have fitted a substantial amount of noise and therefore produce incorrect pseudo-labels with high confidence, leading to severe error propagation. In this work, we propose Meta Self-Refinement (MSR), a noise-resistant learning framework, to effectively combat label noise from weak supervision. Instead of relying on a fixed teacher trained with noisy labels, we encourage the teacher to refine its pseudo-labels. At each training step, MSR performs a meta gradient descent on the current mini-batch to maximize the student performance on a clean validation set. Extensive experimentation on eight NLP benchmarks demonstrates that MSR is robust against label noise in all settings and outperforms state-of-the-art methods by up to 11.4% in accuracy and 9.26% in F1 score.

## introduction fine-tuning pre-trained language models has led to great success across nlp tasks. nonetheless, it still requires a substantial amount of manual labels to achieve satisfying performance on many tasks. in reality, obtaining large amounts of high-quality labels is costly and labor-intensive @xcite . for certain domains, it is even infeasible due to legal issues and lack of data or domain experts. weak supervision is a widely-used approach for reudcing such cost by leveraging labels from weak sources, e,g., heuristic rules, knowledge bases or lower-quality inexpensive crowdsourcing @xcite @xcite . it has raised increasing attention in recent years, and efforts have been made to quantify the progress on weakly supervised learning, like the wrench benchmark @xcite . although weak labels are inexpensive to obtain, they are often noisy and inherit biases from weak sources. training neural networks with weak labels is challenging because of their immense capacity, which leads them to heavily overfit to the noise distribution, resulting in inferior generalization performance @xcite . various approaches have been proposed to tackle this challenge. earlier research focused primarily on simulated noise @xcite , required prior knowledge @xcite or relied on context-free aggregation rules without leveraging modern pre-trained language models @xcite . recently, @xcite proposed a contrastive regularized self-training framework that achieved state-of-the-art (sota) performance in several nlp tasks from the wrench benchmark. it trains a teacher network on weak labels, then iteratively applies the teacher to produce pseudolabels for training a new student model. to prevent error propagation, it filters the pseudo-labels with the model confidence scores and adds contrastive feature regularization to enforce more distinguishable representations. however, we find that this approach is effective on easy tasks but fragile on challenging ones, where the initial teacher model already have memorized a substantial amount of biases with high confidence. consequently, confidence-based filtering is misleading and all future students will be reinforced with these initial wrong biases from the teacher. to address this weakness, one strategy is learning to reweight the pseudo-labels with meta learning @xcite @xcite . by this means, sample weights are dynamically adjusted to minimize the validation loss instead of prefixed with potentially misleading confidence scores. nevertheless, if the initial teacher is weak and mostly produces incorrect pseudo-labels, simply reweighting the labels does not suffice to extract enough useful training signals. in this paper, we propose meta self-refinement (msr) to go one step further. the teacher is jointly trained with a meta objective such that the student, after one gradient step, can achieve better performance on the validation set. in each training step, a copy of the current student performs one step of gradient descent based on the teacher predictions. the teacher will then update itself towards the gradient direction that minimizes the validation loss of the student. finally, the actual student is trained by the updated teacher. in msr, teacher's predictions are iteratively refined, instead of only "reweighted", based on the meta objective. this will enable more efficient data utilization since the teacher still has the opportunity to refine itself to provide the proper training signal, even if its initial output label is wrong. to further stabilize the training, we enhance our framework with confidence filtering when teaching the student and apply a linearly scaled learning rate scheduler to the teacher. in summary, the main contributions are as follows: 1) we propose a meta-learning based selfrefinement framework, msr, that allows robust learning with label noise induced by weak supervision. 2) we analyze and quantify how label noise impacts model predictions and representation learning. we find existing methods become less effective in challenging cases when the label noise can be easily fitted. in contrast, msr is more stable and learns better representation. 3) extensive experiments demonstrate that msr consistently reduces the negative impact of the label noise, matching or outperforming sotas on six sequence classification and two sequence labeling tasks. foot_0 2 related work learning with noisy labels. learning in the presence of label noise is a long-standing problem @xcite . @xcite show that deep neural networks can memorize arbitrary noise during training, resulting in poor generalization. noise-handling techniques -by modeling @xcite @xcite or filtering @xcite the noisy instances -are proposed to conquer the label noise. while being effective, they typically assume that the noise is feature-independent which may oversimplify the noise generation process in realistic settings @xcite . recently, realistic and feature-dependent noise induced by weak supervision has received significant attention. to handle this type of noise, @xcite propose an implication loss that jointly denoises the noisy labels and weak sources. @xcite denoise the weak label by considering the reliability of different weak sources and aggregating them into one cleaned label. @xcite release a benchmark, wrench, including various weakly supervised datasets in both text and image domains. self-training. self-training @xcite is a simple yet effective framework that is commonly used in semi-supervised learning (ssl). it typically trains a teacher model to provide pseudo-labels for the student model. different methods have been proposed for better generalization @xcite @xcite . recently, selftraining has been adopted to tackle weak supervision. @xcite train a teacher network that aggregates weak labels to form highquality pseudo-labels for the student. @xcite ; @xcite initialize the teacher model by training a classifier directly on the weak labels, they apply early stopping to prevent this initial teacher from memorizing the label noise. the student is then trained on the highly confident pseudolabels provided by the teacher. while the core assumption of self-training -that highly confident pseudo-labels are reliable -is generally valid in ssl, it may not be true for feature-dependent noise induced by weak supervision, especially when the noise is easy to learn. in this case, self-training inevitably suffers more from error propagation and fails to train robust models. ## problem formulation let x and y be the feature and label space, respectively. in standard supervised learning, one is given a clean dataset d @xmath0, y i )} n @xmath1, where n is the number of samples. the clean labels y i are supposed to be annotated by human experts. in weak supervision, a dataset is labeled by weak sources rather than humans. weak sources can have diverse forms like lexical rules, knowledge bases, pre-trained models, lower-quality inexpensive crowdsourcing, etc. figure 1 shows an example of text labeled via weak supervision. compared to manual annotations, weak labels contain more mistakes. we denote the dataset labeled by weak sources by d @xmath2, ŷi )} n @xmath3. 2 since weak sources might not cover all data, we may have a set of unlabeled data d u in addition to d w . we use d @xmath4. moreover, as we do not make any assumption on the quality of the weak labels, their distribution can deviate arbitrarily from the distribution of clean labels. learning with only weak labels can lead to unbounded model errors @xcite . hence, following standard practice in weak supervision, we assume the access to a small clean validation set ## meta self-refinement we propose a novel meta-learning based framework, named meta self-refinement (msr), to tackle the label noise induced by weak supervision. in contrast to conventional self-training methods, where the teacher model is fixed after being trained on weakly labeled data, msr enables the teacher to refine itself based on student performance on the clean validation set, yielding higher-quality labels and more accurate confidence estimates. in this section, we first provide an overview of its training objective (section 4.1), then go into the training details (section 4.2). figure 2 illustrates the full training process. ## results comparison with baselines. error decomposition. let y ′ , ŷ, y denote the model prediction, the noisy weak label, and the clean label, respectively. to investigate how the label noise influences the model predictions, we decompose model prediction errors into three types: figure 3 visualizes the three types of errors on three challenging datasets: trec, hausa and conll-03. the blue bars represent model robust- ## conclusion we present msr, a meta-learning based selfrefinement framework that enables robust learning with weak labels. unlike conventional self-training which relies on a fixed teacher, msr dynamically refines the teacher based on the student's performance on the validation set. to further suppress error propagation, we introduce a learning rate scheduler to the teacher and add confidence filtering to the student. we demonstrate that our framework performs on par with or better than current sotas on both sequence classification and labeling tasks.