Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding (2023)

Abstract: The recent success of large pre-trained language models (PLMs) heavily hinges on massive labeled data, which typically produces inferior performance in low-resource scenarios. To remedy this dilemma, we study self-training as one of the predominant semi-supervised learning (SSL) approaches, which utilizes large-scale unlabeled data to generate synthetic examples. However, too many noisy labels will hurt the model performance, and the self-training procedure requires multiple training iterations making it more expensive if all the model parameters of the PLM are updated. This paper presents UPET, a novel Uncertainty-aware Parameter-Efficient self-Training framework to effectively and efficiently address the labeled data scarcity issue. Specifically, we incorporate Monte Carlo (MC) dropout in Bayesian neural network (BNN) to perform uncertainty estimation for the teacher model and then judiciously select reliable pseudo-labeled examples based on confidence and certainty. During the student training, we introduce multiple parameter-efficient learning (PEL) paradigms that allow optimizes only a small percentage of parameters. We also propose a novel Easy-Hard Contrastive Tuning to enhance the robustness and generalization. Extensive experiments over multiple downstream tasks demonstrate that UPET achieves a substantial improvement in terms of performance and efficiency. Our codes and data are released at https: //github.com/wjn1996/UPET.

## introduction pre-trained language models (plms) have become the imperative infrastructure in a series of downstream natural language understanding (nlu) tasks @xcite @xcite , aiming at capturing prior knowledge * corresponding author. by pre-training over large-scale unsupervised corpora and fine-tuning on the target tasks. however, the conventional fine-tuning approaches heavily depend on the time-consuming and labor-intensive process of data annotation, which could be even more bothersome in some real-world scenarios and typically produces inferior performance in few-shot settings @xcite . recently, self-training @xcite has been presented to address the labeled data scarcity issue by leveraging the large-scale unlabeled data in addition to labeled data, which is one of the mature paradigms in semisupervised learning @xcite @xcite @xcite . a teacher model is fine-tuned on the few-shot labeled data, then the pseudo label of each unlabeled example can be generated. after that, a student model can learn the knowledge derived from the large-scale pseudo-labeled data, leading to better performance near to full-supervised learning. previous works typically use self-training in conjunction with large plms to endow the model with the ability of fewshot learning. despite the big success, we observe that there are still two challenges. 1) the pseudolabeled data consists of too many noises, inevitably degrading the model performance due to confirmation bias @xcite . 2) the procedure of self-training is too expensive when updating all parameters of the large plm foot_0 @xcite . fortunately, parameter-efficient learning (pel) opens up the possibility of attaining near state-ofthe-art performance, whilst adding only a few parameters per task @xcite @xcite . notable pel-based methods include ptuning @xcite , prefix-tuning @xcite , adapter @xcite , bitfit @xcite , lora @xcite , etc. yet, it is unclear how these pelbased methods can be applied to self-training. in this paper, we develop a novel uncertaintyaware parameter-efficient self-training framework (upet) for improving self-training through two perspectives, i.e., effectiveness and efficiency. to reach these goals, we respectively present two novel techniques, including reliable example sampling (res) and efficient robust tuning (ert). the goal of res is to explicitly mitigate the effect of label noises. concretely, we obtain the prediction probability distribution over all unlabeled data derived from the teacher model. then, we utilize monte carlo (mc) dropout technique in bayesian neural network (bnn) @xcite to estimate the uncertainty of each unlabeled example. to this end, the example with higher confidence and certainty will be judiciously selected as the reliable pseudo-labeled data. in ert, we aim to leverage pel paradigms to train a robust student model over reliable pseudolabeled data. we design multiple pel-based model architectures for the student model that only need to update a small scope of tunable parameters in plm during iterative self-training. additionally, we introduce easy-hard contrastive tuning to improve the robustness of the parameter-efficient model, which can be viewed as a regularization in the semantic space that keeps the noisy labels away from the reliable examples. we conduct extensive experiments over multiple nlu tasks. results show that upet outperforms strong baselines in terms of both effectiveness and efficiency. the improvement is consistent in different settings with different pel methods and the number of labeled data. our key contributions to this field are summarized as follows: 1) we use parameter-efficient learning of plms in conjunction with uncertainty estimation to form an efficient and effective self-training framework. 2) to better improve the robustness of the parameter-efficient model, we introduce easy-hard contrastive learning. 3) extensive experiments among a wide range of tasks demonstrate that our proposed framework outperforms prevailing strong baselines. parameter-efficient learning. pel is to optimize a small portion of parameters while keeping the model backbone frozen, which aims at improving the training efficiency and preserving the model's effectiveness @xcite . @xcite integrates task-specific neural modules called adapters into plms, and only these adapters are updated during fine-tuning. ptuning @xcite and prefix-tuning @xcite respectively introduce a lightweight prefix module into the input layer and each transformer layer, enabling efficient training over these prefix modules. notable pel-based models also include bitfit @xcite , lora, etc. this paper integrates pel into self-training to improve its efficiency. ## upet: the proposed method we thus propose the upet framework which consists of two novel techniques, i.e., reliable example sampling (res) and efficient robust tuning (ert). the framework overview is illustrated in figure 1 and the detailed algorithm is shown in appendix b. ## conclusion in this paper, we introduce a novel uncertaintyaware parameter-efficient self-training framework (upet) to better improve the effectiveness and efficiency of self-training. in upet, we use uncertainty estimation to judiciously select reliable pseudo-labeled examples to explicitly alleviate the noisy label problem. to make self-training more efficient, we integrate multiple parameter-efficient paradigms into self-training. to further improve the performance, we also present easy-hard contrastive tuning to enhance the robustness and reduce the over-fitting problem. in the future, we will extend our framework to other complex tasks, such as sequence labeling, question answering, etc.