Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach (2021)

Abstract: Fine-tuned pre-trained language models (LMs) have achieved enormous success in many natural language processing (NLP) tasks, but they still require excessive labeled data in the fine-tuning stage. We study the problem of fine-tuning pre-trained LMs using only weak supervision, without any labeled data. This problem is challenging because the high capacity of LMs makes them prone to overfitting the noisy labels generated by weak supervision. To address this problem, we develop a contrastive self-training framework, COSINE, to enable fine-tuning LMs with weak supervision. Underpinned by contrastive regularization and confidence-based reweighting, our framework gradually improves model fitting while effectively suppressing error propagation. Experiments on sequence, token, and sentence pair classification tasks show that our model outperforms the strongest baseline by large margins and achieves competitive performance with fully-supervised fine-tuning methods. Our implementation is available on https://github.com/yueyu1030/COSINE .

## introduction language model (lm) pre-training and fine-tuning achieve state-of-the-art performance in various natural language processing tasks @xcite @xcite . such approaches stack task-specific layers on top of pre-trained language models, e.g., bert @xcite , then fine-tune the models with task-specific data. during fine-tuning, the semantic and syntactic knowledge in the pre-trained lms is adapted for the target task. despite their success, one bottleneck for fine-tuning lms is the requirement of labeled data. when labeled data are scarce, the fine-tuned models often suffer from degraded performance, and the large number of parameters can cause severe overfitting @xcite . ## background in this section, we introduce weak supervision and our problem formulation. weak supervision. instead of using humanannotated data, we obtain labels from weak supervision sources, including keywords and semantic rules foot_1 . from weak supervision sources, each of the input samples @xmath0, where y is the label set and ∅ denotes the sample is not matched by any rules. for samples that are given multiple labels, e.g., matched by multiple rules, we determine their labels by majority voting. problem formulation. we focus on the weaklysupervised classification problems in natural language processing. we consider three types of tasks: sequence classification, token classification, and sentence pair classification. these tasks have a broad scope of applications in nlp, and some examples can be found in formally, the weakly-supervised classification problem is defined as the following: given weaklylabeled samples x @xmath1, y i )} l @xmath2, we seek to learn a classifier f (x; θ) : @xmath3. here @xmath4, 2, • • • , c} is the label set, where c is the number of classes. ## method our classifier @xmath5: bert is a pre-trained language model that outputs hidden representations of input samples, and g is a task-specific classification head that outputs a c-dimensional vector, where each dimension corresponds to the prediction confidence of a specific class. in this paper, we use roberta @xcite as the realization of bert. the framework of cosine is shown in figure 1 . first, cosine initializes the lm with weak labels. in this step, the semantic and syntactic knowledge of the pre-trained lm are transferred to our model. then, it uses contrastive self-training to suppress label noise propagation and continue training. ## conclusion in this paper, we propose a contrastive regularized self-training framework, cosine, for finetuning pre-trained language models with weak supervision. our framework can learn better data representations to ease the classification task, and also efficiently reduce label noise propagation by confidence-based reweighting and regularization. we conduct experiments on various classification tasks, including sequence classification, token classification, and sentence pair classification, and the results demonstrate the efficacy of our model.