Revisiting Self-training for Few-shot Learning of Language Model (2021)

Abstract: As unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. The question is how to effectively make use of such data. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM. Given two views of a text sample via weak and strong augmentation techniques, SFLM generates a pseudo label on the weakly augmented version. Then, the model predicts the same pseudo label when fine-tuned with the strongly augmented version. This simple approach is shown to outperform other state-of-the-art supervised and semi-supervised counterparts on six sentence classification and six sentence-pair classification benchmarking tasks. In addition, SFLM only relies on a few in-domain unlabeled data. We conduct a comprehensive analysis to demonstrate the robustness of our proposed approach under various settings, including augmentation techniques, model scale, and few-shot knowledge transfer across tasks.

## introduction pre-trained language models @xcite @xcite @xcite @xcite have set new state-of-the-art performance in many downstream nlp tasks. however, such performance often relies on large-scale highquality supervision. unfortunately, labeled data are not always available in practice. recently, @xcite study how to facilitate the few-shot learning of language models via the gpt-3 model. it achieves remarkable performance on many nlp datasets without any gradient updates, by incorporating task-specific prompts into the text and reformulating the task as language modeling problems. however, gpt-3 has 175b parameters, that has a footprint too large for many real-world applications. @xcite applies the concept of prompt strategies in gpt-3 to smallfootprint language models, such as bert @xcite and roberta @xcite . after fine-tuned on a few annotated samples, the smallfootprint models exhibit comparable performance to that of the large-footprint gpt-3 model. however, the performance of these models is still lagging behind those under supervised learning, which have a much smaller footprint. intuitively, unlabeled data also carry rich information of downstream tasks and are more available than labelled data. in this paper, we focus on the few-shot learning of language model with a small amount of labeled and unlabeled data. semi-supervised learning benefits from partially labeled datasets. a common implementation of semi-supervised learning is self-training, which leverages supervision signals offered by labeled data to create pseudo-labels for unlabeled data. these pseudo labels serve as additional supervision to refine the models @xcite @xcite @xcite . recent works @xcite apply self-training to language model few-shot learning in an iterative manner whereby multiple generations of models are trained on data pseudo-labeled by the ensemble of previous generations. however, the amount of in-domain unlabeled data required by these methods is quite large, that limits the scope of the applications, especially for low-resource downstream tasks. @xcite try to retrieve more task-relevant unlabeled data from open-domain corpus, but the method depends on a quality sentence encoder. to better address the above issue, we revisit the self-training techniques and introduce a data-efficient few-shot learner of language model (sflm). inspired by recent advances in semi-supervised representation learning for images @xcite , sflm combines pseudolabeling with consistency regularization. next, we briefly describe the workflow. given each unlabeled sentence, we construct two views through weak augmentation (random dropout) and strong augmentation (token masking), respectively. the weakly-augmented view is first passed to a prompt-based language model @xcite to derive the pseudo-label, while the stronglyaugmented view is passed through the model to predict the probability distribution over classes, which is compared to the pseudo-label to derive a cross-entropy loss. this learning procedure encourages the model to capture the information that is almost outside of the data distribution, leading to effective utilization of data. we evaluate sflm on two groups of taskssentence classification and sentence-pair classification. experiments show that our model outperforms other supervised and semi-supervised baselines. we also conduct a detailed analysis of the data efficiency of our model by examining its performance w.r.t various ratios of the amount of the unlabeled data to that of the labelled data. we find out that the performance gain diminishes as more unlabeled data are used. we further extend our method for a more challenging scenario: fewshot transfer across tasks, where the model is first trained on the labeled data of a source task and the unlabeled data of a target task, then evaluated on the target task. we provide the analysis of the factors that affect the model performance to motivate future research. ## methodology problem setup: our goal is to adapt pre-trained language models to downstream tasks in a few-shot setting. the model, m should correctly classify unseen examples leveraging very few labeled data points from each class. let x denote a small set of labeled training data with n samples per class and an unlabeled dataset, u from the same task domain as x . assume that this unlabeled dataset has very limited size µn per class, where µ is the ratio between the size of x and that of u. during training, let each batch consist of b labeled data points, x b , and µb unlabeled data points, u b : figure 1 illustrates the learning process with an example, containing one labeled and three unlabeled data samples. sflm is optimized with following loss function: prompt-based supervised loss: the promptbased supervised loss is motivated by lm-bff @xcite . the classification is reformulated as a language modeling task, in which the probability of class prediction y @xmath0, then we could fine-tune the model with the standard cross-entropy loss: self-training loss: for each unlabeled sentence u i , we obtain the weakly-augmented version α(u i ) and the strongly-augmented version a(u i ), where α and a refers to different augmentation strategies. the self-training process consists of two stages. firstly, we assign a pseudo label to each unlabeled sentence in the batch by computing the output probability distribution corresponding to the weakly-augmented input sentence α(u i ), defined as q @xmath1. the pseudo label, qi , is obtained by @xmath2. secondly, we compute the prompt-based cross-entropy loss between qi and the prediction corresponding to the stronglyaugmented input sentence a(u i ). the self-training loss is defined as, specifically, the surface forms of weaklyaugmented sentences remain unchanged: α(u i ) = u i . for strong augmentation, we randomly replace 15% of the tokens in a(u i ) with the special mask token, @xcite . then, we input α(u i ) and a(u i ) to the language model separately. therefore, the two input sentences will undergo independent dropout operations (0.1 dropout rate by default), which can be considered as part of the data augmentation process (the green arrow in figure 1 ). we empirically show that the performance of our proposed augmentation techniques is superior against other common text augmentation techniques in section 4. self-supervised loss: we also include an auxiliary self-supervised loss term, l ssl , for regularization purpose. the masked language model loss is used for its simplicity and efficiency. ## conclusion and future work in this paper, we present sflm, a simple and effective self-training framework for few-shot learning of language model. our approach addresses the transfer sflm sst mr cr sst mr cr sst -87.3 89.7 -87.3 90.8 mr 90.7 -89.5 91.4 -90.8 cr 89.7 84.5 -90.3 85.5 -