L i ST : Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners (2022)

Abstract: We present a new method LiST for efficient fine-tuning of large pre-trained language models (PLMs) in few-shot learning settings. LiST improves over recent methods that adopt prompt-based fine-tuning (FN) using two key techniques. The first is the use of self-training to leverage large amounts of unlabeled data for prompt-based FN in few-shot settings. We use self-training in conjunction with meta-learning for re-weighting noisy pseudo-prompt labels. Traditionally, self-training is expensive as it requires updating all the model parameters repetitively. Therefore, we use a second technique for light-weight fine-tuning where we introduce a small number of task-specific parameters that are fine-tuned during self-training while keeping the PLM encoder frozen. Our experiments show that LiST can effectively leverage unlabeled data to improve the model performance for few-shot learning. Additionally, the finetuning process is efficient as it only updates a small percentage of the parameters and the overall model footprint is reduced since several tasks can share a common PLM encoder as backbone. We present a comprehensive study on six NLU tasks to validate the effectiveness of LiST. The results show that LiST improves by 35% over classic fine-tuning methods and 6% over prompt-based FN with 96% reduction in number of trainable parameters when fine-tuned with no more than 30 labeled examples from each task. With only 14M tunable parameters, LiST outperforms GPT-3 in-context learning by 33% on few-shot NLU tasks

## introduction large pre-trained language models (plms) have obtained state-of-the-art performance in several natural language understanding tasks @xcite @xcite . despite their remarkable success in fully supervised * most of the work was conducted while the first author was interning at microsoft. 1 list is short for lite prompted self-training. the first one is to leverage self-training with large amounts of unlabeled data from the target domain to improve model adaptation in few-shot settings. prompt-based fine-tuning @xcite have recently shown significant improvements over classic fine-tuning in the few-shot learning setting. in this paper, we demonstrate that self-training with unlabeled data is able to significantly improve prompt-based fine-tuning @xcite where we iteratively update a pair of teacher and student models given natural language prompts and very few labeled examples for the task. since the uncertain teacher in few-shot setting produces noisy pseudo-labels, we further use meta-learning to reweight the pseudo-prompt labels. traditional self-training can be expensive if we have to update all model parameters iteratively. to improve the efficiency of self-training, the second key technique introduces a small number of taskspecific adapter parameters in the plm that are updated with the above technique, while keeping the large plm encoder fixed. we demonstrate such light-weight tuning with self-training to match the model performance where all parameters are tuned. this enables parameter-efficient use of selftraining and reduces the storage cost of the finetuned model since multiple fine-tuned models can now share the same plm as backbone during inference. note that the computational cost of inference is out of the scope of this paper and previous work has studied several ways to address it including techniques like model distillation @xcite and pruning @xcite . we perform extensive experiments in six natural language understanding tasks to demonstrate the effectiveness of list. we devise a comprehensive evaluation framework considering the variance in few-shot performance of plms with different shots, random seeds and splits foot_0 . results show that list improves over traditional and more recent prompt-based fn methods by 35% and 6%, respectively, with 96% reduction in number of trainable parameters given only 30 labeled examples for each downstream task. figure 1 shows the results on mnli @xcite as an example. we compare list with gpt-3 in-context learning outperforming it by 33% as well as several sota few-shot semi-supervised learning approaches with list outperforming the strongest baseline by 6% given only 30 labeled examples for each task. problem statement. each downstream task in our framework consists of very few labeled training examples d t rain k for different shots @xmath0, 20, 30} where ## background on model fine-tuning given a text sequence x or a pair of sequences {x 1 , x 2 } separated by special operators (e.g., @xcite and @xcite ) and a language model encoder enc(θ) parameterized by θ -classic fine-tuning popularized by @xcite leverages hidden state representation h @xcite of the sequence(s) obtained from enc([[cls] x 1 [sep] x 2 [sep]]) as input to a task-specific head sof tmax(w t • h @xcite ) for classification, where @xmath1, are randomly initialized tunable parameters. in the process it updates both task-specific head w and encoder θ parameters jointly. however, this introduces a gap between pretraining and fine-tuning objective with disparate label spaces and additional randomly initiated parameters w introduced for task-specific fine-tuning. this is particularly challenging for few-shot classic fine-tuning, where the limited labeled data is inadequate for adapting the task-specific head and plm weights effectively. prompt-based fn @xcite addresses this gap, by re-formulating the objective as a clozestyle auto-complete task. this is done by adding a phrase (also called prompt) to a sentence like x 1 = "contains no wit, only labored gags" in the form of @xmath2, where ⊕ denotes concatenation of two strings; and output mappings (also called verbalizers) from vocabulary v to the label space y like "{great, terrible}" corresponding to positive and negative classes (refer to figure 3 for an example). the probability of predicting class @xmath3: in this work, we demonstrate lite self-training with unlabeled data to significantly improve prompt fine-tuning of plms in few-shot settings. few-shot adaptation works @xcite @xcite @xcite train models on massive labeled data on source tasks and develop techniques to adapt them to a target task with few-shot labels. for instance, @xcite first trains on miniimagenet with 38,400 labeled examples (64 classes and 600 samples per class). similarly, @xcite first train their models on thousands of labels from the source task to study few-shot target adaptation. in contrast, we focus on single-task true few-shot learning with only 10 to 30 labeled examples available overall and no auxiliary supervision labels. refer to @xcite for an overview of true fewshot learning for nlu. the objective of few-shot adaptation and true few-shot learning is also quite different. the objective of true few-shot learning is to learn a new task with limited labeled data while the objective of few-shot adaptation is to efficiently transfer to a new task/domain with limited labeled data. thus, few-shot adaptation leverages multi-task setting with auxiliary labeled data from the source tasks that are not available in our setting. the most relevant works to our setup @xcite @xcite are used as baselines in our paper. light-weight tuning. standard fine-tuning methods tune all trainable model parameters for every task. recent efforts have focused on lightweight tuning of large plms by updating a small set of parameters while keeping most of parameters in plms frozen, including prefix tuning @xcite , prompt token tuning @xcite and adapter tuning @xcite . all of the above works focus on fully supervised settings with thousands of labeled examples using classic fine-tuning. in contrast, we focus on few-shot learning settings leveraging prompts for model tuning, where we make several observations regarding the design and placement of adapters in few-shot settings in contrast to its resource-rich counterpart. some recent works @xcite pretrain adapters with full supervision with thousands of labeled examples from source tasks for few-shot target adaptation. different from this, we explore adapter tuning for single-task few-shot learning without any auxiliary supervision labels. ## conclusions and future work we develop a new method list for lightweight tuning of large language models in few-shot settings. list uses prompted self-training to learn from large amounts of unlabeled data from target domains. in order to reduce the storage and training cost, list tunes only a small number of adapter parameters with few-shot labels while keeping the large encoder frozen. with only 30 labels for every task, list improves by upto 35% over classic fine-tuning and 6% over prompt-tuning while reducing 96% of the tunable parameters. with significant reduction in the cost of (data) annotation and overall model footprint, list provides an efficient framework towards life-long learning of ai agents @xcite . while adapters reduce storage cost, list does not reduce inference latency given the plm backbone. a future work is to consider combining model compression techniques @xcite with adapters to reduce flops and latency.