PPT : Pre-trained Prompt Tuning for Few-shot Learning (2022)

Abstract: Prompts for pre-trained language models (PLMs) have shown remarkable performance by bridging the gap between pre-training tasks and various downstream tasks. Among these methods, prompt tuning, which freezes PLMs and only tunes soft prompts, provides an efficient and effective solution for adapting large-scale PLMs to downstream tasks. However, prompt tuning is yet to be fully explored. In our pilot experiments, we find that prompt tuning performs comparably with conventional full-model tuning when downstream data are sufficient, whereas it is much worse under few-shot learning settings, which may hinder the application of prompt tuning. We attribute this low performance to the manner of initializing soft prompts. Therefore, in this work, we propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. We name this Pre-trained Prompt Tuning framework “PPT”. To ensure the generalization of PPT, we formulate similar classification tasks into a unified task form and pre-train soft prompts for this unified task. Extensive experiments show that tuning pre-trained prompts for downstream tasks can reach or even outperform full-model fine-tuning under both full-data and few-shot settings. Our approach is effective and efficient for using large-scale PLMs in practice.

## introduction fine-tuning pre-trained language models (plms) @xcite @xcite has made great progress in recent years. by tuning the entire model parameters, the versatile knowledge acquired from large-scale unlabeled corpora can be adapted to handling various nlp tasks and outperform the approach of learning models from scratch @xcite . for simplicity, we name this full-model tuning as "ft". as shown in figure 1 (b ) and (c), there are two mainstream ft approaches. the first one is task-oriented fine-tuning, where a task-specific head is added on top of plms, and the entire model is then fine-tuned by optimizing task-specific objectives on corresponding training data. the second one is prompt-oriented finetuning @xcite , which is inspired by the recent works utilizing language prompts to probe the knowledge in plms @xcite . in promptoriented fine-tuning, data samples are converted to sequences containing prompt tokens, and downstream tasks are formalized as language modeling problems. as shown in figure 1 (c), by adding the prompt "it was x ." to a sentence, we can determine its sentiment polarity with plms by predicting "great" or "terrible" at the mask position. as shown in figure 1 , compared to task-oriented finetuning, prompt-oriented fine-tuning is more similar to the pre-training objectives (masked language modeling), thereby helping to better use knowledge in plms and often obtaining better performance. although ft has shown promising results, with the rapid growth of model scale, fine-tuning and storing the entire large model for each downstream task becomes much more expensive. to address this challenge, @xcite proposes prompt tuning (pt) to adapt large plms to downstream tasks cheaply, as shown in figure 1 (d ). specifically, pt uses soft prompts composed of continuous embeddings instead of hard prompts (discrete language phrases). these continuous prompts are generally randomly initialized and learned end-toend. to avoid storing the entire model for each downstream task, pt freezes all plm parameters and merely tunes soft prompts, without adding any boolq rte cb ccpm c3 cmnli 0 20 40 60 80 100 (t5-xxl model ~11b) (cpm-2 model ~11b) ft pt (a) full-data boolq rte cb ccpm c3 cmnli 0 20 40 60 80 100 (t5-xxl model ~11b) (cpm-2 model ~11b) ft pt (b) few-shot figure 2: comparison between pt and ft. the tuned prompt is composed of 100 learnable embeddings whose dimensions are the same as the token embeddings of plms (4096 dimensions). all these results are based on 11b plms t5 and cpm-2. ft needs to optimize all 11b parameters, while pt only trains about 410k prompt parameters. hence, in this paper, we explore how to use plms for few-shot learning in an efficient and effective manner through pt. specifically, we con-duct pilot experiments to empirically analyze the effectiveness of pt on plms in section 2, which is ignored by most existing works. our discoveries are as follows: (1) the verbalizer choice has a large impact on the performance; (2) simply initializing soft prompts with concrete word embeddings fails to improve the performance, yet (3) combining soft and hard prompts is helpful; and (4) all these methods cannot handle few-shot prompt tuning problems well. the above observations reveal that prompt searching for plms is not trivial, and carefully initialized soft prompt tokens is crucial. to help the model find suitable prompts, we pretrain these tokens with self-supervised tasks on large-scale unlabeled corpora. to ensure the generalization of pre-trained prompts, we group typical classification tasks into three formats: sentencepair classification, multiple-choice classification, and single-text classification, each format corresponding to one self-supervised pre-training task. in addition, we find multiple-choice classification more general among these formats and we can unify all classification tasks to this format. we name this pre-trained prompt tuning framework "ppt". we evaluate ppt on several datasets based on three 11b plms: t5-xxl @xcite , mt5-xxl @xcite and cpm-2 @xcite in few-shot scenarios. experiments show that ppt can not only improve pt by a large margin, reaching or even outperforming ft methods, but also reduce the variance of few-shot learning. besides the effectiveness, ppt also retains the parameter efficiency of pt, which is valuable for future applications on large-scale plms. ## pilot experiments in this section, we present pilot experiments of pt for few-shot learning. we analyze three strategies including hybrid prompt tuning, verbalizer selec- ## conclusion and future work in this paper, we present ppt, a framework that improves prompt tuning for few-shot learning. we propose to firstly unify downstream tasks to several formats. then, we design self-supervised pre-training tasks for each format and pre-train prompts on these tasks. finally, we do prompt tuning on downstream tasks based on the pre-trained initialization. extensive experiments show that our method significantly outperforms other prompt tuning baselines, performing comparable or even better than full-model tuning. there are three important directions for future work: (1) designing unified task formats and the corresponding pre-training objectives for other kinds of tasks such as language generation and relation extraction. (2) evaluating the few-shot performance of other parameter-efficient tuning approaches @xcite and adapting unified task pre-training to them. (3) beyond the soft prompt, studying whether unified task pre-training helps the pre-trained language models itself.