On Transferability of Prompt Tuning for Natural Language Processing (2022)

Abstract: Prompt tuning (PT) is a promising parameter-efficient method to utilize extremely large pre-trained language models (PLMs), which can achieve comparable performance to full-parameter fine-tuning by only tuning a few soft prompts. However, PT requires much more training time than fine-tuning. Intuitively, knowledge transfer can help to improve the efficiency. To explore whether we can improve PT via prompt transfer, we empirically investigate the transferability of soft prompts across different downstream tasks and PLMs in this work. We find that (1) in zero-shot setting, trained soft prompts can effectively transfer to similar tasks on the same PLM and also to other PLMs with a cross-model projector trained on similar tasks; (2) when used as initialization, trained soft prompts of similar tasks and projected prompts of other PLMs can significantly accelerate training and also improve the performance of PT. Moreover, to explore what decides prompt transferability, we investigate various transferability indicators and find that the overlapping rate of activated neurons strongly reflects the transferability, which suggests how the prompts stimulate PLMs is essential. Our findings show that prompt transfer is promising for improving PT, and further research shall focus more on prompts’ stimulation to PLMs. The source code can be obtained from https://github.com/thunlp/Prompt-Transferability .

## introduction pre-trained language models (plms), such as bert @xcite and gpt @xcite have achieved great performance on various natural language processing (nlp) tasks @xcite . recently, people have found that extremely large plms can achieve remarkable improvements, and various large plms are continually developed @xcite @xcite @xcite , which contain up to hundreds of billions of parameters. considering the extremely large scale of these state-of-the-art plms, conventional full-parameter fine-tuning methods become extremely expensive. hence, various parameter-efficient tuning methods @xcite @xcite @xcite are explored, among which prompt tuning (pt) has attracted broad research attention. pt prepends some soft prompts, which are essentially learnable virtual tokens, into the input sequences and only trains them while keeping all the plm's parameters fixed. the training objective is to generate desired outputs in the same way as the pretraining tasks. pt can match the downstream task performance of fine-tuning with only thousands of tunable parameters @xcite when the plm has billions of parameters. although pt is an effective approach to utilizing extremely large plms, it requires much more 7lphv $ffxudf\ )lqhwxqlqj 3urpsw7xqlqj training time than fine-tuning to reach the convergence as shown in figure 2 ; hence, it is worthwhile to explore how to improve the efficiency of pt. in this work, we attempt to improve pt via prompt transfer across different tasks and models. knowledge transfer across tasks @xcite and models @xcite have been widely used to improve the efficiency and effectiveness of nlp systems. intuitively, soft prompts are the only tuned parameters in pt and thus shall concentrate the knowledge required to solve tasks conditioned on plms. thus transferring the trained prompts is promising to accelerate pt. as shown in figure 1 , we empirically analyze the transferability of prompts across different tasks (cross-task transfer setting) and plms (crossmodel transfer setting) in this paper. the empirical analysis is conducted on 17 nlp tasks of 6 types and two representative plm series: roberta @xcite and t5 @xcite . in crosstask transfer, the prompt transfer can be done by directly reusing the trained prompts of the source task on the target task. however, in cross-model transfer, directly reusing prompts is intractable since the semantic spaces of different plms are inconsistent; hence, we develop various prompt projectors to project the soft prompts trained on the source plm to the semantic space of the target plm. we conduct two lines of experiments: furthermore, we explore why can the prompts transfer and what decides their transferability. to this end, we design various prompt similarity metrics from different perspectives and examine how well they can serve as transferability indicators, i.e., how well they correlate with prompt transfer performance. experiments find that our novel method of measuring prompt similarity via model activations in feed-forward layers is better correlated with prompt transferability than prompt embedding distance-based metrics. this suggests the prompts are essentially stimulating plm's inner ability distributing among neurons to do specific nlp tasks, and future prompt transfer works should focus more on how the plms respond to different prompts' stimulation rather than the prompts' embedding properties. to summarize, our contributions are three-fold: (1) we thoroughly analyze the transferability of prompts across different tasks and models, and show that improving pt with prompt transfer is possible and promising. (2) we propose to transfer prompts with initialization, which enhances both pt's efficiency and effectiveness. (3) we explore the effectiveness of various prompt similarity metrics serving as transferability indicators and demonstrate how the prompts stimulate plms to decide the transferability, which may facilitate further transferrable pt research. knowledge transfer cross-task knowledge transfer @xcite has been a long-standing way to improve the effectiveness and efficiency of nlp systems. in the plm era, some works propose to tune the plms on intermediate tasks @xcite @xcite @xcite before fine-tuning on specific target tasks to achieve certain benefits. @xcite empirically analyze the transferability between tasks in this setting. these explorations are all for fine-tuning. considering the potential of pt, we believe the transferability and knowledge transfer methods for pt are worth exploring. as a prior attempt, @xcite demonstrate that pt's cross-domain transferability is stronger than fine-tuning. similar to our work, recent work @xcite ) also explores the cross-task transfer with prompt initialization and prompt similarity metrics based on cosine similarity. however, @xcite focus on improving the effectiveness of pt but we attempt to improve the efficiency. additionally, we explore more transferability indicators, especially the overlapping rate of activated neurons, and also investigate cross-model transfer, which is inspired by previous cross-model knowledge transfer works such as net2net @xcite , knowledge distillation @xcite and knowledge inheritance @xcite . ## preliminary here we introduce the basic knowledge about pt ( § 3.1) as well as the downstream tasks ( § 3.2) and models ( § 3.3) investigated in experiments. ## cross-model transfer we further study the cross-model transferability of soft prompts. intuitively, cross-model transfer allows us to train prompts on a small and computationally efficient plm and use them on a massive and computationally expensive plm, which will be much more efficient and environmentfriendly. we investigate the feasibility of crossmodel transfer on transferring from a source plm (roberta large ) to a larger and heterogeneous target plm (t5 xxl ), which shall be the most difficult setting. appendix c shows the experimental results of other settings. directly reusing trained soft prompts between different plms is infeasible since their embedding spaces are different. hence, we investigate how to do cross-model prompt projection ( § 5.1) and see the transfer performance ( § 5.2). furthermore, we explore to improve pt with cross-model transfer initialization ( § 5.3). ## exploring transferability indicator based on the positive results in cross-task and crossmodel transfer, we explore why the soft prompts can transfer across tasks and what decides the transferability between them, which may shed light on the mechanisms behind pt and help to design transferable pt methods. we explore various prompt similarity metrics and examine how well do they align with the zero-shot transfer performance. if a similarity metric can well indicate transferability, it suggests the factors considered in designing this metric decide the prompt transferability. moreover, the prompt similarity metrics can qualify task similarities using the trained soft prompts as task embeddings and may help in developing cross-task transfer methods. as a straightforward example, if we build a prompt warehouse containing prompts of diverse tasks, we can retrieve prompts of similar tasks for a new task with a certain similarity metric and better improve pt with tpt task . ## conclusion we empirically investigate the transferability of prompts in this paper. in the cross-task setting, we find that soft prompts can transfer to similar tasks without training. in the cross-model setting, we successfully project prompts into the space of other plms. further, we utilize trained prompts of other tasks or other plms as initialization to significantly accelerate training and improve effectiveness. moreover, we explore various prompt transferability indicators and show that how the prompts stimulate plms is important to transferability. we hope the empirical analyses and the model stimulation idea can facilitate further research on transferable and efficient pt. timo schick and hinrich schütze. 2021a. exploiting cloze-questions for few-shot text classification and natural language inference. in proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume, pages 255-269, online. association for computational linguistics. timo schick and hinrich schütze. 2021b. it's not just size that matters: small language models are also few-shot learners. in proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 2339-2352, online. association for computational linguistics. lakshay sharma, laura graesser, nikita nangia, and utku evci. 2019. natural language understanding with the quora question pairs dataset. arxiv preprint, abs/1907.01041. taylor shin, yasaman razeghi, robert l. logan iv, eric wallace, and sameer singh. 2020. autoprompt: eliciting knowledge from language models with automatically generated prompts. in proceedings of the 2020 conference on empirical methods in natural language processing (emnlp), pages 4222-4235, online. association for computational linguistics. richard socher, alex perelygin, jean wu, jason chuang, christopher d. manning, andrew ng, and christopher potts. 2013. recursive deep models for semantic compositionality over a sentiment treebank. in proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642, seattle, washington, usa. association for computational linguistics. charles spearman. 1987. the proof and measurement of association between two things. in proceedings of the american journal of psychology, volume 100, pages 441-471. a basic setup for various tasks a.1 dataset and task sentiment analysis (sa) sa is the task of classifying sentiment polarities for a given sentence. we select imdb (maas et al., 2011), sst-2 (socher et al., 2013), semeval/laptop (pontiki et al., 2014), semeval/restaurant @xcite , movie rationales (movie) @xcite , and tweeteval (tweet) @xcite for our experiments.