M ULTITASK P ROMPT T UNING E NABLES P ARAMETER -E FFICIENT T RANSFER L EARNING
A BSTRACT
Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks.
However, existing methods typically learn soft prompt vectors from scratch, and it has not been clear how to exploit the rich cross task knowledge with prompt vec tors in a multitask learning setting.
We propose multitask prompt tuning
MPT
, which first learns a single transferable prompt by distilling knowledge from mul
- tiple task
-specific source prompts.
We then learn multiplicative low rank updates to this shared prompt to efficiently adapt it to each downstream target task.
Ex tensive experiments on
23 NLP datasets demonstrate that our proposed approach outperforms the state-of-the-art methods, including the full finetuning baseline in some cases, despite only tuning 0 .
035% as many task-specific parameters.
I NTRODUCTION
Finetuning pretrained language models (PLMs) has led to significant improvements across various downstream NLP tasks (Devlin et al., 2019; Howard & Ruder, 2018; Raffel et al., 2020).
However, the conventional paradigm of full task-specific finetuning is difficult to scale to multiple tasks, given that contemporary PLMs can have hundreds of millions (or even billions) of parameters.
There thus has been a growing interest in developing parameter-efficient methods for model tuning (Houlsby et al., 2019; Lester et al., 2021;
Ding et al., 2022), where the goal is to learn only a small number of additional parameters per task while achieving performance comparable to full finetuning.
[Figure]
Prompt tuning (PT), which prepends tunable continuous prompt vectors to the input, has emerged as a promising approach for parameter-efficient transfer learning with
PLMs (Liu et al., 2021a; Li & Liang, 2021;
Lester et al., 2021;
Liu et al., 2022b; 2021b).
PT freezes the PLM parameters and only learns a small set of task-specific prompt vectors.
However, despite their impressive performance, there is still a large gap between prompt tuning and full finetuning (Lester et al., 2021).
Additionally, this approach is sensitive to initialization and often requires more training time than finetuning (Su et al., 2022; Zhong et al., 2022).
Recent work has proposed to address these issues by transferring prompt vectors from various tasks (Su et al., 2022; Zhong et al., 2022).
These methods first train soft prompts on multiple source tasks and then use these pretrained prompts to initialize the prompt for further finetuning on a target task based on a (potentially learned) similarity measure (Vu et al., 2022; Asai et al., internship
Project page: https://zhenwang9102.github.
[Figure]
2022) (see Figure 1, top).
In this paper, we extend this line of work and introduce multitask prompt tuning (MPT), which uses multitask data to learn a single prompt that can be efficiently transferred to target tasks.
While conceptually simple, learning a shared prompt space can be practically challenging as it requires learning commonalities across different source tasks while minimizing interference.
Therefore, we decompose the soft prompt of each source task (which can be represented as a prompt matrix) into a multiplication of a shared matrix and a low-rank task-specific matrix, and find that this decomposition is more effective than simply sharing the prompt matrix across all tasks.
This decomposition is learned through knowledge distillation from soft prompts obtained from regular prompt tuning.
To transfer to new tasks, we perform low-rank multiplicative updates to the shared prompt matrix.
Figure 1 (bottom) illustrates our approach.
Extensive experiments on 23 NLP datasets across diverse tasks demonstrate the effectiveness of our proposed approach over state-of-the-art prompt transfer methods.
On the SuperGLUE benchmark (Wang et al., 2019), MPT with T5-Base (Raffel et al., 2020) yields a 16 .
3% improvement over the vanilla prompt tuning baseline (PT, Lester et al., 2021), and also outperforms the most competitive multitask prompt transfer baseline (ATTEMPT, Asai et al., 2022) despite tuning many fewer task-specific prompt parameters ( 77 .
6 K vs 232 K).
On some benchmarks, MPT exceeds the performance of full finetuning while only requiring 0 .
035% tunable parameters per task (see Figure 2).
We also find that MPT is very effective for few-shot learning with 4 - 32 labels for each target task.
R ELATED W ORK
Parameter-efficient transfer learning.
Parameter-efficient transfer learning for pretrained language models is an active research area (Ding et al., 2022).
Adapter (Houlsby et al., 2019; Mahabadi et al., 2021) and its variants (Hu et al., 2021; Karimi Mahabadi et al., 2021) insert trainable layers, while BitFit (Zaken et al., 2022) only updates the bias parameters without changing any other model parameters.
Diff pruning (Guo et al., 2021) and FISH (Sung et al., 2021) learn sparse updates to the original PLM.
Another popular choice is prompt tuning (Lester et al., 2021) which only updates soft prompt vectors prepended to the input.
Prefix-tuning for optimizing continuous prompts for natural language generation tasks is presented in Li & Liang (2021).
UNIPELT learns to combine different tuning methods via gating mechanism (Mao et al., 2022).
HyperPrompt (He et al., 2022) introduces task-conditioned hyperprompts that condition the model on task-specific information for constructing prompts.
Discrete (i.e., hard) prompts have also been shown to be effective in many cases (Schick & Sch ¨ utze, 2021a;b; Gao et al., 2021; Malkin et al., 2022).
However, our approach is most related to the transferability of prompts (Wang et al., 2021;
Vu et al., 2022; Su et al., 2022), which focuses on boosting the performance of prompt tuning across many tasks.
SPoT (Vu et al., 2022) selects one prompt using a similarity measure, and ATTEMPT (Asai et al., 2022) adopts an attention mechanism over the source prompts to initialize the prompt for a target task.
Unlike existing works, our approach learns a single shared prompt by decomposing and distilling knowledge from source prompts for efficient adaptation to a diverse set of target tasks.
Multitask learning.
Multitask learning, which focuses on simultaneously solving multiple related tasks with a single model, has been studied from multiple perspectives (Zhang & Yang, 2021; Ruder, 2017).
Transferring a model fine-tuned on multiple source tasks to another target task is a common approach to multitask learning (Vu et al., 2020; Raffel et al., 2020; Aghajanyan et al., 2021a;
Zhong et al., 2021; Clark et al., 2019b; ? ).
Few recent works show zero-shot and few-shot transfer capabilities of language models through massive multitask learning over a large number of tasks (Sanh et al., 2022;
Wang et al., 2022; Liu et al., 2022a; Wei et al., 2021).
Designing specific parametersharing strategies is also another recent trend in multitask learning (Ruder et al., 2019; Sun et al., 2020; Misra et al., 2016).
While our proposed approach is inspired by these methods, this paper focuses on multitask prompt transfer for parameter-efficient adaptation of language models, which still remains a challenging and largely understudied problem.
Knowledge distillation.
Knowledge distillation has been used to improve performance and efficiency across many tasks (Gou et al., 2021), including model compression (Hinton et al., 2015; Jiao et al., 2020; Sanh et al., 2019), transfer learning (Furlanello et al., 2018;
Xu et al., 2020), machine translation (Zhou et al., 2019), question answering (Hu et al., 2018), and document retrieval (Shakeri et al., 2019).
Concurrent to our work, PANDA (Zhong et al., 2022) uses knowledge distillation with a new metric to better predict the prompt transferability across different combinations of sourcetarget tasks.
PANDA focuses on transferring from one source task to another target task using a similarity measure (similar to SPoT (Vu et al., 2022)), while our MPT approach leverages multitask learning to better exploit cross-task knowledge for prompt transfer.
A PPROACH
Given a set of source tasks S = {S 1 , S 2 , ..., S κ } and target tasks T = {T 1 , T 2 , ..., T τ } , our goal is to learn a single soft prompt over S that can be adapted to each task T i in a parameter-efficiency way.
Simply training a single soft prompt on S and then finetuning on each T i is sub-optimal as it can fail to leverage commonalities across source tasks while minimizing interference.
To this end, multitask prompt tuning (MPT) aims to compress task-shared knowledge in S into a single prompt matrix ϕ S via knowledge distillation to improve performance on T while filtering out task-specific information that is less useful for transfer learning.
Prompt tuning.
Given a pre-trained language model with parameters Θ and one target task T with training data ( X , Y ) =
{ x i , y i } Ni =1 , the standard approach is to directly finetune all the parameters by maximizing the conditional probability P ( Y | X ; Θ) , which can be parameter-inefficient when considering a group of target tasks T .
An alternative to full finetuning that is more parameterefficient is prompt tuning (PT), which randomly initializes a small number of learnable prompt vectors (i.e., soft prompts) to be prepended to the input embeddings of the PLM while freezing model parameters Θ (Lester et al., 2021;
Liu et al., 2022b).
Formally, for a sequence of input tokens with token embeddings as T =
[ t 1 , t 2 , ..., t n ] ∈ R n × d , PT prepends a learnable prompt matrix
P ∈ R l × d with the same dimension as the token embedding d , where l is a hyperparameter.
PT then optimizes the following loss function with respect to P ,
[Equation]
where the input to the language model is given by the concatenated matrix [ P ; T ] ∈ R (
[Equation]
×
[Equation]
While this approach has been successful on some tasks and models, researchers have observed that vanilla PT can sometimes lead to lower performance (especially on smaller PLMs), be slow to converge, and have high sensitivity to parameter initialization (Lester et al., 2021;
Su et al., 2022;
Zhong et al., 2022).
Recent works address these issues by first training prompts on multiple source tasks, and then using these prompts to initialize the prompts for a target task via some similarity measure (Asai et al., 2022; Vu et al., 2022).
We extend this line of work and propose a framework for transferring multitask knowledge into a single soft prompt to enable more performant and parameterefficient transfer learning to downstream target tasks T .
3. M ULTITASK P ROMPT T UNING
Our proposed framework, dubbed MPT, consists of two stages: source training and target adaptation .
MPT first focuses on source training to generate a single soft prompt matrix to be reused in the second stage for target task adaptation.
Specifically, prompt matrices for the source tasks are decomposed into a task-shared matrix and a low-rank task-specific matrix ( prompt decomposition ), where the former is shared across all tasks.
This decomposition into shared and task-specific component is learned through knowledge distillation.
Once learned, the shared prompt matrix is adapted to a downstream target task via low-rank multiplicative updates.
[Figure]
Prompt decomposition.
The goal of prompt decomposition is to enable efficient knowledge sharing across source tasks S while still allowing each task to maintain its own parameters to encode task-specific knowledge.
We decompose the soft prompt P k for the k -th task into two parts, as shown in Figure 3.
Let P ∗ ∈ R l × d denote the shared prompt across all tasks, and further let u k ∈ R l , v k ∈ R d be the task-specific vectors for each task k .
The task-specific vectors form a rank-one matrix
W k = u k ⊗ v Tk , which has the same dimensions as the shared prompt P ∗ .
The task prompt (cid:98) P for k -th source task is then parameterized as:
[Equation]
where ◦ denotes the Hadamard product between two matrices.
Our parameterization of prompt decomposition is inspired by prior low-rank methods (Li et al., 2018; Aghajanyan et al., 2021b; Wen et al., 2020), such that general information across the set of source tasks S can be captured by “slow” weights P ∗ shared across tasks, while the “fast” weights W k could then encode task-specific knowledge for S k in a low-rank subspace.
Prompt distillation.
Learning the prompt decomposition directly from the multitask datasets S tended to make the shared component P ∗ to overfit to larger tasks.
We found knowledge distillation from separately-trained source prompts to be an effective strategy for learning good decomposable prompts.
Specifically, we first obtain a teacher prompt P ( teacher ) k for the k -th source task by conventional prompt tuning.
We then randomly initialize a corresponding student prompt as (cid:98) P k
=
P ∗ ◦ ( u k ⊗ v Tk ) , where all student prompts share P ∗ and have their own task-specific vectors as described above.
We then use distillation to transfer cross-task knowledge into the shared prompt matrix (Sanh et al., 2019).
The first loss is to match the output probability distributions of students and teachers through minimizing their KL-Divergence with respect to the shared prompt matrix P ∗ and the task specific parameters u k and v k ,
[Equation]
We use a temperature T to control the smoothness of the output distribution for both teacher and student models as p j = 1 Z exp( z j /T ) , where z i is the logit score for class j and Z is the normalization factor.
We also have an additional mean squared loss on teacher model hidden states,
[Equation]
where H ( teacher ) k,i and H k,i denote the hidden states of teacher and student networks respectively, which consist of a sequence of hidden vectors for i -th input.
Such additional distillation loss from intermediate states has been shown to improve results in distilling PLMs (Jiao et al., 2020; Shleifer & Rush, 2020).
The total loss function for training student source prompts for obtaining a single shared prompt to be transferred to the target side is then,
[Equation]
where L PLM = (cid:80) k
∈| S | L k PLM represents the aggregated task losses for all source tasks, and λ is a weight to balance the impact of distillation loss terms.
3. S OURCE T RAINING AND T ARGET A DAPTATION
Training the single source prompt to be transferred to target tasks requires two steps.
First, the teacher prompts for all source tasks are pretrained individually through vanilla prompt tuning.
Then, we conduct multitask training on S
=
{S 1 , . . .
, S κ } to jointly learn the single shared prompt via the knowledge distillation loss function in Equation 5.
We also adopt a simple stochastic task sampling strategy, which dynamically changes the number of tasks per batch.
For each batch of multitask samples, we randomly select a number K from [2 , κ ] first, then randomly choose K tasks from S and their corresponding samples to constitute mini-batches.
Such dynamic task sampling strategies are common in the PLM multitask learning literature (Raffel et al., 2020).
For target adaptation, we initialize the target prompt for target task T t to be the Hadamard product of the shared prompt matrix and the task-specific low-rank prompt matrix, i.e., (cid:98) P t
=
P ∗ ◦ ( u t ⊗ v ⊤ t ) and optimize with the regular task loss in Equation 1 with respect to P ∗ , u t , v t , where we use separate learning rates for P ∗ vs. u t , v t (see Appedix A).
We remark that MPT can also be used for multitask learning on a group of target tasks
T
= {T 1 , T 2 , ..., T τ } , where P ∗ is shared across T .
Parameter-efficiency.
Each task contains the shared prompt l × d that has the same dimensions as a vanilla soft prompt and a smaller number of task-specific vectors ( l + d ) .
Thus, the total number of tunable parameters for a single target task is ( l × d ) + ( l + d ) .
After training, this can further be compressed into a single matrix of size l × d . 2 For a group of target tasks, the total number of tunable parameters is ( l × d ) + ( l + d ) τ , where τ is the number of target tasks.
We list and compare different methods in terms of the number of trainable parameters in Table 1.
E XPERIMENTS
We conduct experiments across a comprehensive range of NLP datasets to show that MPT outperforms strong baselines in both full-dataset (Tables 1, 2) and few-shot (Tables 3, 4) adaptation, while being more parameter-efficient compared to existing methods (Figure 2).
4. E XPERIMENTAL S ETUP
Datasets and tasks.
As in Asai et al.
we evaluate MPT using 6 datasets with more than 100 k annotations as source tasks: MNLI (Williams et al., 2017), QNLI (Demszky et al., 2018),
QQP (Wang et al., 2018), SST-2 (Socher et al., 2013), SQuAD (Rajpurkar et al., 2016), and
ReCoRD (Zhang et al., 2018).
We use 23 datasets from four benchmarks as target tasks: MultiRC (Khashabi et al., 2018), BoolQ (Clark et al., 2019a), WiC (Pilehvar & Camacho-Collados, 2018), WSC (Levesque et al., 2012), and CB (De Marneffe et al., 2019) from SuperGLUE (Wang et al., 2019); RTE (Giampiccolo et al., 2007), CoLA (Warstadt et al., 2019), STS-B (Cer et al., 2017),
MRPC (Dolan & Brockett, 2005), MNLI, QQP, QNLI and SST-2 from GLUE (Wang et al., 2018);
Natural Questions (Kwiatkowski et al., 2019), HotpotQA (Yang et al., 2018), NewsQA (Trischler et al., 2017) and SearchQA (Dunn et al., 2017) from MRQA (Fisch et al., 2019); WinoGrande (Sakaguchi et al., 2021), Yelp-2 (Zhang et al., 2015), SciTail (Khot et al., 2018) and PAWS-Wiki (Zhang et al., 2019) from the “Others” benchmark in (Asai et al., 2022); and E2E (Novikova et al., 2017) and
WebNLG (Gardent et al., 2017) for experiments on adapting to natural language generation tasks.
Models.
Following the standard approach in prompt tuning (Lester et al., 2021; Asai et al., 2022), we mainly experiment using the publicly available pretrained T5-Base model with 220 M parameters (Raffel et al., 2020).
We use 100 prompt vectors for all benchmarks (hence (cid:98) P k ∈ R 100 × d ).
In our ablation study we also consider T5-Small ( 60 M) and T5-Large ( 770 M) models.
Baselines.
We compare MPT with the following baselines.
(1) Full finetuning (FT), where all the model parameters are tuned during adaptation on each downstream task, (2) vanilla prompt tuning (PT) (Lester et al., 2021), where target prompt vectors are initialized by randomly sampled top vocabularies, (3) existing prompt transfer methods, including SPoT (Vu et al., 2022) and
ATTEMPT (Asai et al., 2022), that initialize target prompts by retrieving or aggregating source prompts, (4) popular parameter-efficient methods including Adapters (Houlsby et al., 2019) and
BitFit (Zaken et al., 2022).
On GLUE, we also compare with several state-of-the-art methods that
[Table] [Table]
adapt a pretrained model to all the target tasks using multitask learning, such as HyperFomer (Mahabadi et al., 2021), HyperDecoder (Ivison & Peters, 2022), multitask variants of FT, and Adapters.
We directly quote numbers reported in published papers when possible or use publicly available source code (Karimi Mahabadi et al., 2021; Mahabadi et al., 2021; Asai et al., 2022) under the same backbone and experimental settings for a fair comparison.
Implementation details.
For source training, we train MPT on the mixture of source tasks for 5 epochs with the examples-proportional mixing strategy (Raffel et al., 2020) and stochastic task sampling described in Section 3.2.
For prompt distillation, we calculate the hidden state loss for hidden states from both the encoder and decoder of T5.
For target adaptation, we reuse the shared prompt from MPT and take averaged source task-specific vectors to initialize the target task-specific vector.
We run all the experiments three times with different random seeds and report the mean and standard deviations.
In few-shot experiments, for each number of shots k , we randomly sampletimes from the training set with different random seeds and report the mean performances.
Note that for few-shot learning, the source prompt learning still uses the full set of the source tasks.
See
Appendix A for full experimental setup including hyperparameters.
4. R ESULTS AND A NALYSIS
Full-dataset adaptation.
Tables 1 and 2 show the per-task performance of different methods on all four benchmarks.
As seen from Table 1 (top), MPT establishes new state-of-the-art results for parameter-efficient finetuning on both GLUE and SuperGLUE.
When compared to vanilla
PT (Lester et al., 2021), MPT obtains a relative improvement of 13% on GLUE and 16% on SuperGLUE with the same number of task-specific parameters, highlighting the benefits of transferring knowledge from multiple source target tasks.
MPT also consistently outperforms other parameterefficient methods such as SPoT (Vu et al., 2022), ATTEMPT (Asai et al., 2022), and BitFit (Zaken et al., 2022) despite updating many fewer parameters.
Adapters is the most competitive in terms of average accuracy on both benchmarks, but MPT is far more parameter efficient and requires 4 × fewer task-specific parameters.
More surprisingly, MPT outperforms the full model finetuning baseline on both benchmarks, despite tuning 0 .
035% as many task-specific parameters.
See Figurefor a comparison against different methods in terms of accuracy and parameter-efficiency.
Table 1 (bottom) shows the results when finetuning against a group of target tasks.
ATTEMPT and MPT are particularly performant in this setting, even when compared against state-of-the-art
[Table] [Table]
multitask baselines such as HyperFormer (Mahabadi et al., 2021) and HyperDecoder (Ivison &
Peters, 2022) which train a single model on different target tasks.
This reveals the potential of our
MPT to further leverage multitask knowledge on the target side to enable even more parameterefficient adaptation of pretrained language models
[Table]
Table 2 shows the performance of different methods on the MRQA and Others benchmark.
Our approach significantly improves the average performance of PT by +2 .
8% on MRQA and +13 .
5% on the Others benchmark, while adding only 0 .
01% more task-specific parameters.
Similarly, MPT obtains 85 .
5% average accuracy on WinoGrande, Yelp, SciTail, and PAWS, outperforming BitFit ( 84 .
7% ), which updates 10 × more task-specific parameters.
When we increase the prompt length from 100 to 300, we found an an average improvement of 0 .
8% on MRQA and 0 .
6% on Others, closing the gap between MPT and Adapters.
While our improvements being highly parameterefficient are encouraging, the accuracy gap between MPT and the full finetuning is still significant in MRQA, which indicate opportunities for future work in multitask prompt tuning.
Few-shot adaptation.
Following prior works (Mahabadi et al., 2021; Asai et al., 2022), we conduct few-shot experiments on BoolQ, CB, and SciTail tasks to measure how the pretrained MPT prompts can be generalized to new tasks with only a few training examples available ( k = 4 , 16 , 32 ).
Table 3 shows the results of our approach and other baselines, which includes full-model finetuning,
Adapters, and HyperFormer.
As can be seen from Table 3, vanilla PT struggles for few-shot adaptation (esp.
, CB and SciTail), suggesting that randomly initialized prompts are hard to generalize to new tasks with only a few labeled examples.
SPoT improves the performance of PT on CB and SciTail tasks, and MPT outperforms both PT and SPoT.
We also observe that other methods in Table(Finetuning, Adapters, HyperFormer, and ATTEMPT) have trouble in the few-shot setting.
Tableshows the few-shot learning performance comparison between PT and MPT on all the GLUE and
SuperGLUE tasks.
As shown in Table 4, we can observe that not only MPT outperforms the vanilla
PT by a large margin in most of the datasets, but also MPT can perform very well on many datasets to reach their full-dataset performance with 16 or 32 shots, such as QQP, QNLI, STS-B, and WSC.
These results clearly indicate that MPT can effectively use cross-task knowledge in source tasks to target tasks where there are only a few labeled examples.
Natural language generation tasks.
We next conduct experiments to test whether prompt decomposition learned from source NLU tasks can generalize to target NLG tasks.
We transfer the
T5-Large prompt trained using 6 diverse source tasks to two NLG tasks:
E2E (Novikova et al., 2017) and WebNLG (Gardent et al., 2017).
Table 5 shows that our proposed MPT significantly outperforms standard PT (Lester et al., 2021) on both NLG tasks across all the metrics.
Our BLEU improvements over PT are 3 .
03% and 6 .
25% on E2E and WebNLG tasks respectively, showing the
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
effectiveness of our approach on both NLU (e.g., classification, NLI, QA tasks) and NLG tasks.
This is particularly an impressive result since the source tasks were all NLU tasks,
i
[Figure]
e., MPT can transfer knowledge from NLU tasks to NLG tasks.
Model scaling.
We conduct scaling experiments to analyze how MPT performs with increasing pretrained model sizes on three SuperGLUE tasks as in Asai et al.
Figure 4 (left) shows the performance of MPT as well as full model finetuning (FT), Adapters, prompt tuning (PT), and
ATTEMPT with three different T5 models (T5-Small, T5-Base, T5-Large).
These results show that
MPT is not only able to achieve the best parameter efficiency but also effective across different model scales ranging from 60M to 770M parameters.
Analyzing prompt matrices.
We conduct qualitative analysis on prompts learned using MPT to investigate whether cross-task knowledge is indeed encoded in the task-shared prompt which makes it easier for target tasks to effectively adapt and encode their own knowledge effectively.
Following Vu et al.
(2022), we use the prompt matrices to compute cosine similarities between all target task pairs after adaptation, where each task is represented by the composition of task-shared and task-specific prompts (averaged to obtain a single vector).
Figure 4 (right) shows the visualization of cosine similarity matrices for SPoT and MPT on SuperGLUE tasks.
We find that task embeddings can effectively cluster similar tasks together (e.g., MultiRC is similar to BoolQ).
4. A BLATION S TUDIES
Prompt decomposition and distillation.
Table 6 shows the results on SuperGLUE where we fix all the hyper-parameters across all settings and rerun MPT source training to get various ablated versions of the transferred prompt.
[Table]
To measure the effect of prompt decomposition, we replace the vanilla source prompt with our decomposable prompt with task-shared and task-specific components and train it without prompt distillation (third row in Table 6), which gives us 3 .
5% average performance improvement on SuperGLUE.
This ablation clearly indicates the importance of the prompt decomposition strategy in MPT and demonstrates that the shared component can effectively capture the rich cross-task knowledge that is beneficial to target downstream tasks.
To test the effect of prompt distillation, we train a vanilla prompt shared by all the source tasks with the same training loss of MPT in Equation 5.
The teacher models are kept the same for this ablation and MPT.
Compared with the simple baseline (first row in Table 6), adding prompt distillation (second row) produces 1 .
1% average performance improvement.
We observe that prompt
[Table]
distillation combined with prompt decomposition yields the best average performance of 74 .
1% on the SuperGLUE benchmark.
This confirms that distilling knowledge from separately-trained source prompts is an effective strategy for learning good decomposable prompts
Distillation objective.
We further investigate the individual components of prompt distillation to see their influences on final performance.
We remove the loss of hidden states from Equation 5 and find that it produces an average performance of 73 .
7% on SuperGLUE, verifying the effectiveness of regularizing hidden states in conjunction with logits to reach its full performance, which is consistent with findings in Sanh et al.
Finally, we consider a variant of distillation loss to match the teacher and student prompts directly by adding an MSE loss to minimize the distance between those two prompts.
Replacing our proposed distillation losses with this prompt distance loss and jointly training it with prompt decomposition yields an average SuperGLUE performance of 73 .
6% , which performs worse than the distillation losses based on logits and hidden states.
[Figure]
Prompt length.
While our experiments use l = 100 prompt vectors, we show in Figure 5 that using longer prompts obtains improvements until l = 300 , reaching 76.8% on SuperGLUE.
However, further increasing the prompt length from 300 to 400 lead to an absolute 1.8% drop in accuracy, possibly due to overfitting.
Target adaptation strategy.
When transferring the shared prompt from source to target tasks, we find that only updating task-shared component (i.e., removing task-specific vectors) or only updating task-specific vectors (i.e., freezing task-shared component) produces suboptimal results ( 62 .
5% and 71 .
3% on
SuperGLUE).
This shows importance of updating both components (which have different learning rates) for target adaptation.
Stochastic task sampling.
MPT uses a multitask training strategy in Section 3.2, which stochastically samples a number of tasks within each mini-batch.
Not using stochastic task sampling results in 73 .
7% on SuperGLUE, which shows the slight benefits of this simple multitask training strategy.
Number of source tasks for pretraining.
For our main experiments we selected 6 NLP tasks following Asai et al.
We investigate the effect of more source tasks by incorporatingadditional diverse source tasks on top of the original 6 tasks: topic classification (AGNews (Zhang et al., 2015)), multi-choice QA (CommmonsenseQA (Talmor et al., 2019), OpenBookQA (Mihaylov et al., 2018), ARC (Clark et al., 2018)), adversarial NLI (ANLI (Nie et al., 2020)) and commonsense reasoning (Winogrande (Sakaguchi et al., 2021)).
Table 7 shows the results on MRQA and Others benchmarks.
MPT with 12 tasks is still quite effective for target adaptation on both benchmarks, slightly outperforming MPT trained using 6 tasks.
While it is unclear how much MPT would benefit from even more source tasks, it would be interesting to see whether MPT trained on largescale benchmarks such as CrossFit (Ye et al.
, 2021)—which consist of 160 NLP tasks—can enable even more parameter-efficient (and accurate) transfer learning.
C ONCLUSION
We introduced and studied multitask prompt tuning (MPT), which learns a single transferable prompt by decomposing and distilling knowledge from multiple source tasks as well as their taskspecific source prompts.
MPT decomposes the task prompt by the Hadamard product of a shared prompt matrix and a rank-one task-specific matrix.
The shared component is then transferred and adapted to target tasks to be further tuned.
Empirically we found this approach enables parameterefficient transfer learning to target downstream tasks across diverse NLP benchmarks, even outperforming full finetuning baseline in some cases despite tuning many fewer task-specific parameters.
A CKNOWLEDGEMENTS
ZW sincerely thanks Peihao Wang for the insightful discussion during the internship.
YK was partially supported by an MIT-IBM Watson AI grant.
We also acknowledge support from the IBM
Research AI Hardware Center and the Center for Computational Innovation at Rensselaer Polytechnic Institute for the computational resources on the AiMOS Supercomputer
A E XPERIMENTAL S ETUP
For initial training of source prompts, we train MPT on the mixture of source tasks for 5 epochs with the examples-proportional mixing strategy (Raffel et al., 2020) and stochastic task sampling.
For prompt distillation, we calculate the hidden state loss for hidden states from both the encoder and decoder of T5.
For target adaptation, we reuse the shared prompt from MPT and take averaged source task-specific vectors to initialize the target task-specific vector.
We train 20 epochs on small datasets, 10 epochs on large (more than 10k examples) datasets, and 5 epochs on the MRQA dataset.
We run all the experiments three times with different random seeds and report the mean and standard deviations.
In few-shot experiments, for each number of shots k , we randomly sample 10 times from the training set with different random seeds and report the mean performances.
For few-shot setting, the source prompt learning still uses the full set of the source tasks.
During source training, we set the default learning rate as 0 .
3 for both task-shared and task-specific components.
However, during target adaptation, we use a strategy of two-speed learning rates for those two components, as in Ponti et al.
We set the learning rate to 0 .
3 and 0 .
4 for the taskshared and task-specific components respectively during target task adaptation.
Following Lester et al.
(2021), we set the default number of tunable tokens per each prompt as 100 and initialize the teacher and student prompts by randomly sampling tokens from T5’s vocabulary (Raffel et al., 2020).
We set the default batch size for T5-Base as 32 and for model scaling experiments, the batch sizes for T5-Small and T5-Large are 100, and 12 respectively.
The default input length for most tasks are set to 256 , except MultiRC and MRQA benchmark have the input length as 348 and 512 .
We set distillation loss coefficient λ in Equation 5 to 0 .
9 and keep it fixed for all our experiments.
For all datasets, we use the development set as the testing set if the original testing set is not publicly available.
We split the original development set into the development and testing set if the training set is small; otherwise, we separate a development set from the training set and use the original development set for testing.
We limit the number of training data for Yelp to 100k.