M ULTITASK P ROMPT T UNING E NABLES P ARAMETER -E FFICIENT T RANSFER L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for the efficient adaptation of large language models to multiple downstream tasks.
However, existing methods typically learn soft prompt vectors from scratch, and it has not been clear how to exploit the rich cross-task knowledge in task-specific prompt vectors to improve performance on target downstream tasks.
In this paper, we propose multitask prompt tuning (MPT), which first learns a single transferable prompt by decomposing and distilling knowledge from multiple task-specific source prompts.
We then learn multiplicative low rank updates to this shared prompt to efficiently adapt it to each downstream target task.
Extensive experiments on 21 NLP datasets demonstrate that our proposed approach outperforms the state-of-the-art methods, including the full finetuning baseline in some cases, despite only tuning 0 .
035% as many task-specific parameters.
I NTRODUCTION
Finetuning Pretrained Language Models (PLMs) has led to significant improvements across various downstream NLP tasks (Devlin et al., 2019; Howard & Ruder, 2018; Raffel et al., 2020).
However, the conventional paradigm of full task-specific finetuning is difficult to scale to multiple tasks given that contemporary PLMs can have hundreds of millions (or even billions) of parameters.
There has thus been a growing interest in developing parameter-efficient methods for model tuning (Houlsby et al., 2019; Lester et al., 2021;
Ding et al., 2022), where the goal is to learn only a small number of additional parameters per task while achieving performance comparable to full model finetuning.
[Figure]
Prompt tuning (PT), which prepends continuous prompt vectors to the input, has emerged as a promising approach for parameter-efficient transfer learning with PLMs (Liu et al., 2021a;
Li & Liang, 2021; Lester et al., 2021; Liu et al., 2022b; 2021b).
PT freezes the PLM parameters and only learns a small set of task-specific prompt vectors.
Despite their impressive performance, there is still a large gap between prompt tuning and full finetuning for many models and tasks (Lester et al., 2021).
Prompt vectors trained using task-specific training data only are more sensitive to initialization and require significantly more training time than finetuning (Su et al., 2022; Zhong et al., 2022).
Recent work has proposed to address these issues through transferring prompt vectors from different tasks (Su et al., 2022; Zhong et al., 2022).
These methods first train soft prompts on multiple source tasks and then use these pretrained prompts to initialize the prompt for further finetuning on a target task based on a (potentially learned) similarity measure (Vu et al., 2022; Asai et al., 2022) (see Figure 1, top).
In this paper, we
[Figure]
extend this line of work and introduce multitask prompt tuning (MPT), which uses multitask data to learn a single prompt that can be efficiently transferred to target tasks.
While conceptually simple, learning a shared prompt space can be practically challenging as it requires learning commonalities across source tasks while minimizing interference.
We decompose the soft prompt of each source task (which can be represented as a prompt matrix) as a multiplication of a shared matrix and a lowrank task-specific matrix, and find that this decomposition is more effective than simply sharing the prompt matrix across all tasks.
This decomposition is learned through knowledge distillation from soft prompts obtained from regular prompt tuning.
To transfer to new tasks, we perform low-rank multiplicative updates to the shared prompt matrix.
Figure 1 (bottom) illustrates our approach.
Extensive experiments on 21 NLP datasets across diverse tasks demonstrate the effectiveness of our proposed approach over state-of-the-art prompt transfer methods.
On the SuperGLUE benchmark (Wang et al., 2019), MPT with T5-Base (Raffel et al., 2020) yields a 16 .
3 improvement over the vanilla prompt tuning baseline (PT, Lester et al., 2021), and also outperforms the most competitive multitask prompt transfer baseline (ATTEMPT, Asai et al., 2022) despite tuning much fewer task-specific prompt parameters ( 77 .
6 K vs 232 K).
On some benchmarks, MPT exceeds the performance of full finetuning while only requiring 0 .
035% tunable parameters per task (see Figure 2).
We also discover that MPT is very effective for few-shot learning with 4 - 32 labels.
Finally, ablation studies further show that MPT matches the performance of full finetuning at different model scales ranging from 60 M to 770 M parameters.
Our code and models will be made publicly available.
R ELATED W ORK
Parameter-Efficient Transfer Learning.
Parameter-efficient transfer learning for pretrained language models is an active research area (Ding et al., 2022).
Adapters (Houlsby et al., 2019; Mahabadi et al., 2021) and variants (Hu et al., 2021; Karimi Mahabadi et al., 2021) insert trainable layers, while BitFit (Zaken et al., 2022) only updates the bias parameters without changing any other model parameters.
Diff pruning (Guo et al., 2021) and FISH (Sung et al., 2021) learn sparse updates to the original PLM.
Another popular choice is prompt tuning (Lester et al., 2021) which only updates soft prompts prepended to the input.
Prefix-tuning for optimizing continuous prompts for natural language generation tasks is presented in Li & Liang (2021).
UNIPELT learns to combine different tuning methods via gating mechanism (Mao et al., 2022).
HyperPrompt (He et al., 2022) introduces task-conditioned hyperprompts that conditions the model on task-specific information for constructing prompts.
Discrete (i.e., hard) prompts have also been shown to be effective in many cases (Schick & Sch¨utze, 2021a;b; Gao et al., 2021; Malkin et al., 2022).
However, our approach is most related to the transferability of prompts (Wang et al., 2021;
Vu et al., 2022; Su et al., 2022), which focuses on boosting the performance of prompt tuning across many tasks.
SPoT (Vu et al., 2022) selects one prompt using a similarity measure and ATTEMPT (Asai et al., 2022)
adopts an attention mechanism over the source prompts to initialize the prompt for a target task.
Unlike existing works, our approach learns a single shared prompt by decomposing and distilling knowledge from source prompts in a structured way for efficient adaptation to a diverse set of target tasks.
Multitask Learning.
Multitask learning, which focuses on simultaneously solving multiple related tasks, has been studied from multiple perspectives (Zhang & Yang, 2021; Ruder, 2017).
Transferring a model fine-tuned on multiple source tasks to another target task is a common approach to multitask learning (Vu et al., 2020; Raffel et al., 2020; Aghajanyan et al., 2021a;
Zhong et al., 2021;
Clark et al., 2019b).
Few recent works show zero-shot and few-shot transfer capabilities of language models through massive multitask learning over a large number of tasks (Sanh et al., 2022;
Wang et al., 2022; Liu et al., 2022a; Wei et al., 2021).
Designing specific parameter-sharing strategies is also another recent trend in multitask learning (Ruder et al., 2019; Sun et al., 2020; Misra et al., 2016).
While our proposed approach is inspired by these methods, in this paper we focus on multitask prompt transfer for parameter-efficient adaptation of language models, which still remains as a challenging and largely under-addressed problem.
Knowledge Distillation.
Knowledge distillation has been used to improve performance and efficiency across many tasks (Gou et al., 2021), including model compression (Hinton et al., 2015; Jiao et al., 2020; Sanh et al., 2019), transfer learning (Furlanello et al., 2018;
Xu et al., 2020), machine translation (Zhou et al., 2019), question answering (Hu et al., 2018), and document retrieval (Shakeri et al., 2019).
Concurrent to our work, PANDA (Zhong et al., 2022) uses knowledge distillation with a new metric to better predict the prompt transferability across different combinations of source-target tasks.
This differs from MPT, which uses a prompt decomposition strategy to leverage commonalities across the source tasks while minimizing interference between them.
In addition, PANDA focuses on transferring from one source task to another target task using a similarity measure (similar to SPoT (Vu et al., 2022)), while our MPT approach leverages multitask learning to better exploit the rich cross-task knowledge in prompt transfer.
M ETHODOLOGY
Given a set of source tasks S = {S 1 , S 2 , ..., S κ } and target tasks T = {T 1 , T 2 , ..., T τ } , our goal is to learn a single soft prompt over S that can be efficiently updated to enable better performance on
T .
Simply training a single soft prompt on S is sub-optimal as it can fail to leverage commonalities across source tasks while minimizing interference.
To this end, our multitask prompt tuning (MPT), aims to efficiently compress task-shared knowledge in S into a single prompt ϕ s to improve performance on T while filtering out task-specific information that is less useful for transfer learning.
Prompt Tuning.
Given a pre-trained language model with parameters Θ and one target task T with training data ( X , Y ) =
{ x i , y i } Ni =1 , directly finetuning all the parameters by maximizing the conditional probability P ( Y | X ; Θ) is expensive and often tends to overfit on small datasets.
An alternative to finetuning that is more parameter-efficient is prompt tuning (PT), which randomly initializes a small number of learnable prompt vectors (i.e., soft prompts) to be prepended to the input embeddings of the PLM while freezing model parameters Θ (Lester et al., 2021; Liu et al., 2022b).
Formally, for a sequence of input tokens with token embeddings as x = { t 1 , t 2 , ..., t n } ∈ R n × d , PT prepends the soft prompt P ∈ R l × d with the same dimension as the token embedding d and vector length as l .
Then PT optimizes the following loss function:
[Equation]
with respect to P .
While this approach has been successful on some tasks and models, researchers have observed that vanilla PT can sometimes lead to lower performance (especially on smaller
PLMs), be slow to converge, and have high sensitivity to the initialization (Lester et al., 2021;
Su et al., 2022; Zhong et al., 2022).
Recent works address these issues by first training prompts on multiple source tasks, and then initializing the prompts for a target task via some similarity measure or learned attention (Asai et al., 2022; Vu et al., 2022).
We extend this line of work and propose a novel framework for transferring multitask knowledge into a single soft prompt to enable more performant and parameter-efficient transfer learning to downstream target tasks T .
3. M ULTITASK P ROMPT T UNING
Our proposed framework mainly consists of two stages, source training and target adaptation .
The proposed MPT framework first focuses on the source training to generate a single soft prompt to be reused in the second stage for target task adaptation.
Specifically, task prompts for source tasks are decomposed into a task-shared component and a low-rank task-specific component ( prompt decomposition ), where the former is shared across all tasks, and the latter is task-specific.
We also use prompt distillation to better transfer multitask knowledge to the shared component by distilling knowledge from multiple tasks-specific source prompts.
Once learned, the shared prompt matrix is adapted to a downstream target task via low-rank multiplicative updates.
⊗ ⊗
[Figure]
Prompt Decomposition.
The goal of prompt decomposition is to enable efficient knowledge sharing across
S while still allowing each task to maintain its own parameters to encode task-specific knowledge.
Specifically, we decompose the soft prompt P k for k -th task into two parts, as shown in Figure 3.
Let P ∗ ∈ R l × d denote the shared prompt across all tasks, and further let u k ∈ R l , v k ∈ R d be the task-specific vectors for each task k .
The task-specific vectors form a rank-one matrix W k = u k · v Tk , which has the same dimensions as the shared prompt P ∗ .
The final task prompt (cid:98) P for k -th source task is then parameterized as:
[Equation]
where ◦ denotes the Hadamard product between two matrices.
Our parameterization of prompt decomposition is inspired by prior low-rank methods (Li et al., 2018; Aghajanyan et al., 2021b; Wen et al., 2020), such that general information of S can be captured by “slow” weights P ∗ shared across tasks and “fast” weights W k could encode task-specific knowledge in a low-rank subspace.
Prompt Distillation.
Simply using the mixed golden labels to guide the prompt decomposition directly can bias the shared component to overfit to larger tasks.
We found knowledge distillation from separately-trained source prompts to be an effective strategy for learning good decomposable prompts.
Specifically, we first obtain a teacher prompt P tk for the k -th source task by conventional prompt tuning.
We then randomly initialize a corresponding student prompt as (cid:99) P sk =
P ∗ ◦ ( u k · v Tk ) , where all student prompts share P ∗ and have their own task-specific vectors as described above.
Following Sanh et al.
(2019), we design distillation losses to transfer cross-task knowledge into the shared prompt matrix.
The first loss is to match the output probability distributions of students and teachers through minimizing their KL-Divergence,
[Equation]
We further use a temperature T to control the smoothness of the output distribution for both teacher and student models as p j = 1 Z exp( z j /T ) , where z i is the logit score for class j and Z is the normalization factor.
We also have an additional mean squared loss on teacher model hidden states,
[Equation]
where H tki , H ski denotes the hidden states of teacher and student networks, respectively, consisting of a sequence of hidden vectors for i -th input.
Such additional distillation loss from intermediate states has been shown to improve results in distilling PLMs (Jiao et al., 2020; Shleifer & Rush, 2020).
Finally, our total loss function to train student source prompts for obtaining a single shared prompt to be transferred to the target side is formulated as follows:
[Equation]
where L PLM = (cid:80)
k
L k PLM represents the aggregated task losses for all source tasks, and λ is a weight to balance the impact of distillation loss terms.
3. S OURCE T RAINING AND T ARGET A DAPTATION
Training the single source prompt to be transferred to target tasks requires two steps.
First, the teacher prompts for all source tasks are pretrained individually through vanilla prompt tuning.
Then, we conduct multitask training on S to jointly learn the single shared prompt via the knowledge distillation loss function in Equation 5.
We also adopt a simple stochastic task sampling strategy, which dynamically changes the number of tasks per batch.
In particular, for each batch of multitask samples, we randomly select a number k from [2 , κ ] first, then randomly choose k tasks from S and their corresponding samples to constitute mini-batches.
Such dynamic task sampling strategies are common in the PLM multitask learning literature (Raffel et al., 2020).
For target adaptation, initialize the target prompt to be the Hadamard product of the shared prompt and the low-rank target prompt matrix and optimize with the regular task loss in Equation 1.
We remark that our proposed method can also be used for multitask learning on target tasks to enable more parameter-efficient adaption of pretrained language models.
Parameter-Efficiency.
Our method is parameter-efficient during both source training and target adaptation.
Each task contains the shared prompt l × d that has the same dimensions as a vanilla soft prompt and a smaller number of task-specific vectors ( l + d ) .
Thus, the total number of tunable parameters for a single target task is ( l × d )+( l + d ) .
For a group of target tasks, the total number of tunable parameters is ( l × d ) + ( l + d ) τ , where τ is the number of target tasks.
We list and compare different methods in terms of the number of trainable parameters in Table 1.
E XPERIMENTS
We conduct extensive experiments on 21 diverse NLP datasets to show that MPT outperforms strong baselines in both full-dataset (Tables 1, 2) and few-shot adaptation settings (Table 3), while achieving more parameter-efficiency compared to existing methods (Figure 2).
We also perform comprehensive ablation studies and analysis to better understand the effect of model sizes (Figure 4), prompt length (Figure 5) and different components in our approach (Table 4).
4. E XPERIMENTAL S ETUP
Datasets and Tasks.
As in Asai et al.
we evaluate performance of MPT using 6 datasets with more than 100 k annotations as source tasks (MNLI (Williams et al., 2017), QNLI (Demszky et al., 2018), QQP (Wang et al., 2018), SST-2 (Socher et al., 2013), SQuAD (Rajpurkar et al., 2016), and ReCoRD (Zhang et al., 2018)) and 21 datasets from four benchmarks as target tasks, namely
MultiRC (Khashabi et al., 2018), BoolQ (Clark et al., 2019a), WiC (Pilehvar & Camacho-Collados, 2018), WSC (Levesque et al., 2012), and CB (De Marneffe et al., 2019) from SuperGLUE (Wang et al., 2019); RTE (Giampiccolo et al., 2007), CoLA (Warstadt et al., 2019), STS-B (Cer et al., 2017),
MRPC (Dolan & Brockett, 2005), MNLI, QQP, QNLI and SST-2 from GLUE (Wang et al., 2018);
Natural Questions (Kwiatkowski et al., 2019), HotpotQA (Yang et al., 2018), NewsQA (Trischler et al., 2017) and SearchQA (Dunn et al., 2017) from MRQA (Fisch et al., 2019); WinoGrande (Sakaguchi et al., 2021), Yelp-2 (Zhang et al., 2015), SciTail (Khot et al., 2018) and PAWS-Wiki (Zhang et al., 2019) from the Others benchmark in (Asai et al., 2022).
Models.
Following the standard prompt tuning (Lester et al., 2021; Asai et al., 2022), we mainly experiment using the publicly available pretrained T5-Base model with 220 M parameters (Raffel et al., 2020).
In our ablation we also consider T5-Small, and T5-Large with 60 M, and 770 M parameters respectively, to empirically analyze the effect of model size on MPT performance in Figure4.
Baselines.
We compare our approach with the following baselines.
(1) Full finetuning (FT), where all the model parameters are tuned during adaptation on each downstream task, (2) vanilla prompt tuning (PT) (Lester et al., 2021), where target prompt vectors are initialized by randomly sampled top vocabularies, (3) existing prompt transfer methods, including SPoT (Vu et al., 2022) and
ATTEMPT (Asai et al., 2022), that initialize target prompts by retrieving or aggregating source prompts, (4) popular parameter-efficient methods including Adapter (Houlsby et al., 2019) and BitFit (Zaken et al., 2022).
On GLUE, we also compare with several state-of-the-art methods that adapt a pretrained model to all the target tasks using multitask learning, such as HyperFomer (Mahabadi et al., 2021), HyperDecoder (Ivison & Peters, 2022), multitask variants of FT, and Adapter.
We directly quote numbers reported in published papers when possible or use publicly available source
[Table]
codes (Karimi Mahabadi et al., 2021; Mahabadi et al., 2021; Asai et al., 2022) under the same backbone and experimental settings for a fair comparison.
Implementation Details.
Following (Karimi Mahabadi et al., 2021), for all datasets, we use the development set as the testing set if the original testing set is not publicly available.
We split the original development set into the development and testing set if the training set is small; otherwise, we separate a development set from the training set and use the original development set for testing.
We limit the number of training data for Yelp to 100k.
For source training, we train MPT on the mixture of source tasks for 5 epochs with the examples-proportional mixing strategy (Raffel et al., 2020) and stochastic task sampling described in Section 3.2.
For prompt distillation, we calculate the hidden state loss for hidden states from both the encoder and decoder of T5.
For target adaptation, we reuse the shared prompt from MPT and take averaged source task-specific vectors to initialize the target task-specific vector.
We train 20 epochs on small datasets, 10 epochs on large (more than 10k examples) datasets, and 5 epochs on the MRQA dataset.
We run all the experiments three times with different random seeds and report the mean and standard deviations.
During source training, we set the default learning rate as 0 .
3 for both task-shared and task-specific components.
However, during target adaptation, we use a strategy of two-speed learning rates for those two components, as in Ponti et al.
We set the learning rate to 0 .
3 and 0 .
4 for taskshared and task-specific components during adaptation for each target task.
Following Lester et al.
(2021), we set the default number of tunable tokens per each prompt as 100 and initialize the teacher and student prompts by randomly sampling tokens from T5’s vocabulary (Raffel et al., 2020).
We set the default batch size for T5-Base as 32 and for model scaling experiments, the batch sizes for
T5-Small and T5-Large are 100, and 12 respectively.
The default input length for most tasks are set to 256 , except MultiRC and MRQA benchmark have the input length as 348 and 512 .
We set the distillation loss coefficient λ in Equation 5 to 0 .
9 and keep it fixed for all our experiments.
In few-shot experiments, for each number of shots k , we randomly sample 10 times from the training set with different random seeds and report the mean performances.
Following (Asai et al., 2022), we report our performance BoolQ, CB, and SciTail with same validation and testing sets in full-dataset setting.
Note that for few-shot setting, the source prompt learning still uses the full set of the source tasks.
We use one NVIDIA Tesla V100 GPU for training all our models.
4. R ESULTS AND A NALYSIS
Full-Dataset Adaptation.
Tables
1- 2 show the per-task performance of different methods on all four benchmarks.
As seen from Table 1 (top part), MPT establishes new state-of-the-art results for parameter-efficient finetuning on both GLUE and SuperGLUE.
When compared to vanilla
PT (Lester et al., 2021), MPT obtains more than +10% points improvement on average performance ( +13% on GLUE, +16% on SuperGLUE) with the same number of task-specific parameters, which demonstrates that multitask prompt tuning provides an effective means of improving the performance of PT, especially on small datasets.
MPT consistently outperforms SPoT (Vu et al., 2022) to obtain an average of 85 .
6% on GLUE and 74 .
1% on SuperGLUE, which is +3 .
and +11 .
2% point accuracy improvements respectively.
Furthermore, MPT achieves 2 .
1% and 3 .
6% average accuracy improvements over ATTEMPT (Asai et al., 2022), despite updating 3 × fewer parameters on GLUE and SuperGLUE respectively.
Similarly, when comparing against BitFit (Zaken et al., 2022) which only tunes the bias vectors, MPT outperforms it by +2 .
3% on GLUE and +3 .
6% on
[Table] [Table]
SuperGLUE, while tuning 2 × fewer parameters for each target task.
Among compared methods
[Table]
Adapters is the most competitive in terms of average accuracy on both benchmarks.
MPT also outperforms Adapters (Houlsby et al., 2019) on GLUE with 4 × fewer task-specific parameters.
More surprisingly
[Table]
MPT outperforms the full model finetuning baseline on both benchmarks, despite tuning 0 .
035% as many task-specific parameters (see Figures 2 for a comparison between different methods versus their number of updated parameters on GLUE and SuperGLUE benchmarks).
When compared with state-of-the-art multitask baselines which train a single model on different target tasks, including recent HyperFormer (Mahabadi et al., 2021) and HyperDecoder (Ivison &
Peters, 2022), Table 1 (bottom part) shows that our MPT ∗ (w/ prompt decomposition on target tasks) performs well and also further improves upon the single target task baseline.
This reveals the potential of our method to further leverage multitask knowledge on the target side to enable even more parameter-efficient adaptation of pretrained language models.
Table 2 shows the performance of different methods on the MRQA and Others benchmark.
Our approach significantly improves the average performance of PT by +2 .
8% on MRQA and +13 .
5% on the Others benchmark, while adding only 0 .
01% more task-specific parameters.
Similarly, MPT obtains 85 .
5% average accuracy on WinoGrande, Yelp, SciTail, and PAWS, outperforming BitFit ( 84 .
7% ), which updates 10 × more task-specific parameters.
We further investigate the performance of MPT by increasing prompt length from 100 to 300 and find an average improvement of 0 .
8% on
MRQA and 0 .
6% on Others, closing the gap between MPT and Adapters (e.g., only 0 .
1% difference in Others benchmark: see Appendix C for more details).
While our improvements being highly parameter-efficient are encouraging, the accuracy gap between MPT and the full finetuning is still significant in MRQA, which indicate opportunities for future work in multitask prompt tuning.
Few-Shot Adaptation.
Following prior works (Mahabadi et al., 2021; Asai et al., 2022), in addition to the full dataset adaptation on four benchmarks, we conduct few-shot experiments on BoolQ, CB, and SciTail tasks to measure how pretrained MPT prompts can be generalized to new tasks with only a few training examples available ( k = 4 , 16 , 32 ).
Table 3 shows the results of our approach and other baselines, including full-model finetuning, Adapter, and HyperFormer.
As can be seen from Table 3, vanilla PT performs poorly in few-shot adaptation (esp.
, CB and SciTail), suggesting randomly initialized prompts are hard to generalize to new tasks with only a few shots.
SPoT improves the performance of PT on CB and SciTail tasks, and MPT outperforms both PT and SPoT.
We also observe that other methods in Table 3 (Finetuning, Adapter, HyperFormer, and ATTEMPT) have trouble in the few-shot setting.
These results indicate that MPT can effectively use cross-task knowledge in source tasks to target tasks where there are only a few labeled examples.
Appendix D shows full performance comparison between PT and MPT on all the GLUE and SuperGLUE tasks.
[Figure]
Scaling.
We conduct scaling experiments to analyze how MPT performs with increasing pretrained model sizes on three SuperGLUE tasks as in (Asai et al., 2022).
Figure 4 shows the performance of MPT as well as full model finetuning (FT),
Adapter, prompt tuning (PT), and ATTEMPT with three different T5 models (T5-Small, T5Base, T5-Large).
Performance on WiC, MultiRC, and BoolQ shows that MPT can greatly benefit from scaling up the backbone model and outperforms
PT and ATTEMPT consistently across all model sizes.
These results show that our prompt decomposition strategy is not only able to achieve the best parameter efficiency but also effective across different model scales ranging from 60M to 770M parameters.
[Figure]
In addition to increasing model sizes, we also increase the length of the prompt l to add more parameters and compare it with vanilla
PT on SuperGLUE.
Figure 5 compares PT and MPT over various prompt lengths l = { 100, 200, 300 } .
From Figure 5, we observe that increasing prompt length for PT only produces marginal improvement, which is consistent with recent findings in (Lester et al., 2021; Li & Liang, 2021).
On the other hand, MPT can get consistent improvements till 300 prompt length to reach the average performance of 76 .
8% on SuperGLUE.
However, a further increase of prompt length from 300 to 400 lead to 1 . 8% drop in accuracy, indicating the optimal prompt length to be of 300 for our current experiments.
We leave the in-depth exploration of the relationship between prompt capacity and knowledge transfer to future work.
Analyzing Learned Prompts.
We conduct qualitative analysis on prompts learned using MPT to investigate whether cross-task knowledge is indeed encoded in the task-shared prompt which makes it easier for target tasks to effectively adapt and encode their own knowledge effectively.
[Figure]
Following (Vu et al., 2022), we leverage task embedding to compute cosine similarities between all target task pairs after adaptation, where each task is represented by the composition of task-shared and task-specific prompts (averaged to obtain a single vector).
Figure 6 shows the visualization of cosine similarity matrices for SPoT and MPT on SuperGLUE tasks.
We find that task embeddings can effectively cluster similar tasks together, such as MultiRC is similar to BoolQ, and WiC is also close to both of them.
Moreover, we also observe that MPT has clearer clusters than SPoT, which verifies our hypothesis that MPT helps target tasks to encode task-specific knowledge more effectively.
4. A BLATION S TUDIES
We conduct extensive ablation studies to measure the importance of various components of MPT and justify several important modeling strategies in our framework in the following subsections.
Effectiveness of Prompt Decomposition.
Table 4 presents our ablation studeis on SuperGLUE for testing the effect of prompt decomposition and prompt distillation.
We fix all the hyper-parameters across all settings and rerun MPT source training to get various ablated versions of the transferred prompt.
First, we ablate both prompt decomposition and distillation and initialize a vanilla prompt to be shared across all source tasks (no task-specific vectors).
We train it with the simple mixing of all datasets, then transfer the resulting prompt to target tasks for adaptation.
Table 4 shows that simply training a single soft prompt only produces an average accuracy of 69 .
5% on SuperGLUE (top row), as it fails to leverage commonalities across source tasks while minimizing interference.
To measure the effect of prompt decomposition, we replace the vanilla source prompt with our decomposable prompt with task-shared and task-specific components and train it without prompt distillation (third row in Table 4), which gives us 3 .
5% average performance improvement on SuperGLUE.
This ablation clearly indicates the importance of our proposed prompt decomposition strategy in MPT and demonstrates that the shared component can effectively capture the rich cross-task knowledge to be beneficial to target downstream tasks.
[Table]
Effectiveness of Prompt Distillation.
To test the effect of prompt distillation, we ablate prompt decomposition and train a vanilla prompt shared by all the source tasks with the same training loss of MPT in Equation 5.
The teacher models are kept the same for this ablation and
MPT.
Compared with the simple baseline (first row in Table 4), adding prompt distillation (second row) produces 1 .
1% average performance improvement, which verifies the effectiveness of prompt distillation.
With distillation, the vanilla shared prompt can be trained by more fine-grained learning signals from each task, but without prompt decomposition, knowledge of all the source tasks is entangled together, which may hurt transfer performance on target tasks.
Finally, we observe that prompt distillation combined with prompt decomposition yields the best average performance of 74 .
1% on
SuperGLUE benchmark.
This confirms that distilling knowledge from separately-trained source prompts is an effective strategy for learning good decomposable prompts.
We further investigate the individual components of prompt distillation to see their influences on final performance.
We remove the loss of hidden states from Equation 5 and find that it produces an average performance of 73 .
68% on SuperGLUE, verifying the effectiveness of regularizing hidden states in conjunction with logits to reach its full performance, which is consistent with findings in (Sanh et al., 2019).
Lastly, we consider a variant of distillation loss to match the teacher and student prompts directly by adding an MSE loss to minimize the distance between those two prompts.
Replacing our proposed distillation losses with this prompt distance loss and jointly training it with prompt decomposition yields an average SuperGLUE performance of 73 .
6% , which performs worse than the distillation losses based on logits and hidden states.
Ablation on Target Adaptation Strategies.
When transferring the shared prompt from source training to target tasks, we ablate MPT with the choices of how to tune the task-shared and task-specific components for target tasks.
We find that only updating the task-shared component (i.e., removing target task-specific vectors) or only updating task-specific vectors (i.e., freezing task-shared component) produce unsatisfied results ( 62 .
5% and 71 .
3% SuperGLUE, respectively).
This indicates the significance of keeping both components for prompt decomposition on target adaptation.
Effectiveness of Stochastic Task Sampling.
Inspired by stochastic depth for training deep neural networks (Huang et al., 2016), we propose a multitask training strategy in Section 3.2, which is to stochastically sample a various number of tasks within each mini-batch to help the shared component of MPT be robust to task variances.
Ablating this training strategy produces an average performance on SuperGLUE as 73 .
66% , which verifies the importance of this simple multitask training strategy.
C ONCLUSION
We introduced and studied multitask prompt tuning (MPT), which learns a single transferable prompt by decomposing and distilling knowledge from multiple source tasks as well as their taskspecific source prompts.
MPT decomposes the task prompt by the Hadamard product of a shared prompt matrix and a rank-one task-specific matrix.
The shared component is then transferred and adapted to target tasks to be further tuned.
Empirically we found this approach enables parameterefficient transfer learning to target downstream tasks across diverse NLP benchmarks, even outperforming full finetuning baseline in some cases despite tuning much fewer task-specific parameters.
A MPT FOR NLG T ASKS
[Table]
We follow the standard evaluation protocol of prior works (Asai et al., 2022; Vu et al., 2022) to conduct our experiments on GLUE, SuperGLUE, MRQA and Other benchmarks.
In this section, we conduct additional experiments on NLG tasks by applying T5-MPT source prompt on target
NLG tasks.
In particular, we transfer the T5-Large prompt trained using 6 diverse source tasks used in our current experiments for adaptation to two target tasks, namely E2E (Novikova et al., 2017) and
WebNLG (Gardent et al., 2017).
Table 5 shows that our proposed MPT significantly outperforms standard PT (Lester et al., 2021) on both NLG tasks across all the metrics.
Our BLEU improvements over PT are 3 .
03% and 6 .
25% on E2E and WebNLG tasks respectively, showing the effectiveness of our approach on both NLU (e.g., classification, NLI, QA tasks) and NLG tasks.
This is particularly an impressive result since the source tasks were all NLU tasks, i.e., MPT can transfer knowledge from NLU tasks to NLG tasks.
B MPT WITH MORE NUMBER OF SOURCE TASKS
[Table]
Following (Asai et al., 2022), we select 6 representative NLP tasks as source datsks, including 2 NLI, 1 paraphrase, 1 sentiment analysis and 2 large-scale QA tasks, which are general/diverse enough and can enable knowledge transfer to other tasks.
In this section, we investigate the effect of adding more remotely relevant source tasks to our proposed multitask prompt tuning.
Specifically, we add 6 additional diverse source tasks, including one topic classification (AGNews (Zhang et al., 2015)), three multi-choice QA (CommmonsenseQA (Talmor et al., 2019), OpenBookQA (Mihaylov et al., 2018),
ARC (Clark et al., 2018)), one adversarial NLI (ANLI (Nie et al., 2020)) and one commonsense (winogrande (Sakaguchi et al., 2021)) datasets.
Table 6 shows the results on MRQA and Others benchmarks.
As can be seen, MPT with 12 tasks is still very effective for target adaptation on both benchmarks, slightly outperforming MPT trained using 6 tasks.
Furthermore, it would be compelling to use benchmarks like CrossFit (Ye et al., 2021) consisting of 160 NLP tasks as source tasks for analyzing the performance of MPT on parameter-efficient transfer learning.
While we currently do not possess the compute resources for this extreme large-scale study, we hope to cover this an interesting future work.
Last but not least, we will release pretrained source task prompts and easily extendable code so that it can motivate further studies on task scaling and understanding transferabilty across a more diverse set of source and target tasks.
C P ROMPT S CALING FOR MRQA AND O THERS
Our proposed MPT outperforms both Finetuning and Adapter baslines on GLUE and SuperGLUE benchmarks (Table 1).
However, they’re still better than MPT on MRQA and Others benchmarks (Table 2) with 2832 and 24 times more parameters than MPT, respectively.
While adding the same number of prompt parameters as the Adapter to close the performance gap on MRQA and Others benchmarks is an interesting suggestion, we note that it requires a prompt length of 2400 tokens on
T5-Base, which can be computationally inefficient due to transformer’s quadratic complexity with the input length.
Table 7 shows that increasing prompt length from 100 to 300 yields an average improvement of 0.8% on MRQA and 0.6% on Others, further closing the gap between MPT and
[Table]
Adapters (e
[Table]
g., only 0.1% difference in Others benchmark).
We also test with a prompt length of 400 tokens but did not notice any significant improvements.
We believe this is because the optimal prompt length in our current experiments is around 300 tokens as discussed in our prompt scaling analysis.
Applying MPT for every layer of the pretrained model, instead of only input layer (like
P-Tuning v2 (Liu et al., 2021b)) could be a promising direction to further improve the performance: we leave this as an interesting future work
[Table]
D F EW -S HOT R ESULTS ON GLUE AND S UPER GLUE
[Table]
Following (Asai et al., 2022)
[Table]
we conduct few-shot experiments on BoolQ, CB, and SciTail tasks for a fair and direct comparison with other parameter-efficient methods, namely SpoT (Vu et al., 2022),
ATTEMPT (Asai et al., 2022) and Hyperformer (Mahabadi et al., 2021).
In this section, we conduct more comprehensive few-shot experiments on all the GLUE and SuperGLUE tasks by comparing vanilla PT (Lester et al., 2021) and MPT.
As shown in Table 8, we can observe that not only MPT outperforms the vanilla PT by a large margin in most of the datasets, but also MPT can perform very well on many datasets to reach their full-dataset performance with 16 or 32 shots, such as QQP,
QNLI, STS-B, and WSC.
These results clearly indicate that MPT can effectively use cross-task knowledge in source tasks to target tasks where there are only a few labeled examples.
E G ENERALIZATION OF MPT TO GPT M ODELS
Our current MPT is built on top of prompt tuning (Lester et al., 2021), which is mostly applied to T5 (Raffel et al., 2020).
So, we follow prior works, such as SPoT (Vu et al., 2022) and ATTEMPT (Asai et al., 2022), to conduct experiments on T5-variants.
However, our proposed approach is quite generic and can be applied to both T5 and GPT models.
This is primarily because
MPT only prepends a prompt matrix (i.e., virtual tokens) to the input embedding layer and hence can be adopted to any transformer models (encoder-only, encoder-decoder, or decoder-only), not limited to T5.
Specifically, MPT focuses on decomposing the prompt matrix into task-specific and task-shared components, which introduces minimal intrusion to the backbone model.
Similarly, the distillation part of MPT is also model-agnostic and can be generalized to GPT models.
We leave the extension of MPT to GPT models as an interesting topic for future work.