Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?
Year: 2023

Abstract:
Prompt tuning (PT) which only tunes the embeddings of an additional sequence of tokens per task, keeping the pre-trained language model (PLM) frozen, has shown remarkable performance in few-shot learning. Despite this, PT has been shown to rely heavily on good initialization of the prompt embeddings. In this work, we study meta prompt tuning (MPT) to systematically explore how meta-learning can help improve (if it can) cross-task generalization in PT through learning to initialize the prompt embeddings from other relevant tasks. We empirically analyze a representative set of meta learning algorithms in a wide range of adaptation settings with different source/target task configurations on a large set of few-shot tasks. With extensive experiments and analysis, we demonstrate the effectiveness of MPT. We find the improvement to be significant particularly on classification tasks. For other kinds of tasks such as question answering, we observe that while MPT can outperform PT in most cases, it does not always outperform multi-task learning. We further provide an in-depth analysis from the perspective of task similarity.

heading: 1 Introduction
text: Humans can easily learn to perform new tasks with only few data by leveraging previously acquired knowledge from other relevant tasks. Such capability is a hallmark of human intelligence (Carey and Bartlett, 1978). However, when it comes to the models, they often face over-fitting issues when they are tasked to learn from a few labeled examples (Lake et al., 2017; Linzen, 2020), a problem commonly termed as few-shot learning (FSL).
With the recent advancements in developing large-scale pre-trained language models (PLMs), prompt-based methods have shown promising results in FSL. Brown et al. (2020) show that by virtue of in-context (meta) learning, a frozen GPT3 model can achieve good results on a variety of
few-shot tasks through manually designed prompts, which are task instructions along with a few examples expressed in natural language. However, the performance of in-context learning has been shown to be highly sensitive to the design of such “discrete” prompts (Zhao et al., 2021). It is also limited by the maximum sequence length supported by the PLMs (Li and Liang, 2021). Down this line, efforts have been made on automatically searching and optimizing for discrete prompts (Shin et al., 2020; Schick and Schütze, 2021; Gao et al., 2021).
As an alternative to discrete prompts, recent efforts attempt to learn “soft” prompts that add additional trainable parameters (Liu et al., 2021b; Li and Liang, 2021; Lester et al., 2021), showing better results than discrete prompts (Liu et al., 2021a). Lester et al. (2021) introduce prompt tuning (PT) that prepends a sequence of tunable tokens to the input and optimize their embeddings keeping the PLM frozen. Despite its strong few-shot performance, PT has been shown to be sensitive to the initialization of the embeddings, which might limit its practical application (Qin and Joty, 2022b). To address this, Gu et al. (2022) propose pre-trained
11802
prompt tuning (PPT) to pre-train soft prompts using self-supervised tasks on unlabeled data. It relies on carefully designed pre-training tasks tailored to the downstream tasks, and the pre-training objectives are only applicable to classification tasks. Vu et al. (2022) introduce soft prompt transfer (SPoT), which uses the soft prompts learned from a set of source tasks through multi-task learning to initialize the prompt for a target task. Both PPT and SPoT demonstrate cross-task generalization (Fig. 1) – learning of a new task can benefit from learning of other related tasks (Ye et al., 2021).
In a recent survey, Lee et al. (2022) claim that meta learning (Schmidhuber, 1987) can play an important role for cross-task generalization in NLP.1
Different from multi-task learning which considers the performance on the source tasks to learn the initial parameters, meta learning aims to find initial parameters suitable for adapting to a target few-shot task. Hence, it could outperform multitask learning in several scenarios with full-model finetuning (Dou et al., 2019; Chen et al., 2020b). However, to our knowledge, there is no systematic study on the role of meta learning on PT. In a recent work, Huang et al. (2022) adopt MAML (Finn et al., 2017) for pre-training soft prompts. One major limitation of their study is that it is limited to only one type of meta learning algorithm and only sentiment classification tasks, lacking comprehensive understanding of cross-task generalization. Min et al. (2022) and Chen et al. (2022) show the effectiveness of in-context learning for PLMs, whereas we mainly focus on optimization-based meta learning.
To systematically study meta prompt tuning (MPT) for cross-task generalization, we conduct experiments on a large collection of few-shot tasks involving different types of datasets with a unified text-to-text format (Ye et al., 2021). We investigate a wide range of adaptation settings with different source/target task types, which helps better understand the capability and limitation of meta learning in PT. With extensive experiments, we aim to address the following research questions: • Q1. Can MPT improve cross-task generalization
in PT? Is it better than multi-task learning? • Q2. What happens with more labelled data for
source/target tasks (beyond few-shot settings)?
1Unless otherwise specified, by meta learning in this paper we generally refer to the optimization-based meta learning algorithms, and use more specific names for the other kinds such as in-context learning for black-box meta learning and metric learning for non-parametric meta learning.
• Q3. Does it help with more diverse source tasks? • Q4. Is the performance gain of MPT consistent
across different backbone models? To answer these questions, we empirically an-
alyze MAML (Finn et al., 2017), FoMAML and Reptile (Nichol et al., 2018), which constitute a representative set of meta learning methods. Experimental results show that MPT can indeed help cross-task generalization, e.g., MAML improves the performance of PT by more than 20% on classification tasks. However, we also notice that MPT does not always outperform multi-task learning, especially on non-classification tasks. We provide an in-depth analysis from the perspective of task similarity. As for Q2, we find that MPT does benefit cross-task generalization beyond few-shot settings. For Q3, we observe that increasing the diversity of source tasks does not necessarily improve crosstask generalization. Finally, the consistent gain of MPT across different models shows its robustness to model type and size. In summary, the two main contributions of this work are: • To the best of our knowledge, we are the first
to extensively explore how meta learning helps cross-task generalization in prompt tuning.
• With extensive experiments and analysis, we show the effectiveness and limitation of meta prompt tuning in various source/target settings.

heading: 2 Related Work
text: Few-shot Learning (FSL) FSL aims to learn a task with only a few labeled examples, which often leads to the over-fitting problem. Existing methods to address this problem mainly focus on optimizing the hypothesis space of the few-shot tasks (Triantafillou et al., 2017; Finn et al., 2017; Hu et al., 2018) or augmenting the few-shot data (Gao et al., 2020; Qin and Joty, 2022a). Recently, largescale pre-trained language models (PLMs) have demonstrated strong FSL ability through promptbased methods, including both discrete (Brown et al., 2020; Ding et al., 2022) and soft prompts (Lester et al., 2021).
Prompt-based Learning (PL) PL is a new paradigm which prepends a task-specific template or prompt to the input for learning new tasks (Liu et al., 2021a). Initial PL methods mainly focus on designing, searching or optimizing discrete prompts (Brown et al., 2020; Shin et al., 2020; Gao et al., 2021). However, discrete prompts are
hard to optimize. To solve this, recent PL methods attempt to optimize prompts in a continuous space, i.e., learn soft prompts (Li and Liang, 2021; Liu et al., 2021b; Lester et al., 2021), showing impressive FSL performance (Qin and Joty, 2022b). In addition to prompt design, several recent studies have explored the applications (Zhu et al., 2022; Li et al., 2022; Qin et al., 2023; Zhao et al., 2023) and analysis (Zhong et al., 2021; Le Scao and Rush, 2021) of PL.
Meta Learning Meta Learning or learning to learn, has been applied to boost few-shot performance on various NLP tasks, e.g., relation extraction (Han et al., 2018) and machine translation (Gu et al., 2018). Meta learning algorithms can be divided into three main categories. First, blackbox methods adopt additional meta learners to help adaptation (Santoro et al., 2016; Garnelo et al., 2018; Mishra et al., 2018; Brown et al., 2020). Second, non-parametric methods explore how to learn metrics that can compare the distances between different samples, i.e., learning to compare (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017). Finally, optimization-based methods aim to learn better parameter initialization to effectively and efficiently adapt to unseen tasks, i.e., learning to initialize (Finn et al., 2017; Nichol et al., 2018; Kedia et al., 2021). Lee et al. (2022) claim that meta learning can be effective for cross-task generalization, especially the optimization-based methods. They can be applied to various problems in a model-agnostic way to improve FSL on target tasks with model fine-tuning (Ye et al., 2021).
Summary. Existing work shows that meta learning can improve cross-task few-shot generalization with full model fine-tuning. However, there is no systematic study on whether (and how) meta learning can do so with prompt tuning of PLMs. To fill this research gap, our work provides a comprehensive understanding of the effectiveness and limitation of meta learning in prompt tuning.

heading: 3 Preliminaries
text: In this section, we revisit the basics about prompt tuning and optimization-based meta learning.

heading: 3.1 Prompt Tuning
text: Following Lester et al. (2021), we reframe all tasks into a text-to-text format. Given a training dataset D𝑡𝑟 = {(𝑋1, 𝑌1), ..., (𝑋𝑛, 𝑌𝑛)} for a task T , differ-
ent from traditional model fine-tuning, prompt tuning (PT) is a parameter-efficient learning method which freezes the PLM 𝜃 and prepends the input text 𝑋𝑖 with a sequence of tunable soft tokens 𝑃, parameterized by prompt embeddings 𝜙. The prompt embeddings 𝜙 are initialized from the vocabulary of the PLM and optimized through gradient descent with the following objective:
LT𝜙 = L(𝜙,D𝑡𝑟 ) = − 𝑛∑︁ 𝑖=1 log 𝑝(𝑌𝑖 | [𝑃, 𝑋𝑖], 𝜙, 𝜃) (1)

heading: 3.2 Optimization-based Meta Learning
text: The main goal of optimization-based meta learning (or learning to initialize), is to learn better initial parameters that can effectively and efficiently adapt to a new task T new with limited data. We denote the initial parameters (meta-parameters) as 𝜙∗.
To obtain 𝜙∗, the model needs to learn from a series of meta-training tasks Tmeta = {T1, ...,T𝑛}. The dataset D𝑖 of each task T𝑖 is divided into two disjoint sets: a support set S𝑖 and a query set Q𝑖. The objective for learning 𝜙∗ is
𝜙∗ = arg min 𝜙 ∑︁ T𝑖∈Tmeta L ( 𝜙 − 𝛼∇𝜙L(𝜙,S𝑖)︸ ︷︷ ︸
inner update
,Q𝑖 ) (2)
where L is the objective function defined in Eq. (1), 𝜙 is the set of parameters to meta-learn and 𝛼 is the inner learning rate. Denoting the overall loss as LTmeta𝜙 = ∑ T𝑖∈Tmeta L(𝜙′,Q𝑖) with 𝜙′ being the inner-updated value of 𝜙, we use gradient descent to update 𝜙 further in the meta-training stage:
𝜙 = 𝜙 − 𝛽∇𝜙LTmeta𝜙 (3)
where 𝛽 is the outer learning rate. This is actually the Model-Agnostic Meta-Learning or MAML (Finn et al., 2017). Notice that optimizing Eq. (3) requires calculating second-order gradients, which can be quite memory-consuming. To alleviate this, First-order MAML (FoMAML) and Reptile (Nichol et al., 2018) are proposed to use first-order approximations, allowing lower memory costs.
After the meta-training stage, 𝜙∗ serves as the initial parameters for learning an unseen meta-testing task T new which is usually few-shot.

heading: 4 Approach
text: In this section, we first introduce the problem setting and evaluation metric. Then, we illustrate the key methods for meta prompt tuning (MPT).

heading: 4.1 Problem Setting
text: To evaluate cross-task generalization in prompt tuning, we select a large and diverse collection of fewshot tasks from Ye et al. (2021), covering various types including classification, question answering and generation. We partition the set of all tasks T all into two disjoint parts: source tasks T src and target tasks T tgt. Details of the tasks and partitions are provided later in our experiment setup (§5).
Following Min et al. (2022), we can divide the whole learning process into two stages (Fig. 1):
• Upstream learning on source tasks In this stage, the model has access to T src, which is regarded as meta-training tasks Tmeta in Eq. (2). We divide the dataset D𝑖 of every source task T𝑖 into training (or support) and validation (or query) sets, and conduct optimization-based meta learning or multi-task learning on these sets to obtain metaparameters 𝜙∗. Note that we use both support and query sets for model training in multi-task learning to ensure fair data access for both methods.
• Downstream learning on target tasks After the upstream learning stage, we use the learned meta-parameters 𝜙∗ as the initial point for learning target tasks T tgt. Every target task T𝑘 has its own training set Dtr𝑘 , validation set Dval𝑘 , and test set Dtest𝑘 . The model is required to learn from Dtr𝑘 via prompt tuning and will be evaluated on Dtest𝑘 . The performance on Dval𝑘 is used for hyper-parameters tuning and model selection.
This two-stage learning paradigm can naturally reflect cross-task generalization where the model needs to learn an unseen task given previously acquired knowledge from other tasks.

heading: 4.2 Evaluation Metric
text: We evaluate the model performance on a set of target tasks T tgt. As T tgt may cover various task types, simply averaging the performance of different target tasks is unreasonable. Following Ye et al. (2021), we use average relative gain (ARG) as the main evaluation metric. We first calculate relative gain (RG) for each target task, i.e., relative performance improvement before and after applying the upstream (meta or multi-task) learning on the source tasks. Then we average the relative gains of all target tasks to obtain the final result which indicates the overall performance improvement.

heading: 4.3 Meta Prompt Tuning (MPT)
text: As shown in Fig. 2, the key idea of MPT is to apply optimization-based meta-training as upstream learning to a set of source tasks in order to learn meta parameters, which in this case are prompt embeddings. The learned prompt embeddings serve as the initialization for learning unseen target tasks, referred to as meta-testing or downstream learning.

heading: 4.3.1 Meta-training
text: We meta-train the prompt embeddings on source tasks T src. Without loss of generality, we take MAML (Finn et al., 2017) as an example. For every iteration, we first sample one source task T𝑖 which has a support set S𝑖 and a query set Q𝑖. Then we sample a support batch B𝑠 from S𝑖 and a query batch B𝑞 from Q𝑖. Denoting the trainable prompt embeddings as 𝜙, B𝑠 and B𝑞 are used for one gradient update with the following objective:
L𝑖𝜙 = L(𝜙 − 𝛼∇𝜙L(𝜙,B𝑠),B𝑞) 𝜙 = 𝜙 − 𝛽∇𝜙L𝑖𝜙
(4)
where L is the task loss defined in Eq. (1), and 𝛼 and 𝛽 are inner and outer learning rates, respectively. During the meta-training stage, we iterate over tasks in T src to update prompt embeddings 𝜙 for a fixed number of steps. The learned metaparameters 𝜙∗ is used in the meta-testing stage.

heading: 4.3.2 Meta-testing
text: In meta-testing, the model is expected to learn unseen target tasks T tgt. For each target task T𝑘 , we use the learned meta-parameters 𝜙∗ to initialize the prompt embeddings for the task. Denoting the training set of T𝑘 as Dtr𝑘 , the learning objective during
meta testing is defined as:
L𝜙∗ (Dtr𝑘 ) = − 𝑛∑︁ 𝑖=1 log 𝑝(𝑌𝑖 | [𝑃∗, 𝑋𝑖], 𝜙∗, 𝜃) (5)
where 𝜃 is the frozen PLM, (𝑋𝑖 , 𝑌𝑖) ∼ Dtr𝑘 is a training sample and 𝑃∗ are the prompt tokens.
We evaluate the model with the best validation performance on the test set and calculate average relative gain on the test sets of T tgt.

heading: 5 Experimental Setup
text: We first describe the source/target task partitions, and then introduce methods compared in our work. Finally, we present the implementation details.

heading: 5.1 Task Partitions
text: We experiment with ten different source/target task partitions as shown in Table 1. Depending on the type of the target tasks, we can divide these ten settings into several groups:
• R→R (Random→Random): We first experiment with the R→R setting where both source and target tasks are randomly selected, meaning that they can cover any task type. This setting mimics the learning paradigm of humans and reflects whether cross-task generalization can help obtain a general-purpose few-shot learner.
• X→Cls (X=Cls, Both, Non-Cls): The target tasks involve classification, while the source tasks can be classification, non-classification tasks or both. This setting helps us better understand the influence of the source task distribution.
• X→Non-Cls (X=Cls, Both, Non-Cls): The only difference between this and the previous setting is the type of target tasks. We investigate how
meta learning improves cross-task generalization when target tasks are non-classification tasks.
• X→QA (X=QA, Non-QA): Compared to the previous one, this group is more fine-grained. We only select target tasks from question answering (QA) instead of all non-classification tasks. We conduct experiment on different source task types, including QA and Non-QA tasks.
• NP→P (Non-Paraphrase Cls→Paraphrase): This group has the finest granularity in our setting. We choose paraphrase identification which is a sub-category of classification as the target, and non-paraphrase classification as the source. The final two groups help understand how meta learning performs in more fine-grained scenarios.
Note that we ensure that there is no overlap between the source and target tasks. Following Ye et al. (2021), we use 16 samples per class in the training (or support) and validation (or query) sets for classification tasks, and 32 samples per set for non-classification tasks. For every task, we sample the training and validation sets 5 times with different random seeds to reduce variance in few-shot evaluation and cover more diverse samples in upstream learning. We provide full details of tasks and partitions in Appendix A.1.

heading: 5.2 Methods Compared
text: We mainly use T5-Large (Raffel et al., 2019) as the backbone language model and compare the following methods in our work.
• Prompt Tuning (PT) on target tasks. It is our baseline without the upstream learning. We directly apply PT (Lester et al., 2021) to target tasks and use its performance as the basis for computing average relative gain for other methods.
• Model-Agnostic Meta-Learning (MAML). We apply MAML (Finn et al., 2017) in the upstream learning (meta-training) stage. The learned metaparameters are used to initialize prompt embeddings for learning target tasks.
• First-order MAML (FoMAML) and Reptile. We also investigate two first-order meta learning algorithms: FoMAML (Finn et al., 2017) and Reptile (Nichol et al., 2018). Compared to MAML, they are more memory-efficient.
• Multi-task learning (MTL). We conduct multitask learning on source tasks instead of meta learning to obtain initial parameters. This is a straight-forward yet effective method as demon-
strated by Vu et al. (2022).
• Fine-tuning on target tasks. Fine-tuning is the dominant paradigm where the whole language model is tuned for learning target tasks. We include it to verify whether cross-task generalization can help PT outperform fine-tuning.
In addition, we conduct experiments with different backbone models to verify MPT’s robustness.

heading: 5.3 Implementation Details
text: All our methods are implemented with PyTorch/Transformers library (Wolf et al., 2020). We use higher library (Grefenstette et al., 2019) for higher-order optimization in meta learning methods. The prompt length in PT is set to 100 tokens following Lester et al. (2021). For meta-training, we set the inner and outer learning rates to 3e−5 and 5e−1, respectively. We use 5000 for total training steps. We set the inner batch size to 2, 4 and 4, and inner update steps to 1, 1 and 10 for MAML, FoMAML and Reptile, respectively. For multi-task learning, we set the learning rate, batch size and number of epochs to 5e−1, 4 and 20, respectively. For MAML, we select the inner learning rate from {2e−5, 3e−5, 5e−5}, the outer learning rate from {2e−1, 3e−1, 5e−1}, and total training steps from {2500, 5000, 10000}. We adopt the same three hyperparameters for FoMAML and Reptile. The search range for the inner update steps of Reptile is {2, 4, 6, 8, 10}. For multi-task learning, we select the learning rate from {2e−1, 3e−1, 5e−1}, the batch size from {2, 4, 6, 8}, and the number of epochs from {5, 10, 20}.
For downstream learning, we mainly follow the settings in Ye et al. (2021). For prompt tuning, we select the learning rate from {5e−1, 4e−1, 3e−1, 2e−1} based on the validation performance. For fine-tuning, the search range for the learning rate is {5e−4, 3e−4, 2e−4, 1e−4}. We set the batch size, total training steps and evaluation interval to 8, 3000 and 50, respectively.
Since it is infeasible to search for optimal hyperparameters for each of the meta- and multi-task learning methods in each of the settings, we select them based on the R→R setting. We randomly select 5 tasks that are not in the source and target sets as validation tasks for hyperparameter search. The hyperparameters with best validation performance (ARG) are used for upstream learning. We select the inner learning rate, the outer learning rate and total training steps for MAML and adopt the same
three hyperparameters for FoMAML and Reptile.

heading: 6 Results and Analysis
text: We now address the four research questions asked before in §1 with empirical results.
Q1. Can meta prompt tuning improve cross-task generalization? Is it better than multi-task learning?
The ARG of different methods w.r.t. PT in various settings are shown in Table 2; more detailed results on every target task are in Appendix A.2.
• MPT can indeed help cross-task generalization. From the results in Table 2, we observe that MPT outperforms the baseline PT in most cases with +ve ARG scores. Out of 30 different runs for three meta learning methods in ten different settings (see the 1st block of results), MPT achieves better performance than PT in 23 runs, demonstrating its effectiveness in cross-task generalization.
For the R→R setting, MAML achieves the best performance, showing that it is a good generalpurpose few-shot learner. For adapting to classification tasks, MAML outperforms PT by 20.16% if the prompt embeddings are initialized from other classification tasks. The results in a more finegrained setting (NP→P) also indicate the ability of MAML to learn classification tasks. While Reptile performs the best (20.44%) in this setting, MAML still outperforms PT by a large margin (11.14%).
However, as shown in Table 2, MAML falls behind FoMAML when adapting to non-classification tasks. Among the three meta learning methods, FoMAML achieves the best performance (9.81%) on non-classification target tasks in the Both→NonCls setting, showing effective knowledge transfer. We observe similar results in more fine-grained settings QA/Non-QA→QA, where FoMAML outperforms MAML and Reptile significantly. While Reptile is claimed empirically to be better than MAML/FoMAML (Lee et al., 2022), it falls short of MAML/FoMAML in many cases. This might be because MAML and FoMAML are more similar compared to Reptile from a gradient perspective (Nichol et al., 2018). And since the hyperparameter search is done based on MAML (§5.3), which means Reptile’s method may be suboptimal.
In addition, we can see that meta learning helps PT outperform fine-tuning in several settings including Cls→Cls (MAML, FoMAML), Both→Cls
(FoMAML) and NP→P (MAML, Reptile), which demonstrates the superiority of MPT.
• MPT does not always outperform multi-task learning (MTL). While meta learning is specifically designed for quickly adapting to unseen target tasks, it does not always outperform MTL in PT. From Table 2, we can observe that MTL achieves better performance than MPT in many cases, especially on non-classification target tasks. We analyze the reasons as follows:
• Meta learning methods have been shown to be highly sensitive to the hyperparameters (Antoniou et al., 2019), which we could not tune exhaustively due to memory/time constraints (see Appendix A.5 for hyperparameter sensitivity analysis). As mentioned in §5.3, we select the hyperparameters of MAML using the R→R setting, and then use the same hyperparameters for all meta learning methods in all settings, which might limit the performance of MPT.
• There might be less shared structure (or features) among non-classification tasks compared to classification. The classification tasks mostly involve sentence-level classification and in some cases the task labels correlate well (e.g., AG News and DBpedia). Thus, they share some common semantics in both source and target tasks. The model can learn similar patterns (inferring the label of the entire input sentence) during both metatraining and meta-testing stages, enabling better knowledge transfer. The non-classification set on the other hand can include different types of tasks such as QA and summarization; modeling them typically requires a Seq2Seq formulation. These tasks typically lack shared task semantics. For example, the structure of QA is context + question + answer, requiring reasoning ability. In contrast, the structure of summarization is long document + short summary, requiring summarizing ability. Although it has been shown that
QA can help summarization in content selection (Arumae and Liu, 2019), it is more difficult for MPT to capture transferable knowledge as success of meta learning eventually depends on how much the tasks share (Finn, 2022). To provide an in-depth analysis of the differ-
ence between classification and non-classification tasks, we consider from the perspective of task similarity. We follow Lin et al. (2022) which shows that the correlation between input subspaces (the norm of projected subspace onto the other subspace) for two tasks can serve as the similarity score between them. We randomly pick 5 (cls,cls) task pairs as similar tasks. For dissimilar tasks, we randomly pick 5 (QA, summarization) task pairs. The average similarity score for similar task pairs is 0.768 while for dissimilar task pairs the score is only 0.306 (see Appendix A.6 for detailed results), which verifies that classification tasks share more structure than non-classification tasks.
Given the performance gap between MPT and MTL in some settings, we believe that exploring more advanced MPT methods could be a promising research direction.
Q2. What happens with more labelled data for source/target tasks (beyond few-shot settings)?
As mentioned in §5.1, we mainly explore how MPT improves cross-task generalization when both the source and target tasks are few-shot, which corresponds to the way humans learn (Lake et al., 2017). We used 16 samples per class for classification tasks, and 32 samples per dataset for nonclassification tasks. To validate whether more labelled data for source/target tasks can influence the performance of MPT, we conduct controlled experiments with {32, 64, 128, all} samples per class for source/target tasks in the Cls→Cls setting.
• Source We report the results of MAML and MTL with more labelled data for the source tasks in Fig. 3. We can observe that: (i) MPT outperforms PT (ARG = 0) and MTL in all cases including using the full dataset, showing its robustness to data sizes. (ii) Increasing the number of samples in source tasks does not necessarily lead to better cross-task generalization for MPT. The best ARG is achieved for 16-shot rather than the full dataset, which justifies using few-shot source tasks. (iii) The performance of MTL improves with more data for source tasks, showing a different learning pattern from MPT.
• Target Table 3 shows the results for increasing the number of examples in target tasks. We can see that: (i) The performance gain of MPT is evident even using the full dataset (3.27%), demonstrating that it does help cross-task generalization beyond few-shot. (ii) MPT outperforms MTL by a large margin in all settings. (iii) MTL is unstable in terms of ARG scores; while it outperforms PT in 64-shot (1.96%) and all samples (0.53%), it falls behind PT in all other settings, indicating that MPT is a better choice when adapting to classification tasks.
Method MAML FoMAML Reptile MTL Fine-tuning
T5-Large 11.14 −0.68 20.44 −5.19 1.56 T5-Base 9.24 4.15 7.96 1.64 7.41 T5-XLarge 14.35 2.46 10.74 5.72 −9.61 BART-Large 7.63 1.16 8.94 −2.37 2.74 GPT2-Large 3.19 −2.68 4.62 −1.43 3.75
MPT aims to learn to initialize the prompt embeddings from source tasks, which may cover different types. We hypothesize that the diversity of source tasks might influence its performance. To verify this, we analyze the influence of different source task selections on the same target tasks in two settings: varying the type and number of tasks.
• Type of tasks. The results of learning from different types of source tasks are reported in Table 2. The performance of MPT on non-classification target tasks improves when using more diverse source tasks, e.g., from Non-Cls/Cls→Non-Cls to Both→Non-Cls. However, for adapting to classification task, the best ARG is achieved when all source tasks are classification, i.e., the Cls→Cls setting. Hence, we can conclude that increasing the type diversity of source tasks does not necessarily improve cross-task generalization, which is consistent with the finding in Ye et al. (2021).
• Number of tasks. To investigate the impact of the number of source tasks, we conduct controlled experiments on {12, 24} source tasks sampled from the original 45 source tasks in the Cls→Cls setting (see Appendix A.4 for a full list). From Table 4, we can observe that the performance of MPT keeps improving as the number of source tasks increases, showing better cross-task generalization.
It is worthwhile to note that while our work provides some insights on the choice of source tasks, more systematic studies on how to select the most suitable source tasks given a set of target tasks are needed. We hope that future analysis can provide a more comprehensive understanding of the relationship between source and target tasks.
Q4. Is the performance gain of MPT consistent across different backbone language models?
Our experiments and analysis so far use T5-Large as the backbone model. To verify whether the performance gain of MPT is consistent across different backbone models, we extend the experiments to T5Base, T5-XLarge, BART-Large and GPT2-Large in the NP→P setting. From the results shown in Table 5, we can see that MPT still outperforms PT and MTL by a large margin when using other PLMs as the backbone model, showing its robustness to model size and type. In addition, the consistent gain of MPT with T5-XLarge could also verify the effectiveness of MPT for huge PLMs which have been shown to perform better in prompt tuning (Lester et al., 2021).

heading: 6.1 Further Analysis

heading: Prompt tuning (PT) vs. Fine-tuning (FT).
text: While PT shows strong few-shot learning ability, FT remains the dominant paradigm. As shown in Table 2, FT outperforms PT when adapting to classification tasks even in few-shot settings, which might be because PT has only a few tunable parameters. Though MPT is based on PT, its performance gain over FT in all cases suggests that it can learn to initialize the prompt embeddings from source tasks, enabling effective knowledge transfer.
Case Study To take a closer look at the influence of different source task types on a particular target task, we further conduct a case study where we ensure that the task under consideration appears in the target task partitions.2 Results are shown in Table 6; for example, the first block indicates that Amazon_Polarity appears as a target task in both R→R and Cls→Cls settings. We can observe that there is no consistent conclusion on how we should choose the source tasks for a specific target task,
2As before, we ensure it does not appear in the source.
which is consistent with our view in Q3.

heading: 7 Conclusion
text: In this paper, we have introduced meta prompt tuning (MPT), which learns to initialize the prompt embeddings for adapting to a target task. We have identified key research questions and systematically studied where and how meta learning can improve cross-task generalization in prompt tuning. We have empirically analyzed a representative set of meta learning methods in a variety of adaptation settings on a large, diverse collection of few-shot tasks. Extensive experimental results and analysis verify the effectiveness of MPT. Given the findings, in the future, we would like to explore more advanced meta learning algorithms which can consistently outperform multi-task learning.

heading: Limitations
text: Although comprehensive, our study of MPT in this work has couple of limitations: • As mentioned in §5.3, because of infeasiblity
to search for optimal hyperparameters for each of the meta learning methods in each of the ten settings, we choose to use the R→R setting as our main representative setting. This could be one of the reasons for MPT underperforming MTL in some non-classification tasks (noted in §6-Q1).
• We mainly focus on how upstream meta learning can improve the performance on target tasks. However, meta learning also enables faster convergence. We leave how it could help reduce the convergence time of PT as future work. Aside from that, meta prompt tuning (MPT) as a
method has a limitation that it is Memory-intensive. Optimization-based meta learning methods, especially MAML, are memory-intensive, which limits the tuning of the inner batch size and inner update steps (§5.3). One potential solution is to build more memory-efficient meta learning libraries.

heading: A Appendix

heading: A.1 Task List
text: We report the full list of tasks used in ten different settings in Table 9. All tasks are taken from CROSSFIT (Ye et al., 2021).

heading: A.2 Relative gain of Every Target Task
text: We mainly report average relative gain (ARG) in our experiments (§6). In this section, we show detailed relative gain of each target task in Fig. 4 ∼ Fig. 13.

heading: A.3 Absolute Scores for Every Target Task
text: We show detailed absolute scores for each target task in Fig. 14 ∼ Fig. 23.

heading: A.4 Details of Sampled Tasks
text: We sample {12, 24} tasks from the original 45 source tasks in the Cls→Cls setting to investigate the influence of the number of source tasks. The details of sampled tasks are shown in Table 10.

heading: A.5 Hyperparameter Sensitivity Analysis
text: As mentioned in §5.3, for MAML, we select the inner learning rate from {2e−5, 3e−5, 5e−5}, the outer learning rate from {2e−1, 3e−1, 5e−1}, and total training steps from {2500, 5000, 10000} in the R→R setting. The best validation performance (10.14% ARG) is achieved with {3e−5, 5e−1, 5000}, while the worst validation ARG is −16.21% when using {5e−5, 2e−1, 2500}. We can see that MPT is quite sensitive to hyperparameters. It performs even worse than PT with inappropriate hyperparameters.

heading: A.6 Task Similarity Analysis
text: As discussed in §6, we use the correlation between input subspaces for two tasks as the similarity score between them. Detailed results of randomly picked similar and dissimilar task pairs are shown in Table 7.

heading: A.7 Pilot Experiments on Prompt Transfer
text: We conduct some pilot experiments to explore the soft prompt transferability between different source tasks and a given single target task. We randomly pick 3 target tasks in the R→R setting and conduct prompt tuning on these tasks to obtain their corresponding prompt embeddings {𝑃1𝑡 , 𝑃2𝑡 , 𝑃3𝑡 }. We then conduct prompt tuning on 30 randomly selected source tasks to obtain the soft prompts {𝑃1𝑠 , ..., 𝑃30𝑠 }.
As shown in Lin et al. (2022), the correlation between input subspaces (the norm of projected subspace onto the other subspace) for two tasks could serve as the similarity score between them, which may also indicate the transferability. For each source/target task, we regard the soft prompt as the task embedding (Zhou et al., 2022) and obtain its subspace by Singular Value Decomposition (SVD) following Saha et al. (2021). We then calculate the correlation scores between a given target task and all source tasks following Lin et al. (2022).
Finally, for each target task, we apply MPT with 3 different sets of source tasks: (i) 5 source tasks with the highest correlation scores, (ii) 5 randomly picked source tasks, and (iii) 5 source tasks with the lowest correlation scores. The relative gain of every target task is shown in Table 8. We can observe that using 5 source tasks with the highest correlation scores achieves better performance than the other two settings, indicating that input subspaces could be used to measure the soft prompt transferability between different source tasks and a given single target task.
Note that current experiments and analysis are for a single target task. For the average performance of many target tasks, we need more exploration.
Task Name Reference
eli5-eli5 Fan et al. 2019 ethos-race Mollas et al. 2020 tweet_qa Xiong et al. 2019 tweet_eval-stance_hillary Barbieri et al. 2020 piqa Bisk et al. 2020 acronym_identification Pouran Ben Veyseh et al. 2020 wiki_split Botha et al. 2018 scitail Khot et al. 2018 emotion Saravia et al. 2018 medical_questions_pairs McCreery et al. 2020 blimp-anaphor_gender_agreement Warstadt et al. 2020 sciq Welbl et al. 2017 paws Zhang et al. 2019 yelp_review_full Zhang et al. 2015; (link) freebase_qa Jiang et al. 2019 anli Nie et al. 2020 quartz-with_knowledge Tafjord et al. 2019b hatexplain Mathew et al. 2020 yahoo_answers_topics (link) search_qa Dunn et al. 2017 tweet_eval-stance_feminist Barbieri et al. 2020 codah Chen et al. 2019 lama-squad Petroni et al. 2019, 2020 superglue-record Zhang et al. 2018 spider Yu et al. 2018 mc_taco Zhou et al. 2019 glue-mrpc Dolan and Brockett 2005 kilt_fever Thorne et al. 2018 eli5-asks qa Fan et al. 2019 imdb Maas et al. 2011 tweet_eval-stance_abortion Barbieri et al. 2020 aqua_rat Ling et al. 2017 duorc Saha et al. 2018 lama-trex Petroni et al. 2019, 2020 tweet_eval-stance_atheism Barbieri et al. 2020 ropes Lin et al. 2019 squad-no_context Rajpurkar et al. 2016 superglue-rte Dagan et al. 2005 qasc Khot et al. 2020 hate_speech_offensive Davidson et al. 2017 trec-finegrained Li and Roth 2002; Hovy et al. 2001 glue-wnli Levesque et al. 2012 yelp_polarity Zhang et al. 2015; (link) kilt_hotpotqa Yang et al. 2018 glue-sst2 Socher et al. 2013 xsum Narayan et al. 2018 tweet_eval-offensive Barbieri et al. 2020 aeslc Zhang and Tetreault 2019 emo Chatterjee et al. 2019 hellaswag Zellers et al. 2019 social_i_qa Sap et al. 2019 kilt_wow Dinan et al. 2019 scicite Cohan et al. 2019 superglue-wsc Levesque et al. 2012 hate_speech18 de Gibert et al. 2018 adversarialqa Bartolo et al. 2020 break-QDMR Wolfson et al. 2020 dream Sun et al. 2019 circa Louis et al. 2020 wiki_qa Yang et al. 2015 ethos-directed_vs_generalized Mollas et al. 2020 wiqa Tandon et al. 2019 poem_sentiment Sheng and Uthus 2020 kilt_ay2 Hoffart et al. 2011 cosmos_qa Huang et al. 2019 reddit_tifu-title Kim et al. 2019 superglue-cb de Marneffe et al. 2019 kilt_nq Kwiatkowski et al. 2019 quarel Tafjord et al. 2019a race-high Lai et al. 2017 wino_grande Sakaguchi et al. 2020 break-QDMR-high-level Wolfson et al. 2020 tweet_eval-irony Barbieri et al. 2020 liar Wang 2017 openbookqa Mihaylov et al. 2018 superglue-multirc Khashabi et al. 2018 race-middle Lai et al. 2017 quoref Dasigi et al. 2019 cos_e Rajani et al. 2019 reddit_tifu-tldr Kim et al. 2019 ai2_arc Clark et al. 2018 quail Rogers et al. 2020 crawl_domain Zhang et al. 2020 glue-cola Warstadt et al. 2019
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: section 6 and limitations
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Left blank.
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: Left blank.

heading: 3 B1. Did you cite the creators of artifacts you used?
text: section 5
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. open-source
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. open-source
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? datasets are provided by previous work
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. section 5
C 3 Did you run computational experiments?

heading: Left blank.

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? section 5
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? section 5

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? section 6
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants?

heading: Left blank.
text: D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We leave how it could help reduce the convergence time of PT as future work. Aside from that, meta prompt tuning (MPT) as a
method has a limitation that it is Memory-intensive. Optimization-based meta learning methods, especially MAML, are memory-intensive, which limits the tuning of the inner batch size and inner update steps (§5. 3). One potential solution is to build more memory-efficient meta learning libraries