Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning? (2023)

Abstract: Prompt tuning (PT) which only tunes the embeddings of an additional sequence of tokens per task, keeping the pre-trained language model (PLM) frozen, has shown remarkable performance in few-shot learning. Despite this, PT has been shown to rely heavily on good initialization of the prompt embeddings. In this work, we study meta prompt tuning (MPT) to systematically explore how meta-learning can help improve (if it can) cross-task generalization in PT through learning to initialize the prompt embeddings from other relevant tasks. We empirically analyze a representative set of meta learning algorithms in a wide range of adaptation settings with different source/target task configurations on a large set of few-shot tasks. With extensive experiments and analysis, we demonstrate the effectiveness of MPT. We find the improvement to be significant particularly on classification tasks. For other kinds of tasks such as question answering, we observe that while MPT can outperform PT in most cases, it does not always outperform multi-task learning. We further provide an in-depth analysis from the perspective of task similarity.

## introduction humans can easily learn to perform new tasks with only few data by leveraging previously acquired knowledge from other relevant tasks. such capability is a hallmark of human intelligence @xcite . however, when it comes to the models, they often face over-fitting issues when they are tasked to learn from a few labeled examples @xcite , a problem commonly termed as few-shot learning @xcite . with the recent advancements in developing large-scale pre-trained language models (plms), prompt-based methods have shown promising results in fsl. @xcite show that by virtue of in-context (meta) learning, a frozen gpt-3 model can achieve good results on a variety of few-shot tasks through manually designed prompts, which are task instructions along with a few examples expressed in natural language. however, the performance of in-context learning has been shown to be highly sensitive to the design of such "discrete" prompts @xcite . it is also limited by the maximum sequence length supported by the plms @xcite . down this line, efforts have been made on automatically searching and optimizing for discrete prompts @xcite @xcite . as an alternative to discrete prompts, recent efforts attempt to learn "soft" prompts that add additional trainable parameters @xcite @xcite , showing better results than discrete prompts @xcite . @xcite introduce prompt tuning (pt) that prepends a sequence of tunable tokens to the input and optimize their embeddings keeping the plm frozen. despite its strong few-shot performance, pt has been shown to be sensitive to the initialization of the embeddings, which might limit its practical application @xcite . to address this, @xcite propose pre-trained prompt tuning (ppt) to pre-train soft prompts using self-supervised tasks on unlabeled data. it relies on carefully designed pre-training tasks tailored to the downstream tasks, and the pre-training objectives are only applicable to classification tasks. @xcite introduce soft prompt transfer (spot), which uses the soft prompts learned from a set of source tasks through multi-task learning to initialize the prompt for a target task. both ppt and spot demonstrate cross-task generalization (fig. 1 ) -learning of a new task can benefit from learning of other related tasks @xcite . in a recent survey, @xcite claim that meta learning @xcite can play an important role for cross-task generalization in nlp. 1 different from multi-task learning which considers the performance on the source tasks to learn the initial parameters, meta learning aims to find initial parameters suitable for adapting to a target few-shot task. hence, it could outperform multitask learning in several scenarios with full-model finetuning @xcite . however, to our knowledge, there is no systematic study on the role of meta learning on pt. in a recent work, @xcite adopt maml @xcite for pre-training soft prompts. one major limitation of their study is that it is limited to only one type of meta learning algorithm and only sentiment classification tasks, lacking comprehensive understanding of cross-task generalization. @xcite and @xcite show the effectiveness of in-context learning for plms, whereas we mainly focus on optimization-based meta learning. to systematically study meta prompt tuning (mpt) for cross-task generalization, we conduct experiments on a large collection of few-shot tasks involving different types of datasets with a unified text-to-text format @xcite . we investigate a wide range of adaptation settings with different source/target task types, which helps better understand the capability and limitation of meta learning in pt. with extensive experiments, we aim to address the following research questions: • q1. can mpt improve cross-task generalization in pt? is it better than multi-task learning? • q2. what happens with more labelled data for source/target tasks (beyond few-shot settings)? to answer these questions, we empirically analyze maml @xcite , fomaml and reptile @xcite , which constitute a representative set of meta learning methods. experimental results show that mpt can indeed help cross-task generalization, e.g., maml improves the performance of pt by more than 20% on classification tasks. however, we also notice that mpt does not always outperform multi-task learning, especially on non-classification tasks. we provide an in-depth analysis from the perspective of task similarity. as for q2, we find that mpt does benefit cross-task generalization beyond few-shot settings. for q3, we observe that increasing the diversity of source tasks does not necessarily improve crosstask generalization. finally, the consistent gain of mpt across different models shows its robustness to model type and size. in summary, the two main contributions of this work are: • to the best of our knowledge, we are the first to extensively explore how meta learning helps cross-task generalization in prompt tuning. ## related work few-shot learning (fsl) fsl aims to learn a task with only a few labeled examples, which often leads to the over-fitting problem. existing methods to address this problem mainly focus on optimizing the hypothesis space of the few-shot tasks @xcite @xcite or augmenting the few-shot data @xcite . recently, largescale pre-trained language models (plms) have demonstrated strong fsl ability through promptbased methods, including both discrete @xcite and soft prompts @xcite . meta learning meta learning or learning to learn, has been applied to boost few-shot performance on various nlp tasks, e.g., relation extraction @xcite and machine translation @xcite . meta learning algorithms can be divided into three main categories. first, blackbox methods adopt additional meta learners to help adaptation @xcite @xcite . second, non-parametric methods explore how to learn metrics that can compare the distances between different samples, i.e., learning to compare @xcite @xcite . finally, optimization-based methods aim to learn better parameter initialization to effectively and efficiently adapt to unseen tasks, i.e., learning to initialize @xcite @xcite . @xcite claim that meta learning can be effective for cross-task generalization, especially the optimization-based methods. they can be applied to various problems in a model-agnostic way to improve fsl on target tasks with model fine-tuning @xcite . ## preliminaries in this section, we revisit the basics about prompt tuning and optimization-based meta learning. ## approach in this section, we first introduce the problem setting and evaluation metric. then, we illustrate the key methods for meta prompt tuning (mpt). ## experimental setup we first describe the source/target task partitions, and then introduce methods compared in our work. finally, we present the implementation details. ## results and analysis we now address the four research questions asked before in §1 with empirical results. q1. can meta prompt tuning improve cross-task generalization? is it better than multi-task learning? the arg of different methods w.r.t. pt in various settings are shown in for the @xmath0, maml achieves the best performance, showing that it is a good generalpurpose few-shot learner. for adapting to classification tasks, maml outperforms pt by 20.16% if the prompt embeddings are initialized from other classification tasks. the results in a more finegrained setting (@xmath1. while reptile performs the best (20.44%) in this setting, maml still outperforms pt by a large margin (11.14%). however, as shown in in addition, we can see that meta learning helps pt outperform fine-tuning in several settings including @xmath4, fomaml), @xmath5, reptile), which demonstrates the superiority of mpt. from to provide an in-depth analysis of the difference between classification and non-classification tasks, we consider from the perspective of task similarity. we follow @xcite which shows that the correlation between input subspaces (the norm of projected subspace onto the other subspace) for two tasks can serve as the similarity score between them. we randomly pick 5 (cls,cls) task pairs as similar tasks. for dissimilar tasks, we randomly pick 5 (qa, summarization) task pairs. the average similarity score for similar task pairs is 0.768 while for dissimilar task pairs the score is only 0.306 (see appendix a.6 for detailed results), which verifies that classification tasks share more structure than non-classification tasks. given the performance gap between mpt and mtl in some settings, we believe that exploring more advanced mpt methods could be a promising research direction. as mentioned in §5.1, we mainly explore how mpt improves cross-task generalization when both the source and target tasks are few-shot, which corresponds to the way humans learn @xcite . we used 16 samples per class for classification tasks, and 32 samples per dataset for nonclassification tasks. to validate whether more labelled data for source/target tasks can influence the performance of mpt, we conduct controlled experiments with {32, 64, 128, all} samples per class for source/target tasks in the @xmath6. mpt aims to learn to initialize the prompt embeddings from source tasks, which may cover different types. we hypothesize that the diversity of source tasks might influence its performance. to verify this, we analyze the influence of different source task selections on the same target tasks in two settings: varying the type and number of tasks. the performance of mpt on non-classification target tasks improves when using more diverse source tasks, e.g., from non-cls/@xmath7. however, for adapting to classification task, the best arg is achieved when all source tasks are classification, i.e., the @xmath8. hence, we can conclude that increasing the type diversity of source tasks does not necessarily improve cross-task generalization, which is consistent with the finding in @xcite . it is worthwhile to note that while our work provides some insights on the choice of source tasks, more systematic studies on how to select the most suitable source tasks given a set of target tasks are needed. we hope that future analysis can provide a more comprehensive understanding of the relationship between source and target tasks. q4. is the performance gain of mpt consistent across different backbone language models? our experiments and analysis so far use t5-large as the backbone model. to verify whether the performance gain of mpt is consistent across different backbone models, we extend the experiments to t5-base, t5-xlarge, bart-large and gpt2-large in the @xmath9. from the results shown in ## conclusion in this paper, we have introduced meta prompt tuning (mpt), which learns to initialize the prompt embeddings for adapting to a target task. we have identified key research questions and systematically studied where and how meta learning can improve cross-task generalization in prompt tuning. we have empirically analyzed a representative set of meta learning methods in a variety of adaptation settings on a large, diverse collection of few-shot tasks. extensive experimental results and analysis verify the effectiveness of mpt. given the findings, in the future, we would like to explore more advanced meta learning algorithms which can consistently outperform multi-task learning.