M eta P rompting: Learning to Learn Better Prompts (2022)

Abstract: Prompting method is regarded as one of the crucial progress for few-shot nature language processing. Recent research on prompting moves from discrete tokens based “hard prompts” to continuous “soft prompts”, which employ learnable vectors as pseudo prompt tokens and achieve better performance. Though showing promising prospects, these soft-prompting methods are observed to rely heavily on good initialization to take effect. Unfortunately, obtaining a perfect initialization for soft prompts requires understanding of inner language models working and elaborate design, which is no easy task and has to restart from scratch for each new task. To remedy this, we propose a generalized soft prompting method called MetaPrompting, which adopts the well-recognized model-agnostic meta-learning algorithm to automatically find better prompt initialization that facilitates fast adaptation to new prompting tasks. Extensive experiments show MetaPrompting tackles soft prompt initialization problem and brings significant improvement on three different datasets (over 7 points improvement in accuracy for 1-shot setting), achieving new state-of-the-art performance.

## introduction enabling models to learn from a few labeled examples, i.e., few-shot learning (fsl), is one of the key steps toward more human-like artificial intelligence. recently, taking advantage of largescale pretrained language models (plm) @xcite , prompting-based methods achieve impressive results for few-shot learning of natural language processing (nlp) @xcite @xcite . prompting-based methods insert a piece of text, i.e. prompts, to the input examples, so that the fewshot task can be formulated as a (masked) language x denotes the query sentence, and z is learnable pseudo tokens in soft prompts. ϕ represents all trainable parameters. metaprompting exploits optimizationbased meta-learning to find an initialization ϕ meta that facilitates better and faster adaptation to new tasks. modeling problem. for example, say we want to classify the sentiment of the book review "i will never read it again.", we can append a prompt "it was" to the sentence, getting "i will never read it again. it was". it is natural to expect a higher probability from the plm to generate "terrible" than "great" then. such converting bridges the gap between pre-training and target tasks. consequently, it has better transferability and less dependence on target task data. the performance of prompting methods is found to be greatly affected by the design of prompts @xcite . that is, a good prompt makes significant difference. early attempts take manuallydesigned prompts or search prompts automatically. @xcite and @xcite explore to automatically identify label words. in pursuit of better performance compared to handpicked prompts, @xcite proposes lm-bff to search both prompt templates and label words. autoprompt @xcite leverages gradient-based searching to find out the best prompts. these prompts consist of discrete tokens, which limits the prompt search space. to further liberate the potential of prompts, recent works employ learnable vectors as prompt content and learn optimal prompts in continuous space, which is so-called "soft prompts" @xcite . since they no longer require prompts to be composed of real words, these methods greatly expand the possibilities of prompts and thus achieve better performance @xcite . however, despite the promising prospects of soft prompts, learning a good prompt is still far from trivial. because soft-prompts search for optimal solutions in an infinite continuous space, the choice of the starting point for the search (i.e., prompt initialization) becomes crucial. soft-prompt is observed to be more sensitive to different initialization than discrete prompts in low data setting @xcite . unfortunately, creating a perfect prompt initialization requires both understanding of lms' inner workings and trial-anderror. @xcite initialize soft prompt with the token embeddings of hand-crafted prompt directly. @xcite search discrete tokens as better initialization, which shows better performance. what's worse is that these initializations are task-bounded. every time we confront a new task, the costly process of initialization design has to start from scratch. in this paper, to tackle the above issues, we let loose the prompt design of a specific task, but instead focus on obtaining a task general prompt initialization that facilitates faster and better adaptation to new prompting tasks. recently proposed optimization-based meta-learning algorithms, such as maml @xcite and reptile @xcite , achieve better adaption by learning a parameter initialization. following their essence, we tackle soft prompt initialization problem by proposing metaprompting, which is a generalized soft prompting method powered by meta-learning algorithms. metaprompting learns general metaknowledge from source domain tasks to form a better soft prompt initialization, and thus adapts faster and better across various target domain tasks (see figure 1 ). extensive experiments show that metaprompting achieves promising performance with desired robustness. we summarize the main contribution of this paper as follows: all code and data will be publicly available at https://github.com/dousia/metaprompting . ## preliminaries and related works in this section, we review related work and provide preliminaries about language model prompting and meta-learning. ## method since prompt-based methods, especially those adopting soft prompts, are very sensitive to parameter initialization @xcite , we introduce optimization-based meta-learning methods into prompting methods to find better initialization points for prompt-based models and further explore their capabilities in fewshot scenarios. in this section, we first introduce the prompt-based model tuning process used in our method ( §3.1), and then describe how to construct meta prompting tasks ( §3.2). finally, we elaborate and formulate the meta prompting objective and parameter updating strategies ( §3.3 and §3.4). ## experiment we conduct experiments by evaluating the proposed methods on four widely-used benchmark datasets with various low resource settings. ## conclusion in this paper, we introduce a generalized optimization-based meta-learning approach metaprompting for few-shot nlp problems. utilizing sampled meta tasks and meta-learning-based optimization, metaprompting learns to find an initialization that alleviates soft prompt initialization problem, and allows better and faster adaption to new tasks. extensive experiments on four few-shot learning benchmarks show that metaprompting significantly outperforms vanilla soft-prompting models and strong meta-learning baselines, achieving new state-of-the-art performance.