Meta-Learning of Prompt Generation for Lightweight Prompt Engineering on Language-Model-as-a-Service (2023)

Abstract: Recently, many companies have been providing the capabilities of large language models as services. These Language-Model-as-a-Service (LMaaS) offerings support a variety of user tasks through in-context learning from prompts, which include instructions and demonstrations of the task. However, for users, manually crafting prompts or running automatic prompt tuning methods themselves can be demanding. Despite these challenges, LMaaS providers do not offer automatic prompt engineering methods as part of their services. One of the major obstacles to deploying them on an LMaaS is the heavy computational costs associated with automatic prompt engineering methods. These methods are typically designed to iterate through tens of thousands of examples, which impose unaffordable overheads for LMaaS providers. In this paper, we introduce MetaL-Prompt, a novel lightweight automatic prompt generation method for LMaaS. MetaL-Prompt meta-trains a prompt generation model (PGM) to enable robust learning by the language model from the contexts created by the generated prompts (i.e., in-context learning). Thanks to our meta-learning approach, a PGM can generate prompts for unseen tasks without requiring additional training for those specific tasks. Furthermore, the PGM can generate prompts with a single forward pass, significantly reducing computational costs compared to previous methods. We evaluate MetaL-Prompt on a range of unseen tasks and find that it improves performance by up to 19.4% in terms of mean F1 score on QA datasets compared to the state-of-the-art baseline P-tuning, with limited computational cost.

## introduction many companies have recently succeeded to train large-scale language models (lms), and have provided these models' capabilities as services @xcite @xcite . to support diverse user tasks, such language-model-as-a-services (lmaass) leverage in-context learning, where a model learns the task from a prompt (i.e., demonstration examples or a natural language instruction) provided by the user. while in-context learning has shown remarkable performances and flexibility on various natural language processing (nlp) tasks, it still suffers from unstable performances @xcite @xcite , which motivates research on automatic prompting methods. one of the major automatic prompt tuning methods is gradient-based prompt tuning @xcite @xcite , which updates a parameterized prompt with gradients. on the other hand, gradient-free prompt tuning tackles the problem without leveraging gradients. some gradient-free methods generate prompts with pre-trained lms @xcite or task-specific reinforcement learning (rl) agents @xcite , while others tune prompts using evolutionary search @xcite . even though such methods have shown outstanding performances, these methods typically require too heavy computation costs to be deployed on lmaass. in this paper, we present metal-prompt, a novel and lightweight prompt engineering method that leverages meta-learning for prompt generation. metal-prompt effectively trains a prompt generation model (pgm), which is based on an lm, to facilitate robust learning of an lm from the contexts (referred to as in-context learning) created by the generated prompts (section 3.1). the key advantage of our meta-learning approach is that it enables the pgm to generate prompts for an extensive range of unseen tasks without requiring additional task-specific training of the model. we also propose trainable padding (section 3.2) to further enhance prompt generation efficiency by reducing the multi-pass generation process to a single forward pass. as a result, metal-prompt significantly reduces the computational cost compared to previous methods that typically involve thousands of forward or backward passes. prompts generated by the pgm also can be additionally tuned with gradient-based prompt tuning for further improved prompt qualities if such computation costs are allowed. in order to utilize metal-prompt for an lmaas, the model provider initiates the process by training a pgm using our meta-learning algorithm in the offline stage (figure 1 (a) ). once the service is operational, the trained pgm is employed to generate appropriate prompts from few-shot examples provided by a user, without the need for additional training (figure 1 (b) ). the generated prompt is then saved and consistently reused to fulfill any subsequent user requests for the same task (figure 1 (c) ). in addition, we delve into the generation of continuous prompts (section 3.3), which are realvalued vectors similar to word embeddings, in addition to the conventional discrete prompts expressed in natural language. despite the proven effectiveness of continuous prompts in gradientbased prompt tuning @xcite @xcite , prior prompt generation methods @xcite @xcite have not yet explored their potential. we compare three different approaches to continuous prompt generation and one approach to discrete prompt generation using pgms. through empirical analysis, we demonstrate that continuous prompts yield superior performance compared to discrete prompts, thanks to their enhanced expressiveness. to the best of our knowledge, metal-prompt stands as the pioneering approach that employs meta-learning for prompt generation utilizing fewshot examples. while flipped @xcite also employs meta-training for prompt generation, it is tailored for a new inference method that selects the label most likely to generate a pre-defined task instruction. flipped's model scores alignment between a single data instance and a pre-defined task instruction, and it is not explicitly designed to generate prompts from few-shot examples. other related approaches such as p-tuning v2 @xcite , mpt @xcite , and metaprompting @xcite ) also leverage meta-learning or multi-task learning for effective prompt engineering. however, their focus lies in learning improved initialization for gradient-based prompt tuning rather than the direct generation of prompts. we evaluate metal-prompt in diverse metalearning settings where there is no overlap between training datasets and test datasets (i.e., unseen tasks). on qa datasets, metal-prompt shows up to 19.4% gain in mean f1 score compared to the state-of-the-art prompt engineering methods, p-tuning, when only generated prompts without demonstrations are given for in-context learning. one of the most popular methods to adapt the lm to a user's task on lmaas is in-context learning, where a prompt -a (natural language) task instruction or/and demonstration examples for the task -is given to help the lm understand the task. manual prompt tuning was initially studied to support in-context learning. however, manually crafted prompts require extensive human efforts to devise and sometimes show unstable and unaccountable behavior. ## metal-prompt to address the challenges associated with automatic prompt tuning in lmaas, we introduce metal-prompt, a meta-learning approach for a lightweight prompt generation. in this approach, we meta-train a prompt generation model (pgm) (section 3.1), which is initialized with a target language model (lm), with the objective of generating prompts that enhance the lm's contextual learning capabilities. we refer to this training process as meta-learning because the pgm learns generation of prompts that effectively induce meaningful contexts for the target lm to learn from (i.e., learningto-learn). we also propose the use of trainable padding (section 3.2) to alleviate the overhead of the prompt generation process, which requires multiple forward passes, during the meta-learning. additionally, we explore various types of prompts that pgms can generate, as discussed in section 3.3, which have not been explored in previous prompt generation methods. the overall workflow of an lmaas with metal-prompt is depicted in figure 1 . initially, the model provider trains a pgm using metal-prompt (figure 1 (a) ). importantly, this training process does not impact the ongoing service as the training occurs prior to the service initiation. during the service, when a user provides a set of few-shot examples for the user's specific task, the prompt generation model generates a prompt based on these few-shot examples (figure 1 (b) ). this generated prompt is then saved and associated with the user's future requests. finally, when the user submits a request pertaining to the task, the prompt is composed to the query, and the composed input is fed into the lm to generate the response (figure 1 (c)). metal-prompt offers significant advancements over prior works in two key aspects: prompt quality and computation cost for prompt generation. the prompts that metal-prompt generate empirically demonstrate more accurate or comparable prediction quality compared to prompts crafted by previous gradient-based or gradient-free approaches @xcite @xcite , given limited computation budgets. in terms of computational efficiency, metal-prompt surpasses previous methods by requiring only a single forward pass for prompt generation, as explained in section 3.2. this streamlined approach is highly productive, especially when contrasted with prior approaches that necessitate an extensive number of forward or backward passes to tune or generate a prompt for a single task, as discussed in section 2. the reduction in computation cost achieved by metal-prompt significantly enhances its practicality and efficiency with lmaas. ## experimental setup in this section, we present the setup of our experiments, which includes datasets, training and evaluation details, baselines, and models. ## experimental results we evaluate metal-prompt on the setup presented in section 4. we first show the performance of metal-prompt when the generated or tuned prompt is solely provided without additional demonstrations (section 5.1). we also provide experimental computation costs to prove metal-prompt's costefficiency in (section 5.2). then, we explore a trade-off between the number of examples used for in-context learning demonstrations for lms and those for prompt generation (section 5.3). we additionally validate whether the pgms are capable of generalizing to inference settings that involve a different number of demonstrations from training settings (section 5.4). finally, we compare the performances of prompt designs we presented in section 3.3 (section 5.5). ## conclusion in this paper, we propose metal-prompt, a novel and lightweight prompting method for lmaas. metal-prompt meta-trains a prompt generation model (pgm) for better in-context learning from the generated prompt, accompanying trainable padding for more efficient prompt generation. thanks to the meta-learning, the trained pgm does not require additional training for unseen user tasks, and it can generate a prompt with a single forward pass, allowing additional tuning of the prompt. moreover, we explore the generation of continuous prompts using a pgm, which has not yet been discussed in the previous prompt generation studies. with the proposed designs, metal-prompt achieves performance gains over five baselines up to 19.4% for mean f1 score on qa datasets with less computation cost for prompt generation than the baselines. the results support the efficiency of metal-prompt in terms of model performance and computational cost.