Meta-Learning of Prompt Generation for Lightweight Prompt Engineering on Language-Model-as-a-Service
Authors: Hyeonmin Ha
Jihye Lee
Wookje Han
Byung-Gon Chun
Venue: EMNLP 2023 Findings (2023)
Keywords: prompt tuning
prompt engineering
in-context learning
language model

Abstract: Recently, many companies have been providing the capabilities of large language models as services.
These Language-Model-as-a-Service (LMaaS) offerings support a variety of user tasks through in-context learning from prompts, which include instructions and demonstrations of the task.
However, for users, manually crafting prompts or running automatic prompt tuning methods themselves can be demanding.
Despite these challenges, LMaaS providers do not offer automatic prompt engineering methods as part of their services.
One of the major obstacles to deploying them on an LMaaS is the heavy computational costs associated with automatic prompt engineering methods.
These methods are typically designed to iterate through tens of thousands of examples, which impose unaffordable overheads for LMaaS providers.
In this paper, we introduce MetaL-Prompt, a novel lightweight automatic prompt generation method for LMaaS. MetaL-Prompt meta-trains a prompt generation model (PGM) to enable robust learning by the language model from the contexts created by the generated prompts (i.e., in-context learning). Thanks to our meta-learning approach, a PGM can generate prompts for unseen tasks without requiring additional training for those specific tasks.
Furthermore, the PGM can generate prompts with a single forward pass, significantly reducing computational costs compared to previous methods. We evaluate MetaL-Prompt on a range of unseen tasks and find that it improves performance by up to 19.4\% in terms of mean F1 score on QA datasets compared to the state-of-the-art baseline P-tuning, with limited computational cost.

Meta-Learning of Prompt Generation for Lightweight Prompt Engineering on -eguuage-Modl--s----Seee
Hyeonmin Ha!
Jihye Lee²
Wookje Han
Byung-Gon Chun
'FriendliA
Seoul National
University
CCoumbia University
hyeonmin. ha@friendliiic... ai
(mmodestas taa, bgchun
wookje han@columbi. edu
Abstract
Recently, many companies have been provid- ing the capabilities of large language models as services. These
(b) Examples
Prompt Generation Model
(LMaaS) offerings support a variety of user tasks through in-context learning from prompts, which include instructions and demonstrations of the task. However, for users, manually craft- ing prompts or running automatic prompt tun- ing methods themselves can demanding. Despite these challenges, LMaaS providers do not offer automatic prompt engineering methods as part of their services. One of the major obstacles to deploying them LMaaS the heavy computational costs as- sociated with automatic prompt engineering methods. These methods are typically designed to iterate through tens of thousands of exam- ples, which impose unaffordable overheads for LMaaS providers. In this paper, we introduce Merap,-Prompt, novel lightweight automatic prompt generation method for LMaaS. MetaL- Prompt meta-trains prompt generation model (PGM) to enable robust learning by the lan- guage model from the contexts created by the generated prompts (i.e., in-context learning). Thanks to our meta-learning approach, PGM can generate prompts for unseen tasks with- out requiring additional training for those spe- cific tasks. Furthermore, the PGM can gen- erate prompts with a single forward pass, sig- nificantly reducing computational costs com- pared to previous methods. We evaluate MetaL- Prompt on a range of unseen tasks and find that it improves performance by up 19.4% in terms of mean F1 score on QA datascts com- pared to the state-of-the-ee baseline P-tuning, with limited computational cost.
Prompt A
Prompi B
Language Model
Requests
Each LM
Each task
Each requeet
Figure workflow of MetaL-Prompt on LMaaS.
Language-Noodel-----Srvvvee (LMaaSs) leverage in-context learning, where model learns the task from prompt (i.c., demonstration examples natural language instruction) provided by the user. While in-context learning has shown remarkable performances and flexibility on various natural lan- guage processing (NLP) tasks, it still suffers from unstable performances (Lu et al., 2022; Perez et al., 2021: Zhao et al., 2021), which motivates research on automatic prompting methods.
One of the major automatic prompt tuning methods is gradient-based prompt tuning (Liu et al., 2021: Lester al., 2021: Li and Liang, 2021). which updates a parameterized prompt with gradients. the other hand, gradient- free prompt tuning tackles the problem without leverag- ing gradients. Some gradient-free methods gen- erate prompts with pre-trained LMs (Honovich et al., 2022; Zhou et al., 2022) or task-specific reinforeement learning (RL) agents (Deng et al., 2022), while others tune prompts using evolution- ary search (Sun et al., 2022b,a). Even though such methods have shown outstanding performances, these methods typically require too heavy compu- tation costs to be deployed on LMaaSs.
1 Introduction
Many companies have recently succeeded to train large-scale language models (LMs), and have pro- vided these models' capabilities as services (Ope- nAI, 2020; Cohere, 2021; A121Labs, 2021; An- thropic, 2023). To support diverse user tasks, such

---

In this paper, we present MetaL-Prompt, novel and lightweight prompt engineering method that leverages meta-learning for prompt generation. Metal-Proompt effectively trains a prompt genera- tion model (PGM), which is based on an LM, to facilitate robust learning of an LM from the con- texts (referred to as in-context learning) created by the generated prompts (Section 3.1). The key advantage of our meta-learning approach is that it enables the PGM to generate prompts for an exten- sive range of unseen tasks without requiring addi- tional task-specific training of the model. We also propose trainable padding (Section 3.2) to further enhance prompt generation efficiency by reducing the multi-pass generation process to single for- ward pass. As a result, Metal-Prompt significantly reduces the computational cost compared to pre- vious methods that typically involve thousands of forward or backward passes. Prompts generated by the PGM also can be additionally tuned with gradient-based prompt tuning for further improved prompt qualities if such computation costs are al- lowed.
To the best of our knowledge, MetaL-Prompp stands as the pionecring approach that employs meta-learning for prompt generation utilizing few- shot examples. While FLIPPED (Ye et al., 2023) also employs meta-training for prompt generation, it is tailored for a new inference method that selects the label most likely to generate pre-defined task instruction. FLIPPED's model scores alignment between single data instance and pre-defined task instruction, and it is not explicitly designed generate prompts from few-shot examples. Other related approaches such as P-tuning v2 (Liu et al., 2022), MPT (Wang et al., 2023), and MetaPrompt- ing (Hou et al., 2022b) also leverage meta-learning or multi-task learning for effective prompt engi- neering. However, their focus lies in learning im- proved initialization for gradient- based prompt tun- ing rather than the direct generation of prompts.
We evaluate Metal_-Prompt in diverse meta- learning settings where there is no overlap be- tween training datasets and test datasets (i.e., un- seen tasks). On QA datasets, MetaL-Prompt shows up to 19.4% gain in mean Fl score compared to the state-of-the-art prompt engineering methods, P-tuning, when only generated prompts without demonstrations are given for in-context learning.
In order to utilize MetaL-Prompt for an LMaaS, the model provider initiates the process by training a PGM using our meta-learning algorithm in the offline stage (Figure 1 (a)). Once the service is operational, the trained PGM is employed to gen- erate appropriate prompts from few-shot examples provided by user, without the need for additional training (Figure (b)). The generated prompt is then saved and consistently reused to fulfill any sub- sequent user requests for the same task (Figure 1 (c)).
Background and Related Work
Language Model as a Service As large-scale language models (LMs) show astonishing per- formances across different tasks, several compa- nies train such models and provide them as ser- vices (OpenAI, 2020; Cohere, 2021; Al21 2021; Anthropic, 2023) which is often called Moniage-Modellas---SSevvi (LMaaS) (Sun al., 2022b). On an LMaaS, user sends an input text to the service via the API, then she can obtain cor- responding outputs from the LM.
In addition, we delve into the generation of continuous prompts (Section 3.3), which are real- valued vectors similar to word embeddings, in addition the conventional discrete prompts ex- pressed in natural language. Despite the proven effectiveness of continuous prompts in gradient- based prompt tuning (Lester et al., 2021; Liu et al., 2021; Li and Liang, 2021; Sun et al., 2022a), prior prompt generation methods (Honovich et al., 2022; Zhou et al. 2022; Deng et al., 2022) have not yet ex- plored their potential. We compare three different approaches to continuous prompt generation and one approach to discrete prompt generation using PGMs. Through empirical analysis, we demon- strate that continuous prompts yield superior per- formance compared to discrete prompts, thanks to their enhanced expressiveness.
One of the most popular methods to adapt the LM to user's task on LMaaS is in-context learn- ing, where prompt- instruction or/and demonstration examples for the task - is given to help the LM understand the task. Manual prompt tuning was initially studied to support in-context learning. However, manually crafted prompts require extensive human efforts to devise and sometimes show unstable and unac- countable behavior.
(natural language) task
Automatic Prompt Tuning To resolve the problems, recent studies suggest techniques to au- tomatically search for optimal prompts. One of the

---

Method
# examples
few-shot settings, we report the costs given 16 ex- amples. Otherwise, we provide the actual costs as demonstrated the papers' experiments. Con- sidering the vast number of users that an LMaaS serves, each with unique tasks, processing such a large number of examples for a single task becomes excessively burdensome. This highlights the need for lightweight prompt tuning method specifically tailored for LMaaS.
P-tuning (Liu al., 2021)
56,000
SoftPrompt (Lester et al., 2021)
960,000
RLPrompt (Deng et al., 2022)
96,000 192.000
TEMPERA (Zhang al., 2022)
65,536
MctaPrompting (Hou al., 2022b)
240
APE (Zhou al., 2022)
12,800
BBTv2 (Sun et al., 2022a)
256,000
Clip-Tuning (Chai et al., 2022)
16,000 32,000
BDPL (Diao et al., 2023)
32,000 128,000
Prefix-tuning (Li and Liang, 2021)
626,590
MetaL-Prompt (ours)
16
Note that MetaPrompting and MctaL-Prommp (ours) require meta-learning, which is preliminary process for prompt engineering and is conducted once and not for each task. Since the meta-learning processes introduce constant costs independent of the number of users, do not count the meta- learning processes as the costs in Table which demonstrate the costs and the scalability of the baselines with respect to the number of users.
Table I: Number of examples that each prompt tuning
method is required to process for each task in the exper-
iments of the original papers.
widely used approaches is a gradient-based method, which keeps updating prompt tokens or parame- terized prompt embeddings leveraging gradients (Liu et al., 2021, 2022; Lester et al., 2021; Li and Liang. 2021; Shin al., 2020; Hou al., 2022b). P-tuning v2 (Liu et al., 2022), MPT (Wang al., 2023), and MctaPrompting (Hou et al., 2022b) pro- posed methods to search for better initialization of the gradient-based prompt tuning using multi- task learning or meta-learning. The other line of work is a gradient-free ffree method, which generates prompts using pre- mreitrained LMs (Hou et al., 2022a; Honovich et al., 2022: Zhou al., 2022), trains RL agents to generate prompts (Deng et al., 2022; Zhang et al., 2022), uses search-based optimiza- tion (Prasad et al., 2022; Sun et al., 2022b,a; Chai et al., 2022) or uses a gradient estimator (Diao et al., 2023).
3 MetaL-Prompt
To address the challenges associated with auto- matic prompt tuning in LMaaS, we introduce MetaL-Prompt, meta-learning approach for lightweight prompt generation. In this approach, we meta-train a prompt generation model (PGM) (Section 3.1), which is initialized with target lan- guage model (LM), with the objective of generating prompts that enhance the LM's contextual learning capabilities. We refer to this training process as meta-learning because the PGM learns generation of prompts that effectively induce meaningful con- texts for the target LM to learn from (i.e., learning- to-learn). We also propose the use of trainable padding (Section 3.2) to alleviate the overhead of the prompt generation process, which requires mul- tiple forward passes, during the meta-learning. Ad- ditionally, we explore various types of prompts that PGMs can generate, as discussed in Scction 3.3, which have not been explored in previous prompt generation methods.
Automatic Prompt Tuning on LMaaSs Even though automatic prompt tuning methods have proven to be effective, their adoption in Modiuugg--Moel--s-----
(LMaaS) is cur- rently lacking. While users can attempt to cre- ate prompts by running gradient-free prompt tun- ing methods using the provided LMaaS APIs, this approach poses challenges, particularly for non- experts who may find it difficult to deploy and exe- cute such methods in their private environments.
The overall workflow of an LMaaS with MetaL- Prompt is depicted in Figure 1. Initially, the model provider trains a PGM using MetaL-Prompt (Fig- ure (a)). Importantly, this training process does not impact the ongoing service as the training oc- curs prior to the service initiation. During the ser- vice, when user provides set of few-shot exam- ples for the user's specific task, the prompt gen- eration model generates a prompt based on these few-shot examples (Figure (b)). This generated prompt is then saved and associated with the user's 's
The primary obstacle to integrating both gradient-based and gradient-free prompt tuning methods into LMaaS lies in their substantial com- putational costs. In Table 1, we present the number of examples (often tens of thousands) that an LM needs to process in order to optimize prompt for each task using various prompt engineering meth- ods. In cases where the original papers present

---

Prompt Generation Model ($ 3.1)
LM
as follows:
Trainable Padding ($ 3.2)
X ?
x
Brd
arg max Σ (X) X 01) 0-
0p
Q
n.giiv
0 nigle
Layer 75
(1)
K
Prompt ($ 3.3)
where Or and Op are the parameters of the LM and the PGM respectively, yi is the ex- pected answer for the input text Zt. and x!" {xf vho
Attention & MLP
Attention MLP
X
Inibialize
Head
o v.-1--.3 and X { Vio If,, vf..... are concatenations of ex- amples. Xp and XL consist of examples from various NLP tasks. The prompt generation pro- cess for can be realized different ways, and one straightforward example is choosing the most probable next tokens using probabilities predicted by the PGM. Further details regarding the prompt generation process will be discussed in-depth in Section 3.3.
CC
Figure 2: An illustration of meta-learning of a prompt generation model.
future requests. Finally, when the user submits request pertaining to the task, the prompt is com- posed to the query, and the composed input is fed into the LM to generate the response (Figure 1 (c)).
The parameter Op is initialized with target LM to leverage its existing understanding of various NLP tasks. In essence, MetaL-Prompt employs Equation I to adapt the LM and obtain PGM. For this adaptation process, we utilize LoRA (Hu al., 2022), parameter-efficine instead of full-parameter fine-tuning. Note that the original LM is frozen and only PGM is tuned during the meta-learning.
MetaL-Prompp offers significant advancements over prior works in two key aspects: prompt qual- ity and computation cost for prompt generation. The prompts that MetaL-Prompt generate empir- ically demonstrate more accurate or comparable prediction quality compared to prompts crafted by previous gradient-based or gradient-free ap- proaches (Liu et al., 2021; Lester et al., 2021; Sun et al., 2022a; Deng al., 2022), given limited com- putation budgets. In terms of computational ef- ficiency, MetaL-Prompt surpasses previous meth- ods by requiring only a single forward pass for prompt generation, as explained in Section 3.2. This streamlined approach is highly productive, especially when contrasted with prior approaches that necessitate extensive number of forward backward passes to tune generate a prompt for a single task, as discussed in section 2. The reduction in computation cost achieved by Metal.- Prompt significantly enhances its practicality and efficiency with LMaaS.
fine-tuning method,
During the service phase following meta- lcarning, when a user provides a set of sho- ex- amples, MetaL-Prompt divides them into two sub- sets: X and X The PGM utilizes X ate an appropriate prompt. This generated prompt is then combined with the additional demonstration examples X and the composed input is used process future user requests. that X can be an empty set. In such cases, only the prompt and the input from the user's request are fed to the LM. This configuration enables the fastest inference speed due to the shorter sequence lengths.
P to gener-
important to note
A prompt generated by the PGM can be directly utilized for LM prompting. However, for cases where additional computational cost is acceptable, we can enhance the prompt by applying gradient- based prompt tuning methods (Lester et al., 2021; Liu al., 2021; Li and Liang, 2021; Shin et al., 2020), In this scenario, the generated prompt serves as an initialization of gradient-based prompt tuning. We observe that this additional tuning process can further improve the performance of the prompt.
3.1 Prompt generation model (PGM)
MetaL-Prompt employs meta-training to train prompt generation model (PGM), enabling the cre- ation of prompts that enhance in-context learning of the target language model (LM) across diverse tasks. The objective function utilized by MetaL- Prompt, as depicted in Figure 2, can be expressed

---

3.2 Trainable padding
LMs, the input hidden states from the head layer have the same representations as the word embed- dings. This makes them valid inputs (i.e., prompts) for the LM.
Generative language models, such as GPT- 3 (Brown et al., 2020), typically predict one token time within given context, necessitating forward passes to generate tokens. This multi- pass generation process can not only introduce in- efficiencies existing serving systems (Yu et al., 2022) but also cause extra overheads to train PGMs with the objective outlined in Equation 1, which also includes generation processes.
Prefix adds prompt before the keys and val- ues of transformers (Li and Liang, 2021), rather than prepends it to the imputs like the others. As depicted in Figure 2, we extract the keys and values of self-attention layers from each layer of PGM at the position of the trainable padding, and prepend them to the keys and values of target LM in the corresponding layers. Prefix demonstrates the best performance among all (Section 5.5), and there- fore, we utilize Prefix in the following experiments (section 5).
We tackle this challenge by proposing trainable padding, which inspired by special tokens of re- cent LMs and gradient-basee prompt tuning (Lester et al., 2021; Liu et al., 2021). ) As depicted in Fig- ure 2, MetaL-Prompt appends trainable embed- dings to the given examples X², which are then fed to the PGM as part of the input. This enables the PGM to generate multiple prompt tokens si- multaneously by leveraging the hidden states cor- responding to each padding position. Additionally, we reparameterize the trainable padding similar to other prompt tuning methods (Liu et al., 2021; Li and Liang, 2021; Liu et al., 2022; Hou et al., 2022b), specifically employing LSTMs following the methodology of P-tuning (Liu et al., 2021).
Experimental setup
In this section, we present the setup of our experi- ments, which includes datasets, training and evalu- ation details, baselines, and models.
4.1 Dataset
We conduct experiments to evaluate the perfor- mance of MetaL-Prompt using the combination of CrossFIT (Ye et al., 2021) and UNIFIED QA (Khashabi et al., 2020), which consists of 142 diverse datasets. Specifically, we adopt three differ- ent task settings from MetalCL (Min et al., 2022), cls -> cls, HR LRR, and Q4 Q4. Each setting defines two disjoint sets meta- meaan-le
3.3 Prompt design
In this section, we discuss four prompt designs Discrete, Weighted Sum, Hidden State, and Prefix and their generation using a PGM. While cxisting prompt generation methods have primarily focused on discrete prompts in natural language, we extend our investigation to include real-valued prompts (i.e., continuous prompts) as previous prompt tun- ing methods (Liu et al., 2021; Li and Liang, 2021; Sun et al., 2022a) have demonstrated their effec- tiveness.
datasets and evaluation datasets - and brief expla- nations of the settings are as follows.
Classification to classification (cls - cls):
Both the meta-learning datasets and the evaluation datasets encompass classification tasks.
QA to QA (QA + QA): Both the meta- learning datasets and the evaluation datasets consist of Duestion-Answering (QA) tasks.
Diserete is a prompt that consists of the most probable natural language tokens predicted by a PGM. To train the PGM for Discrete, we adopt Gumbel-Softmax reparameterization with discretization ("Straight-throughh trick).
High Resource + Low Resource (HR - LR): In this particular setting, each meta-learning dataset comprises over 10,000 training examples, while each evaluation dataset consists of fewer than 10,000 examples.
Weighted Sum a continuous prompt obtained by multiplying the token probability predicted by a PGM with the word embeddings of a target LMM It represents the babbability-wighhee sum of the word embeddings.
4.2 Evaluation
We use Macro-F1, which is a better measure than accuracy imbalanced datasets, as evaluation metric and report the mean scores obtained from four distinct runs. For each run, Metal.-Prompt and the basclines are given a different set of 16 examples sampled from the training split of the
Hidden State directly uses the input hidden states of the head layer in PGM as a continu- ous prompt. As word embedding layers and head layers typically share the same parameters in recent

---

evaluation dataset for prompt generation tuning. MetaL-Prompt trains only single PGM for all of the runs, but the PGM generates unique prompts for each run by leveraging distinct example sets. Furthermore, we explore the impact of varying ex- ample sizes and provide the corresponding results. Here, np denotes the number of examples used for prompt generation, and nı represents the num- ber of examples utilized for in-context learning demonstrations in addition to the tuned or gener- ated prompts.
parameters), GPT2-XL (1.5B parameters) (Rad- ford et al., 2019). and Llama2 (7B parameters). The motivation behind the model choice that au- toregressive models are widely utilized for LMaaS, such as OpenAl API. However, Metal_-Prompt is not limited to such models. It can support other LMs such as sequence-to-sene -equeenc models as well.
5 Experimental Results
We evaluate MetaL-Prompt on the setup presented in section 4. We first show the performance of Metal-Proompt when the generated or tuned prompt is solely provided without additional demonstra- tions (Section 5.1). We also provide experimental computation costs to prove MetaL-Prompt" 's cost- efficiency in (Section 5.2). Then, we explore a trade-off between the number of examples used for in-context learning demonstrations for LMs and those for prompt generation (Section 5.3). We ad- ditionally validate whether the PGMs are capable of generalizing inference settings that involve different number of demonstrations from train- ing settings (Section 5.4). Finally, we compare the performances of prompt designs we presented Scction 3.3 (Section 5.5).
4.3 Training
MetaL- Metal-Proompt trains PGM on the meta-learning datasets of each setting. Subsequently, for eval- uation, the trained PGM is employed to generate prompts from np examples of the unseen evalu- ation datasets. As we mentioned in Section 3.1, the generated prompts can be further tuned with SGD as the previous gradient-base bbasee tuning meth- ods. If not specified, we do not adopt the additional tuning.
As described in Section 3.1, Metal.-Promp con- catenates examples to form X." for prompt genera- tion and X for inference, respectively. not men- tioned, we use the same np and ri with the evalua- tion settings to construct X." and X for alignment between the training and evaluation settings. More details for the training of MetaL-Prompt and the baselines are described in Appendix A and Ap- pendix B.
5.1 Prompt-only In-context Learning
We present the performance of the prompts gen- erated by MetaL-Prompl without extra demonstra- tions for in-context learning, In this setting, all 16 examples from task are used for prompt gener- ation tuning. The inputs are composed of only the prompt and test inputs without any additional demonstrations (np = 16, n,, = -). This setting is the most practical setting because it keeps the shortest input length. Short sequence length leads to low latency, small hidden state cache (i.e., key and value cache of transformer-basee models for demonstrations or prompts), low monetary cost for users.
4.4 Baselines
We compare MetaL-Prompt against five different prompt tuning baselines - P-tuning (Liu et al., 2021), SoftPrompt (Lester et ul., 2021), Prefix- tuming (Li and Liang, 2021), RLPrompt (Deng et al., 2022), and BBTv2 (Sun et al., 2022a). To assess the effectiveness of the baselines on LMaaS, we evaluate their performance under constrained cost settings, roughly 10 epochs of forward and backward passes. When MetaL-Prompp adopts the further tuning of the generated prompts, we rum 9 epochs of the tuning to meet the computation budget, considering the prompt generation costs. More details regarding the computation costs can be found in Appendix B.2.
As demonstrated in Table 2, MetaL-Prompt ex- hibits superior performance compared to the base- lines, except Prefix-tuning. It achieves notable improvements of up to 19.4% on QA tasks, sur- passing the state-of-thh-an method (i.e., P-tuning), even with significantly lower computational costs requiring only a single forward pass. Interestingly, Prefix-tuning showcases exceptional efficacy in the limited budget setting compared to other base- lines. However, Metal Prompp outperforms Prefix- tuning on QA tasks, and maintains comparable
4.5 Models
To evaluate MetaL-Prompt and the baselines, we employ autoregressive LMs: GPT2-Large (762M

---

Model
Setting
BBT2
RLPrompt
SoftP
P-tuning
Prefix
MetaL
MetaL
(+Tune)
Large
cls-
22.43
23.13
28.53
34.01
39.71
35.62
44.63
HR-+LR
24.90
26.62
30.56
34.33
30.12
35.46
QA-QA
24.92
20.66
27.14
26.52
28.94
30.61
Avg.
24.08
25.27
30.57
33.52
31.56
36.90
XL
cls xcls
24.92
23.02
27.24
30.25
36.76
36.85
46.00
HR -LR
25.31
24.33
29.36
34.15
30.05
34.75
QA QQA
25.28
20.74
26.64
25.70
31.81
34.03
Avg.
25.17
24.10
28.75
32.20
32.90
38.26
Table 2: Comparison of evaluation results on the prompt-only setting between MetaL--Prompt and the baselines. All examples are used for prompt generation or tuning, no additional demonstration is provided for in-context learning.
Model
Setting
Prefix
Metal.
Computation costs
Method
Our constrained costs
Original (Table 1) (hrs)
Llama-2 7B
cls-+cls
34.03
47.29
(Appendix B.2) (ms)
Table 3: Comparison of evaluation results on the prompt- only setting and Llama-2 (7B) between Metal-Promppt and prefix-tuning
Prefix
26.4
SoftP
1880
50.1
P-tuning
1987
BBT2
3916
RLPrompt
19042
Metal.
2.11e-5 (76 ms)
performance on GPT2-XL, the larger model, and classification tasks. Notably, MetaL-Prompt bene- fits from its advantagcously low computation costs while still delivering competitive results. In Ta- ble 3, We additionally compare the performances of Prefix-tuning and Metal-Prrompt on larger model, Llama-2 (7B parameters). using the cls + cls setting. MetaL-Prompt outperforms our leading bascline, Prefix-tuning, on the Llama-2.
Table 4: Experimental computation costs of Metal.
Prompt and our baselines with our constrained compu- tation budget and the original budgets.
cacy on larger models, while properly benefits from additional gradient-based tuning. This means that MetaL-Prompp is expected to deliver improved per- formance on larger models, which is more prac- tical, whereas other gradient-based methods may encounter limitations when applied such models. This scalability advantage positions MetaL-Prompt as a favorable choice for scenarios involving larger models.
Furthermore, discussed in Section 3.1, we can enable MetaL-Prompt utilize comparable compu- tation costs to the baselines (Section 4.4) by incor- porating additional tuning of the generated prompts. indi-
This variant, referred as (eetaL(+Tinze cated in Table 2. When considering the fair compu- tation costs, Merompt consistently achieves superior performance across various task settings and models.
5.2 Experimental computation costs for automatic prompt engineering methods
To demonstrate the efficiency of MetaL-Prrompt in computation costs, and provide experimental evidences to support computation costs of our prac- tical setting (Scction 4.4 and Appendix B.2), we measure the actual computation costs for MetaL- Prompt and our baselines. We conduct these ex- periments on GPT2-XL model, single NVIDIA A100 80GB SXM, and randomly selected dataset, WIKI-QA, with batch size of 16.
Lastly, it is important to highlight that the gradient-based prompt tuning methods Soft- Prompt, P-tuning, and Prefix-tuning - demon- strate diminishing performance as models increase in size particularly on the classification tasks. We attribute this phenomenon to the convergence speed, Due to the smaller size of the model and the prompts, the prompts tuned by these methods con- verge more rapidly the smaller model, allowing them to reach better performance within the con- strained computational budgets.
We first measure the average time consumed for each forward and backward pass. Using the aver- age time measured for each forward and backward pass above, we compute the actual costs required
Conversely, MetaL-Prompt showcases its effi-

---

Example Splits
Train Setting
Test Setting
Method
(16, 0)
(12, 4)
(16, 0)
(12, 4)
(8. 8)
cls cls
cls->cl
P-tuning
30.25
29.87
(16. 0)
36.85
31.09
27.75
Prefix-tuning
36.76
37.07
(12, 4)
32.46
39.33
39.38
MetaL-Proompt
36.85
39.33
QA-
QA-QA
(16, 0)
31.81
26.51
27.35
P-tuning 26.64
26.53
(12, 4)
23.56
30.49
31.31
Prefix-tuning 25.70
26.17
MetaL- 31.81
30.49
Table 6: Results for transferability of the PGMs. We evaluate the PGMs for GPT2-XI. on the test settings different from the training settings.
Table 5: Comparison of the performances on various distribution of examples for prompting and demonstra- tion. We use GPT2-XL for the LM and PGMs.
demonstrations, thereby enhancing performance. It is worth mentioning that Prefix-tuning also benefits from these budgetary allocations in both settings, whereas P-tuning does not exhibit the same advan- tage.
process each task. We report two costs: our practical scenario and the settings from the original papers (Table 1).
As described in the table, the baselines consume up several days in the original settings, and even in our practical setting, they consume several sec- onds. Considering these costs are measured in GPT2-XL, prompt tuning on larger models such as GPT-3 (Brown et al., 2020) requires much more costs, and the gap between MetaL-Prompy and the baselines will be larger.
5.4 Transferability to different test settings
In the previous section, we have discussed that Metal.-Prompt can further enhance performance by tailoring example splits through increased com- putation or spatial budgets. In this section, we further explore that PGM trained with specific training setting (i.e., particular example split) is still available in different test settings. We evaluate two PGMs trained for GPT2- XL where (n-, n..) is (16, 0). (12, 4). These models represent train- ing without demonstrations and with additional demonstrations respectively. We evaluate the mod- els on cls->cls and QA-QA with three test set- tings where (mp,nL) is (16, 0), (12, 4), and (8, 8).
5.3 Additional demonstrations with generated prompts
Although prompt-only is the most cost-efficient set- ting, model providers or users may be willing to allocate expanded computation budgets for infer- ence larger spatial budgets (i.e., larger key/value cache) further increase the performance. For such situations, we explore the effect of addi- tional demonstrations combined with the generated prompt. We keep the number examples for each task the same but vary the ratio between n. and n- We evaluate Mrrmpt and two baselines, P- tuning (Liu et al., 2021) and Prefix-tuning (Li and Liang, 2021), which are the most powerful base- lines in Section 5.1, on settings where (np, nr.) is (16, 0) and (12,4) 4).
As presented in Table 6, PGMs trained without demonstrations (i.e., (np, nL.) (16, 0)) does not generalize to the other test settings. The perfor- mance decreases when examples are provided with an LM as demonstrations instead of solely utilized for prompt generation. Interestingly, PGMs trained with (np,, = (12, where demonstrations are considered, show marginal or no degradation when transferred to the other test settings with demon- strations.
Table 5 presents the results of Metal_Prrompt and the baselines when additional demonstrations are provided. Notably, when given these extra demonstrations, Metal_-Proppt demonstrates fur- ther performance improvements, particularly in the cls->cls setting, with enhancements of up to 6.7%. Hence, depending on tasks, a model provider may opt allocate a larger computation budget for in- ference on longer sequences or allocate additional spatial resources to cache the hidden states of the
In summary, PGMs trained without demonstra- tions can not be transferred to test settings with demonstrations, and vice versa. From this obser- vation, notice that there exists significant dis- parity between suitable prompts that are soley used without demonstrations and those that are paired with demonstrations. Exploration of this gap will be an interesting future work.

---

Method
Disc
WS
HS
Prefix
and it can generate prompt with a single for- ward pass, allowing additional tuning of the prompt. Moreover, we explore the generation of continu- ous prompts using a PGM, which has not yet been discussed in the previous prompt generation stud- ies. With the proposed designs, MetaL-Promppi achieves performance gains over five baselines up to 19.4% for mean F1 score on QA datasets with less computation cost for prompt generation than the baselines. The results support the efficiency of MetaL_Prommpt in terms of model performance and computational cost.
F1
31.78
33.13
31.19
36.85
Table 7: Ablation studies on effect our prompt design. We evaluate various prompt designs GPT2- XL and cls.
5.5 Comparison between various prompt designs
this section, we compare performances of di- verse prompt designs listed in Section 3.3: Discrete, Weighted Sum (WS), Hidden State (HS), and Prefix. Discrete is prompt consisting of natural language tokens, whereas Weighted Sum, Hidden State, and Prefix represent continuous prompts, which are real-valued prompts. We compare the designs on cls->cls where (np; n.) (16, 0).
Limitations
Flexibility on the number of demonstrations As highlighted Section 5.4, it is important to note that a prompt generation model (PGM) trained on specific training setting (i.e., example split) can- not be directly transferred to different test settings. Consequently, when we aim to enhance prediction quality through the inclusion of additional demon- strations as on the cls-+cls setting in Section 5.4. we require multiple PGMs for each specific setting, allowing for flexible trade-off between inference speed and prediction quality by adjusting the num- ber of demonstrations. However, training and man- aging multiple PGMs pose challenges for LMaaS providers.
As depicted in Table 7, Prefix exhibits the best performance among the approaches due to its abil- ity to prepend to each layer, resulting in a larger prompt size while maintaining the same prompt length. This larger prompt size provides Prefix with richer expressiveness compared to the other methods. Weighted Sum also demonstrates im- proved performance, benefiting from its enhanced expressiveness compared to the discrete prompt, which consists of natural language tokens.
However, Hidden State displays degenerated per- formance, even when compared to the discrete prompts. As discussed in Section 3.3, the input hidden states of the head layer have representations similar to word embeddings, but they not iden- tical, particularly in terms of the scale of the val- ues. This discrepancy may cause the Hidden State prompts to deviate from the manifold of the word embeddings. We hypothesize that the degraded per- formance of Hidden State prompts is a result of this discrepancy. Consequently, incorporating an addi- tional scaler to mitigate the discrepancy is expected be beneficial in improving the performance of Hidden State prompts.
Prompt generation with more examples In our experiments, the prompt generation mod- els (PGMs) are constrained by sequence size lim- its. Consequently, if user provides an excessive number of examples, the PGMs may be unable to process such a large set if the concatenation of the examples exceeds the sequence size limit.
However, it is worth noting that recent language models have started to adopt longer sequence sizes, which helps alleviate this limitation. The incorpo- ration of longer sequence sizes enables PGMs to handle larger sets of examples more effectively.
Additionally, we explore an iterative approach to improving prompts by concatenating previ- ously generated prompt with new examples. This concatenated context is then used as input to the PGM to generate an enhanced prompt, allowing the PGM to accommodate arbitrary number of examples by iteratively processing the example sub- sets. This iterative approach enhances the scalabil- ity of prompt generation, empowering PGMs process varying numbers of examples effectively.
6 Conclusion
In this paper, we propose MetaL-Prompt, novel and lightweight prompting method for LMaaS. MetaL-Prompt meta-trains prompt generation model (PGM) for better in-context learning from the generated prompt, accompanying trainable padding for more efficient prompt generation. Thanks to the meta-learning, the trained PGM does not require additional training for unseen user tasks,

---

References
Bairu Hou, Joe O'Connor, Jacob Andreas, Shiyu Chang, and Yang Zhang. 2022a. Promptboosting: blx text classification with ten forward passes. arXiv preprint 2rx.v:2212.09257.
Al21Labs. 2021. Ai21 studio. https://www. ai21. com/studio.
Anthropic. 2023. Anthropic api. cnttropic..oompprdduee com/product.
https://wm/
Yutai Hou, Hongyuan Dong. Xinghao Wang, Bohan Li, and Wanxiang Che. 2022b. MetaPrompting: Learn- ing to learn better prompts. In Proceedings of the 29th International Conference on Computational Lin- guisties, pages
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah. Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learness In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang. Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
HuggingFace. 2019.
Distilgpt2. wwwginngfac.co//dle co/distilgtt...
https: //
Daniel Khashabi. Sewon Min, Tushar Khot, Ashish Sabharwal. Oyvind Tafjord. Peter Clark, and Han- naneh Hajishirzi. 2020. UNIFIEDQA: Crossing for- mat boundaries with a single QA system. In Find- ings of the Association for Computational Linguistics: EMNLP 2020, pages 1896-1907, Online. Association for Computational Linguistics.
Yekun Chai, Shuohuan Wang. Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2022. Clip-tuning: Towards derivative-free prompt learning with a mixture of rewards. In Findings of the Association for Compu- lational Linguistics: EMNLP 2022, pages 108-117, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Brian Lester, Rami AI- Al-Rfou, and Noah Constant. 2021.
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020. Tabfact: A large-scale dataset for basee fact verification. In Interna- tional Conference on Learning Representations.
The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pages 3045-3059.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume 1: Lonng Papers). pages 4582- 4597.
Cohere. 2021. Cohere api. https://coheree
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yi- han Wang, Han Guo, Tianmin Shu, Meng Song, Fric Xing. and Zhiting Hu. 2022. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 3369-333 3391, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning: Prompt tuning can be comparable to fine- fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pupers), pages 61 61-68. Dublin, Ireland. Association for Computational Lin- guistics.
Shizhe Diao, Zhichao Huang. Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, and Tong Zhang. 2023. Black- box prompt learning for pre-trained language models. Transactions on Machine Learning Research.
Nikolaus Hansen, Sibylle D. Müller, and Petros Koumoutsakos. 2003. Reducing the time complex- ity of the derandomized evolution strategy with co- variance matrix adaptation (cma-es). Evolutionary Computation, 1-11:1-18.
Xiao Liu, Yanan Zheng, Zhengxiao Du Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021. Gpt understands, too. arXiv preprint arXiv:2103. 10385.
Ilya Loshchilov and Frank Hutter, 2019, Decoupled weight decay regularization. In International Confer- ence on Learning Representations.
Nikolaus Hansen and Andreas Ostermeier. 2001, Com- pletely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 159-195.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. Fantastically ordered prompts and where to find them: Overcoming few- shot prompt order sensitivity. In Proceedings of the
Or Honovich, Uri Shaham, Samuel R. Bowman, and Omer Levy. 2022. Instruction induction: From few examples to natural language task descriptions.

---

60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 8086-8098.
Seonghyeon Ye. Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo. 2023. Guess the instruction! flipped learning makes language models stronger zero-shot learners. In ICLR 2023 Workshop on Math- ematical and Empirical Understanding of Founda- tion Models.
Sewon Min, Mike Lewis. Luke Zettlemoyer, and Han- naneh Hajishirzi. 2022. Metaicl: Learning to learn in context. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2791-2809.
Gyeong-In Yu, Joo Seong Jeong. Geon- Kim, Soo- jeong Kim, and Byung-Gor Gon Chun. 2022. Orea: A distributed serving system for Transformer-Bast -Bsse generative models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages Association.
OpenAI. 2020. Openai api. https://openai.. api/.
Carlshad, CA. USENIX
Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021, True few-shot learning with language models. Ad- vances neural information processing systems. 34:11054-1107..
Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. 2022. Tem- pera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211. 1.th:2211.111900
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. 2022. Grips: Gradient-free, edit-based in- struction search for prompting large language mod- els.
Xiang Zhang. Junbo Zhao, and Yann LeCun. 2015a. Character-level Convolutional Networks for Text Classification. arXiv: 1509.01626 [cs].
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015b, Character-level convolutional networks for text clas- sification. In NIPS.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9.
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improv- ing few-shot performance of language models. In In- teratiinnl Conference on Machine Learning, pages 12697-12706.. PMLR.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. In NeurlPS EMC² Workshop.
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. In NewrlPS 2022 Foundation Mod- els for Decision Making Workshop.
Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with au- tomatically generated prompts. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222-4235.
Tianxiang Sun, Zhengfu He, Hong Qian, Yunhua Zhou, Xuanjing Huang, and Xipeng Qiu. 2022a. Bbtv2: Towards a gradient-free free future with large language models. In Proceedings of EMNLP.
Tianxiang Sun. Yunfan Shao. Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022b. Black-box tuning for sateige-oooll------seee
In International Con- ference on Machine Learning, pages 20841-20853 PMLR.
Zhen Wang, Rameswar Panda, Leonid Karlinsky, Roge- rio Feris, Huan Sun, Yoon Kim. 2023. Multitask prompt tuning enables parameter-effiieet transfer learning. In The Eleventh International Conference on Learning Representations.
Qinyuan Ye. Bill Yuchen Lin, and Xiang Ren. 2021. Crossfit: A few-shot learning challenge for cross- task generalization in nlp. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7163-7189.

---

B.1 Hyperparameters
Split
Setting
Meta-learning
Eval
In order to train the baselines, we use the hyper- parameters listed in Table 9. For P-tuning (Liu et al., 2021), SoftPrompt (Lester al., 2021) and Prefix-tuning (Li and Liang, 2021), we tune learn- ing rates based on F1 scores of three classification tasks - AG News (Zhang et al., 2015b), Yelp Po- larity (Zhang et al., 20) and TabFact (Chen et al., 2020) that have the largest test splits among clas- sification datasets, and we borrow other hyperpa- rameters from the original paper. We set prompt length of 20.
cls-+cls
43
20
HR +LR
61
26
QA >QA
37
22
Table 8: The number of datasets for each task setting. There is no overlap between the meta-learning datasets and the evaluation datasets in each setting.
A Training Details of MetaL-Prompp
Metapt-Prompt trains PGM on the meta-learning datasets of each setting (i.e., cls-tels, HR LLR, and QA-QA), using up to 16,384 training exam- ples per dataset in accordance with MetaICL (Min et al., 2022).
To train a policy model for RLPrompt (Deng et al., 2022), we adhere to the hyperparameters de- scribed in the original paper. In our experiments, we employ distilGPT-2 (HuggingFace,, 2019; Sanh et al., 2019). as policy model for generating op- timized prompts following the original paper. To maintain consistency with the recommendations of Deng et al. (2022), we set the prompt length Given that BBTv2 (Sun et al., 2022a) leverages the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) (Hansen and Ostermeier, 2001; Hansen et al., 2003) during its training, adopt all hyperparameters for CMA-ES outlined by Sun et al. (2022a), with the exception of the population size and a budget limit the computation cost. In line with Sun et al. (2022a), we set the prompt length to 50.
These settings cover a total of 133 unique tasks, which is significantly larger than the number of tasks explored in previous prompt tuning meth- ods (Sun et al., 2022a,b; Deng et al., 2022; Prasad et al., 2022; Honovich et al., 2022; Zhou et al., 2022). The statistics of each setting are described in Table 8.
To train the prompt generation models (PGMs) of MetaL-Prompt, we utilize the AdamW opti- mizer (Loshchilov and Hutter, 2019) along with linear learning rate decay. The learning rate is initialized at 0.0001 for HR -RR and 0.0002 for cls->els and AAA without employing a learn- ing rate warmup. The training epochs for cls >cls, HR-LR, and QA-QA are set to 10, 6, and 8, respectively. The prompt length for MetaL-Prompt is 20. For the additional tuning of the generated prompts, employ an initial learning rate of 0.02 and tune them for epochs align with the com- putation costs associated with the gradient- based prompt tuning baselines (Section B.2). It is impor- tant to note that this additional tuning process is performed on the few-shot examples utilized by the PGM for prompt generation.
B.2 Computation costs
In order to efficiently support Language-as-a- Service (LMaaS) with thousands of diverse user tasks, it is crucial to minimize the computation costs associated with each task. Considering our method MetaL-Prompt requires only a single for- ward pass with trainable padding, we adjust the number of forward and backward passes for each baseline to ensure fair comparisons, as described in Table 10.
For P-tuning, SoftPrompt and Prefix-tuning, we train our baselines for 10 epochs with a batch size of 16. implies that, when considering one for- ward pass for each sequence, cumulative total of 160 forward and backward passes are needed.
B Baselines
We evaluate five baselines and compare perfor- mance with MetaL-Prompt. P-tuning (Liu et al., 2021), SoftPrompt (Lester al., 2021), and Prefix- tuning (Li and Liang, 2021) are gradient-based prompt tuning methods. RLPrompt (Deng et al.. 2022) trains an RL agent to generate prompts for each task. Lastly, BBTv2 (Sun et al., 2022a) opti- mizes prompts using evolutionary search.
In accordance with the cost limit, RLPrompt is trained for 5 epochs. During training, RL- Prompt updates its policy model using reward sig- nals, which are logarithmic probabilities of multi- ple prompts, predicted by a target language model (LM). Specifically, each prompt is concatenated

---

Method
P-tuning
SoftPrompt
Prefix-tuning
BBTv2
RLPrompt
Optimizer
AdamW
AdamW
AdamW
Adam
Learning Rate
1.6e-4 4
4e-5
2e-4
5e-5
Learning Rate Schedule
Linear
Linear
Linear
Constant
Epncc
10
10
10
240
5
Batch Size
16
Prompt Length
20
20
20
5
Table 9: Hyperparameters for the baselines.
Method
# of forward
of backward
P-tuning
160
160
SoftPrompt
160
160
Prefix-tuning
160
160
RLPrompt
320 x
BBTv2
3,880
Table 10: Number of forward and backward passes for the baselines. Compared to the baselines, MetaL- Prompt requires only single forward pass. As this number pertains to single task, it will be linearly in- creased when dealing with wide range of user tasks LMaaS.
with every sequence and passed through the target LM to obtain rewards for updating the policy model. To calculate these rewards, we utilize 4 prompts and batch size of 16, requiring minimum of 64 forward passes per epoch.
Additionally, address the limitations of RL- Prompt, which only supports single-token labels, we multiply the batch size by the number of classes to accommodate classification tasks with multi- token labels. This implies that we need mini- mum of 320 forward passes, which will be further multiplied by the number of classes (a) for each task. It is important backward passes of the policy model, as solely updates small MLP layers (Deng et al., 2022) just before the head layer.
that we do consider
To assess BBTv2 in a constrained budget sce- nario, we designate 240 forward passes per batch for GPT2-XL and 180 forward passes for GPT2- Large, taking into account the number of hidden layers in each model (e.g., 48 layers for GPT2-XL) and selected popsize of 5. With batch size of 16 utilized by BBTv2, this translates to total of 3,840 forward passes for GPT2-XL and 2,880 forward passes for GPT2-Large.


Reviews:
[{"review_id": "llHrkQfg3O", "reviewer": "Reviewer_SZaX", "summary": "", "questions": "In line 169, \"We do not count the meta-learning processes as the costs to show the scalabilities of the baselines.\" This sentence is confusing, can you paraphrase it?\n\nIn line 276, in the Equation 1, can you give any introduction and explanation about the $y_i$ ? \n\nIn line 338, \"This enables the PGM to generate multiple prompt tokens simultaneously by leveraging the hidden states corresponding to each padding position.\" How does process work? Can you give more details?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## MetaL-Prompt on a larger model\n\nTo prove the real-world applicability of MetaL-Prompt, we additionally compare the performances (averaged F1 score) of Prefix-tuning and MetaL-Prompt on a larger model, Llama-2 (7B parameters) [1], using the cls→cls setting. \nFor Llama-2, We follow the same settings with Section 5.1, Appendix A, and Appendix B2, but we adjust the learning rate to 4e-4 for MetaL-Prompt and 5e-5 for Prefix-tuning. Please note that the results of GPT-2 models match those we reported in our paper.\n\n\n|  Model | Prefix | MetaL |\n| --------- |--------:|--------:|\n| GPT2-Large (0.77B)  | 39.71 | 35.62 | \n| GPT2-XL (1.5B) |  36.76 | 36.85 |\n| Llama-2 (7B) | 34.03 | 47.29 |\n\nMetaL-Prompt outperforms our leading baseline, Prefix-tuning, on the Llama-2. Notably, this experiment supports our investigation in Section 5.1 on Prefix-tuning's diminishing performance as models increase in size. This is attributed to the inherent challenge of effectively training a continuous prompt with a larger parameter space within our practical training configuration, consequently constraining the number of feasible training steps. Conversely, MetaL-Prompt showcases its efficacy in such settings.\n\n## Technical depth\n\nOur technical foundation can be attributed to our unique bi-level approach employed in the training of our prompt generation model (PGM). Our training scheme involves a synergistic combination of two core challenges: prompt generation and in-context learning. This strategy empowers us to enable fully gradient-based training for the PGM, eliminating the need for supervision on prompts. In contrast to MetaL-Prompt, RLPrompt [2] and TEMPERA [3] resort to partially gradient-based training using ad-hoc signals (i.e., rewards), which does not leverage gradients for in-context learning. FLIPPED [4] heavily relies on supervised prompt information. \n\n\n## Baselines\n\nOur experiments encompass a diverse array of methodologies --- RL-based, search-based, and gradient-based approaches, which also covers both major categories, gradient-based and gradient-free methods.\n\nThank you for noting the missing reference, SUPMER [5]. Unlike MetaL-Prompt, SUPMER necessitates mandatory training from a non-task-specific meta-initialized prompt, which is acquired to enhance gradient-based prompt tuning. However, this requirement places a substantial burden on deployment due to the need for a training system and the associated computation costs. In contrast, MetaL-Prompt directly generates task-specific prompts, resulting in more lightweight costs. We will compare the performances of SUPMER and MetaL-Prompt in the revision.\n\n\n## Clarification and writing\n\n* Computation cost of MetaL-Prompt\n    * We exclude the meta-learning process cost in Table 1. This emphasizes the scalability gap regarding user numbers between previous methods and MetaL-Prompt. Our training precedes LMaaS deployment, resulting in a constant cost unaffected by user numbers.\n* $y_i$ in Equation 1\n    * $y_i$ represents the expected answer for the input text $x_i$. For example, in natural language inference, $y_i$ denotes the verbalization of entailment, neutral, or contradiction, and in QA tasks, $y_i$ is the answer for the question $x_i$.\n* Mechanism of trainable padding\n    * Decoder transformer models, as well as Encoder-Decoder transformer models, are designed to generate a single token given input text by calculating probability only on the final token position. Trainable paddings enable computing embeddings or probabilities for each padding position.\n* In the upcoming revision, we will also address duplicated explanations and presentations of the diagrams and equations, incorporating the clarifications mentioned earlier.\n\n\n\n## Code and data release\n\nThe datasets utilized in our paper align with those employed by MetaICL [6]. These datasets are publicly accessible through the MetaICL repository [7]. As stated in the paper, we have adhered to the identical task settings with MetaICL. Moreover, we are in the process of preparing the release of our code after the anonymity period.\n\n\n[1] Touvron, Hugo, et al. \"Llama 2: Open foundation and fine-tuned chat models.\" 2023.\n\n[2] Deng, Mingkai, et al. \"Rlprompt: Optimizing discrete text prompts with reinforcement learning.\" EMNLP 2022.\n\n[3] Zhang, Tianjun, et al. \"Tempera: Test-time prompting via reinforcement learning.\" ICLR 2023.\n\n[4] Ye, Seonghyeon, et al. \"Guess the instruction! making language models stronger zero-shot learners.\" ICLR 2023.\n\n[5] Pan, Kaihang, et al. \"Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization.\" 2023.\n\n[6] Min, Sewon, et al. \"Metaicl: Learning to learn in context.\" NAACL 2022.\n\n[7] https://github.com/facebookresearch/MetaICL", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper showcases a substantial advancement in experimental outcomes, signifying its robustness and credibility. Simultaneously, the research problem's high practical relevance underscores its potential real-world impact, making the paper's contributions valuable for both academia and practical applications.", "reasons_to_reject": "1. The paper appears to have limited innovation and technical depth, leading to repeated restatements in various sections to compensate for this deficiency. Furthermore, the overall effort appears to be insufficient. The writing quality is moderate, and the graphical representations are relatively basic and lacking refinement, which may impede comprehension.\n\n2. The paper's lack of a substantial baseline comparison undermines the persuasiveness of its experimental results, consequently casting doubt on the validity of the significant improvements claimed. Despite the presence of only one formula in the entire manuscript, this equation remains inadequately explained.\n\n3. Regrettably, Sections 3.1 and 3.2, which encompass the most crucial methodology, suffer from a lack of clarity, detailed explanation, and adequate length, with a surplus of repetition and redundancy in less critical areas. Both the diagram clarity and corresponding textual descriptions are lacking, with a repeated focus on supposed advantages.\n\n4. Furthermore, the discrepancy between the paper's reliance on smaller models for experimentation and the industry's practical utilization of larger models diminishes the real-world applicability and relevance of the paper's experimental outcomes. \n\n5. No code and data provided. Experimental results are not reproducible.\n\nConsequently, due to these inadequacies and inconsistencies, the paper is not deemed suitable for publication.", "questions_to_authors": "In line 169, \"We do not count the meta-learning processes as the costs to show the scalabilities of the baselines.\" This sentence is confusing, can you paraphrase it?\n\nIn line 276, in the Equation 1, can you give any introduction and explanation about the $y_i$ ? \n\nIn line 338, \"This enables the PGM to generate multiple prompt tokens simultaneously by leveraging the hidden states corresponding to each padding position.\" How does process work? Can you give more details?\n", "paper_topic_and_main_contributions": "The paper presents a novel framework called MetaL-Prompt to solve the issue of absence of prompt tuning methods on LMaaS platforms. \nThe key insight is the  lightweight automatic prompt generation method where an LM robustly learns from contexts induced by the created prompt, in a manner of learning to learn. Experiments on diverse unseen tasks show the superiority of the proposed model.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization."}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. The diagrams presented are somewhat basic and lack refinement, which might hinder easy comprehension. I would recommend enhancing their detail for better clarity.\n\n2. The pivotal methodology section of the paper appears to lack clarity, attention to detail, and sufficient elaboration. Furthermore, there seems to be a surplus of repetition and redundancy in less crucial areas. There is room for improvement in addressing these issues."}}, {"review_id": "7RyL8aBwTj", "reviewer": "Reviewer_Y1iN", "summary": "", "questions": "- Since PGM operates without direct supervision, it remains uncertain whether it can generate coherent and meaningful words. Can you offer some discrete samples of words generated by PGM?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## MetaL-Prompt on a larger model\n\nTo prove the real-world applicability of MetaL-Prompt, we additionally compare the performances (averaged F1 score) of Prefix-tuning and MetaL-Prompt on a larger model, Llama-2 (7B parameters) [1], using the cls→cls setting. \nFor Llama-2, We follow the same settings with Section 5.1, Appendix A, and Appendix B2, but we adjust the learning rate to 4e-4 for MetaL-Prompt and 5e-5 for Prefix-tuning. Please note that the results of GPT-2 models match those we reported in our paper.\n\n\n|  Model | Prefix | MetaL |\n| --------- |--------:|--------:|\n| GPT2-Large (0.77B)  | 39.71 | 35.62 | \n| GPT2-XL (1.5B) |  36.76 | 36.85 |\n| Llama-2 (7B) | 34.03 | 47.29 |\n\nMetaL-Prompt outperforms our leading baseline, Prefix-tuning, on the Llama-2. Notably, this experiment supports our investigation in Section 5.1 on Prefix-tuning's diminishing performance as models increase in size. This is attributed to the inherent challenge of effectively training a continuous prompt with a larger parameter space within our practical training configuration, consequently constraining the number of feasible training steps. Conversely, MetaL-Prompt showcases its efficacy in such settings.\n\n\n\n## High-resource setting\n\nDevelopers are continually introducing LMs with longer sequence capabilities, like Anthropic's Claude 2 [2], accommodating up to 100K tokens. This feature enables MetaL-Prompt to effectively leverage a substantial example pool for prompt generation. Although Prefix-tuning may still outpace MetaL-Prompt in the utilization of high data resources on these models, it's vital to consider that Prefix-tuning necessitates a separate training system, distinct from the inference service. It also incurs higher computational costs, as discussed in Section 2, restraining Prefix-tuning's scalability.\n\nShould the feasibility of such auxiliary deployment and the associated computational overhead be acknowledged, MetaL-Prompt can seamlessly incorporate additional gradient-based prompt refinement, as outlined in Sections 3.1 and 5.1. The ensuing performance enhancement becomes apparent in the \"MetaL (+Tune)\" results presented in Table 2.\n\n\n## Discrete prompt examples\n\nWe present concrete examples of prompts that a PGM generates. The PGM is based on GPT2-XL, and is trained to generate “Weighted Sum” prompts since “Prefix” is hard to interpret. To make the continuous prompts human-readable, each prompt embedding $p_i$ is interpreted to the token whose weight $w_{i,j}$ is the highest in the vocabulary $\\mathcal{V}$, and we also report the weights. Note that the sum of the weights for each prompt embedding equals 1: $\\sum_{j}{w_{i,j}} = 1 (j < |\\mathcal{V}|)$.\n\nMedical Question Pairs (MQP) [3]\n\n```\n['itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:0.99', 'itional:0.22', ' Diversity:0.05', ' Diversity:0.06', ' Diss:0.15', ' Similar:0.78', '\\xa0:0.74', ' \\xa0:0.47', ' \\xa0:0.95', ' \\xa0:0.97', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0']\n```\n\nAG News [4]\n```\n['itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:0.19', ' Sports:0.08', ' Sports:0.1', ' Sports:0.19', ' Sci:0.35', '\\xa0:0.77', '\\n\\xa0:0.73', ' \\xa0:0.93', ' \\xa0:0.96', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0']\n```\n\nSuperGLUE CommitmentBank (CB) [5]\n```\n['itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:1.0', 'itional:1.0', 'anim:0.41', 'anim:0.07', 'anim:0.06', ' hypothesis:0.05', ' contradiction:0.69', '\\xa0:0.6', '\\n\\xa0:0.75', ' \\xa0:0.9', ' \\xa0:0.96', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0', 'ib:1.0']\n```\n\nRemarkably, we notice a consistent mapping of the beginnings and endings of prompts to frequently occurring tokens, \"itional\" and \"ib,\" both of which bear remarkably high weights. We hypothesize that these prompt embeddings potentially signal the commencement of a sequence and serve as a delimiter between the prompt and a query example. Additionally, our observations indicate a slight bias towards the verbalized classes of each dataset within the midsection of the prompts. However, the weights associated with these tokens are not overly substantial, suggesting that each embedding is a collaborative representation of multiple tokens.\n\n\n[1] Touvron, Hugo, et al. \"Llama 2: Open foundation and fine-tuned chat models.\" 2023.\n\n[2] https://www.anthropic.com/index/claude-2\n\n[3] McCreery, Clara H., et al. \"Effective transfer learning for identifying similar questions: matching user questions to COVID-19 FAQs.\" ACM 2020.\n\n[4] Zhang, Xiang, Junbo Zhao, and Yann LeCun. \"Character-level convolutional networks for text classification.\" NIPS 2015.\n\n[5] De Marneffe, Marie-Catherine, Mandy Simons, and Judith Tonhauser. \"The commitmentbank: Investigating projection in naturally occurring discourse.\" Sinn und Bedeutung. Vol. 23. No. 2. 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The method establishes a connection between the meta soft-prompt and the discrete demonstrations designed for a specific task, which is inspiring.\n\n- The method yields comparable results without the need for fine-tuning and surpasses existing prompt generation methods when fine-tuned with considerable computing resources.\n\n- The paper extensively investigates the utilization of hidden states acquired from special tokens, which act as prompts.\n\n- Supplementary experiments are detailed, particularly focusing on the varied proportions of prompts used for PGM and LM during training and testing, thereby enhancing the effectiveness of PGM.\n", "reasons_to_reject": "- If possible, the existing method can be easily extended to a larger model. GPT2 is slightly weak as a basic model, and the method will be more convincing under a more powerful base model.\n\n- This method demonstrates clear advantages under low-resource conditions. However, as an in-context learning approach, when abundant samples are accessible, prefix-tuning might be a more straightforward alternative, while this method could face challenges in allocating a large number of demonstrations within a single sample.\n\n", "questions_to_authors": "- Since PGM operates without direct supervision, it remains uncertain whether it can generate coherent and meaningful words. Can you offer some discrete samples of words generated by PGM?", "paper_topic_and_main_contributions": "This paper introduces MetaL-Prompt, a prompt generation method that addresses the challenges of the lack of an automatic prompt generation method in existing language models as a service (LMaaS) and the need for large samples for prompt tuning. The approach involves utilizing two identical Language Models (LMs), one functioning as a prompt generator and the other as a regular response generator. The weights of the LMs are frozen, and only one LoRA module is trained in the Prompt Generation Model (PGM), along with the embedding of padding as a special token. During the main experiment, the hidden states of these special tokens in PGM serve as key-value pairs to interact with the regular LM. The experimental results demonstrate that MetaL-Prompt outperforms the previous PGMs.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wQVbFK7OiC", "reviewer": "Reviewer_fHF6", "summary": "", "questions": "1. Could you provide more experimental evidences to support \"computation cost for prompt generation\" in Appendix B2?\n2. Do you have some experimental results to argue \"It can support other LMs such as sequence-to-sequence models as well.”?\n3. Could you explain more about X_i^P and X_i^L mentioned in line 279-280 ? How do different settings (not only the size of sets, e.g., (16, 0) or (12, 4), but also the actual assignment, e.g., x -> X_i^P or x -> X_i^L) affect the performance of PGMs?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## OpenPrompt\n\nOpenPrompt [1] is an integrated framework for prompt learning, which implements and collects various modules, such as training schemes, prompt templates or verbalizers. MetaL-Prompt is orthogonal to OpenPrompt and MetaL-Prompt can be a module for the framework as some of our baselines, P-tuning and Prefix-tuning.\n\nSince OpenPrompt is a collection of various modules, the availability of OpenPrompt on LMaaS depends on the modules. If an LMaaS decides to use Prefix-tuning, it has to afford the heavy computation overhead as we described in Section 2. Accordingly, OpenPrompt can employ MetaL-Prompt for its module to benefit from our light-weight prompt engineering.\n\n\n## Case studies of MetaL-Prompt\n\nIn a practical use case, we showcase the real-world applicability of MetaL-Prompt. Imagine a novelist aiming to translate their English work into another language while retaining a distinct style and tone that aligns with their unique narrative. Leveraging MetaL-Prompt within an LMaaS framework, the novelist compiles a set of specific translation examples (e.g., 16 English-Korean pairs) to define their desired style. By inputting these examples, the system generates a customized prompt that encapsulates the author's requirements. This prompt empowers the novelist to effortlessly translate numerous sentences from their work. Similarly, the author can also translate character-specific lines by generating prompts tailored to each character's persona. This seamless integration of MetaL-Prompt not only facilitates accurate translation but also facilitates the infusion of the author's creative essence into the translated narrative.\n\n\n## Experimental computation costs for automatic prompt engineering methods.\n\nTo measure experimental computation costs for Appendix B2, we first measure the average time consumed for each forward and backward pass. We conduct these experiments on GPT2-XL model, a single NVIDIA A100 80GB SXM, and a randomly selected dataset, WIKI-QA, with a batch size of 16. \n\nUsing the average time measured for each forward and backward pass above, we compute the actual costs required to process each task. We report two costs on each baseline for our practical setting (Appendix B2) and the setting from the original paper (Table 1 in Section 2). For the original setting of Prefix-tuning, we use the number of training steps that the authors used for DART [2] dataset.\n\n|  | Appendix B2 (ms) |\tSection 2 (hrs) |\n| ----------------|-----------------:|---------------:|\n| Prefix-tuning\t| 1517 |26.40216 |\n| SoftPrompt\t| 1880 |50.13719 |\n| P-tuning\t| 1987 | 3.0912 |\n| BBT2\t        | 3916 | 6.96164 |\n| RLPrompt\t| 19042 | 4.18795 |\n| MetaL-Prompt | 76 | 2.11e-5 (76 ms) |\n\nAs described in the table above, the baselines consume up to several days in the original settings, and even in our practical setting, they consume several seconds. Considering these costs are measured in GPT2-XL, prompt tuning on larger models such as GPT-3 requires much more costs, and the gap between MetaL-Prompt and the baselines will be larger.\n\n## How different data assignments affect the performance\n\nWe show the performance variance according to distribution of data into $X^P$ and $X^L$. We first randomly sample 16 examples for each eval dataset from the cls→cls setting. Then, for each run, we randomly distribute the data into $X^P$ and $X^L$, and $|X^P| = 12$ and $|X^L| = 4$. This shows $1.04$ of standard deviation in F1 score.\n\n## Sequence-to-sequence model\n\nWe will present experimental results with sequence-to-sequence models in the revision.\n\n\n[1] Ding, Ning, et al. \"Openprompt: An open-source framework for prompt-learning.\" ACL 2022.\n\n[2] Nan, Linyong, et al. \"Dart: Open-domain structured data record to text generation.\" NAACL 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is written clearly.\n2. Appropriate and clear diagrams make the method understandable.\n3. The experimental results and analysis are comprehensive and convincing. \n4. The computation cost of the proposed method is much smaller compared to other methods, without the degradation of performance.\n\n", "reasons_to_reject": "1. The paper didn’t mention OpenPrompt (https://github.com/thunlp/OpenPrompt), which has the same functionality as PGMs. The authors would explain why OpenPrompt can’t solve the task.\n2. Case studies are missing. They could be more helpful to understand the contribution of the paper.\n", "questions_to_authors": "1. Could you provide more experimental evidences to support \"computation cost for prompt generation\" in Appendix B2?\n2. Do you have some experimental results to argue \"It can support other LMs such as sequence-to-sequence models as well.”?\n3. Could you explain more about X_i^P and X_i^L mentioned in line 279-280 ? How do different settings (not only the size of sets, e.g., (16, 0) or (12, 4), but also the actual assignment, e.g., x -> X_i^P or x -> X_i^L) affect the performance of PGMs?\n", "paper_topic_and_main_contributions": "The paper proposes a meta-learning approach named MetaL-Prompt for prompt generation, which enables PGM to generate hints for a series of unseen tasks without additional training. MetaL-Prompt achieved the better result on QA datasets compared to the state-of-the-art baselines, even with such a small computation cost.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces MetaL-Prompt, a prompt generation method that addresses the challenges of the lack of an automatic prompt generation method in existing language models as a service (LMaaS) and the need for large samples for prompt tuning. With this meta-learning based approach, a prompt generator model can generate prompts for a series of unseen tasks without additional training. 

Pros:

Well motivated with a timely and a very relevant research topic

The paper establishes a connection between the meta soft-prompt and the discrete demonstrations designed for a specific task

Yields comparable results without the need for fine-tuning and surpasses existing prompt generation methods when fine-tuned with considerable computing resources.

The idea of utilizing hidden states acquired from special tokens as prompts is interesting 

Cons:

The authors have used GPT-2 large and XL .. these are both fairly weak and outdated models and significantly smaller in size compared to SoTA models today. In order to make this work more relevant and more convincing the authors need to show that their proposed method works even for large enough LLMs 

The paper would benefit from some case studies to understand the effectiveness of the method 

This method demonstrates clear advantages under low-resource conditions. But it would be good to understand how it performs under the more common high resource scenario where abundant samples are available (this point also has been raised by multiple reviewers)
Decision: Accept-Findings