Prompt Injection: Parameterization of Fixed Inputs
Authors: Eunbi Choi
Yongrae Jo
Joel Jang
Minjoon Seo
Venue: NeurIPS 2022 Submitted (2022)
Keywords: Prompt
Injection
Parameterization
Language Model
Inference Efficiency
Distillation

Abstract: Recent works have shown that attaching prompts to the input is effective at conditioning Language Models (LM) to perform specific tasks. However, prompts are always included in the input text during inference, thus incurring substantial computational and memory overhead. Also, there is currently no straightforward method of utilizing prompts that are longer than the maximum input length of the LMs without incurring additional costs during inference. We propose Prompt Injection (PI), a novel formulation of injecting the prompt into the parameters of an LM to be an efficient alternative to attaching fixed prompts to the input. We show that in scenarios with long fixed prompts, PI can be up to 280 times more efficient in terms of total FLOPs than previous approaches. We further explore methodologies for PI and show promising results in persona-dependent conversation, semantic parsing, and zero-shot learning with task instructions. Through these explorations, we show that PI can be a promising direction for conditioning language models, especially in scenarios with long and fixed prompts.

Prompt Injection: Parameterization of Fixed Inputs
Anonymous Author(s) Affiliation Address
email
Abstract
Recent works have shown that attaching prompts to the input is effective at con- ditioning Language Models (LM) to perform specific tasks. However, prompts are always included in the input text during inference, thus incurring substantial computational and memory overhead. Also, there is currently straightforward method of utilizing prompts that are longer than the maximum input length of the LMs without incurring additional costs during inference. We propose Prompt Injection (PI), a novel formulation of injecting the prompt into the parameters of an LM to be an efficient alternative attaching fixed prompts to the input. We show that in scenarios with long fixed prompts, PI can be up to 280 times more efficient in terms of total FLOPs than previous approaches. We further explore methodologies for PI and show promising results in persona-dependen conversation, semantic parsing, and zero-shot leamnnng with task instructions. Through these explorations, we show that PI can be a promising direction for conditioning language models, especially in scenarios with long and fixed prompts
3
5
10
11
12
13
14
Introduction
15
Contemporary works with large Language Models (LMs) 3 32 23 5 28 have shown that attaching prompts (also referred to as prefixes) to the input is effective at conditioning LMs to perform specific tasks. During training, LMS trained condition the given prompts in hopes of generalizing to unseen prompts during inference. Unseen prompts can be a persona for persona-dependent conversation (39], database schema for semantic parsing [10], and task instruction for zero-shot [23]. In these tasks, a new prompt is fixed to the input at every
17
19
20
21
learning with task instructions inference. For instance, in persona-dependent conversation 39
22
18 33 persona description is appended to the dialogue history, so that the LM can always be conditioned on the persona. For another example, semantic parsing, the LM is conditioned on the database schema well as natural language questions to generalize to a new database [37] (3). Lastly, zero-shot learning with task instructions 332 23 involves adding natural language instructions to the inputs for adapting LMs to novel tasks.
23
24
25
26
27
However, concatenating prompts to input sequences for prompt-dependent inference has two major limitations. (1) During inference, prompts are always included in the input text and thus incur computational and memory overhead [16]. (2) It challenging fit long text such the detailed description of a persona as a prompt into Transformer-based models whose input lengths are often
29
30
31
32 fixed [27]. For instance, in persona-dependent conversation, the model constantly refers to the persona
33 description along with the dialogue history [35 (2). as shown in the left side of Figure [ Moreover,
34 in real world scenarios, persona may consist of long detailed text description of character or
35 person, not just few profile sentences. Naively concatenating long prompts to the input sequences is
36 challenging due to the quadratic cost in time and memory of Transformer- bassd architectures with
'Code used for the experiments and demo available at this link
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

agent's persona
Inject persona advance
am artist. four recently got a. cat. I enjoy walking exercise. llve watcing Game of Throree.
Holio! How are you today?
Concat persona input
Hello! How you today?
agents persona
lam an artist. have four children. recently got cat. enjoy walking for exerciso. loove watching Game Thrones. How are you today?
you
Great, thanks I My children and were just about to watch Game Thrones.
Great, thanks My children and I were just about to watch Game of Thrones.
LM with Explicit Prompt (Prev)
LM with Prompt Injection (Ours)
Figure 1: Prompt Injection example on a persona-dependere conversation. The left side presents an inference procedure of a previous approach where the persona (prompt) is concatenated to every input. The right side describes Prompt Injection, where the persona is injected into the model in advance, so that the model is able to generate responses without constantly referring the persona description. Thus, Prompt Injection approach takes less time to generate responses than the previous method.
regard to the input sequence length. Other approaches specialized for long inputs || such as 38
Ausioonin-Decodee [12), or those that augment the LM with a retrieval mechanism 9 may be used but still come with increased overall memory and computations, ultimately leading to delay in generating responses. This problem becomes critical in situations where the LMs are deployed, and fast inference speed is required.
40
41
42 In this work, we formulate novel problem called Prompt Injection (PI), where we attempt to inject a given prompt into the parameters of an LM to address the two limitations mentioned above. With outputs without the computational overhead of appending fixed prompts at inference time (the right side of Figure and it also enables the injection of longer prompts in a wholistic way. More specifically, we first show that PI is much more efficient (up to 280 times) in terms of total FLOPs compared to previous approaches that may be used for handling
44 PI, LMs can produce prompt-dependee
46
48
long prompts such as Fusion-in- in-Decoder 12] or Linear Transformer [3]. Next, we explore different methodologies as baselines for PI, including the continued pre- pre-trrining approach on the prompt as 50 well as a novel distillation approach called Pseudo-INput Generation (PING), in order to analyze
51 what components are effective for successful PI. We apply these PI methods to three different tasks
52 with fixed prompts: persona-dependent conversation, semantic parsing, and zero-shot learning with instructions. We compare the methods against LMs with explicit prompts the upper bound (i.c.. unconstrained) as well as the LM without both the prompt and PI as the lower bound. Experimental results show meaningful improvements with respect the lower bound, but also exhibit non- con-trivial gap with the upper bound. Despite the performance gap, we still believe that PI is a direction worth 56
53
54
55
57 exploring considering the computational benefit of the injection, especially since inference speed is critical in real world applications.
58
59 In sum, our main contributions are three folds:
60
We formally define the Prompt Injection (PI) formulation and demonstrate its necessity in terms of computation and memory efficiency, especially in scenarios with long prompts.
61
62
We explore baseline approaches for PI, showing that performance can approach the upper bound (unconstrained) performance in some cases.
64
We show that the injection of long prompts (e.g., detailed description of persona) can be achieved through PT and show efficiency in comparison with previous methods, being up to 280 times more efficient during inference.
85

---

67
Through this work, we hope the community explores PI as an efficient alternative for performing prompt-dependent tasks.
68
Related Work
69
70 Prompting Prompting an emerging paradigm for modeling L.Ms, especially for few-shot and
71 zero-shot learning [20 3 21 24
32 23 With the help of appropriate prompts, can exploit 72 knowledge learned by a pre-trained LM and manipulate the LM's behavior. The benefit of prompting
73 is that the pre-trained LM can adapt to new scenarios with few no labeled training data. However,
74 for the in-context learning scenario, processing prompts that involve many training examples for each inference incurs substantial computational and memory overhead [66] Given training data, Liu et al. 76 16 replace in-context learning with fine-tur tuning a small set of parameters for tackling the above
77 issue. Prompt Injection also tackles the same issue but assumes stricter scenario where there no training data for the given prompt.
78
79
Efficient Transformers for Long Inputs One can consider using efficient Transformer- [2] architectures for handling long input sequences (27]. The main challenge of using vanilla Trans- former architecture is the quadratic cost time and memory with regard to the input sequence length due to the self-attention operation. There has been a surge of recent works addressing this They are primarily dedicated to improving either the efficiency of the self-attention mechanism or the general efficiency of the Transformer architecture through sparse mod- els. Our Prompt Injection approach tackles the efficiency problem of performing prompt-dependent tasks by keeping the input sequences short (without prompts), bounding the time and memory complexity to a constant invariant of the length of the prompt.
81
83
problem (6 38 I 13 40 g
85
87
is challenging, 89 but it enables the chatbot to deliver more personal, specific, consistent, and engaging conversa-
88 Persona-dependent Conversation Endowing a chabot with a persona
39 1
90 tions (39 and gain user trust 17 25 19. To achieve this, previous works have attached a persona to
91 the dialog history at every inference time, so that the model can always be conditioned on the persona. However, given long persona description, this approach brings the critical problem of increased overall memory and computations, resulting in delayed response generation. An LM augmented with retrieval mechanism [9] may be used but still comes with non-trivial computational overhead. Prompt Injection allows dialogue agent to generate responses without persona description the explicit input once the persona is injected.
92
94
96
97
Semantic Parsing Semantic parsing is the task of mapping a natural language query into SQL query executable on a database. Recently, the community has focused more on cross-doman (cross- database) semantic parsing, where models are trained and tested on different domains (databases) [37]. The domain comain-adaptation setup introduces many generalization challenges, such as non-explicit column names and domain-specifie phrases 10. and recent works concatenate the natural language query with the serialized database schema as the input to address the problem [26] 7 33). With Prompt
98
99
100
101
102
103
Injection, the model adapted to a new database schema in advance, so that it can map natural
language queries to SQL queries on the new database without explicitly referring to the schema during inference.
105
106
Zero-shot Learning with Task Instructions Recent works 2 32 have addressed zero-shot generalization to new tasks [3] [14] by multi-task prompted training. With multi-task prompted training, the models learn to use task instructions as prompts to generalize to unseen tasks. It is demonstrated that this approach improves generalization ability to novel tasks and offers an effective substitute for unsupervised language model pre- traitrainnn.. Through Prompt Injection, the LM can be aware of a novel task instruction before performing the task and thus does not require the instruction, which can be lengthy, make predictions.
107
108
109
110
112
3 Prompt Injection
113
114
In this section, we formally define Prompt Injection (PI) as a task and describe the benefits of the formulation. Prompt-dependent generation is a task of generating an output sequence y that is
3

---

116 proper response to the input sequence a and coherent to the prompt 2. Utilizing the prompt during inference, the generated sentence is obtained by y f(z.z) where f denotes an LM such as 118 T5 and GPT-2. Prompt Injection (PI), i.e., parameterization of prompts, allows LMs to perform prompt-dependent generation without using prompts during inference. To achieve this, we need to design Pl method H to inject a prompt into an LM f. The process of PI can be represented as
117
119
120
[= = (z)
where f= denotes an I.M injected with the prompt. Then the prompt-dependent output sequence can be obtained by y = (=x)).
122
123
Pl can also be applied for long prompts whose length exceeds the LM's input sequence length. Given
124
a long prompt z, we decompose into multiple sub-prompts {zi] each of which fits the LM's input
125
length, i.e., 2 = Zi:n [21: $2;... =n]- Then the PI process can be executed iteratively, injecting
126
each sub-prompt sequentially while the LM is aware of the previous sub-prompts:
fz, = H(21.
(2)
fx H (22, fx1)
(3)
fmm. H (Zus fmim-1)
(4)
127
The above formulation can be seen as a high-level abstraction of iterative PI that we aim to ap- proximate. In practice, in order fully inject Zim. we repeat (2)-(4) multiple times (i.e., multiple epochs).
128
129
Why is Prompt Injection necessary? Prompt Injection brings definite advantages when applied to prompt-dependent tasks. The previous approach of appending prompts to the input sequences has the drawback of the model repeatedly referring to the prompt each inference time. This becomes critical in scenarios requiring long prompts, as Transformer architecture has quadratic computational and memory costs due to the limitation of the self-attention operation. We propose PI solution
131
132
139
134
135
to this computation bottleneck. Once a prompt is injected into the LM in advance, the LM no longer needs to refer the prompt during inference. As a result, the model's input length remains independent of the length of prompts and is able to utilize prompts of any length efficiently. We discuss the efficiency gain of PI in Section 6.
138
137
138
139
Evaluation Metric for Prompt Injection PI can be evaluated by the evaluation metric of the fixed prompt-dependent task at hand. We also introduce metric called the Prompt Injection score (PI score) to measure the degree of injection. The metric is agnostic of the target task by comparing the results with that of LM given actual prompts during inference. Let Xw/ prompt denote the LM's task score with the prompt as an additional input (upper bound) and Xw/o prompt denote the LM's task score without the prompt (lower bound). We define PI score as the min- max scaling score of XPI. where XPI represents the score of the LM on the target task after PI, i.e., PI score max(0, XpI Xwwo prompt) / (X.r prompt Xa//o prompt). We limit using PI only in situations where Xw/ prompt > Xxw/o prompt because there is no reason to inject a prompt if task performance degrades when using the prompt. Even if the range of individual task scores may vary from task to task, PI score represents the overall injection effectiveness of the PI methods, agnostic of the individual task score range.
141
142
143
144
145
148
147
148
149
150
4 Methods for Prompt Injection
151
152
In this section, we explore methods of Prompt Injection (PI) that can address prompt-dependent tasks without accessing the prompt during inference. To achieve this, the model should be trained to store the prompt in its parameters. This can be seen as parameterizing the prompt into the model instead of feeding the prompt explicitly to the model. This is challenging as the prompt is unseen to the model and has no corresponding training data. Section 4. baseline method by continued pre-training is introduced, followed by a method for improving the baseline with curriculum learning. Section 4.2 presents a novel distillation-bal based method called Pseudo-INput Generation (PING) that learns to generate pseudo-inputs to inject novel prompts.
153
154
155
196
157
158

---

How departments there?
How many singers there?
SELECT COUNT() FROM
SELECT COUNT() FROM
Task Input
Pseudo Input
Output
Output
Input Generator
Input Generator
Distillation
Teacher
Student
Prompt
Prompt
Prompt
Pseudo Input
Pseudo Input
department id name creation, ranking num, employees....
department |
singer | singer Jd. name, birth, year, not worth millins... How may singers are there?
|singer singer, name
How may singers are there?
Phase 1: Generator Training
Phase 2: Distillation
Figure 2: Illustration of the Pseudo-INpel Generation (PING). During Phase 1, an input generator trained with the task-specific training data. The inputs are prompts task, and the outputs are task inputs corresponding the prompt. Input and output examples applied to semantic parsing shown. During Phase 2, the input generator generates pseudo-inputs from the given target prompt, which are used to distill knowledge from the teacher to the student. Blue square boxes indicate frozen parameters; yellow rounded boxes indicate unfrozen parameters.
180
4.1 Continued Pre-training
We establish the Continued Pre-training method straightforward baseline for PI. This method injects prompts into the parameters of an LM by continuing with the pre-training objective of the LM on the target prompt. The pre-training objective is straightforward option as it works in unsupervised manner. In our experiments, we leverage the pre-trained T5 model [2] and thus use the masked language modeling objective which is the pre-training objective of T5. Following Raffel et al. [21]. we randomly replace 15% of a given prompt with special mask tokens; then, the model is
162
184
186
trained to predict the sequence of masked tokens. In this process, the model learns about the prompt the same way the model learns knowledge during the pre-training stage.
188
169
Curriculum learning We further investigate the baseline method by leveraging curricula 2 during continued pre-training. We set the mask ratio the difficulty criteria [34] and gradually increase the ratio throughout the Continued Pre-training. As the mask ratio increases, the model should predict more masked tokens given less context. With curriculum learning, gradually better adapt to the prompt, improving its prompt-dependent task performance. Throughout the experiments, we increase the mask ratio linearly from 15% to 30%, 50%, and 70% and report the best score.
171
expect the LM to
179
175
176
4.2
Pseudo-INput Generation (PING)
177
The purpose of is to inject a prompt into the parameters of an LM which can also be done indirectly through distillation. In this subsection, we propose novel distillation-based method called Pseudo- INput Generation (PING) that distills a novel prompt into a student LM that does not have access to the prompt through a teacher LM that does have access to the prompt. In order for distillation, pscudo-inputs are needed since we assume a scenario where the prompt to be injected has never been seen during training and does not have separate training data. An overview of PING is illustrated in Figure As shown in the figure, during Phase 1, an input generator is trained with the task- mask-specific training data. When given a prompt of the task as the input. the generator expected to generate the task inputs that correspond to the prompt. During Phase 2, the input generator is frozen and is used to generate pseudo-inputs from the unseen prompt, which are then given the teacher together with the prompt, while only the pseudo-inputs are given to the student. This way, the student learns to follow
178
179
180
182
183
194
185
186
188
the teacher and is able to learn about the prompt indirectly. We believe that this is the first work that aims to distill knowledge with different inputs for the teacher and the student.
5

---

Experimental Setup
190
191 In this section, we explain the experimental setups in detail. All experiments are performed with the T5-base 21 (220M parameters) model unless noted otherwise.
192
193
5.1
Prompt-dependent tasks
194
In order to evaluate the effectiveness of Prompt Injection (PI) methods, we select three prompt- dependent beaks--ppronn-depeendee conversation, semantic parsing, and zero-shot learning with task instructions; all these tasks require fixed prompts during inference. Fixed prompts come in the form of persona in persona-dependent conversation [39], database schema semantic parsing [[O, and task instruction in zero-shot learning with task instructions [23] As described in the introduction and Section 3 when PI is applied for these tasks, there would be apparent benefits in real world scenarios. For instance, PI eliminates the need to repeatedly include persona descriptions in the input during inference when serving a conversational model a specific personality. With these tasks, not only the performance of the baseline PI methods is evaluated, but also the significance of PI is emphasized by comparison with the (unconstrained) previous approaches that concatenate prompts to the input.
195
197
199
201
203
205
5.2
Datasets
Following datasets of prompt-dependent tasks mentioned in Section 5.1 are utilized to evaluate Prompt Injection (PI).
207
208
is a crowd-sourced dataset intended for training agents to perform engaging and personal chit-chat by comprising the dialogues to be grounded on specific personas. They crowdsourced 1,155 unique personas, each with five profile sentences and 162,064 utterances over 10,907 dialogues. For each dialogue, two speakers have 6-8 turn conversation conditioned on a given persona. The task is measured via perplexity (PPL). We randomly select 100 dialogues from the validation set as persona-dependent conversation benchmark for testing PI. The persona descriptions are 60 tokens long on average.
PERSONA-CHAT PERSONA-CHAL
209
210
212
214
215
Spider
Spider 37 a large cross-domain semantic parsing and eext-to-SLL dataset for developing
216
natural language interfaces to cross cross-domain databases. It includes 10,181 questions, 5,693 unique SQL queries, and 200 database schemas covering 138 different domains. Models must generalize to new database schemas as well as new queries to perform well on it. Evaluation metrics include Exact Matching (EM) and Execution Accuracy (EA). We utilize the dev set containing 20 databases with
217
219
about 50 questions per datahase semantic parsing benchmark for PI. The database schemas range in length from 55 to 430 token lengths.
221
222
WSC/ / RTE/ COPA For the task of zero-shot task generalization. Raffel al. (1] have trained the LM on diverse set of tasks and evaluated on held-out group of tasks to evaluate generalization performance. We choose coreference resolution, natural language inference, and sentence completion tasks, three out of their four held-out tasks, and test PI on WSC (Winograd Schema Challenge), RTE (Recognizing Textual Entailment), and COPA (Choice of Plausible Alternatives) datasets [30] All of
223
224
225
226
227
these tasks are binary classification tasks. We utilize task instructions (prompts) of WSC, RTE, and
228
COPA provided from Raffel et al. [21] and report average task scores of using task instructions. The task instructions are comprised of about 20-30 tokens.
230
5.3
Implementation Details
For the Continued Pre- Pre-training method (Section 4.1 we use the Adam optimizer [15 with constant learning rate le-4 and batch size 8. We perform 5-20 steps of injection. For PING (Section (42). input generators are trained on each tasks for 1-2 epochs. We use KL-divergence for distilling the last layer's 's output of the decoder and perform 10-40 steps of injection. Diverse aeeudo-inputs are generated by sampling each token from the output probability distribution of the decoder. For all of experiments except for zero-shot generalization, we use single 16GB T4 GPU. For zero-shot generalization, we use 4 32GB V100 GPUs.
232
233
234
296

---

Table Inference efficiency of different models that can be used for performing prompt-dependent inference. We depict how many times PI is efficient in comparison with the other approaches inside parenthesis. When there out-of-memory (OOM) using the 16GB T4 GPU, we estimate the FL.PSs in italics assuming a linear correlation between prompt length and FL.OPs.
Model
Prompt Length
FLOPs (G)
Latency (s)
T5 w/ PI
0.7k
0.58
T5
512
7.2k (x10.3)
1.09 (x 1.9)
512 x
14.6k (x21.0)
2.38 (x4.1)
512 x
OOM
T5 W/ FID
512
7.2k (x10.3)
1.09 (x1.9)
512 x 2
14.0k (x20.2)
1.54 (x2.6)
512 x
27.6k (x39.8)
2.87 (x4.9)
512 x 8
54.9k (x79.2)
5.87 (x10.0)
512 x 28
OOM (x280)
LINEAR-
512
9.5k (x13.8)
1.58 (x2.7)
TRANSFORMER
512 x
16.1k (x23.2)
2.62 (x4.5)
512 x
29.2k (x42.2)
4.74 (x8.1)
512
55.6k (x80.1)
9.11 (x15.6)
512 X 28
OOM (x280)
238 In order for injection and comparison with upper-bound and lower-bound performance, we first need two different versions of the LM adapted to the given task. For the task of persona-dependent conversation and semantic parsing, one (upper bound) is fine-tuned together with prompts since prompts explicitly used during inference, while the other (lower bound) is fine-tuned on the task without being given the prompt. We perform PI on the lower-bound LM since we also assume having access to prompts during inference.
240
242
For the zero-shot learning task, we modify the prompts developed by Raffel et [21 in the form of a fixed prompt. Their prompts have placeholders such as Premise, and 246 Hypothesis. We replace the placeholders with fixed words such as "Premise" and "Hypoth- esis", then append the actual content to the prompt in key-value format. For example, 248 the original is If {Premise} is true, is it also true that (Hypothesis)?, then the converted prompt is If "Premise" is true, is it also true that "Hypothesis"? 250 Premise: : (Premise} Hypothesis: (Hypothesis)- This ensures that the prompt is fixed, which can be injected with PI. We use the TO-3B LM checkpoint for the zero-shot generalization.
244
6 Experimental Results
In this section, first explore the inference efficiency of models performing prompt-dependent tasks and show that Prompt Injection (PI) leads to meaningful computational efficiency. Then the baseline and proposed methods are tested and compared datasets discussed in Section 5.2 The results indicate that the Pseudo-INpur Generation (PING) method achieves the best performance among PI methods, sometimes even outperforming the unconstrained upper bound, which uses explicit prompts during inference. In Section 6.3, we provide a concrete instance of injecting a real persona description into a conversational model, demonstrating the feasibility of long prompt injection.
254
256
257
258
259
6.1
Inference Efficiency
The comparison of inference efficiency of model with PI, a baseline model that naively concatenates
262
prompts to the input, Busion-in-Decode (FiD) [12, and Linear Transformer [33 are shown in
We consider FiD as one of the options for processing long inputs because it processes 264 long input sequences by encoding chunks of input sequences separately, reducing the quadratic
Table I
complexity to linear. Linear Transformer also reduces the complexity to linear by lincarizing the
7

---

Table Prompt Injection performance on three prompt-dependent tasks. w/ PROMPT stands for the upper bound (unconstrained) method, which uses the prompt during inference by appending it to the input. w/o PROMPT depicts the lower bound method of not utilizing the prompts at all. Lastly, we show three w/ PI methods: CP and CP w/ CURR stand for the Continued Pre-training (baseline) and the Continued Pre-training with curricular, respectively, as explained in Section 4.1 our novel proposed method utilizing distillation.
PING depicts
Dialogue
Semantic Parsing
Task Generalization
PERSONA-CHAN
Spider
WSC
RTE
COРA
PPL (U)
PI Score
EM EA PI Score ACC PI Score ACC PI Score ACC PI Score
W/ PROMPT
8.83
57.9 61.3
63.6
67.9
67.3
W/O PROMPT
W/O PI
11.01
14.5 15.1
44.0
64.2
60.0
W/ PI
CP
10.85
0.073
16.9 17.5
0.054
54.5
0.536
67.7
0.946
64.8
0.658
CP w/ CURR
10.61
0.183
17.7
18.4
0.072
50.8
0.347
68.2
1.08
64.1
0.562
PING
9.82
0.546
36.6
41.7
0.507
63.7
1.005
64.2
0
60.6
0.082
266 attention mechanism. We measure FLOPs and forward propagation latency via DeepSpeed Flops profiler using single 16GB T4 GPU.
267
As shown in Table 1 T5 w/ P is much more efficient than other models, especially as assume a longer prompt length. This is because the efficiency of PI remains the same independent of the prompt length while the costs of others increase linearly. Specifically, when the prompt length is 8 times the model's max input sequence length, one can achieve 80x computational efficiency in terms of FLOPs by applying PI. Furthermore, in a scenario where the prompt length is 28x the model's max input sequence length (shown in Section 6.3 when trying to utilize a long persona that is over 13,000 token length long), previous approaches show an out-of-memory (OOM) issue using
289
271
273
274
275
the I 16GB T4 GPU, and it is impossible to utilize them. PI is estimated to be 280x more efficient in
278
terms of total FLOPs if there no 0OM issue.
277
6.2
Task Performance
278
In Table 2. we report the task performance obtained by applying different PI methods on three prompt-dependent tasks. PI scores are also obtained as introduced in Section For all of PI methods, we observe an overall increase in performance compared to w/o PROMPT, indicating successful injection of prompts into the parameters of the model through PI methods.
279
280
281
282
For the results, while CP gives modest performance improvement over w/o PROMPT, the results of CP w/ CURR show that leveraging curricula during continued pre-training is effective in some cases. CP w/ CURR performs better compared to CP in PERSONA-CHAT, Spider, and RTE; it even outperforms w/ PROMPT in RTE. On the other hand, PING significantly improves performance from CP in PERSONA-CHHE CHAT,, Spider, and WSC, outperforming w/ PROMPT in WSC. This sheds light on the possibility that PI may be able to reach the upper bound (unconstrained) performance. However, the results show at the same time that there is still gap between the performance of PI methods and the upper bound w/ PROMPT that needs be bridged future work.
284
285
286
287
288
289
280
We find that the performance of different methods depends on the complexity of the input sequence structure. We believe that PING achieves good performance in PERSONA-CHAT, Spider, and WSC because those datasets have relatively simple input sequences (short utterances; simple query; sentence and two words, respectively). In datasets with many components or multiple complex sentences (e.g., COPA and RTE), the low quality of generated pseudo-inputs degrades the performance of PING. On the other hand, CP and CP w/ CURR perform better in datasets with complex structure. These findings encourage the community to explore a more integral PI method that can cover different
282
294
295
296
297
datasets.
8

---

Actual Utterances
what do you do?
i work for Tesla, my main concern is the battery pack.
Elon Musk Q37521
caee ceo?
founder of Tesla and the Tech Officer.
of the world's best places what's your mind?
[..]
the people Mars.
technology entrepreneur, investor, and engineer. am the founder, CEO, and lead designer of SpaceX; co founder, CEO, and product architect of Tesla, Inc. L.J
what's your plan?
to go to Mars, it sounds beautiful. what is your plan Mars?
IIl also like to go to Mars!
Mars would be absolutely wonderfull Mars may be the answer to some the most important questions humanity has answered so far.
Figure real world example of Prompt Injection with a long prompt. (Left) The process of injecting Wikipedia article describing person (Elon Musk) into model with PI. The article is more than 13,000 tokens long. (Right) Actual conversation between the persona injected model and human that is hand-picked.
6.3 Long Prompts Injection
To demonstrate the effectiveness of PI injection of long prompts into LMs, show how the method works with real world example. We pick a Wikipedia page (Elon Musk), considering it as long persona description, and inject the entire article (over 13,000 tokens) into LM trained with PERSONA-CHAT. Here, we use T5-large as base model and apply PING.
300
302
Figure shows actual instance of interactions with the LM that underwent PI through PING. The responses show the LM successfully reflecting the description of the person on the Wikipedia page without having the description appended to the input. Moreover, the inference of PI is 280 more computationally efficient in terms of FLOPs than the baseline, as shown Section 6.1 Lastly, we provide live demo to allow interactions with an LM injected with the persona of Elon Musk.
304
306
7
Conclusion
309
Limitations and Future Work While Prompt Injection (PI) enables performing prompt-dependent
310
tasks efficiently, there are limitations that needs to be addressed in future work. In particular, the current PI methods cause task performance degradation. Moreover, the computational costs needed for the injection of prompts into the model parameters have not been extensively considered. For example, when considering previous conversation history as prompts be injected in multi-turn conversation setting, fast injection may also be requirement for real-world application. Updating adding a relatively small number of parameters 31 may be potential avenue for addressing the problems.
311
313
315
317
In this paper, we propose Prompt Injection (PI), a novel formulation of injecting the prompt into the parameters of an LM, as an efficient alternative to attaching fixed prompts to the inputs for prompt- dependent tasks. Through experiments, we show that PI is much more computationally efficient (up to 280 times) in terms of total FI.OPs for handling long prompts compared to the previous alternatives. further explore baseline methodologies for and find that Pseudo-INput Generation (PING), distillation-bases approach, shows promising results in persona-dependent conversation, semantic parsing, and zero-shot learning with task instructions. Through the explorations, we show that PI can be promising direction for conditioning language models with prompts, especially in scenarios with long and fixed prompts. To this end, we hope the community explores PI for achieving both performance and efficiency on prompt-dependent tasks.
318
319
321
323
325
327 References
328
[1] Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer. ArXiv, abs/2004.05150, 2020.

---

330
[2]
Yoshua Bengio, Jéröme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML '09, 2009.
331
332
[3] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Rad- ford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. ArXiv, abs/2005. 14165, 2020.
334
336
338
339
[4] Daniel Fernando Campos. Curriculum learning for language modeling. ArXiv, abs/2108.021700 2021.
[5] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Baindoor Rao, Parker Barnes, Yi Tay. Noam M. Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du Benton C. Hutchin- son, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Gareía, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim. Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz. Erica Oliveira Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathleen S. Weier-Hellstemy Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways. ArXiv, abs/2204.02311. 1. 2022.
342
344
345
346
347
348
3/9
350
351
352
353
354
[6] Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoe V. Le, and Ruslan Salakhut- dinov. Transformer-xl: Attentive language models beyond fixed-length context. In ACL, 2019.
355
356
357
[7] Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, and Matthew Richardson. Structure-grounder pretraining for text-to-sql. ArXiv, abs/2010.12773, 2021.
359
380
[8] Mandy Guo, Joshua Ainslic, David C. Uthus, Santiago Ontañón, Jianmo Ni, Yun-Hsuan Sung. and Yinfci Yang. Longt5: Efficient text-to-text transformer for long sequences. ArXiv, abs/2112.07911 2021.
382
363
[9] Seungju Han, Beomsu Kim, Jin Yong Yoo, Seokjun Seo, Sangbum Kim, Enkhbayar Erdenee, and Buru Chang. Meet your favorite character: Open-domain chatbot mimicking fictional characters with only a few utterances. arXiv preprint arXiv:2204. 10825, 2022.
365
386
[10] Moshe Hazoom, Vibhor Malik, and Ben Bogin. Text-to-sql in the wild: A naturally-occurring dataset based on stack exchange data. ArXiv, abs/2106.0555 05006, 2021.
367
388
[1]] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. ArXiv, abs/2106 09685, 2021.
389
370
371
[12] Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In EACL, 2021.
372
373
[13] Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Franccois Fleuret. Transformers are mns: Fast autoregressive transformers with linear attention. ArXiv, abs/2006.16236. 2020.
374
375
[14] Boseop Kim, Hyoungseok Kim, Sang- Woo Lee, Gichang Lee, Donghyun Kwak, Dong Hyeon Jeon, Sunghyun Park, Sung ju Kim, Seonhoon Kim, Dong Hyung Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, SukHyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Dong hyun Ham, Do- Hyoung Park, Min Young Lee, Jacwoo Kang, Inho Kang, Jung- Woo Ha, Woo Chul Park.
376
377
379
10

---

381
and Nako Sung. What changes can large-scale language models bring? intensive study on hyperclova: Billions-scale korean generative pretrained transformers. ArXiv, abs/2109.04650, 2021.
382
[15] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6680, 2015.
385
386
[16] Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. Few-shot parameter-effiiiet fine-tuning is better and cheaper than in-context learning. 2022.
387
388
[17] Qian Liu, Yihong Chen, B. Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang. You impress me: Dialogue generation via mutual persona perception. ArXiv, abs/2004.05388, 2020.
390
382
[18] Mirre-Emmanaee Mazaré, Samuel Humeau, Martin Raison, and Antoine Bordes. Training millions of personalized dialogue agents. In EMNLP, 2018.
393
384
[19] Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. Assigning personal- ity/profile to a chatting machine for coherent conversation generation. In IJCAI, 2018.
385
396
[0
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models unsupervised multitask learners. 2019.
398
[2]] Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. ArXiv, abs/1910. 10683, 2020.
389
400
401
[22] Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric Michael Smith, Y.-Lan Boureau, and Jason Weston. Recipes for building an open-domain chatbot. In EACL, 2021.
403
404
[23] Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang A. Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M SAIFUL BARI, Canwen Xu, Urmish Thakker, Shanya Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang. Trishala Neeraj. Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries. Ryan Techan, Stella Rose Biderman, Leo Gao, T. G. Owe Bers, Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization. ArXiv, abs/21 10.08207, 2021.
405
406
407
408
409
410
411
412
[24]
Timo Schick and Hinrich Schütze. It's not just size that matters: Small language models are also few-shot learners. ArXiv, abs/2009.07118, 2021.
414
415
[25] Haoyu Song, Weinan Zhang, Yiming Cui, Dong Wang, and Ting Liu. Exploiting persona information for diverse generation of conversational responses. In IJCAI, 2019.
416
417
[26]
Alane Suhr, Ming- Wei Chang, Peter Shaw, and Kenton Lee. Exploring unexplored generaliza- tion challenges for cross-database semantic parsing. In ACL, 2020.
[27]
Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey.
420
ACM Computing Surveys (CSUR), 2022.
421
[28]
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam M. Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du Yaguang Li, Hongrae Lee, Huaixiu Zheng, Amin Ghafouri. Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin. Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Chin Ching Chang, I. A. Krivokon, Willard James Rusch, Mare Pickett, Kathleen S. Meier-Helllstern Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Hartz Søraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravindran Rajakumar, Alena Butryna, Matthew Lamm, V. O. Kuzmina, Joseph Fenton, Aaron
422
423
424
425
426
427
428
11

---

430
Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Areas Claire Cui, Marian Croak, Ed Chi, and Quoc Le. Lamda: Language models for dialog applications. ArXiv, abs/2201.08239, 2022.
431
[29] Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. ArXiv, abs/1706.03762, 2017.
434
[30]
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for general-purpont language understanding systems. In NeurlPS, 2019.
437
438
439
[3]] Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. K-adapter: Infusing knowledge into pre-trained models with adapters. In FINDINGS, 2021.
440
441
442
[32] Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners. ArXiv, abs/2109.01652, 2021.
444
445
[33] Sean Welleck, Jason Weston, Arthur D. Szlam, and Kyunghyun Cho. Dialogue natural language inference. In ACI., 2019.
[34] Alexander Wettig, Tianyu Gao, Zexuan Zhong, and Danqi Chen. Should you mask 15% in masked language modeling? arXiv preprint arXiv:2202.08000 2022.
448
449
[35]
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. Transfertransfo: A trans- fer learning approach for neural network based conversational agents. ArXiv, abs/1901.08144 2019.
450
[36] Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. Unifiedskg: Unifying and mulli-tasking structured knowledge grounding with text-to-text language models. ArXiv, abs/2201.05900 2022.
453
455
457
458
[37]
Tao Yu, Rui Zhang, Kai-Chou Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Z Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir R. Radev. Spider: A large-scale hamen-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In EMNLP, 2018
459
480
461
462
[38] Manzil Zaheer. Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. ArXiv, abs/2007. 14062, 2020.
464
465
[39] Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur D. Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: have dog, do you have pets too? In ACL, 2018.
[40] Chen Zhu, Wei Ping, Chaowei Xiao, Mohammad Shoeybi, Tom Goldstein, Anima Anandkumar, and Bryan Catanzaro. Long-short transformer: Efficient transformers for language and vision. ArXiv, abs/2107.02191 2021.
468
469
12


Reviews:
[{"review_id": "SbZG6-bGTzp", "reviewer": "Reviewer_AvLV", "summary": "This paper addresses the problem in prompt learning that prompts are always included in the input text, which can cause computational and memory overhead.\nTo this end, this paper proposes prompt injection, a method that injects the prompt into the parameters of a pre-trained language model (PLM).\nThey show that the prompt injection method can be 280 times more efficient compared with previous approaches.", "questions": "NA", "limitations": "NA", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. This paper proposes an interesting question that many conditional text generation tasks usually include an instance-specific prompt for each data, for example the persona profile for persona dialogue generation.\nHowever, such prompt can cause computational costs and make it difficult for models to capture the key information during inference, in particular when the prompt is very long.\nTherefore, this paper proposes a new method call Prompt Inject (PI) to store the prompts in the PLM instead of concating it to the input.\nTo my knowledge, the idea is very novel and interesting.\n\n2. The experiments are solid.\nThis paper conducts experiments on persona dialogue generation, Text-2-SQL and zero-shot task generalization.\nThe experiments show that the PI can speed up the inference time and improve the lower bound of the models.\n\n3. This paper is well written with proper citation and clarification\n\nWeakness:\nA minor note would be that the prompt in this paper is different from the prompt in most sense in other NLP paper, such prefix tuning, prompt tuning, etc.\nSo I  would suggest that the authors could give a more clear explanation on this issue in the introduction.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9DpOJxKvDuG", "reviewer": "Reviewer_XfKG", "summary": "The paper is about prompt injection, which aims to be an efficient alternative to attaching fixed prompts to the input. Prompt injection is sample efficient and worth it when there are long prompts. This can be useful if there is a detailed description of a personalized chatbot, for example.\n\n**Update after response.**\n\nWith the additional experiment, I've increased my score to 5.", "questions": "How does the proposed method compare with other baselines such as context distillation? (Askell et al, https://arxiv.org/pdf/2112.00861.pdf)\nAre there other approaches where this approach might be especially useful? As the main benefit is inference-time efficiency, one might imagine that the WSC, RTE, COPA, and SPIDER tasks are not the prime candidates for this approach. The dialogue eval is good but it only uses one dataset. ", "limitations": "Seems fine to me", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: Paper tackles the problem of using fixed prompts many times for different examples.\nQuality: Experiments done in the paper are OK but not particularly stunning. One way to improve the experiments could have been to pick a setting where it was crucial for long contexts (like summarization, or long dialogues), and show that with this method it was possible to use longer contexts which lead to substantially better performance. Or show that it works over a broad range of settings.\nClarity: The paper is relatively clear.\nSignificance: The paper proposes a method for prepending fixed prompts in a computationally efficient manner. \n\nWhile the paper is a nice start, I think it still leaves a lot to be desired. There is enough substance to warrant an ACL short paper, but I am not sure about a NeurIPS paper. Some examples of how I think the paper could be stronger:\n- Empirical results showing that PING works across a very wide range of tasks with good performance. \n- Comparison with other methods that try to do similar things.\n- Show for tasks that require long inputs and long prompts, PING enables substantial performance improvements. \n- Overall, experiments on settings where fast inference is crucial might make it more clear to the reader why it is important to trade off this complexity from using PING for faster inference. \n\nMinor note: Line 22 should be Sanh et al, not Raffel et al., right?\nSome further related work might have been missed, such as context distillation (https://arxiv.org/pdf/2112.00861.pdf)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2bLCYKYSN5", "reviewer": "Reviewer_y47p", "summary": "This paper proposes a (self-proclaimed) novel approach called prompt injection, which injects the declarative knowledge of a prompt into the language model parameters by distilling a teacher model with the prompt into a student teacher without prompt on automatically generated inputs. The papers show that the injection is successful across a wide & diverse range of tasks and the approach saves inference time compute. ", "questions": "- How many pseudo inputs did the paper distill on? I would imagine that the gap between the student and teacher should drastically decrease when you increase the number of pseudo inputs. \n- How does the gap between student & teacher change w.r.t. model size? Should we expect future larger models to become better at this? \n- Can you show some qualitative examples where you claimed that \"the low quality of the pseudo generated inputs lead to lower performance\"? How about the diversity of the pseudo inputs? ", "limitations": "Yes.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- This paper elaborates on the impactful idea of \"prompt-injection\", and show that this is broadly useful for a wide range of tasks. \n- This paper also sets up a reasonable evaluation framework that could be used to test future methods.\n- This paper is generally well-written. \n\nWeakness:\n- The idea of prompt-injection is not novel, and various previous works have explored this idea. For example, [1] and \"context distillation\" in [2]. Please tune down the claim on novelty in the presentation. That said, I would still vote for acceptance of this paper since the other paper did not focus on the framing of prompt-injection, and this paper could be useful for the community to cite. \n- I wished that there were more experiments and ablations, given that the contribution of this paper is more about \"studying an existing algorithm (PING) and figure out its basic property\", rather than \"proposing a drastically novel algorithm\". \n\n[1] Towards Zero-Label Language Learning\n[2] A General Language Assistant as a Laboratory for Alignment", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject