Can discrete information extraction prompts generalize across language models?
Authors: Nathanaël Carraz Rakotonirina
Roberto Dessi
Fabio Petroni
Sebastian Riedel
Marco Baroni
Venue: ICLR 2023 poster (2023)
Keywords: prompting
prompt analysis
language model interfaces
prompt generalizations

Abstract: We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information. After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts on the slot-filling task, we demonstrate a drop in performance for AutoPrompt prompts learned on a model and tested on another. We introduce a way to induce prompts by mixing language models at training time that results in prompts that generalize well across models. We conduct an extensive analysis of the induced prompts, finding that the more general prompts include a larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens. Our work provides preliminary evidence that it's possible to generate discrete prompts that can be induced once and used with a number of different models, and gives insights on the properties characterizing such prompts.

Published as a conference paper at ICLR 2023
CAN DISCRETE INFORMATION EXTRACTION PROMPTS GENERALIZE ACROSS LANGUAGE MODELS?
Nathanaël Carraz Rakotonirina,' Roberto Dessi, 1.2 Fabio Petroni, Sebastian Riedel,* Marco Baroni 155 Universitat Pompeu Fabra, "Meta AI, Saaaaya AI, *University College London, SICREA
ABSTRACT
We study whether automatically-induced prompts that effectively extract informa- tion from a language model can also be used, out-of-the-box, to probe other lan- guage models for the same information. After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts the slot-filling task, we demonstrate drop in performance for Au- toPrompt prompts learned on a model and tested on another. We introduce a way to induce prompts by mixing language models training time that results in prompts that generalize well across models. We conduct an extensive analysis of the induced prompts, finding that the more general prompts include larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens. Our work pro- vides preliminary evidence that it's possible to generate discrete prompts that can be induced once and used with a number of different models, and gives insights on the properties characterizing such prompts
INTRODUCTION
NLP has shifted to a paradigm where very large pre-trained language models (LMs) are adapted to downstream tasks through relatively minor updates (Bommasani al. 2021 Liu al. 2021). In the most extreme case, task adaptation does not require modifying the LM or even accessing its internals all, but simply formulating linguistic query that elicits appropriate, task-specific response by the model (Petroni et al. 2019a Radford et al. 2019). This has promising practical applications, as one could easily imagine proprietary LMs only exposing a hanuuull-aguugg-aaaee interface, with downstream agents extracting the information they need by formulating the appropriate queries" In this scenario, one fundamental question is how robust the querying protocol to changes in the underlying L.M. On the one hand, the same downstream agent might want to query multiple LMs. On the other, if the LM provider updates the model, this should not break the downstream pipeline. On more theoretical level, the properties of an emergent robust protocol might give insights on the general language processing capabilities of neural networks, and how they relate to natural language.
We present a systematic study of the extent to which LM query protocols, that, following current usage, call prompting methods, generalize across LMs. Extending and confirming prior results, we find that discrete prompts that are automatically induced through an existing optimization proce- dure (Shin et al.] (2020) outperform manually and semi-manually crafted prompts, reaching good performance level when tested with the same LM used for prompt induction. While the automati- cally induced discrete prompts also generalize better to other LMs than (semi- maanual prompts and currently popular "soft" prompts, their overall generalization performance is quite poor. We next show that a simple change to the original training procedure, namely using more than one LM at prompt induction time, leads to discrete prompts that better generalize to new L.Ms. The proposed procedure, however, is brittle, crucially relying on the "right" choice of LMs to mix prompt in- duction. We finally conduct the first extensive analysis of automatically induced discrete prompts,
'The code reproduce our analysis is available at generall on
"As concrete example, one of the most powerful current LMs, GPT3, is only available via a text-hased API hhttss:/bbet openai www.///wvvviii..

---

Published as a conference paper at ICLR 2023
LM1
Florence
LM1
Forence
LM2
Bostom
LM2
forence
professionology
Dantapowiord pool club in [MASK]
Japan
bused [MASK]
LMB
Florence
plecestfalmn prompt induced on LMI LMG
tree
UM4
Forence
Figure I:
Cartoon summary of our main results. Prompts induced using single language model have a significant drop of performance when used to query other models. The problem is alleviated when prompts are exposed multiple models in the induction phase. Subtle but consistent differ- ences in the nature of the induced prompts also emerge.
tentatively identifying set of properties characterizing the more general prompts, such as a higher incidence of existing English words and robustness token shuffling and deletion.
RELATED WORK
Prior work such as Petroni et al. (2019a) and Radford et al. (2019) demonstrated that LMs can be directly adapted to new tasks through appropriate querying methods. This led to explosion of work on so-called "prompt engineering" (see Liu et al. 2021 for a thorough review). Much of this work focuses crafting appropriate manual or semi manual prompts and/or tuning LMs to better respond to such prompts (e.g., [Schick& & Schütze 2021; Sanh et al., 2022). Going beyond SiiinpttssSShieeeeee manual prompts, [Shin al. (2020) introduced the AutoPrompt algorithm generate prompts using
gradient-guided search, and demonstrated that such prompts often outperform manual ones. While automatically induced prompts suffer of issues such now-interpretability, we think it is important to continue focusing on them because, besides their better performance (a result we confirm here for AutoPrompt across range of LMs), they are more promising than manual prompts terms of scalability, especially in contexts in which it is not sufficient to formulate a single prompt template for a whole task, but each input query demands a distinct prompt formulation (Zhang et al.) (022).
Concurrent and later work has proposed to replace discrete strings, such those generated by AutoPrompt, with sequences of arbitrary vectors from the LM'ss embedding space (Lester et al,) 2021 Zhong et al., (222)) We confirm here that these continuous, or "soft" prompts outperform AutoPrompt when trained and tested on the same LM. However, they cannot be used in our envis- aged Lul scenario. First, they require access to a model inner representations, beyond the standard natural language querying interface, so that embeddings can be passed as input. Second, continuous prompts, by their nature, won't generalize out-of out-of-the-bon the-box to other LMs. Trivially, they can't generalize across models with different embedding dimensionality. Even when models share dimensionality, there is reason why the absolute position of vector in the embedding space of model should meaningfully transfer to another model. Discretizing soft prompt tokens to their near- est vocabulary neighbours in order to overcome these issues does help either. Khashabi et al. (2021) demonstrated that it is possible to find well-performing soft prompts whose nearest neighbor projections are arbitrarily fixed discrete tokens. Appendix ellaborates on the failure of soft prompts to generalize across models, as well as the problematic behaviour of discretized soft prompts.
We are not aware of much previous work that has addressed the challenge of LM-to-LM transfer- ability. Wallace et al. (2019) studied this problem in the context of textual adversarial attacks (that can seen as a special case of prompting, and indeed their attack method closely related to AutoPrompt). Similarly to us, they notice some performance drop when transferring adversarial "triggers" to different LMs, and they show that this can be mitigated by an ensembling approach where two triggers generated using variants of the same LM combined.
Su et al. 2022 study LM-to-LM transferability in the context of continuous prompts. Since, as just discussed, such prompts not directly transferable, they induce projection from the embedding space of the source LM to that of the target LM, thus considering very different scenario from the type of "out-offthee-ox" transferability are interested in here.
2

---

Published as a conference paper at ICLR 2023
3 EXPERIMENTAL SETUP
3.1 DATA
We focus on the task of slot-filll fott-filiing which, since its introduction in LM evaluation through the LAMA
benchmark (Petroni et al. (2019)). has been extensively used to probe the knowledge contained in LMs (AIKhamissi et al. 2022). More specifically, we use the T-ReX split (Elsahar et al. 2018)) of LAMA. Each fact in T-ReX is represented as a triple (subjecl, relation, object)-for example, (Dante, place of birth, Florence). LMs are queried using cloze prompts as in "Dante was born in place
A LM is said have properly stored fact if can successfully predict the ground-truth object given a prompt and the corresponding subject. We have decided to focus primarily on this task because the prompts convey actual semantic information (characterizing the relation between the subject and the object) rather than just metalinguistic information (as would be the case, for example, for a machine-translation prompt, which might express something like: "translate the next sentence from English to French"). Furthermore, the task requires learning different prompt for each relation, which can be seen as a first step toward fully flexible prompts that would change with each single input (Zhang et al. 2022).
The LAMA test set contains 41 relations, each with up to 1,000 facts. We also evaluate on the more challenging LAMA-UHN subset (Poerner et al. 2019), addressing some of the weaknesses of the original LAMA, in Appendix D All prompting methods are trained using the training data collected by [Shin et al. (2020), which include 1,000 facts for each relation type, drawn from either the original T-REx dataset or Wikidata. LAMA also provides manual prompts, which we will use in our experiments.
Since each LM class in our experiments (see Section 3.2 below) has its own vocabulary, a common subset must be used for fair comparison. This is obtained from the intersection of the vocabularies of all models considered. Furthermore, the training and test datasets are filtered to ensure that each object is included in the common vocabulary. There are 11,511 case-sensitive items in the common vocabulary. The filtered test set contains 25,358 facts, while the training set contains 25,247 facts.
We evaluate prompting methods using micro-avert aicrreevee accuracy (precision 1).
3.2 LANGUAGE MODELS
Our experiments cover the three main types LM. We use pre-trained LMs without any kind of parameter updating.
the LMs considered in this study.
Masked LMs
They produce representations using both the left and right context. Given se- quence x = Jn they estimate the probability of token æ; given its left and right context p(ax;) = p(x; ( (f,, Ii 1i4 I..,,..., T.).
Left-to-right LMs ity to a sequence of tokens. Given a sequence tokens æ [z], ..., &n], left-to-right LMs assign probability p(æ) to the sequence using the chain rule p(æ) II, p( (1 21,, 1t.)).
They predict the next token conditioned on previous ones or assign a probabil-
Sequence- LMs They are composed of a bidirectional encoder that uses both left and right context and a left-to-right decoder that do not share parameters.
3.3 PROMPT INDUCTION METHODS
Prompts are either manually crafted or generated automatically by prompting methods. this study, have selected different prompting methods that are representative of semi-man remi-manual, discrete and continuous induction methods, respectively. They have been shown to perform well on the slot filling task and associated code is publicty available.
LPAQA Starting from seed manual prompts, Jiang et al. (2020) generate diverse candidate prompt set using mining- and paraphrasing- based methods. For each relation, the best perform- ing candidate the training data is selected. To improve performance, the authors also propose
3

---

Published as a conference paper at ICLR 2023
Table Pre-trained language models considered in this study.
Model
Type
#Parameters
Training Corpus
BART Devlin al. 2019
Masked
110M
BERTLARGE Devin et al. 2019
Masked
340M
Wikipedia (en) & BookCorpus (16GB)
DistilBERT Sanh a.. 2019
Masked
RoBERTaBASE Lin et al. 2019
Masked
125M
Wikipeddi (en) & BookCorpus & CC-News & Open WehText Stories (160GB)
RoBERTaLARGE [Lu et al.
20T9
Masked
355M
DisalRoBERaa (Sanh ci al.
2019
Masked
82M
OpenWebText (38GB)
GPT2 (Radford ct al. 2019
Left-to-right
117M
GPT2MEDIUM (Kadford cl 2019
Left-to-right
345M
WebText (40GB)
GPT2LARGE Radiord cl al. 2019
Left-to-right
774M
GPT2x1 Radlord ct Bl. 2019
Left-to-right
1.5B
BARRTASE Lewis et al.
2019
Seq2seq
140M
Wikipedia (en) & BookCorpus & CC-News & OpenWehText & Stories (160GB)
BART (Lewis et al. 2019
Seq2seq
400M
TTSMALL et al.
2020
Seq2seq
60M
TSRASE (Rallel ct al.
2020
Seq2sey
220M
C4 & Wiki-DPR (765GB)
TSLARGE [Kaltel 2020
Sey2sey
770M
prompt ensembling. However, ensembling tends to increase performance independently of the un- derlying prompting method (see Appendix(A). Consequently, we will only focus on the top-1 prompt selection method here. We consider LPAQA semi-manual method because it needs to be seeded with manual prompts, and mining retrieves further human-generated strings.
AutoPrompt It is an automated method proposed by Shin et al. (020)) to generate discrete prompts using gradient-guides search (Wallace al. 2019). The prompts are composed of a se- quence of tokens selected from the vocabulary of the LM. The number of tokens is pre-defined. The process is divided into two phases. For each specific token position, a of candidates that maxi- mize the likelihood on a batch of training data is first created. Then, the candidates are re-evaluated on different batch, with the best one retained. Even though the generated prompts are less inter- pretable, they perform better than manual prompts. In our experiments, use 5-token prompts and run the algorithm for 1,000 iterations.
OptiPrompt Zhong et al. (2021) propose an automated method to generate continuous prompts. They dense vectors in the embedding space of the LM that are learned using gradient descent on separate training dataset. Except for the learning rate, which is increased to 3e-2 for the T5 models for proper convergence, we the same hyperparameters as the original implementation. We initialize vectors randomly.
RESULTS AND ANALYSIS
4.1 PROMPTING METHOD PERFORMANCE
We start by evaluating the performance of the different prompting methods Prompts are induced with specific LM and then evaluated by retrieving objects from the same LM. Table 2 summarizes the results. For reference, majority-class baseline always picking the most common object for each relation reaches 26.91% % accuracy (note that this baseline has partial access to the ground truth in order to retrieve the most common object of each relation). The random baseline is virtually at 0%.
AutoPrompt clearly outperforms LAMA and LPAQA, although it lags behind OptiPrompt. We thus confirm that soft prompts are the way to go if you have access model embedding space and are not interested in generalization across models. If either of these conditions is not met, AutoPrompt preferable to manual and remi-manual prompts.
Masked models tend to perform better as source LMs. This could be attributed to the fact that they were pre-trained with a fillin-thheblank task (Devlin et al.) 2019), which exactly how the slot filling task is formulated.
Following Petroni et al. 2019h), when testing manual and LPAQA prompts with left-to-right I.Ms, only the tokens before [MASK] used. [MASK] is always the last AutoPrompt and OptiPrompt token.
4

---

Published as a conference paper at ICLR 2023
Table Comparison of different prompting methods using micro-averaged accuracy (precision @ 1).
Model
LAMA
LPAQA
AutoPrompt
OptiPrompt
BERT BASE
34.82
41.18
50.09
48.26
BERT
35.81
42.15
49.52
50.88
DistilBERT
6.75
13.14
29.79
44.76
ROBERTaBASE
26.36
32.80
39.63
44.73
RoBERTaLARGE
31.63
40.54
44.12
47.39
DistilRoBERTa
23.80
32.43
41.17
44.21
GPT2
7.23
9.63
11.36
39.28
GET2MEDIUM
13.74
18.29
18.59
38.43
GPT2LARGE
15.50
19.97
12.91
44.14
GPT2x1.
16.98
21.31
15.42
47.76
BARTBASE BASE
22.95
32.43
39.63
43.05
LARTLARGE
27.07
36.78
26.56
45.06
T5SMALL
14.94
20.94
31.44
28.10
TARASE
24.35
32.70
39.59
40,51
TSLARGE
28.21
36.07
42.00
44.42
average
22.00
28.69
32.62
43.39
St dev
9.17
10.56
13.12
5,40
Source LM
"Germany encompassed disneyland state seat tariffs
Target LM
Dataset
<Fremcu capilal Parai <Spain cappiall Madi d> <Japan, capital Tokyo>
AutoPrompt generation
Information extraction
"Berlinn"
Prompt generalization. For each relation, prompt is induced from the source LM. At test time, the prompt is tested on the target LM.
Figure 2:
4.2 AUTOPROMPT GENERALIZATION
In this section, investigate AutoPrompt's ability generalize across different LMs Prompts induced using a Source LM and then evaluated on Targer L.M, which can be the same or different from the Source (Figure 2
The results, relative to single-model LM performance, are shown in Figure 3 AutoPrompt generally performs best when the Source and Target LMs the same, shown by the fact that off-diagonal values are mostly negative. The performance gap gets bigger as we transfer across different LM types. Prompts are generally more stable across different sizes the same model, such BERT and BERTLARGE- The drop in generalization performance of left-to-right models is less dramatic simply because, for these models, the original sone--suure--nn--taae target performance is already low.
We also verify the impact of model size generalization. For each source model, we define the generalization drop score the average of the corresponding column in Figure measures the average drop in accuracy when prompts from source model are tested on a different target, with respect to the original same-source-ande -anrttatrre accuracy. We discovered that performance drop and source model size are highly correlated (0.6). We do not have clear explanation for this correlation, but we think it is an intriguing observation that should be further studied in the prompt analysis literature.
'Appendix B confirms that OptiPrompt prompts do not generalize well. Appendix shows that AutoPrompt outperforms I. IPAQA also in terms of generalization.
5

---

Published as a conference paper at ICLR 2023
BERT base
0
-13
-9
29
-39
-20
26
-32
42
43
29
45
33
36
29
40
BERT large
3
34
-18
-8
40
26
29
32
-33
DistiIBERT
-14
-19
-24
28
-15
22
-28
-27
25
23
26
-5
RoBERTa base
-7
-12
-10
-1
.17
-18
-27
288
-14
25
23
23
19
RoBERTa large
6
-13
10
0
-6
-20
33 -32
-18
25
266
23
-18
20
DistilRoBERTa
99
-14
-12
88
-22
0
-18
-23
-31
-30
-14
-22
-22
27
-24
Target LM
GPT2
-7
GPT2 medium
4
6
-1
-7
-1
4
-1
0
9
6
1
8
-10
10
6
GPT2 large
GPT2 XL
3
-1
4
4
2
-0
2
0
5
2
4
4
-2
BART
-11
-16
-23
-23
-18
BART large
0
-
-10
-13
-5
2
4
-14
-12
-0
-12
-13
6
-16
-16
-18
-18
T5 base
-10
-18
9
-21
-26
-16
-15
-18
-25
-24
.18
-14
0
-13
large
-10
-15
-21
-24
-14
-13
-13
-24
-23
-18
-18
-21
-15
BERT base
BERT large
DistIBERT
ROBERTa base
RoBERTa large
DistiRoBBRRa
GPT2
GPT2 medium
GPT2 large
BART large
T5 small
base
T5 large
GPT2
BART
Source LM
Figure 3: AutoPrompt relative performance across LMs. Each column represents a Source LM and
each Target LM. Each value represents the difference between the accuracy achieved by the Target LM when using prompts induced with the Source LM and the accuracy obtained when Target also used for training.
4.3 MIXED-TRAINING AUTOPROMPT GENERALIZATION
We propose simple modification to AutoPrompt training generate prompts that generalize better. Recall that the AutoPrompt algorithm involves two phases: one in which candidate prompts are generated, and one in which the prompts are evaluated. Rather than relying the same model for the two phases, we now use two different LMs. The first model, which we call the generator, proposes of candidates. Then, the second model, that we call the evaluator, evaluates the candidates and chooses the best one.
To avoid combinatorial explosion, we focus combining the single LMs of each class that per- formed best in the same-source-and-tare setup (Table abbove). For each pair, we arbitrarily use setup) generator. We deliberately avoid the best of the two LMs (in the sane-sourceeand-bae (Figure 3), as in realistic settings we might picking models based on generalization performance have advance access the new architectures we need generalize to.
comparess the performance of standard and mixed AutoPrompt (we extend these experiments LAMA-UHN in Appendix DD.. The BERT L mix has the highest average accuracy. Although it does not perform as well as BERT on the BERT models, it transfers better to all the other models (including the RoBERTa family). This mixed AutoPrompt variant even outperforms the best seq2seq model T5LARGE all sequence-to-sequence models (including T5LARGE itself). It also outperforms GPT2MEDIUM on two GPT2 variants.
If these results are very encouraging, Table 3 also shows that simply mixing models does not guar- antee good generalization. When we replace BERTBASE with T5LARGE as generator, generalization performance actually worse than when using T5LARGE alone, and combining BERT BASE as gener- ator with GPT2 as evaluator leads to minor generalization improvements compared to using BERT BASE alone. Some preliminary insights the best mixing strategy offered in Appendix

---

Published as a conference paper at ICLR 2023
Table AutoPrompt mixed training. The first three columns report generalization accuracy for the single best LM in each class; the next three columns evaluate their combination.
Target
BERT
AAGG
BE.KTnsss/SSAAwG
BERTBASE
50.09
18.02
20.83
41.13
38.64
16.47
BERT
47.01
22.54
26.43
40.51
39,6
16.29
DistiBERT
15.75
3.65
RoBERTauast
32.31
21.31
20.28
36.01
30.56
17.24
RoBERTILAaGE
37.79
24.07
26.06
38.63
34.24
21.16
DiRII
31.71
18.28
17.24
33.49
28.53
16.58
GPT2
4.70
9.71
6.04
10.19
7.53
8.12
GPT2MEDRIM
14.38
18.59
12.51
16.29
22.49
16.47
GPT2:
17.95
15.68
13.52
22.33
19.95
14.84
GPTZXL
18.34
15.02
13.09
19.74
23.19
BARTEAE
26.98
24.11
21.62
34.57
31.62
18.84
BARTLARGE
26.73
25.20
20.32
33.73
29.15
18.94
T3MALL
15.78
16.23
11.89
19.31
18.28
7.99
T5
29.26
22.04
26.58
32.06
TSLAKE
32.32
28.90
42.00
44.51
35.71
27.22
Average
26.87
18.9
18.88
26.99
training LM(s)
semantic overlap
real-word ratio
shuflled accuracy
non non-nrrmalized
ratio
BERT BASE
5.3
81.7
11.5 (3.1)
23.0 (6.2)
GPT2MEDIUM
0.97
68.8
6,0 (1.2)
32.4 (6.7)
TALARGE
2.43*
71.9
15.1 (2.6)
(6.1)
BERT BASE / LARGE
3.29*
86.0
11.5 (3.1)
27.9 (7.4)
BERT BASE GPTP MEDMMI
3.51
88.6
9.5 (3.1)
24.5 (8.0)
#PLARGE/GTT2MEDE
1.44
73.3
9.7 (2.4)
(9.0)
Table 4: AutoPrompt prompt analysis. The semantic overlap column reports the I-score for the dif- ference in semantic overlap between matching and mismatched prompts (see text for explanation), with marking significant scores at a=0.05. The real-word ratio column reports percentage ratios of corpus-attested English words among space-/punctuation- mark delimited tokens appearing prompt set. The shuffied accuracy columns report percentage accuracy after token shuffling, divided by the original accuracy in the ratio column (averages of 10 random shufflings with standard devia- tions in parenthesis).
4.4 PROMPT ANALYSIS
We study the prompts generated by AutoPrompt through single-model and mixed training, looking for differences that might hint how the latter generalize better. Since the prompt examples in Table (Appendii F) suggest that there are no huge differences directly visible to the "naked eye". we undertake a quantitative analysis led by following hypotheses.
1) Each LM has its own peculiarities, but they all share English training language. Prompts op- timized on a single model might overfit the quirks of that model, but mixed-training prompts might capture more general properties of English that are shared across models. We thus hypothesize that prompts that generalize better will have a larger semantic overlap with manually crafted English prompts. 2) Modern LMs sub-word tokenization strategies that differ from model to model. AutoPrompt is thus free to combine sub-words into non-word sequences (e.g., wwwllmgrapherrs, pub- lishedioon in Table S that might in be tokenized differently by different models, leading to inter-model brittleness. We thus conjecture that prompts that generalize better will contain a larger proportion of real English words. 3) Natural languages rely on word order to express meaning, but it's less clear that LMs are capturing genuine syntactic rules (Sinha et al. (022T). It's more likely that, to the extent that prompts crucially rely on token order, this is exploiting statistical co-occurrence quirks of specific LMs. We thus conjecture that "bag-of-token" prompt sequence that does not require the tokens to and, consequently, generalizing prompts will be more robust token shuffligg. 4) On related point, single-moded optimiizd prompts might concentrate information in the slots the source model most sensitive but such slots might vary from model (type) to model (type). We thus conjecture that generalizing prompts will distribute information more evenly across tokens and thus they will be robust single-token deletion.
any special order will be more general than one where order matters

---

Published as a conference paper at ICLR 2023
Semantic overlap with English The manual LAMA prompts our English reference point. We measure semantic overlap as the cosine between a vector representing an Prompt--eeneratee prompt and a LAMA prompt. Specifically, we use fastText (Bojanowski et al.) 2017) to represent prompts, both because it offers independent representations from those of any of the LMs we are comparing, and because it relies vocabulary- and ookenization-indeenddde quence representations. Instead of reporting difficull-t--nnterrret absolute cosines, we report the t-score for the cosine difference between cases where AutoPrompt prompts are compared to the LAMA prompts for the same T-ReX relation, and cases where AutoPrompt prompts are compared to different-relation LAMA prompts. In other words, the larger this value is, the clearer the dif- ference in semantic overlap is between meaningful and random prompt comparisons. Results are in the first column of Table 4 There is strong correlation (>0.9 Pearson) between a prompt se- mantic overlap with English and its accuracy when tested on the model used as source/generator during training. This is encouraging in itself, suggesting that more effective AutoPrompt prompts also more semantically transparent. However, our hypothesis that better generalization implies higher semantic overlap is disproven: there is a clear decrease in overlap between BERT BASE -baed prompts and the better generalizing ones obtained through BERTBASE/ L mixed-training.
n-gram-based
Real-word ratio We verify whether a space- or concctuuatio-maarkes delimited character sequence in prompt is an existing English word by checking if it appears in the list of 56k words occurring at least Ik times in the uk WaC corpus (Baroni et al. 2009). This corpus was not used for training any of the models, thus minimizing biases towards any of them. The minimum occurrence threshold was determined by manual inspection, observing that rarer strings tend not to be real words but "corpus detritus" (numbers, dates, code fragments, typos). The second column of Table 4 reports percentage attested-word ratios among the tokens produced by AutoPrompt for the whole T-ReX relation set in various training setups. For reference, this ratio is at 99.4% for the manual LAMA prompts. The generalizing BERTRASE/TSIAN this metric, tentatively confirming our hypothesis that more word-like strings will transfer better across models. Note however that SPPTTAAS//TTT generalize better than BERTBASE-basce training alone, features prompts sporting an even higher proportion of existing words. So, the latter might be common property of sixeed-rraning-inducee prompts, but not one that automatically entails better generalization.
setup clearly outperforms single-model training on BERTBASE on
mixed-trainins setup that does not
Shufffing We shuffle the tokens in each prompt, and compute the resulting T-ReX accuracy when retrieving information from the LM used as source/generator during AutoPrompt training. To tease the effect of shuffling apart from the absolute performance of prompt set, we also report the ratio of accuracy after shuffling to accuracy with unshuffled prompts. We repeat the shuffling experiment 10 times, and report averaged accuracies/ratios and standard deviations in the last two columns of T] By superficially eyeing AutoPrompt prompts such as those in Table 8 one could think they are bags of tokens, but the ratios show that token order matters, as there always a big drop in per- formance after shuffling. Indeed, by closer inspection of the prompts in Table we notice that some of them do have a sentence flavor (c[X) teaches modelling frescoes downtown in [YJ"). Our hypoth- BASE/TSTAAAR
esis that the generalizing ones is confirmed (compare the ratios of these setups). The extra robustness of BERTBASE/TSLAE might be due to the fact that T5LARGE itself is particularly robust to shuffling, and it remains to be seen if for mixed-training prompts there is a causal relationship between robustness to shuffling and generalization. Note that the relatively high ratio of 2PT2MEDIUM might be due to a flooring effect, as this setup has low accuracy before and after shuffling.
prompts are less order-sensitive than the BERT BASE
Token deletion To verify if information is equally distributed across the 5 tokens of an Auto- Prompt prompt, for each prompt set we compute T-ReX accuracy (retrieving information from the matching source/generator model) after removing a single token, repeating the experiment for each token position. Figure that BERT MIRRTBASE-baSe prompts are concentrating lot of information towards the end of the sequence, and in particular on the last token, with a huge drop in performance if the latter is deleted. While the same tendency to pack information towards the end of the se- quence is present in the better- genterrgnnrrlizing (especially when taking into account the fact that the latter prompts start from lower full-sequence accuracy). Figure 5 in Appendix showss that all generated prompts have a tendency to pack more information at the end of the sequence (which is remarkable, as their encoders are based on the fully
BERTBASE/TSLAUE
prompts, the drop is less dramatic
8

---

Published as a conference paper at ICLR 2023
BERT BASE
(h) BERT BASE/ TSAAGG
Figure 4: Percentage accuracy after dropping the token in each position of an AutoPrompt-generaaee 5-token prompt. The dashed horizontal line marks full-sequence accuracy.
symmetric transformer architecture), but LALARGE generates prompts that are less affected by single- token deletion. The higher robustness of BERT BARTT/TAS/SAAGG compared to BERT BASE BASE might thus be inherited from T5LARGE, and further studies should ascertain whether there is causal relation be- tween smoother information distribution and better generalization for mixed seedd-taiinng prompts.
DISCUSSION
Take-home points Automatically induced discrete prompts, such those derived with the Auto- Prompt algorithm, strike a good balance between (semi-) prompts, that they outperform in retrieval quality and (potentially) scalability, and soft prompts, that only be used out-of-the-box on the model they were induced from, and require access to inner model structures to function. How- ever, the standard AutoPrompt method must be adapted get good performance across language models. In particular, a simple modification in which AutoPrompt is trained using two language models leads to prompts that better generalize further models.
The better-generalizing prompts induced in this way look quite similar to prompts induced with the standard method. However, probing analysis suggests that there are systematic differences, and in particular that the generalizing prompts tend to feature larger proportion of existing English words, and to more robust to ablations that probe sensitivity to token order and asymmetries in information distribution.
In sum, our results suggest that it is viable use learning- discrete prompts that can be employed to extract information from variety of pre-trained language models, only requiring access to their standard discrete-string interface.
beasedngbbased based algorithm to generate "universal"
Limitations and directions for further work Our results based on single task, slot filling, and a single data-set (T-ReX). We believe this is a good starting point, because in slot filling a separate, semantically contentful prompt must be learned for each relation, but future work should extend the investigation to different tasks, including tasks where successful knowledge retrieval requires more than recalling single word, in the LAMA/T-ReX setup we used.
We used single discrete prompt induction algorithm, AutoPrompt, confirming that it is generat- ing high-quality prompts. However, this method generates single fixed prompt for each task or sub-task. True scalability will only be achieved with prompting methods that can generate an appro- priate query for each different input they receive, and we intend design discrete- discrete-prompt-inne proppp--inumtttiio algorithms matching this desideratum (see Haviv et al. (2021) for step in this direction). Another reason to focus on algorithm development is that the single-model comparison between AutoPrompt and the OptiPrompt soft prompts shows there is still large room to improve retrieval quality.
Our analysis revealed systematic differences between the best model-specific and generalizing prompts. However, it is not clear that any of these differences is causally connected with generaliza- tion improvement. In future work, we would like to better understand the relation between properties such robustness to shuffling and generalization. A particular exciting direction is to favour the emergence such properties through appropriate auxiliary functions at prompt-induction time, and verify whether they lead to further improvements in generalization performance.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We thank the reviewers for constructive feedback. We thank the members of the UPF COLT group for discussion and advice. UPF has received funding from the European Research Council (ERC) under the European Union's 's Horizon 2020 research and innovation programme (grant agreement No. 101019291). This paper reflects the authors' view only, and the funding agency not responsible for any use that may be made of the information it contains. Fabio Petroni conducted work on the project while at Meta AI.
REPRODUCIBILITY STATEMENT
All datasets, pre-trained models and prompting methods used in this paper are publicly available. We use the same hyperparameters as the original implementation of the prompting methods unless it is clearly specified in the text. Code to reproduce the results as well as the common vocabulary and the filtered datasets will be shared upon acceptance.
ETHICS STATEMENT
Prompting relies on pre-trained language models, and thus inherits most ethical risks inherent in such models (Weidinger al. 2022). As are not introducing new models, are not adding new potential issues tied to LMs.
As the examples in Table 8 show, automated prompting methods tend produce opaque prompts. This characteristic might be exploited for harmful purposes, such as adversarial attacks in which model is "triggered" to produce unwanted information through apparently innocuous prompt (Wlllace et al. 2019). We believe that this provides further motivation for our focus on discrete prompts. that are more directly Section [.44] that there is very high correlation (>.9) between the quality of antomatiicall--nndeee prompts the relevant information retrieval task and the degree of semantic transparency of the prompts. If this result could be extended, it would constitute very good news on the interpretability front, suggesting that very high-quality prompts will also be more iitereiitterpetabble, making harder to exploit opaque prompts for harmful purposes.
readable than soft prompts. We showed in particular in
REFERENCES
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models as knowledge bases. arXiv preprint arXiv:2204.06031, 2022.
Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. The WaCky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43(3):209-22) 226, 2009.
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov, Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics. 5:135-146,
2017.
Rishi Bommasani, Drew Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shya- mal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji. Annie Chen, Kathleen Creel, Jared Davis, Dorottya Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Hender- son, John Hewitt, Daniel Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard. Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khat- tab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Lad- hak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Li, Xuechen Li, Tengyu Ma. Ali Malik, Christopher Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed
10

---

Published as a conference paper at ICLR 2023
Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishman Srinivasan, Alex Tamkin, Rohan Taori, Armin Thomas, Florian Tramér, Rose Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Xie, Michi- hiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. https: /wtpiaaxivvv//va org/ abs/ /tles/2200 2108 07258 2021.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT; Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL, pp. 4171-4186, Minncapolis, MN, 2019.
Hady Elsahar, Pavlos Vougiouklis,, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, and Elena Simperl. T-rex: A large scale alignment of natural language with knowledge base triples. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), 2018.
Tatsunori B Hashimoto, David Alvarez-Melis and Tommi S Jaakkola. Word embeddings as metric recovery in semantic spaces. Transactions of the Association for Computational Linguistics, 4: 273-286, 2016.
Adi Haviv, Jonathan Berant, and Amir Globerson. BERTese: Learning speak to BERT. In Proceedings of EACL, pp. 3618-3623, Online, 2021.
Zhengbao Jiang, Frank Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics, Linguititoo 8:423-438, 2020.
Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sameer Singh, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, al. Prompt wayward- ness: The curious case of discretized interpretation of continuous prompts. arXiv preprint arXiv:2112.00844, 2021.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for pfrfmeter-efficient prompt tuning. In Proceedings of EMNLP, pp. 3045-3059, Punta Cana, Dominican Republic, 2021.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequunee pre- training for natural language generation, translation, and comprehension. arXiv preprint 1rXiv:1910.13400 2019.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. htttpp://arxivvorrrte
abs/ 2107. 13586 2021.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1907.116944 2019.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. Distributed representa- tions of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013.
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv: 1909. 1et.::1999000666 01066, 2019a.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? In Proceedings EMNLP, pp. 2463- 2473, Hong Kong, China, 2019b.
Nina Poerner, Ulli Waltinger, and Hinrich Schütze. Bert is not knowledge base (yet): Factual knowledge vs. name- name-based reasoning in unsupervised qa. arXiv preprint 1xXiv:1911.0363 3, 2019.
11

---

Published as a conference paper at ICLR 2023
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21 21111111:11111111111 2020.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint 1910.0091000111110 2019.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, De- bajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang. Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang. Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander Rush. Multitask prompted train- ing enables zero-shot task generalization. online: https: https::/openrett wwwperevee....e.coooo.opp. net htte//grouppicc ICLR. cc 2tt0////o/feeecceecooooeoo
Proceedings of ICLR, Online, 2022. Published
Timo Schiek and Hinrich Schütze. It's not just size that matters: Small language models are also few-shot learners. In Proceedings of NAACL, pp. 2339-2352, Online, 2021.
Thibault Sellam, Steve Yadlowsky. Jason Wei, Naomi Saphra. Alexander D' A'mour, Tal Linzen, Jasmijn Bastings, Iulia Ture, Jacob Eisenstein, Dipanjan Das, et The multiberts: Bert repro- ductions for robustness analysis. arXiv preprint 1161:22106.1616,, 2021.
Taylor Shin, Yasaman Razeghi, Robert Logan IV, Eric Wallace, and Sameer Singh. AutoPrompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of EMNLP, pp. 4225, virtual conference, 2020.
Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, and Douwe Kiela. Masked language modeling and the distributional hypothesis: Order word matters pre-training for little. In Proceedings of EMNLP, pp. 1888-29133 Punta Cana, Dominican Republic, 2021.
Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, and Jie Zhou. On transferability of prompt tuning for natural language processing. In Proceedings of NAACL. pp. WA, 2022.
Seattle,
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing NLP. In Proceedings of EMNLP, pp. 2153-2162, Hong Kong, China, 2019.
Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. Taxonomy of risks posed by language models. In Proceedings of FAceT, pp. 214-229, Seoul, Korea, 2022.
Yue Zhang, Hongliang Fei, Dingcheng Li, and Ping Li. PromptGen: Automatically generate prompts using generative models. In Findings of NAACI, pp. 30-37, Seattle, WA, 2022.
Zexuan Zhong, Dan Friedman, and Dangi Chen. Factual probing is [mask]: Learning vs. learning to recall. arXiv preprint arXiv:2104.05240, 2021.
A PROMPT ENSEMBLING
Discrete prompt ensembling for knowledge extraction introduced by Jiang al. (2020) using LPAQA. Instead of selecting the top-1 prompt from a set of candidates. multiple prompts are com- bined in order to improve performance. The authors argue that ensembling improves the extraction
12

---

Published as conference paper at ICLR 2023
Table Direct OptiPrompt transfer across LMs with same embedding dimension.
Source
BERT BASE
DistiBBERT
RoBERTaBASE
BoRRR
GPT2
BART BASK
Target
BERT
48.25
33.71
0.62
1.24
1.02
1.00
0.19
DistiBBRT
11.89
46.41
0.97
0.83
1.64
0.81
0.64
ROBERTaBASE
2.27
2.02
46.17
41.04
2.67
3.13
4.90
DistiRRoBERTa
2.12
1.88
39.86
45.26
2.48
2.87
3.81
GPT2
0.01
0.04
0.15
0.01
40.57
0.00
0.00
BARTBASE
1.20
0.72
1.02
0.05
0.60
44.3
0.27
T5DASE
4.89
2.89
4.92
42.08
of knowledge that appeared in different contexts within the training data. Since the idea of ensem- bling is distinct from the specifics of the LPAQA method, in this appendix check if ensembling also helps AutoPrompt. Specifically we experiment with the optimized ensembling approach used with LPAQA by Jiang et al. (2020), which associates learnable weights to each prompt.
For ensembling purposes, multiple candidate prompts for each relation are required. One way to obtain these candidate prompts for AutoPrompt is by inducing prompts from instances of the same model trained with different seeds. Considering the computational cost of retraining the models our- selves, we use the already available MultiBERT collection (Sellam et al.) 2021), which contains instances of BERT BASE trained with different seeds. Unfortunately, unlike the models in Table I the MultiBERTs were trained on uncased data, which means we can only evaluate on the MultiBERTss themselves. Specifically, we seed-0 MultiBERT as target model, and obtain the AutoPromph prompts from the remaining MultiBERTs. For LPAQA, we use seed-O MultiBERT for training and testing, obtaining the candidate prompts with the standard LPAQA mining and paraphrasing proce- dure.
First, confirm that in this different setup AutoPrompt is still outperforming LPAQA in top-1 eval- uation (39.40% vs. 15% T-ReX accuracies, respectively). Second, ensembling helps both mod- els. However, ensembled LPAQA barely reaches the performance of top-1 Automprompt (39.21%), whereas ensembled AutoPrompt further improves to 42.43% accuracy.
B SOFT PROMPT GENERALIZATION
Transferring soft prompts across models is straightforward as with discrete prompts. In or- der to transfer the vectors, the source and target must have the same embedding dimension. We experiment with the subset of models that have a common embedding dimension of 768. Table 5] confirms that soft prompts generalize extremely poorly. The only exception is decent student/teacher transfers for distilled models, probably because their embeddings were initialized with those of the corresponding teacher models.
more general approach consists in discretizing the soft prompts to their nearest vocabulary neigh- bors (Mikolov al.) 2013 Hashimoto et al. 2016), and using the resulting vocabulary item quences as prompts. OptiPrompt vectors can be initialized with random numbers, random vocab- ulary embeddings manual prompts. When initialized with random vocabulary embeddings manual prompts, the nearest neighbor projections of the resulting vectors are always identical to the initialization tokens, making the process vacuous. However, when the vectors initialized to random numbers, the nearest neighbors of the optimized vectors are the same for multiple rela- tions 5 As expected, unlike the corresponding soft prompts, these discretized prompts perform very poorly. For example, when training on BERT BASE, OptiPrompt soft prompts achieve 48.26% T-ReX accuracy. The corresponding discretized prompts only achieve 1.21 12%% accuracy.
'This might be related to the "prompt waywardness" phenomenon observed by Khashabi et al. 22021). who showed that, for any arhitrary embedding, is possible to find well-performing soft prompt that has that embedding its nearest neighhor.
13

---

Published as a conference paper at ICLR 2023
Table 6: Comparison of LPAQA and AutoPrompt across LMs. Each method trained on the LM
leading to the best same-source-anee -odure-and--rrge performance.
Source
LPAQA
AutoPrompt BERT BASE
Target
BERT
LARGE
BERTBASE BASE
38.01
50.09
BERT
42.14
47.01
DistilBERT
7.24
15.75
RoBERTaBASE
27.68
32.31
RoBERTa
33.68
37.79
DistilRoBERTa
22.21
31.71
GPT2
7.88
4.70
GPT2MEDIUM
14.88
14.38
GPT2LARGE
17.15
17.95
GPT2xL
18.01
18.34
BART BASE
26.74
28.98
BARTLARGE LARGE
28.47
26.73
T5
16.38
15.78
TSRSS
27.76
29.26
LARAG
31.20
32.32
Average
23.96
26.87
C LPAQA GENERALIZATION
AutoPrompt is better than LPAQA in the same-source-and -angttreend setup, demonstrated in Table Table 6 further shows that AutoPrompt also generalizes better than LPAQA to other target models (we train the latter BERT since this is the LPAQA setup reaching the best same-source- and-target accuracy).
D LAMA-UHN RESULTS
LAMA-UHN (UnHelpfulNames) (Poerner et al. 2019) is subset of LAMA where facts that can be answered based on entity names alone were removed. The dataset is built using two heuristics. The first one filters facts whose object is case-insensitive substring of the subject entity name. The second heuristic deletes triple a given model can infer information such as native language, nationality place of birth using the subject's surface form only. We use the original dataset with BERT BASE as filtering model.
Table shows that the generalization results still hold. The difference between BERT BASE and BERT L.LAAGE is smaller, possibly due to a ceiling effect with this harder data-set, but (once we exclude the non-generalizing same-source- and and-tre BERTBASE case), it is statistically significant (paired r-test, a=0.55..
E CHOOSING MODELS FOR MIXED-TRAINING
We study to what extent same-targed larget-same-souree accuracies of component models are good predic- tors of mixture generalization accuracy. To avoid combinatorial explosion, we fix BERT BASE (the best model in the same-target-same--ouree setup) either generator or evaluator, and pair it with 15 different LMs. The correlation across 15 mixtures and at 0.49 when it is fixed evaluator. Overall, this suggests that single-model setup performance good predictor of mixture quality. We attribute the lower score with -eRT-as-generator to mild ceiling effect in this case.
0.22 when BARTBASE fixed as generator,
Indeed, we observe that, on average, performance is better when using BERT BASE as generator than evaluator. This is likely due to the fact that the generator model does most of the "heavy lifting" in the mixture, as it the one finding candidate prompts, that are then simply reranked by the evaluator. It thus makes sense the best single-model-see system used the generator in mixture.
14

---

Published as a conference paper at ICLR 2023
Table 7: AutoPrompt generalization evaluated on LAMA-UHN.
Source
BERT
15 BERRTnsnrTSSy.a BERTTxxs/GPP2nnm
BER BASE
39,50
11.09
15.00
11.03
28.01
13.98
AACE
20.73
DistiBERT
13.51
3.02
3.54
13.11
11.6
2,88
RoBERTauss
25.35
16.57
14.82
28.23
23.52
RoBERTALAaGE
30.82
18.91
20.14
31.62
28.18
18.14
Dist
24.98
14.17
11.71
24.61
21.03
13.01
GPT2
2.34
6.65
3.68
6.48
1.83
3.22
GPT2MDDUM
9.28
14.19
9.43
10.09
15.02
11.96
GPTALARGE
11.63
10.87
GPT2XL
11.02
9.78
9.57
12.54
16.36
13.48
BAR BASE
22.25
18.81
15.18
LARTLARGE
19.62
18.74
13.80
25.07
21.24
14.53
T5RMAL
7.56
T3 T3BASE
20.36
13.71
15.19
25.98
21.19
11.74
LLARGE
23.46
20.70
31.48
34.25
25.84
Average
19.90
13.70
13.19
21.72
19.54
12,06
Finally, when fixing BART as generator, we observe that, in the majority of cases, obtain better generalization results by using different evaluator than BERT BaSe itself, suggesting that mixed-training generalizes better compared to single model training even when randomly selecting one of the two models.
F PROMPT EXAMPLES
Table 8 shows manual (LAMA) and AutoPrompt prompts for randomly picked set of T-ReX re- lations. We present examples for AutoPrompt trained on BERTBASE, the best choice the same- source-and-taree setup, and for mixed training using BERTBASE as generator model and T5LARGE as evaluator BAERT mance. For comparison, also present prompts obtained using GPT2MEDIUM source model, since this a setup with relatively low performance in both the same-source and-target and gener- alization evaluations.
since this combination results in the best generalization perfor-
We note that the two hetter models generally produce prompts that have at least some degree of semantic/lexical relevance the relation. This might be as well, but much more opaquely so. Interestingly, for the two better models at least, the most transparent information is concentrated towards the end of the string. For example, the BERTBASE location prompt ends with headquartered in, and the corresponding BERT BARTTBAS//SSARCE prompt ends with headquarters in. The BERT BASE manufacturer prompt ends with marketed by, and the corresponding BERT RARRTTRA//SSARRR prompt ends with maker. Both these observations (higher se- mantic relevance of better-performing prompts and higher information concentration at the end of the prompt string) are confirmed by the quantitative analyses reported in Section 4.4 of the main text.
degree the case for GPT2
By qualitative inspection, the only noticeable difference between the BERTBASE prompts (better in the non-generalizing setup) and the BERTBASE/TSLARRE ones (better at generalization) that the latter contain higher proportion of well-formed English words (as opposed to concatenations of word pieces that do not result existing words). Again, this is quantitatively confirmed by the analysis in Section 4.4
15

---

Published as a conference paper at ICLR 2023
Table 8: Example prompts for random T-ReX relation subset. The Manual column shows manually- manually-crafied ce---rraeeed LAMA prompts. The following two columns show prompts generated by AutoPrompt using source models BERTBASE and GPT2MEDUM respectively The last column shows prompt generated through AutoPrompi mixed training, using BERTEASE as generator model and T5LARGE as evaluator model. In all cases, [X] and [Y] stand for the subject and object slots.
[X] teaches modelling frescoes downtown in [Y]
[X] devised by billion fielding
[X] dynasty mosques helped peacefully spread [Y]
[X] streamed world semifinal
[X] specialiseding!! based headquarters in [Y]
badminton representing [X]
BERT RASTETTAT/S TSLARGE
maker [Y]
[X] (eS00waggnttnnoooossaapapaabbe (Y).
[X] pseudonym dual (Paris and [Y]
[/guu u918d) ( ufffd constitutes orthodox [Y]
[X]estyle rivalry (Germany and [Y]
[X] infographic biking & accommodation in [Y]
GPT22EDDEM
[X] believe unidentified fundamental
[X] comes versa publishedtoon outside [Y]
[X] contrasts slalomgraphers
[X]lland flees exilelessly
[X] isn isn altirudes binary marketed by [Y]
religion opposite [Y]
headquartered in [Y]
downtown [Y]
BERTBASS
[X] was created in [Y]
[X] is produced by [Y]
[X] is affiliated with
[X] is located in [Y]
[X] died [Y]
the [Y] religion
Manual
country of origin
place of death
manufacturer
relation
location
religion
16

---

Published as a conference paper at ICLR 2023
BERT BASE
(h) GPT2MEDIUM
L5LARGE
(d) BERT BASE/ /ASTETAAS///AAAEE LARRGE
(e) BERT BAse/GPT2MEDDCH
(1) LARGEE L GPT2MEDDIM
Figure 5: Percentage accuracy after dropping the token in each position af AuoPrompt-eenneatte
5-token prompt. The dashed horizontal line marks full-sequence accuracy.
G TOKEN DELETION FULL RESULTS
Figure reports the T-ReX accuracy for AutoPrompt prompt sets obtained using different LMs or LM combinations, always tested on the LM used for AutoPrompt training (the generator LM in mixed-training setups), when one token at a time is removed from the full prompt sequence.
17


Reviews:
[{"review_id": "zywvlr2roX", "reviewer": "Reviewer_FEH5", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies whether prompts that effectively extract information from a language model can also be used to probe other language models for the same information. The authors show that this is indeed not the case. They propose an approach to induce prompts by mixing language models at training time and show that in this way they can generalize well across models. They evaluate on the slot-filling task which is good to investigate the knowledge contained in language models. They provide extensive analysis and insight into the generalization of these large language models and how to effectively probe information from them.\n", "summary_of_the_review": "This work is well-motivated, clearly explained, and sufficiently supported by analysis. There are some questions for the authors (see section Strength And Weaknesses), however overall it's a valuable contribution to the field. ", "strength_and_weaknesses": "- The motivation of this paper is clear and a worthwhile topic to investigate. The experiment setup and settings are good and the evaluation is sound. The authors provide ample analysis on their findings.\n\n- The way this paper trains and evaluates, it would need to learn a different prompt for each relation in the slot-filling task. The authors use the original (manual) prompts provided by LAMA in their experiments. Isn't that limiting the findings? Have the authors looked into other prompt templates?\n\n- The authors look at three types of language models and various prompt induction methods. The model they choose however is on the [relatively] smaller side. Do the authors think model size and the number of parameters is also a contributor to how these models learn and output knowledge?  Have the authors looked into other bigger models and if not what is their hypothesis about these model's behaviour and how their findings generalize to larger models?\n\n- Table 3 shows that by simply mixing the models we would not achieve good generalization. Do the authors have a hypothesis why that's the case or have they looked further into it?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Dzxnj0myiZ", "reviewer": "Reviewer_9ydY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method  for automated prompt generation using a combination of Language Models. It shows that certain source LM combinations are more succesful than others in outperforming autoprompts trained on single ML.  The paper also includes a thorough analysis of potential reasons.\n\nAdditionally,  the paper provides an overview of the currentpormting approaches and povides their comparison, describing the recommended areas of use and limitations.\n\nContributions:\n* a novel method for automated prompt generation from a mix of LMs\n* comparison of promtpting approaches\n* analysis of effects and limitations of the newly proposed method\n\nI think that in addition to the proposed method, the paper may serve as great practical reference on prompting methods so it has  a potential for citation and community impact in various aspects. ", "summary_of_the_review": "It's an interesting solid paper with moderate novelty. Lean to accept", "strength_and_weaknesses": "Strength:\n* introduced a novel method and clearly showed its benefits and limitations\n* thorough analysis of the potential underlying linguistic reasons (rather than treating as a black-box)\n* clear narration\n* comprehensive overview of related works\n* solid grounding for experimentation\n\nWeaknesses:\n* while the paper doesn't present break-through findings, it is a solid and sufficiently impactful research", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "C6kVTahHseK", "reviewer": "Reviewer_wUgX", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information. After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts on the slot-filling task, the authors demonstrate a drop in performance for AutoPrompt prompts learned on one model and tested on another. The authors introduce a way to induce prompts by mixing language models at training time the results in prompts that generalize well across models. The authors conduct an extensive analysis of the induced prompts, finding that the more general prompts include a larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens.", "summary_of_the_review": "Please refer to the points in the above sections to improve the paper.", "strength_and_weaknesses": "Pros: The authors present a systematic study of the extent to which language models (LM) query protocols, that, following current\nusage, the authors call prompting methods, generalize across LMs.  The authors conduct the extensive analysis of automatically induced discrete prompts, tentatively identifying a set of properties characterizing the more general prompts, such as a higher incidence of existing English words and robustness to token shuffling and deletion. The paper is well-written and easy to understand. There are comprehensive experimental results presented by the authors.\n\nCons: The contribution of this paper to the new methodology for natural language processing is limited. As a result, the analysis in the paper is also limited by mainly utilizing the existing methods.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "6gIrhea3p5T", "reviewer": "Reviewer_DprU", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper verifies whether automatically induced prompts can use the same information to apply to other models. It turns out that automatically induced prompts by AutoPrompt [Shin et al. 2020] outperforms manual and semi manual methods in slot-filling tasks, and verifies that automatically induced prompts can learn from one model and test in another, and the effect is not good. In addition, also found that more general prompts often contain some attributes.", "summary_of_the_review": "The paper verifies whether automatically induced prompts can use the same information to apply to other models. It turns out that automatically induced prompts by AutoPrompt [Shin et al. 2020] outperforms manual and semi manual methods in slot-filling tasks, and verifies that automatically induced prompts can learn from one model and test in another, and the effect is not good. In addition, also found that more general prompts often contain some attributes. The content of the paper is clearly stated, however, generally, the work lacks novelty. Although the detailed analysis seems to make some sense, the content is not enough for a ICRL research paper, even for application track. ", "strength_and_weaknesses": "Strength: The paper is easy to follow.\nWeaknesses: the work lacks novelty, and one can barely find origin idea proposed in the paper. Although the detailed analysis seems to make some sense, the content seems not enough for a regular research paper, even for application track.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster