Original: C AN DISCRETE INFORMATION EXTRACTION PROMPTS GENERALIZE ACROSS LANGUAGE MODELS ?
Revised: C AN DISCRETE INFORMATION EXTRACTION PROMPTS GENERALIZE ACROSS LANGUAGE MODELS ?

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information.
Revised: We study whether automatically-induced prompts that effectively extract information from a language model can also be used, out-of-the-box, to probe other language models for the same information.

Original: After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts on the slot-filling task, we demonstrate a drop in performance for AutoPrompt prompts learned on a model and tested on another.
Revised: After confirming that discrete prompts induced with the AutoPrompt algorithm outperform manual and semi-manual prompts on the slot-filling task, we demonstrate a drop in performance for AutoPrompt prompts learned on a model and tested on another.

Original: We introduce a way to induce prompts by mixing language models at training time that results in prompts that generalize well across models.
Revised: We introduce a way to induce prompts by mixing language models at training time that results in prompts that generalize well across models.

Original: We conduct an extensive analysis of the induced prompts, finding that the more general prompts include a larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens.
Revised: We conduct an extensive analysis of the induced prompts, finding that the more general prompts include a larger proportion of existing English words and have a less order-dependent and more uniform distribution of information across their component tokens.

Original: Our work provides preliminary evidence that it’s possible to generate discrete prompts that can be induced once and used with a number of different models, and gives insights on the properties characterizing such prompts.
Revised: Our work provides preliminary evidence that it’s possible to generate discrete prompts that can be induced once and used with a number of different models, and gives insights on the properties characterizing such prompts.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: NLP has shifted to a paradigm where very large pre-trained language models (LMs) are adapted to downstream tasks through relatively minor updates (Bommasani et al., 2021; Liu et al., 2021).
Revised: NLP has shifted to a paradigm where very large pre-trained language models (LMs) are adapted to downstream tasks through relatively minor updates (Bommasani et al., 2021; Liu et al., 2021).

Original: In the most extreme case, task adaptation does not require modifying the LM or even accessing its internals at all, but simply formulating a linguistic query that elicits an appropriate, task-specific response by the model (Petroni et al., 2019a;
Revised: In the most extreme case, task adaptation does not require modifying the LM or even accessing its internals at all but simply formulating a linguistic query that elicits an appropriate, task-specific response by the model (Petroni et al., 2019a;

Original: Radford et al., 2019).
Revised: Radford et al., 2019).

Original: This has promising practical applications, as one could easily imagine proprietary LMs only exposing a natural-language-based interface, with downstream agents extracting the information they need by formulating the appropriate queries.
Revised: This has promising practical applications, as one could easily imagine proprietary LMs only exposing a natural-language-based interface, with downstream agents extracting the information they need by formulating the appropriate queries.

Original: In this scenario, one fundamental question is how robust the querying protocol is to changes in the underlying LM.
Revised: In this scenario, one fundamental question is how robust the querying protocol is to changes in the underlying LM.

Original: On the one hand, the same downstream agent might want to query multiple LMs.
Revised: On the one hand, the same downstream agent might want to query multiple LMs.

Original: On the other, if the LM provider updates the model, this should not break the downstream pipeline.
Revised: On the other, if the LM provider updates the model, this should not break the downstream pipeline.

Original: On a more theoretical level, the properties of an emergent robust protocol might give us insights on the general language processing capabilities of neural networks, and how they relate to natural language.
Revised: On a more theoretical level, the properties of an emergent robust protocol might give us insights on the general language processing capabilities of neural networks, and how they relate to natural language.

Original: We present a systematic study of the extent to which LM query protocols, that, following current usage, we call prompting methods , generalize across LMs.
Revised: We present a systematic study of the extent to which LM query protocols, that, following current usage, we call prompting methods , generalize across LMs.

Original: Extending and confirming prior results, we find that discrete prompts that are automatically induced through an existing optimization procedure (Shin et al., 2020) outperform manually and semi-manually crafted prompts, reaching a good performance level when tested with the same LM used for prompt induction .
Revised: Extending and confirming prior results, we find that discrete prompts that are automatically induced through an existing optimization procedure (Shin et al., 2020) outperform manually and semi-manually crafted prompts, reaching a good performance level when tested with the same LM used for prompt induction .

Original: While the automatically induced discrete prompts also generalize better to other LMs than (semi-)manual prompts and currently popular “soft” prompts, their overall generalization performance is quite poor.
Revised: While the automatically induced discrete prompts also generalize better to other LMs than (semi-)manual prompts and currently popular “soft” prompts, their overall generalization performance is quite poor.

Original: We next show that a simple change to the original training procedure, namely using more than one LM at prompt induction time, leads to discrete prompts that better generalize to new LMs.
Revised: We next show that a simple change to the original training procedure, namely using more than one LM at prompt induction time, leads to discrete prompts that better generalize to new LMs.

Original: The proposed procedure, however, is brittle, crucially relying on the “right” choice of LMs to mix at prompt induction.
Revised: The proposed procedure, however, is brittle, crucially relying on the “right” choice of LMs to mix at prompt induction.

Original: We finally conduct the first extensive analysis of automatically induced discrete prompts,  tentatively identifying a set of properties characterizing the more general prompts, such as a higher incidence of existing English words and robustness to token shuffling and deletion.
Revised: We finally conduct the first extensive analysis of automatically induced discrete prompts, generalization tentatively identifying a set of properties characterizing the more general prompts, such as a higher incidence of existing English words and robustness to token shuffling and deletion.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Prior work such as Petroni et al. (2019a) and Radford et al.
Revised: Prior work such as Petroni et al. (2019a) and Radford et al.

Original: (2019) demonstrated that LMs can be directly adapted to new tasks through appropriate querying methods.
Revised: (2019) demonstrated that LMs can be directly adapted to new tasks through appropriate querying methods.

Original: This led to an explosion of work on so-called “prompt engineering” (see Liu et al., 2021, for a thorough review).
Revised: This led to an explosion of work on so-called “prompt engineering” (see Liu et al., 2021, for a thorough review).

Original: Much of this work focuses on crafting appropriate manual or semi-manual prompts and/or on tuning LMs to better respond to such prompts (e.g., Schick & Sch ¨ utze, 2021; Sanh et al., 2022).
Revised: Much of this work focuses on crafting appropriate manual or semi-manual prompts and/or on tuning LMs to better respond to such prompts (e.g., Schick & Sch ¨ utze, 2021; Sanh et al., 2022).

Original: Going beyond manual prompts, Shin et al.
Revised: Going beyond manual prompts, Shin et al.

Original: (2020) introduced the AutoPrompt algorithm to generate prompts using gradient-guided search, and demonstrated that such prompts often outperform manual ones.
Revised: (2020) introduced the AutoPrompt algorithm to generate prompts using gradient-guided search, and demonstrated that such prompts often outperform manual ones.

Original: While automatically induced prompts suffer of issues such as low-interpretability, we think it is important to continue focusing on them because, besides their better performance (a result we confirm here for AutoPrompt across a range of LMs), they are more promising than manual prompts in terms of scalability, especially in contexts in which it is not sufficient to formulate a single prompt template for a whole task, but each input query demands a distinct prompt formulation (Zhang et al., 2022).
Revised: While automatically induced prompts suffer of issues such as low-interpretability, we think it is important to continue focusing on them because, besides their better performance (a result we confirm here for AutoPrompt across a range of LMs), they are more promising than manual prompts in terms of scalability, especially in contexts in which it is not sufficient to formulate a single prompt template for a whole task, but each input query demands a distinct prompt formulation (Zhang et al., 2022).

Original: Concurrent and later work has proposed to replace discrete strings, such as those generated by
Revised: Concurrent and later work has proposed to replace discrete strings, such as those generated by

Original: AutoPrompt, with sequences of arbitrary vectors from the LM’s embedding space (Lester et al., 2021; Zhong et al., 2021).
Revised: AutoPrompt, with sequences of arbitrary vectors from the LM’s embedding space (Lester et al., 2021; Zhong et al., 2021).

Original: We confirm here that these continuous, or “soft” prompts outperform
Revised: We confirm here that these continuous, or “soft” prompts outperform

Original: AutoPrompt when trained and tested on the same LM.
Revised: AutoPrompt when trained and tested on the same LM.

Original: However, they cannot be used in our envisaged multiple-LM scenario.
Revised: However, they cannot be used in our envisaged multiple-LM scenario.

Original: First, they require access to a model inner representations, beyond the standard natural language querying interface, so that embeddings can be passed as input.
Revised: First, they require access to a model inner representations, beyond the standard natural language querying interface, so that embeddings can be passed as input.

Original: Second, continuous prompts, by their nature, won’t generalize out-of-the-box to other LMs.
Revised: Second, continuous prompts, by their nature, won’t generalize out-of-the-box to other LMs.

Original: Trivially, they can’t generalize across models with different embedding dimensionality.
Revised: Trivially, they can’t generalize across models with different embedding dimensionality.

Original: Even when models share dimensionality, there is no reason why the absolute position of a vector in the embedding space of a model should meaningfully transfer to another model.
Revised: Even when models share dimensionality, there is no reason why the absolute position of a vector in the embedding space of a model should meaningfully transfer to another model.

Original: Discretizing soft prompt tokens to their nearest vocabulary neighbours in order to overcome these issues does not help either.
Revised: Discretizing soft prompt tokens to their nearest vocabulary neighbours in order to overcome these issues does not help either.

Original: Khashabi et al.
Revised: Khashabi et al.

Original: (2021) demonstrated that it is possible to find well-performing soft prompts whose nearest neighbor projections are arbitrarily fixed discrete tokens.
Revised: (2021) demonstrated that it is possible to find well-performing soft prompts whose nearest neighbor projections are arbitrarily fixed discrete tokens.

Original: Appendix B elaborates on the failure of soft prompts to generalize across models, as well as the problematic behaviour of discretized soft prompts.
Revised: Appendix B elaborates on the failure of soft prompts to generalize across models, as well as the problematic behaviour of discretized soft prompts.

Original: We are not aware of much previous work that has addressed the challenge of LM-to-LM transferability.
Revised: We are not aware of much previous work that has addressed the challenge of LM-to-LM transferability.

Original: Wallace et al.
Revised: Wallace et al.

Original: (2019) studied this problem in the context of textual adversarial attacks (that can be seen as a special case of prompting, and indeed their attack method is closely related to
Revised: (2019) studied this problem in the context of textual adversarial attacks (that can be seen as a special case of prompting, and indeed their attack method is closely related to

Original: AutoPrompt).
Revised: AutoPrompt).

Original: Similarly to us, they notice some performance drop when transferring adversarial “triggers” to different LMs, and they show that this can be mitigated by an ensembling approach where two triggers generated using variants of the same LM are combined.
Revised: Similarly to us, they notice some performance drop when transferring adversarial “triggers” to different LMs, and they show that this can be mitigated by an ensembling approach where two triggers generated using variants of the same LM are combined.

Original: Su et al.
Revised: Su et al.

Original: (2022) study LM-to-LM transferability in the context of continuous prompts.
Revised: (2022) study LM-to-LM transferability in the context of continuous prompts.

Original: Since, as we just discussed, such prompts are not directly transferable, they induce a projection from the embedding space of the source LM to that of the target LM, thus considering a very different scenario from the type of “out-of-the-box” transferability we are interested in here.
Revised: Since, as we just discussed, such prompts are not directly transferable, they induce a projection from the embedding space of the source LM to that of the target LM, thus considering a very different scenario from the type of “out-of-the-box” transferability we are interested in here.

Original:  E XPERIMENTAL S ETUP 3. D ATA
Revised:  E XPERIMENTAL S ETUP 3. D ATA

Original: We focus on the task of slot-filling which, since its introduction in LM evaluation through the LAMA benchmark (Petroni et al., 2019a), has been extensively used to probe the knowledge contained in
Revised: We focus on the task of slot-filling which, since its introduction in LM evaluation through the LAMA benchmark (Petroni et al., 2019a), has been extensively used to probe the knowledge contained in

Original: LMs (AlKhamissi et al., 2022).
Revised: LMs (AlKhamissi et al., 2022).

Original: More specifically, we use the T-ReX split (Elsahar et al., 2018) of LAMA.
Revised: More specifically, we use the T-ReX split (Elsahar et al., 2018) of LAMA.

Original: Each fact in T-ReX is represented as a triple ⟨ subject, relation, object ⟩ —for example, ⟨ Dante , place of birth , Florence ⟩ .
Revised: Each fact in T-ReX is represented as a triple ⟨ subject, relation, object ⟩ —for example, ⟨ Dante , place of birth , Florence ⟩ .

Original: LMs are queried using cloze prompts as in “ Dante was born in .”
Revised: LMs are queried using cloze prompts as in “ Dante was born in .”

Original: A LM is said to have properly stored a fact if it can successfully predict the ground-truth object given a prompt and the corresponding subject.
Revised: A LM is said to have properly stored a fact if it can successfully predict the ground-truth object given a prompt and the corresponding subject.

Original: We have decided to focus primarily on this task because the prompts convey actual semantic information (characterizing the relation between the subject and the object) rather than just metalinguistic information (as would be the case, for example, for a machine-translation prompt, which might express something like: ”translate the next sentence from English to French”).
Revised: We have decided to focus primarily on this task because the prompts convey actual semantic information (characterizing the relation between the subject and the object) rather than just metalinguistic information (as would be the case, for example, for a machine-translation prompt, which might express something like: ”translate the next sentence from English to French”).

Original: Furthermore, the task requires learning a different prompt for each relation, which can be seen as a first step toward fully flexible prompts that would change with each single input (Zhang et al., 2022).
Revised: Furthermore, the task requires learning a different prompt for each relation, which can be seen as a first step toward fully flexible prompts that would change with each single input (Zhang et al., 2022).

Original: The LAMA test set contains 41 relations, each with up to 1,000 facts.
Revised: The LAMA test set contains 41 relations, each with up to 1,000 facts.

Original: We also evaluate on the more challenging LAMA-UHN subset (Poerner et al., 2019), addressing some of the weaknesses of the original LAMA, in Appendix D. All prompting methods are trained using the training data collected by Shin et al.
Revised: We also evaluate on the more challenging LAMA-UHN subset (Poerner et al., 2019), addressing some of the weaknesses of the original LAMA, in Appendix D. All prompting methods are trained using the training data collected by Shin et al.

Original: (2020), which include 1,000 facts for each relation type, drawn from either the original T-REx dataset or Wikidata.
Revised: (2020), which include 1,000 facts for each relation type, drawn from either the original T-REx dataset or Wikidata.

Original: LAMA also provides manual prompts, which we will use in our experiments.
Revised: LAMA also provides manual prompts, which we will use in our experiments.

Original: Since each LM class in our experiments (see Section 3.2 below) has its own vocabulary, a common subset must be used for fair comparison.
Revised: Since each LM class in our experiments (see Section 3.2 below) has its own vocabulary, a common subset must be used for fair comparison.

Original: This is obtained from the intersection of the vocabularies of all models considered.
Revised: This is obtained from the intersection of the vocabularies of all models considered.

Original: Furthermore, the training and test datasets are filtered to ensure that each object is included in the common vocabulary.
Revised: Furthermore, the training and test datasets are filtered to ensure that each object is included in the common vocabulary.

Original: There are 11,511 case-sensitive items in the common vocabulary.
Revised: There are 11,511 case-sensitive items in the common vocabulary.

Original: The filtered test set contains 25,358 facts, while the training set contains 25,247 facts.
Revised: The filtered test set contains 25,358 facts, while the training set contains 25,247 facts.

Original: We evaluate prompting methods using micro-averaged accuracy (precision@1).
Revised: We evaluate prompting methods using micro-averaged accuracy (precision@1).

Original: 3. L ANGUAGE M ODELS
Revised: 3. L ANGUAGE M ODELS

Original: Our experiments cover the three main types of LM.
Revised: Our experiments cover the three main types of LM.

Original: We use pre-trained LMs without any kind of parameter updating.
Revised: We use pre-trained LMs without any kind of parameter updating.

Original: Table 1 shows the LMs considered in this study.
Revised: Table 1 shows the LMs considered in this study.

Original: Masked LMs They produce representations using both the left and right context.
Revised: Masked LMs They produce representations using both the left and right context.

Original: Given a sequence x = [ x 1 , ..., x n ] , they estimate the probability of a token x i given its left and right context p ( x i ) =
Revised: Given a sequence x = [ x 1 , ..., x n ] , they estimate the probability of a token x i given its left and right context p ( x i ) =

Original: p ( x i | x 1 , ..., x i − 1 , x i +1 , ..., x n ) .
Revised: p ( x i | x 1 , ..., x i − 1 , x i +1 , ..., x n ) .

Original: Left-to-right LMs They predict the next token conditioned on previous ones or assign a probability to a sequence of tokens.
Revised: Left-to-right LMs They predict the next token conditioned on previous ones or assign a probability to a sequence of tokens.

Original: Given a sequence of tokens x = [ x 1 , ..., x n ] , left-to-right LMs assign a probability p ( x ) to the sequence using the chain rule p ( x ) =
Revised: Given a sequence of tokens x = [ x 1 , ..., x n ] , left-to-right LMs assign a probability p ( x ) to the sequence using the chain rule p ( x ) =

Original: (cid:81) t p ( x t | x 1 , ..., x t − 1 ) .
Revised: (cid:81) t p ( x t | x 1 , ..., x t − 1 ) .

Original: Sequence-to-sequence LMs They are composed of a bidirectional encoder that uses both left and right context and a left-to-right decoder that do not share parameters.
Revised: Sequence-to-sequence LMs They are composed of a bidirectional encoder that uses both left and right context and a left-to-right decoder that do not share parameters.

Original: 3. P ROMPT INDUCTION METHODS
Revised: 3. P ROMPT INDUCTION METHODS

Original: Prompts are either manually crafted or generated automatically by prompting methods.
Revised: Prompts are either manually crafted or generated automatically by prompting methods.

Original: In this study, we have selected 3 different prompting methods that are representative of semi-manual, discrete and continuous induction methods, respectively.
Revised: In this study, we have selected 3 different prompting methods that are representative of semi-manual, discrete and continuous induction methods, respectively.

Original: They have been shown to perform well on the slot filling task and associated code is publicly available.
Revised: They have been shown to perform well on the slot filling task and associated code is publicly available.

Original: LPAQA Starting from seed manual prompts, Jiang et al.
Revised: LPAQA Starting from seed manual prompts, Jiang et al.

Original: (2020) generate a diverse candidate prompt set using mining- and paraphrasing-based methods.
Revised: (2020) generate a diverse candidate prompt set using mining- and paraphrasing-based methods.

Original: For each relation, the best performing candidate on the training data is selected.
Revised: For each relation, the best performing candidate on the training data is selected.

Original: To improve performance, the authors also propose
Revised: To improve performance, the authors also propose

Original: [Table]
Revised: [Table]

Original: prompt ensembling.
Revised: prompt ensembling.

Original: However, ensembling tends to increase performance independently of the underlying prompting method (see Appendix A).
Revised: However, ensembling tends to increase performance independently of the underlying prompting method (see Appendix A).

Original: Consequently, we will only focus on the top-1 prompt selection method here.
Revised: Consequently, we will only focus on the top-1 prompt selection method here.

Original: We consider LPAQA a semi-manual method because it needs to be seeded with manual prompts
Revised: We consider LPAQA a semi-manual method because it needs to be seeded with manual prompts

Original: [Table]
Revised: [Table]

Original: and mining retrieves further human-generated strings
Revised: and mining retrieves further human-generated strings

Original: [Table]
Revised: [Table]

Original: AutoPrompt It is an automated method proposed by Shin et al.
Revised: AutoPrompt It is an automated method proposed by Shin et al.

Original: (2020) to generate discrete prompts using gradient-guided search (Wallace et al., 2019).
Revised: (2020) to generate discrete prompts using gradient-guided search (Wallace et al., 2019).

Original: The prompts are composed of a sequence of tokens selected from the vocabulary of the LM.
Revised: The prompts are composed of a sequence of tokens selected from the vocabulary of the LM.

Original: The number of tokens is pre-defined.
Revised: The number of tokens is pre-defined.

Original: The process is divided into two phases.
Revised: The process is divided into two phases.

Original: For each specific token position, a set of candidates that maximize the likelihood on a batch of training data is first created.
Revised: For each specific token position, a set of candidates that maximize the likelihood on a batch of training data is first created.

Original: Then, the candidates are re-evaluated on a different batch, with the best one retained.
Revised: Then, the candidates are re-evaluated on a different batch, with the best one retained.

Original: Even though the generated prompts are less interpretable, they perform better than manual prompts.
Revised: Even though the generated prompts are less interpretable, they perform better than manual prompts.

Original: In our experiments, we use 5-token prompts and run the algorithm for 1,000 iterations.
Revised: In our experiments, we use 5-token prompts and run the algorithm for 1,000 iterations.

Original: OptiPrompt Zhong et al.
Revised: OptiPrompt Zhong et al.

Original: (2021) propose an automated method to generate continuous prompts.
Revised: (2021) propose an automated method to generate continuous prompts.

Original: They are dense vectors in the embedding space of the LM that are learned using gradient descent on a separate training dataset.
Revised: They are dense vectors in the embedding space of the LM that are learned using gradient descent on a separate training dataset.

Original: Except for the learning rate, which is increased to 3e-2 for the T models for proper convergence, we use the same hyperparameters as the original implementation.
Revised: Except for the learning rate, which is increased to 3e-2 for the T models for proper convergence, we use the same hyperparameters as the original implementation.

Original: We initialize vectors randomly.
Revised: We initialize vectors randomly.

Original:  R ESULTS AND ANALYSIS 4. P ROMPTING M ETHOD P ERFORMANCE
Revised:  R ESULTS AND ANALYSIS 4. P ROMPTING M ETHOD P ERFORMANCE

Original: We start by evaluating the performance of the different prompting methods.
Revised: We start by evaluating the performance of the different prompting methods.

Original: 2 Prompts are induced with a specific LM and then evaluated by retrieving objects from the same LM.
Revised: 3 Prompts are induced with a specific LM and then evaluated by retrieving objects from the same LM.

Original: Table 2 summarizes the results.
Revised: Table 2 summarizes the results.

Original: For reference, a majority-class baseline always picking the most common object for each relation reaches 26.91% accuracy (note that this baseline has partial access to the ground truth in order to retrieve the most common object of each relation).
Revised: For reference, a majority-class baseline always picking the most common object for each relation reaches 26.91% accuracy (note that this baseline has partial access to the ground truth in order to retrieve the most common object of each relation).

Original: The random baseline is virtually at 0%.
Revised: The random baseline is virtually at 0%.

Original: AutoPrompt clearly outperforms LAMA and LPAQA, although it lags behind OptiPrompt.
Revised: AutoPrompt clearly outperforms LAMA and LPAQA, although it lags behind OptiPrompt.

Original: We thus confirm that soft prompts are the way to go if you have access to a model embedding space and are not interested in generalization across model.
Revised: We thus confirm that soft prompts are the way to go if you have access to a model embedding space and are not interested in generalization across models.

Original: If either of these conditions is not met, AutoPrompt is preferable to manual and semi-manual prompts.
Revised: If either of these conditions is not met, AutoPrompt is preferable to manual and semi-manual prompts.

Original: Masked models tend to perform better as source LMs.
Revised: Masked models tend to perform better as source LMs.

Original: This could be attributed to the fact that they were pre-trained with a fill-in-the-blank task (Devlin et al., 2019), which is exactly how the slot filling task is formulated.
Revised: This could be attributed to the fact that they were pre-trained with a fill-in-the-blank task (Devlin et al., 2019), which is exactly how the slot filling task is formulated.

Original: [Table]
Revised: [Table]

Original: 4. A UTO P ROMPT G ENERALIZATION
Revised: 4. A UTO P ROMPT G ENERALIZATION

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we investigate AutoPrompt’s ability to generalize across different LMs.
Revised: we investigate AutoPrompt’s ability to generalize across different LMs.

Original: 3 Prompts are induced using a Source LM and then evaluated on a Target LM, which can be the same or different from the Source (Figure 2).
Revised: 4 Prompts are induced using a Source LM and then evaluated on a Target LM, which can be the same or different from the Source (Figure 2).

Original: The results, relative to single-model LM performance, are shown in Figure 3.
Revised: The results, relative to single-model LM performance, are shown in Figure 3.

Original: AutoPrompt generally performs best when the Source and Target LMs are the same, as shown by the fact that off-diagonal values are mostly negative.
Revised: AutoPrompt generally performs best when the Source and Target LMs are the same, as shown by the fact that off-diagonal values are mostly negative.

Original: The performance gap gets bigger as we transfer across different LM types.
Revised: The performance gap gets bigger as we transfer across different LM types.

Original: Prompts are generally more stable across different sizes of the same model, such as BERT BASE and BERT LARGE .
Revised: Prompts are generally more stable across different sizes of the same model, such as BERT BASE and BERT LARGE .

Original: The drop in generalization performance of Left-to-right models is less dramatic simply because, for these models, the original same-source-and-target performance is already low.
Revised: The drop in generalization performance of left-to-right models is less dramatic simply because, for these models, the original same-source-and-target performance is already low.

Revised: We also verify the impact of model size on generalization.

Revised: For each source model, we define the generalization drop score as the average of the corresponding column in Figure 3.

Revised: It measures the average drop in accuracy when prompts from a source model are tested on a different target, with respect to the original same-source-and-target accuracy.

Revised: We discovered that performance drop and source model size are highly correlated (0.6).

Revised: We do not have a clear explanation for this correlation, but we think it is an intriguing observation that should be further studied in the prompt analysis literature.

Original: 4. M IXED - TRAINING A UTO P ROMPT G ENERALIZATION
Revised: 4. M IXED - TRAINING A UTO P ROMPT G ENERALIZATION

Original: We propose a simple modification to AutoPrompt training to generate prompts that generalize better.
Revised: We propose a simple modification to AutoPrompt training to generate prompts that generalize better.

Original: Recall that the AutoPrompt algorithm involves two phases: one in which candidate prompts are generated, and one in which the prompts are evaluated.
Revised: Recall that the AutoPrompt algorithm involves two phases: one in which candidate prompts are generated, and one in which the prompts are evaluated.

Original: Rather than relying on the same model for the two phases, we now use two different LMs.
Revised: Rather than relying on the same model for the two phases, we now use two different LMs.

Original: The first model, which we call the generator ,terms proposes a set of candidates.
Revised: The first model, which we call the generator , proposes a set of candidates.

Original: Then, the second model, that we call the evaluator , evaluates the candidates and chooses the best one.
Revised: Then, the second model, that we call the evaluator , evaluates the candidates and chooses the best one.

Original: To avoid a combinatorial explosion, we focus on combining the single LMs of each class that performed best in the same-source-and-target setup (Table 2 above).
Revised: To avoid a combinatorial explosion, we focus on combining the single LMs of each class that performed best in the same-source-and-target setup (Table 2 above).

Original: For each pair, we arbitrarily use the best of the two LMs (in the same-source-and-target setup) as generator.
Revised: For each pair, we arbitrarily use the best of the two LMs (in the same-source-and-target setup) as generator.

Original: We deliberately avoid picking models based on generalization performance (Figure 3), as in a realistic settings we might not have advance access to the new architectures we need to generalize to.
Revised: We deliberately avoid picking models based on generalization performance (Figure 3), as in a realistic settings we might not have advance access to the new architectures we need to generalize to.

Original: Table 3 compares the performance of standard and mixed AutoPrompt (we extend these experiments to LAMA-UHN in Appendix D).
Revised: Table 3 compares the performance of standard and mixed AutoPrompt (we extend these experiments to LAMA-UHN in Appendix D).

Original: The BERT BASE /T

Revised: The BERT BASE /T

Original: LARGE mix has the highest average accuracy.
Revised: LARGE mix has the highest average accuracy.

Original: Although it does not perform as well as BERT BASE on the BERT models, it transfers better to all the other models (including the RoBERTa family).
Revised: Although it does not perform as well as BERT BASE on the BERT models, it transfers better to all the other models (including the RoBERTa family).

Original: This mixed AutoPrompt variant even outperforms the best seq2seq model T5 LARGE on all sequence-to-sequence models (including T5 LARGE itself).
Revised: This mixed AutoPrompt variant even outperforms the best seq2seq model T5 LARGE on all sequence-to-sequence models (including T5 LARGE itself).

Original: It also outperforms GPT2 MEDIUM on two GPT2 variants.
Revised: It also outperforms GPT2 MEDIUM on two GPT2 variants.

Original: If these results are very encouraging, Table 3 also shows that simply mixing models does not guarantee good generalization.
Revised: If these results are very encouraging, Table 3 also shows that simply mixing models does not guarantee good generalization.

Original: When we replace BERT BASE with T5 LARGE as generator, generalization performance is actually worse than when using T5 LARGE alone, and combining BERT BASE as generator with GPT2 MEDIUM as evaluator leads to minor generalization improvements compared to using
Revised: When we replace BERT BASE with T5 LARGE as generator, generalization performance is actually worse than when using T5 LARGE alone, and combining BERT BASE as generator with GPT2 MEDIUM as evaluator leads to minor generalization improvements compared to using

Original: BERT BASE alone.
Revised: BERT BASE alone.

Original: Future research should aim at a better understanding of which factors determine the generalization success of specific combinations of LM.
Revised: Some preliminary insights on the best mixing strategy are offered in Appendix E. [Table]

Original: 4. P ROMPT ANALYSIS
Revised: 4. P ROMPT ANALYSIS

Original: We study the prompts generated by AutoPrompt through single-model and mixed training, looking for differences that might hint at how the latter generalize better.
Revised: We study the prompts generated by AutoPrompt through single-model and mixed training, looking for differences that might hint at how the latter generalize better.

Original: Since the prompt examples in Table 8 (Appendix E) suggest that there are no huge differences directly visible to the “naked eye”, we undertake a quantitative analysis led by the following hypotheses.[Table]
Revised: Since the prompt examples in Table 8 (Appendix F) suggest that there are no huge differences directly visible to the “naked eye”, we undertake a quantitative analysis led by the following hypotheses.

Original: 1) Each LM has its own peculiarities, but they all share English as training language.
Revised: 1) Each LM has its own peculiarities, but they all share English as training language.

Original: Prompts optimized on a single model might overfit the quirks of that model, but mixed-training prompts might capture more general properties of English that are shared across models.
Revised: Prompts optimized on a single model might overfit the quirks of that model, but mixed-training prompts might capture more general properties of English that are shared across models.

Original: We thus hypothesize that prompts that generalize better will have a larger semantic overlap with manually crafted English prompts .
Revised: We thus hypothesize that prompts that generalize better will have a larger semantic overlap with manually crafted English prompts .

Original: Modern LMs use sub-word tokenization strategies that differ from model to model.
Revised: Modern LMs use sub-word tokenization strategies that differ from model to model.

Original: AutoPrompt is thus free to combine sub-words into non-word sequences (e.g., slalomgraphers , publishedtoon in Table 8) that might in turn be tokenized differently by different models, leading to inter-model brittleness.
Revised: AutoPrompt is thus free to combine sub-words into non-word sequences (e.g., slalomgraphers , publishedtoon in Table 8) that might in turn be tokenized differently by different models, leading to inter-model brittleness.

Original: We thus conjecture that prompts that generalize better will contain a larger proportion of real English words .
Revised: We thus conjecture that prompts that generalize better will contain a larger proportion of real English words .

Original: 3) Natural languages rely on word order to express meaning, but it’s less clear that LMs are capturing genuine syntactic rules (Sinha et al., 2021).
Revised: 3) Natural languages rely on word order to express meaning, but it’s less clear that LMs are capturing genuine syntactic rules (Sinha et al., 2021).

Original: It’s more likely that, to the extent that prompts crucially rely on token order, this is exploiting statistical co-occurrence quirks of specific LMs.
Revised: It’s more likely that, to the extent that prompts crucially rely on token order, this is exploiting statistical co-occurrence quirks of specific LMs.

Original: We thus conjecture that a “bag-of-token” prompt sequence that does not require the tokens to be in any special order will be more general than one where order matters and, consequently, generalizing prompts will be more robust to token shuffling .
Revised: We thus conjecture that a “bag-of-token” prompt sequence that does not require the tokens to be in any special order will be more general than one where order matters and, consequently, generalizing prompts will be more robust to token shuffling .

Original: On a related point, single-model-optimized prompts might concentrate information in the slots the source model is most sensitive to, but such slots might vary from model (type) to model (type).
Revised: On a related point, single-model-optimized prompts might concentrate information in the slots the source model is most sensitive to, but such slots might vary from model (type) to model (type).

Original: We thus conjecture that generalizing prompts will distribute information more evenly across tokens and thus they will be more robust to single-token deletion .
Revised: We thus conjecture that generalizing prompts will distribute information more evenly across tokens and thus they will be more robust to single-token deletion .

Original: Semantic overlap with English The manual LAMA prompts are our English reference point.
Revised: Semantic overlap with English The manual LAMA prompts are our English reference point.

Original: We measure semantic overlap as the cosine between a vector representing an AutoPrompt-generated prompt and a LAMA prompt.
Revised: We measure semantic overlap as the cosine between a vector representing an AutoPrompt-generated prompt and a LAMA prompt.

Original: Specifically, we use fastText (Bojanowski et al., 2017) to represent prompts, both because it offers independent representations from those of any of the LMs we are comparing, and because it relies on vocabulary- and tokenization-independent n-gram-based se- quence representations.
Revised: Specifically, we use fastText (Bojanowski et al., 2017) to represent prompts, both because it offers independent representations from those of any of the LMs we are comparing, and because it relies on vocabulary- and tokenization-independent n-gram-based sequence representations.

Original: Instead of reporting difficult-to-interpret absolute cosines, we report the t -score for the cosine difference between cases where AutoPrompt prompts are compared to the
Revised: Instead of reporting difficult-to-interpret absolute cosines, we report the t -score for the cosine difference between cases where AutoPrompt prompts are compared to the

Original: LAMA prompts for the same T-ReX relation, and cases where AutoPrompt prompts are compared to different-relation LAMA prompts.
Revised: LAMA prompts for the same T-ReX relation, and cases where AutoPrompt prompts are compared to different-relation LAMA prompts.

Original: In other words, the larger this value is, the clearer the difference in semantic overlap is between meaningful and random prompt comparisons.
Revised: In other words, the larger this value is, the clearer the difference in semantic overlap is between meaningful and random prompt comparisons.

Original: Results are in the first column of Table 4.
Revised: Results are in the first column of Table 4.

Original: There is a strong correlation ( > 0 .
Revised: There is a strong correlation ( > 0 .

Original: 9 Pearson) between a prompt semantic overlap with English and its accuracy when tested on the model used as source/generator during training.
Revised: 9 Pearson) between a prompt semantic overlap with English and its accuracy when tested on the model used as source/generator during training.

Original: This is encouraging in itself, suggesting that more effective AutoPrompt prompts are also more semantically transparent.
Revised: This is encouraging in itself, suggesting that more effective AutoPrompt prompts are also more semantically transparent.

Original: However, our hypothesis that better generalization implies higher semantic overlap is disproven: there is a clear decrease in overlap between BERT BASE -based prompts and the better generalizing ones obtained through BERT BASE /T

Revised: However, our hypothesis that better generalization implies higher semantic overlap is disproven: there is a clear decrease in overlap between BERT BASE -based prompts and the better generalizing ones obtained through BERT BASE /T

Original: LARGE mixed-training.
Revised: LARGE mixed-training.

Original: Real-word ratio We verify whether a space- or punctuation-marked delimited character sequence in a prompt is an existing English word by checking if it appears in the list of 56k words occurring at least 1k times in the ukWaC corpus (Baroni et al., 2009).
Revised: Real-word ratio We verify whether a space- or punctuation-marked delimited character sequence in a prompt is an existing English word by checking if it appears in the list of 56k words occurring at least 1k times in the ukWaC corpus (Baroni et al., 2009).

Original: This corpus was not used for training any of the models, thus minimizing biases towards any of them.
Revised: This corpus was not used for training any of the models, thus minimizing biases towards any of them.

Original: The minimum occurrence threshold was determined by manual inspection, observing that rarer strings tend not to be real words but “corpus detritus” (numbers, dates, code fragments, typos).
Revised: The minimum occurrence threshold was determined by manual inspection, observing that rarer strings tend not to be real words but “corpus detritus” (numbers, dates, code fragments, typos).

Original: The second column of Table 4 reports percentage attested-word ratios among the tokens produced by AutoPrompt for the whole T-ReX relation set in various training setups.
Revised: The second column of Table 4 reports percentage attested-word ratios among the tokens produced by AutoPrompt for the whole T-ReX relation set in various training setups.

Original: For reference, this ratio is at 99.4% for the manual LAMA prompts.
Revised: For reference, this ratio is at 99.4% for the manual LAMA prompts.

Original: The generalizing BERT BASE /T

Revised: The generalizing BERT BASE /T

Original: LARGE setup clearly outperforms single-model training on BERT BASE on this metric, tentatively confirming our hypothesis that more word-like strings will transfer better across models.
Revised: LARGE setup clearly outperforms single-model training on BERT BASE on this metric, tentatively confirming our hypothesis that more word-like strings will transfer better across models.

Original: Note however that BERT BASE /GPT

Revised: Note however that BERT BASE /GPT

Original: MEDIUM , a mixed-training setup that does not generalize better than BERT BASE -based training alone, features prompts sporting an even higher proportion of existing words.
Revised: MEDIUM , a mixed-training setup that does not generalize better than BERT BASE -based training alone, features prompts sporting an even higher proportion of existing words.

Original: So, the latter might be a common property of mixed-training-induced prompts, but not one that automatically entails better generalization.
Revised: So, the latter might be a common property of mixed-training-induced prompts, but not one that automatically entails better generalization.

Original: Shuffling We shuffle the tokens in each prompt, and compute the resulting T-ReX accuracy when retrieving information from the LM used as source/generator during AutoPrompt training.
Revised: Shuffling We shuffle the tokens in each prompt, and compute the resulting T-ReX accuracy when retrieving information from the LM used as source/generator during AutoPrompt training.

Original: To tease the effect of shuffling apart from the absolute performance of a prompt set, we also report the ratio of accuracy after shuffling to accuracy with unshuffled prompts.
Revised: To tease the effect of shuffling apart from the absolute performance of a prompt set, we also report the ratio of accuracy after shuffling to accuracy with unshuffled prompts.

Original: We repeat the shuffling experiment 10 times, and report averaged accuracies/ratios and standard deviations in the last two columns of
Revised: We repeat the shuffling experiment 10 times, and report averaged accuracies/ratios and standard deviations in the last two columns of

Original: Table 4.
Revised: Table 4.

Original: By superficially eyeing AutoPrompt prompts such as those in Table 8, one could think they are bags of tokens, but the ratios show that token order matters, as there is always a big drop in performance after shuffling.
Revised: By superficially eyeing AutoPrompt prompts such as those in Table 8, one could think they are bags of tokens, but the ratios show that token order matters, as there is always a big drop in performance after shuffling.

Original: Indeed, by closer inspection of the prompts in Table 8, we notice that some of them do have a sentence flavor (‘ [X] teaches modelling frescoes downtown in [Y] ”).
Revised: Indeed, by closer inspection of the prompts in Table 8, we notice that some of them do have a sentence flavor (‘ [X] teaches modelling frescoes downtown in [Y] ”).

Original: Our hypothesis that the generalizing BERT BASE /T

Revised: Our hypothesis that the generalizing BERT BASE /T

Original: LARGE prompts are less order-sensitive than the BERT BASE ones is confirmed (comparein particular the ratios of these setups).
Revised: LARGE prompts are less order-sensitive than the BERT BASE ones is confirmed (compare the ratios of these setups).

Original: The extra robustness of BERT BASE /T

Revised: The extra robustness of BERT BASE /T

Original: LARGE might be due to the fact that T5 LARGE itself is particularly robust to shuffling, and it remains to be seen if  there is a causal relationship between robustness to shuffling and generalization for mixed-training prompts.
Revised: LARGE might be due to the fact that T5 LARGE itself is particularly robust to shuffling, and it remains to be seen if for mixed-training prompts there is a causal relationship between robustness to shuffling and generalization.

Original: Note that the relatively high ratio of GPT2 MEDIUM might be due to a flooring effect, as this setup has low accuracy before and after shuffling.
Revised: Note that the relatively high ratio of GPT2 MEDIUM might be due to a flooring effect, as this setup has low accuracy before and after shuffling.

Original: Token deletion To verify if information is equally distributed across the 5 tokens of an AutoPrompt prompt, for each prompt set we compute T-ReX accuracy (retrieving information from the matching source/generator model) after removing a single token, repeating the experiment for each token position.
Revised: Token deletion To verify if information is equally distributed across the 5 tokens of an AutoPrompt prompt, for each prompt set we compute T-ReX accuracy (retrieving information from the matching source/generator model) after removing a single token, repeating the experiment for each token position.

Original: Figure 4 reveals that BERT BASE -based prompts are concentrating a lot of information towards the end of the sequence, and in particular on the last token, with a huge drop in performance if the latter is deleted.
Revised: Figure 4 reveals that BERT BASE -based prompts are concentrating a lot of information towards the end of the sequence, and in particular on the last token, with a huge drop in performance if the latter is deleted.

Original: While the same tendency to pack information towards the end of the sequence is present in the better-generalizing BERT BASE /T

Revised: While the same tendency to pack information towards the end of the sequence is present in the better-generalizing BERT BASE /T

Original: LARGE prompts, the drop is less dramatic (especially when taking into account the fact that the latter prompts start from a lower full-sequence accuracy).
Revised: LARGE prompts, the drop is less dramatic (especially when taking into account the fact that the latter prompts start from a lower full-sequence accuracy).

Original: Figure 5 in Appendix F shows that all generated prompts have a tendency to pack more information at the end of the sequence (which is remarkable, as their encoders are based on the fully  symmetric transformer architecture), but T5 LARGE generates prompts that are less affected by singletoken deletion.
Revised: Figure 5 in Appendix G shows that all generated prompts have a tendency to pack more information at the end of the sequence (which is remarkable, as their encoders are based on the fully [Figure] symmetric transformer architecture), but T5 LARGE generates prompts that are less affected by singletoken deletion.

Original: The higher robustness of BERT BASE /T

Revised: The higher robustness of BERT BASE /T

Original: LARGE compared to BERT BASE might thus be inherited from T5 LARGE , and further studies should ascertain whether there is a causal relation between smoother information distribution and better generalization for mixed-training prompts.[Figure]
Revised: LARGE compared to BERT BASE might thus be inherited from T5 LARGE , and further studies should ascertain whether there is a causal relation between smoother information distribution and better generalization for mixed-training prompts.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: Take-home points Automatically induced discrete prompts, such as those derived with the AutoPrompt algorithm, strike a good balance between (semi-)manual prompts, that they outperform in retrieval quality and (potentially) scalability, and soft prompts, that can only be used out-of-the-box on the model they were induced from, and require access to inner model structures to function.
Revised: Take-home points Automatically induced discrete prompts, such as those derived with the AutoPrompt algorithm, strike a good balance between (semi-)manual prompts, that they outperform in retrieval quality and (potentially) scalability, and soft prompts, that can only be used out-of-the-box on the model they were induced from, and require access to inner model structures to function.

Original: However, the standard AutoPrompt method must be adapted to get good performance across language models.
Revised: However, the standard AutoPrompt method must be adapted to get good performance across language models.

Original: In particular, a simple modification in which AutoPrompt is trained using two language models leads to prompts that better generalize to further models.
Revised: In particular, a simple modification in which AutoPrompt is trained using two language models leads to prompts that better generalize to further models.

Original: The better-generalizing prompts induced in this way look quite similar to prompts induced with the standard method.
Revised: The better-generalizing prompts induced in this way look quite similar to prompts induced with the standard method.

Original: However, a probing analysis suggests that there are systematic differences, and in particular that the generalizing prompts tend to feature a larger proportion of existing English words, and to be more robust to ablations that probe sensitivity to token order and asymmetries in information distribution.
Revised: However, a probing analysis suggests that there are systematic differences, and in particular that the generalizing prompts tend to feature a larger proportion of existing English words, and to be more robust to ablations that probe sensitivity to token order and asymmetries in information distribution.

Original: In sum, our results suggest that it is viable to use a learning-based algorithm to generate “universal” discrete prompts that can be employed to extract information from a variety of pre-trained language models, only requiring access to their standard discrete-string interface.
Revised: In sum, our results suggest that it is viable to use a learning-based algorithm to generate “universal” discrete prompts that can be employed to extract information from a variety of pre-trained language models, only requiring access to their standard discrete-string interface.

Original: Limitations and directions for further work Our results are based on a single task, slot filling, and a single data-set (T-ReX).
Revised: Limitations and directions for further work Our results are based on a single task, slot filling, and a single data-set (T-ReX).

Original: We believe this is a good starting point, because in slot filling a separate, semantically contentful prompt must be learned for each relation, but future work should extend the investigation to different tasks, including tasks where successful knowledge retrieval requires more than recalling a single word, as in the LAMA/T-ReX setup we used.
Revised: We believe this is a good starting point, because in slot filling a separate, semantically contentful prompt must be learned for each relation, but future work should extend the investigation to different tasks, including tasks where successful knowledge retrieval requires more than recalling a single word, as in the LAMA/T-ReX setup we used.

Original: We used a single discrete prompt induction algorithm, AutoPrompt, confirming that it is generating high-quality prompts.
Revised: We used a single discrete prompt induction algorithm, AutoPrompt, confirming that it is generating high-quality prompts.

Original: However, this method generates a single fixed prompt for each task or sub-task.
Revised: However, this method generates a single fixed prompt for each task or sub-task.

Original: True scalability will only be achieved with prompting methods that can generate an appropriate query for each different input they receive, and we intend to design discrete-prompt-induction algorithms matching this desideratum (see Haviv et al.
Revised: True scalability will only be achieved with prompting methods that can generate an appropriate query for each different input they receive, and we intend to design discrete-prompt-induction algorithms matching this desideratum (see Haviv et al.

Original: (2021) for a step in this direction).
Revised: (2021) for a step in this direction).

Original: Another reason to focus on algorithm development is that the single-model comparison between AutoPrompt and the OptiPrompt soft prompts shows there is still large room to improve retrieval quality.
Revised: Another reason to focus on algorithm development is that the single-model comparison between AutoPrompt and the OptiPrompt soft prompts shows there is still large room to improve retrieval quality.

Original: Our analysis revealed systematic differences between the best model-specific and generalizing prompts.
Revised: Our analysis revealed systematic differences between the best model-specific and generalizing prompts.

Original: However, it is not clear that any of these differences is causally connected with generalization improvement.
Revised: However, it is not clear that any of these differences is causally connected with generalization improvement.

Original: In future work, we would like to better understand the relation between properties such as robustness to shuffling and generalization.
Revised: In future work, we would like to better understand the relation between properties such as robustness to shuffling and generalization.

Original: A particular exciting direction is to favour the emergence of such properties through appropriate auxiliary functions at prompt-induction time, and verify whether they lead to further improvements in generalization performance. 
Revised: A particular exciting direction is to favour the emergence of such properties through appropriate auxiliary functions at prompt-induction time, and verify whether they lead to further improvements in generalization performance. A CKNOWLEDGMENTS

Revised: We thank the reviewers for constructive feedback.

Revised: We thank the members of the UPF COLT group for discussion and advice.

Original:  R EPRODUCIBILITY S TATEMENT
Revised: The project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101019291). This paper reflects the authors’ view only, and the funding agency is not responsible for any use that may be made of the information it contains. Fabio Petroni conducted work on the project while at Meta AI. R EPRODUCIBILITY S TATEMENT

Original: All datasets, pre-trained models and prompting methods used in this paper are publicly available.
Revised: All datasets, pre-trained models and prompting methods used in this paper are publicly available.

Original: We use the same hyperparameters as the original implementation of the prompting methods unless it is clearly specified in the text.
Revised: We use the same hyperparameters as the original implementation of the prompting methods unless it is clearly specified in the text.

Original: Code to reproduce the results as well as the common vocabulary and the filtered datasets will be shared upon acceptance.
Revised: Code to reproduce the results as well as the common vocabulary and the filtered datasets will be shared upon acceptance.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: Prompting relies on pre-trained language models, and thus inherits most ethical risks inherent in such models (Weidinger et al., 2022).
Revised: Prompting relies on pre-trained language models, and thus inherits most ethical risks inherent in such models (Weidinger et al., 2022).

Original: As we are not introducing new models, we are not adding new potential issues tied to LMs.
Revised: As we are not introducing new models, we are not adding new potential issues tied to LMs.

Original: To the contrary, developing more general prompts might help in scenarios where an underlying LM needs to be retired and replaced with a new one due to ethical issues.

Original: As the examples in Table 8 show, automated prompting methods tend to produce opaque prompts.
Revised: As the examples in Table 8 show, automated prompting methods tend to produce opaque prompts.

Original: This characteristic might be exploited for harmful purposes, such as adversarial attacks in which a model is “triggered” to produce unwanted information through an apparently innocuous prompt (Wallace et al., 2019).
Revised: This characteristic might be exploited for harmful purposes, such as adversarial attacks in which a model is “triggered” to produce unwanted information through an apparently innocuous prompt (Wallace et al., 2019).

Original: We believe that this provides further motivation for our focus on discrete prompts, that are more directly human-readable than soft prompts.
Revised: We believe that this provides further motivation for our focus on discrete prompts, that are more directly human-readable than soft prompts.

Original: We showed in particular in
Revised: We showed in particular in

Original: Section 4.4 that there is a very high correlation ( >. 9 ) between the quality of automatically-induced prompts in the relevant information retrieval task and the degree of semantic transparency of the prompts.
Revised: Section 4.4 that there is a very high correlation ( >. 9 ) between the quality of automatically-induced prompts in the relevant information retrieval task and the degree of semantic transparency of the prompts.

Original: If this result could be extended, it would constitute very good news on the interpretability front, suggesting that very high-quality prompts will also be more human-interpretable, making it harder to exploit opaque prompts for harmful purposes.
Revised: If this result could be extended, it would constitute very good news on the interpretability front, suggesting that very high-quality prompts will also be more human-interpretable, making it harder to exploit opaque prompts for harmful purposes.

Original: A P ROMPT ENSEMBLING
Revised: A P ROMPT ENSEMBLING

Original: Discrete prompt ensembling for knowledge extraction was introduced by Jiang et al.
Revised: Discrete prompt ensembling for knowledge extraction was introduced by Jiang et al.

Original: (2020) using
Revised: (2020) using

Original: LPAQA.
Revised: LPAQA.

Original: Instead of selecting the top-1 prompt from a set of candidates, multiple prompts are combined in order to improve performance.
Revised: Instead of selecting the top-1 prompt from a set of candidates, multiple prompts are combined in order to improve performance.

Original: The authors argue that ensembling improves the extraction  of knowledge that appeared in different contexts within the training data.
Revised: The authors argue that ensembling improves the extraction [Table] of knowledge that appeared in different contexts within the training data.

Original: Since the idea of ensembling is distinct from the specifics of the LPAQA method, in this appendix we check if ensembling also helps AutoPrompt.
Revised: Since the idea of ensembling is distinct from the specifics of the LPAQA method [Table] in this appendix we check if ensembling also helps AutoPrompt.

Original: Specifically we experiment with the optimized ensembling approach used with LPAQA by Jiang et al.
Revised: Specifically we experiment with the optimized ensembling approach used with LPAQA by Jiang et al.

Original: (2020), which associates learnable weights to each prompt.
Revised: (2020), which associates learnable weights to each prompt.

Original: For ensembling purposes, multiple candidate prompts for each relation are required.
Revised: For ensembling purposes, multiple candidate prompts for each relation are required.

Original: One way to obtain these candidate prompts for AutoPrompt is by inducing prompts from instances of the same[Table] model trained with different seeds.
Revised: One way to obtain these candidate prompts for AutoPrompt is by inducing prompts from instances of the same model trained with different seeds.

Original: Considering the computational cost of retraining the models ourselves

Original: [Table] we use the already available MultiBERT collection (Sellam et al., 2021) [Table] which containsinstances of BERT BASE trained with different seeds.
Revised: Considering the computational cost of retraining the models ourselves, we use the already available MultiBERT collection (Sellam et al., 2021), which containsinstances of BERT BASE trained with different seeds.

Original: Unfortunately, unlike the models in Table [Table] the
Revised: Unfortunately, unlike the models in Table 1, the

Original: MultiBERTs were trained on uncased data [Table] which means we can only evaluate on the MultiBERTs themselves.
Revised: MultiBERTs were trained on uncased data, which means we can only evaluate on the MultiBERTs themselves.

Original: Specifically, we use seed-0 MultiBERT as our target model
Revised: Specifically, we use seed-0 MultiBERT as our target model

Original: [Table]
Revised: [Table]

Original: and obtain the AutoPrompt prompts from the remaining MultiBERTs.
Revised: and obtain the AutoPrompt prompts from the remaining MultiBERTs.

Original: For LPAQA, we use seed-0 MultiBERT for training and testing, getting the candidate prompts with the standard LPAQA mining and paraphrasing procedure.
Revised: For LPAQA, we use seed-0 MultiBERT for training and testing, obtaining the candidate prompts with the standard LPAQA mining and paraphrasing procedure.

Original: First, we confirm that in this different setup AutoPrompt is still outperforming LPAQA in top-1 evaluation (39.40% vs. 35.17% T-ReX accuracies, respectively).
Revised: First, we confirm that in this different setup AutoPrompt is still outperforming LPAQA in top-1 evaluation (39.40% vs. 35.17% T-ReX accuracies, respectively).

Original: Second, ensembling helps both models.
Revised: Second, ensembling helps both models.

Original: However, ensembled LPAQA barely reaches the performance of top-1 Automprompt (39.21%), whereas ensembled AutoPrompt further improves to 42.43% accuracy.
Revised: However, ensembled LPAQA barely reaches the performance of top-1 Automprompt (39.21%), whereas ensembled AutoPrompt further improves to 42.43% accuracy.

Original: B S OFT PROMPT GENERALIZATION
Revised: B S OFT PROMPT GENERALIZATION

Original: Transferring soft prompts across models is not as straightforward as with discrete prompts.
Revised: Transferring soft prompts across models is not as straightforward as with discrete prompts.

Original: In order to transfer the vectors, the source and target must have the same embedding dimension.
Revised: In order to transfer the vectors, the source and target must have the same embedding dimension.

Original: We experiment with the subset of models that have a common embedding dimension of 768.
Revised: We experiment with the subset of models that have a common embedding dimension of 768.

Original: Tableconfirms that soft prompts generalize extremely poorly.
Revised: Tableconfirms that soft prompts generalize extremely poorly.

Original: The only exception is decent student/teacher transfers for distilled models, probably because their embeddings were initialized with those of the corresponding teacher models.
Revised: The only exception is decent student/teacher transfers for distilled models, probably because their embeddings were initialized with those of the corresponding teacher models.

Original: A more general approach consists in discretizing the soft prompts to their nearest vocabulary neighbors (Mikolov et al., 2013; Hashimoto et al., 2016), and using the resulting vocabulary item sequences as prompts.
Revised: A more general approach consists in discretizing the soft prompts to their nearest vocabulary neighbors (Mikolov et al., 2013; Hashimoto et al., 2016), and using the resulting vocabulary item sequences as prompts.

Original: OptiPrompt vectors can be initialized with random numbers, random vocabulary embeddings or manual prompts.
Revised: OptiPrompt vectors can be initialized with random numbers, random vocabulary embeddings or manual prompts.

Original: When initialized with random vocabulary embeddings or manual prompts, the nearest neighbor projections of the resulting vectors are always identical to the initialization tokens, making the process vacuous.
Revised: When initialized with random vocabulary embeddings or manual prompts, the nearest neighbor projections of the resulting vectors are always identical to the initialization tokens, making the process vacuous.

Original: However, when the vectors are initialized to random numbers, the nearest neighbors of the optimized vectors are the same for multiple relations.
Revised: However, when the vectors are initialized to random numbers, the nearest neighbors of the optimized vectors are the same for multiple relations.

Original: 4 As expected, unlike the corresponding soft prompts, these discretized prompts perform very poorly.
Revised: 5 As expected, unlike the corresponding soft prompts, these discretized prompts perform very poorly.

Original: For example, when training on BERT BASE , OptiPrompt soft prompts achieve 48.26% T-ReX accuracy.
Revised: For example, when training on BERT BASE , OptiPrompt soft prompts achieve 48.26% T-ReX accuracy.

Original: The corresponding discretized prompts only achieve 1.21% accuracy. 
Revised: The corresponding discretized prompts only achieve 1.21% accuracy. [Table]

Original: C LPAQA G ENERALIZATION
Revised: C LPAQA G ENERALIZATION

Original: AutoPrompt is better than LPAQA in the same-source-and-target setup, as demonstrated in Table 2.
Revised: AutoPrompt is better than LPAQA in the same-source-and-target setup, as demonstrated in Table 2.

Original: Table 6 further shows that AutoPrompt also generalizes better than LPAQA to other target models (we train the latter on BERT LARGE , since this is the LPAQA setup reaching the best same-sourceand-target accuracy).[Table] [Table]
Revised: Table 6 further shows that AutoPrompt also generalizes better than LPAQA to other target models (we train the latter on BERT LARGE , since this is the LPAQA setup reaching the best same-sourceand-target accuracy).

Original: D LAMA-UHN R ESULTS
Revised: D LAMA-UHN R ESULTS

Original: LAMA-UHN (UnHelpfulNames) (Poerner et al., 2019) is a subset of LAMA where facts that can be answered based on entity names alone were removed.
Revised: LAMA-UHN (UnHelpfulNames) (Poerner et al., 2019) is a subset of LAMA where facts that can be answered based on entity names alone were removed.

Original: The dataset is built using two heuristics.
Revised: The dataset is built using two heuristics.

Original: The first one filters facts whose object is a case-insensitive substring of the subject entity name.
Revised: The first one filters facts whose object is a case-insensitive substring of the subject entity name.

Original: The second heuristic deletes a triple if a given model can infer information such as native language, nationality or place of birth using the subject’s surface form only.
Revised: The second heuristic deletes a triple if a given model can infer information such as native language, nationality or place of birth using the subject’s surface form only.

Original: We use the original dataset with
Revised: We use the original dataset with

Original: BERT BASE as filtering model.
Revised: BERT BASE as filtering model.

Original: Table 7 shows that the generalization results still hold.
Revised: Table 7 shows that the generalization results still hold.

Original: The difference between BERT BASE and
Revised: The difference between BERT BASE and

Original: BERT BASE /T

Revised: BERT BASE /T

Original: LARGE is smaller, possibly due to a ceiling effect with this harder data-set, but (once we exclude the non-generalizing same-source-and-target BERT BASE case), it is statistically significant (paired t -test, α =0 . 05 ).
Revised: LARGE is smaller, possibly due to a ceiling effect with this harder data-set, but (once we exclude the non-generalizing same-source-and-target BERT BASE case), it is statistically significant (paired t -test, α =0 . 05 ).

Original: E P ROMPT EXAMPLES
Revised: E C HOOSING MODELS FOR MIXED - TRAINING

Revised: We study to what extent same-target-same-source accuracies of component models are good predictors of mixture generalization accuracy. To avoid a combinatorial explosion, we fix BERT BASE (the best model in the same-target-same-source setup) as either generator or evaluator, and pair it withdifferent LMs. The correlation across 15 mixtures is at 0.22 when BERT BASE is fixed as generator, and at 0.49 when it is fixed as evaluator.

Revised: Overall, this suggests that single-model setup performance is a good predictor of mixture quality.

Revised: We attribute the lower score with BERT-as-generator to a mild ceiling effect in this case.

Revised: Indeed, we observe that, on average, performance is better when using BERT BASE as generator than as evaluator.

Revised: This is likely due to the fact that the generator model does most of the “heavy lifting” in the mixture, as it is the one finding candidate prompts, that are then simply reranked by the evaluator.

Revised: It thus makes sense to use the best single-model-setup system used as the generator in a mixture.

Revised: [Table]

Revised: Finally

Revised: [Table]

Revised: when fixing BERT BASE as generator

Revised: [Table]

Revised: we observe that, in the majority of cases, we obtain better generalization results by using a different evaluator than BERT BASE itself, suggesting that mixed-training generalizes better compared to single model training even when randomly selecting one of the two models.

Revised: F P ROMPT EXAMPLES

Original: Table 8 shows manual (LAMA) and AutoPrompt prompts for a randomly picked set of T-ReX relations.
Revised: Table 8 shows manual (LAMA) and AutoPrompt prompts for a randomly picked set of T-ReX relations.

Original: We present examples for AutoPrompt trained on BERT BASE , the best choice in the samesource-and-target setup, and for mixed training using BERT BASE as generator model and T5 LARGE as evaluator (BERT BASE /T5 LARGE ), since this combination results in the best generalization performance.
Revised: We present examples for AutoPrompt trained on BERT BASE , the best choice in the samesource-and-target setup, and for mixed training using BERT BASE as generator model and T5 LARGE as evaluator (BERT BASE /T5 LARGE ), since this combination results in the best generalization performance.

Original: For comparison, we also present prompts obtained using GPT2 MEDIUM as source model, since this is a setup with relatively low performance in both the same-source-and-target and generalization evaluations.
Revised: For comparison, we also present prompts obtained using GPT2 MEDIUM as source model, since this is a setup with relatively low performance in both the same-source-and-target and generalization evaluations.

Original: We note that the two better models generally produce prompts that have at least some degree of semantic/lexical relevance to the relation.
Revised: We note that the two better models generally produce prompts that have at least some degree of semantic/lexical relevance to the relation.

Original: This might be to some degree the case for GPT2 MEDIUM as well, but much more opaquely so.
Revised: This might be to some degree the case for GPT2 MEDIUM as well, but much more opaquely so.

Original: Interestingly, for the two better models at least, the most transparent information is concentrated towards the end of the string.
Revised: Interestingly, for the two better models at least, the most transparent information is concentrated towards the end of the string.

Original: For example, the BERT BASE location prompt ends with headquartered in , and the corresponding BERT BASE /T

Revised: For example, the BERT BASE location prompt ends with headquartered in , and the corresponding BERT BASE /T

Original: LARGE prompt ends with headquarters in .
Revised: LARGE prompt ends with headquarters in .

Original: The BERT BASE manufacturer prompt ends with marketed by , and the corresponding BERT BASE /T

Revised: The BERT BASE manufacturer prompt ends with marketed by , and the corresponding BERT BASE /T

Original: LARGE prompt ends with maker .
Revised: LARGE prompt ends with maker .

Original: Both these observations (higher semantic relevance of better-performing prompts and higher information concentration at the end of the prompt string) are confirmed by the quantitative analyses reported in Section 4.4 of the main text.
Revised: Both these observations (higher semantic relevance of better-performing prompts and higher information concentration at the end of the prompt string) are confirmed by the quantitative analyses reported in Section 4.4 of the main text.

Original: By qualitative inspection, the only noticeable difference between the BERT BASE prompts (better in the non-generalizing setup) and the BERT BASE /T

Revised: By qualitative inspection, the only noticeable difference between the BERT BASE prompts (better in the non-generalizing setup) and the BERT BASE /T

Original: LARGE ones (better at generalization) is that the latter contain a higher proportion of well-formed English words (as opposed to concatenations of word pieces that do not result in existing words).
Revised: LARGE ones (better at generalization) is that the latter contain a higher proportion of well-formed English words (as opposed to concatenations of word pieces that do not result in existing words).

Original: Again, this is quantitatively confirmed by the analysis in Section 4.
Revised: Again, this is quantitatively confirmed by the analysis in Section 4.

Original: re l a t i o n
Revised: re l a t i o n

Original: M a nu a l
Revised: M a nu a l

Original: BE R
Revised: BE R

Original: T B A S E
Revised: T B A S E

Original: G P T 2 M E D
Revised: G P T 2 M E D

Original: I U M
Revised: I U M

Original: BE R
Revised: BE R

Original: T B A S E / T 5 L
Revised: T B A S E / T 5 L

Original: AR G E p l a ce
Revised: AR G E p l a ce

Original: o f d e a t h
Revised: o f d e a t h

Original: [ X ] d i e d i n [ Y ] [ X ] ll a nd fl ee s e x il e l e ss l y
Revised: [ X ] d i e d i n [ Y ] [ X ] ll a nd fl ee s e x il e l e ss l y

Original: [ X ] p s e udony m du a l (
Revised: [ X ] p s e udony m du a l (

Original: P a r i s a nd [ Y ] [ X ] t eac
Revised: P a r i s a nd [ Y ] [ X ] t eac

Original: h e
Revised: h e

Original: s m od e lli ng
Revised: s m od e lli ng

Original: fr e s c
Revised: fr e s c

Original: o e s do w n t
Revised: o e s do w n t

Original: o w n [ Y ] do w n t
Revised: o w n [ Y ] do w n t

Original: o w n i n [ Y ] r e li g i on [ X ] i s a f fi li a t e d w it
Revised: o w n i n [ Y ] r e li g i on [ X ] i s a f fi li a t e d w it

Original: h
Revised: h

Original: [ X ] b e li e v e un i d e n ti fi e d f und a m e n t a l [ X ] ” : ” / gnu \ u9 f 8d \ u fff d c on s tit u t e s [ X ] dyn a s t
Revised: [ X ] b e li e v e un i d e n ti fi e d f und a m e n t a l [ X ] ” : ” / gnu \ u9 f 8d \ u fff d c on s tit u t e s [ X ] dyn a s t

Original: y m o s qu e s
Revised: y m o s qu e s

Original: h e l p e d t h e [
Revised: h e l p e d t h e [

Original: Y ] r e li
Revised: Y ] r e li

Original: g
Revised: g

Original: i on r e li
Revised: i on r e li

Original: g i on oppo s it e
Revised: g i on oppo s it e

Original: [ Y ] o r t hodox [ Y ] p eace f
Revised: [ Y ] o r t hodox [ Y ] p eace f

Original: u ll y s p
Revised: u ll y s p

Original: r
Revised: r

Original: ea d [ Y ] l o c a ti on [ X ] i s l o ca t e d i n [ Y ] [ X ] c on t r a s t s s l a l o m g r a ph e r s
Revised: ea d [ Y ] l o c a ti on [ X ] i s l o ca t e d i n [ Y ] [ X ] c on t r a s t s s l a l o m g r a ph e r s

Original: [ X ] i n f og r a ph
Revised: [ X ] i n f og r a ph

Original: i c b i k
Revised: i c b i k

Original: i ng & [ X ] s p
Revised: i ng & [ X ] s p

Original: ec i a li s e d i ng l y b a s e d h ea dqu a r t e
Revised: ec i a li s e d i ng l y b a s e d h ea dqu a r t e

Original: r e d i n [ Y ] acc o mm od a ti on i n [ Y ] h ea dqu a r t e r
Revised: r e d i n [ Y ] acc o mm od a ti on i n [ Y ] h ea dqu a r t e r

Original: s i n [ Y ] c oun t
Revised: s i n [ Y ] c oun t

Original: r y o f
Revised: r y o f

Original: o r
Revised: o r

Original: i g i n [ X ] w a s c r ea t
Revised: i g i n [ X ] w a s c r ea t

Original: e
Revised: e

Original: d i n [ Y ] [ X ] c
Revised: d i n [ Y ] [ X ] c

Original: o m e s v e r s a pub li s h e d t oon
Revised: o m e s v e r s a pub li s h e d t oon

Original: [ X ] e s t y l e r
Revised: [ X ] e s t y l e r

Original: i v a l r y
Revised: i v a l r y

Original: [ G e r m a ny a n d [ Y ] [ X ] s t r ea m e d w o r l d
Revised: [ G e r m a ny a n d [ Y ] [ X ] s t r ea m e d w o r l d

Original: s e m i fi n a l ou t s i d e
Revised: s e m i fi n a l ou t s i d e

Original: [ Y ] b a d
Revised: [ Y ] b a d

Original: m i n t on r e p r e s e n ti ng
Revised: m i n t on r e p r e s e n ti ng

Original: [ Y ] m anu f a c t u r e r
Revised: [ Y ] m anu f a c t u r e r

Original: [ X ] i s p r odu ce d by [ Y ]
Revised: [ X ] i s p r odu ce d by [ Y ]

Original: [ X ] i s n a ltit
Revised: [ X ] i s n a ltit

Original: ud e s b i n a r y
Revised: ud e s b i n a r y

Original: [ X ] 650 w a g e n
Revised: [ X ] 650 w a g e n

Original: J ohn s on a s a k
Revised: J ohn s on a s a k

Original: i
Revised: i

Original: p
Revised: p

Original: h
Revised: h

Original: a b e t [ Y ] .
Revised: a b e t [ Y ] .

Original: [ X ] d e v i s e d by b illi on fi e l d i ng m a r k e t e d by [ Y ] m a k e r
Revised: [ X ] d e v i s e d by b illi on fi e l d i ng m a r k e t e d by [ Y ] m a k e r

Original: [ Y ]
Revised: [ Y ]

Original: T a b l e 8 :
Revised: T a b l e 8 :

Original: E x a m p l e p r o m p t s f o r a r a ndo m
Revised: E x a m p l e p r o m p t s f o r a r a ndo m

Original: T - R e
Revised: T - R e

Original: X r e l a ti on s ub s e t .
Revised: X r e l a ti on s ub s e t .

Original: T h e
Revised: T h e

Original: M a nu a l c o l u m n s ho w s m a nu a ll
Revised: M a nu a l c o l u m n s ho w s m a nu a ll

Original: y - c
Revised: y - c

Original: r a f t e d
Revised: r a f t e d

Original: L A
Revised: L A

Original: M A p r o m p t s .
Revised: M A p r o m p t s .

Original: T h
Revised: T h

Original: e f o ll o w i ng t
Revised: e f o ll o w i ng t

Original: w o c
Revised: w o c

Original: o l u m n s s ho
Revised: o l u m n s s ho

Original: w p r o m
Revised: w p r o m

Original: p t s g e n e r a t e d by
Revised: p t s g e n e r a t e d by

Original: A u t o
Revised: A u t o

Original: P r o m p t u s i
Revised: P r o m p t u s i

Original: ng a s s ou r ce m od e l s
Revised: ng a s s ou r ce m od e l s

Original: B E R
Revised: B E R

Original: T B A S
Revised: T B A S

Original: E a nd
Revised: E a nd

Original: G P T 2 M E D
Revised: G P T 2 M E D

Original: I U M , r e s p ec ti v e l y .
Revised: I U M , r e s p ec ti v e l y .

Original: T h
Revised: T h

Original: e l a s t c o l u m n s ho w s p
Revised: e l a s t c o l u m n s ho w s p

Original: r
Revised: r

Original: o m p t g e n e r a t e d t h r ough
Revised: o m p t g e n e r a t e d t h r ough

Original: A u t o
Revised: A u t o

Original: P r o m p t m i x e d t r a i n i ng , u s i ng
Revised: P r o m p t m i x e d t r a i n i ng , u s i ng

Original: B E R
Revised: B E R

Original: T B A S
Revised: T B A S

Original: E a s g e n e r a t
Revised: E a s g e n e r a t

Original: o r m od e l a nd
Revised: o r m od e l a nd

Original: T 5 L A
Revised: T 5 L A

Original: R G E a s e v a l u a t
Revised: R G E a s e v a l u a t

Original: o r m od e l .
Revised: o r m od e l .

Original: I n a ll ca s e s , [ X ] a nd [ Y ] s t a nd f o r t h e s ub j ec t a nd ob j ec t s l o t s .
Revised: I n a ll ca s e s , [ X ] a nd [ Y ] s t a nd f o r t h e s ub j ec t a nd ob j ec t s l o t s .

Original: [Figure]
Revised: [Figure]

Original: F T OKEN DELETION FULL RESULTS
Revised: G T OKEN DELETION FULL RESULTS

Original: Figure 5 reports the T-ReX accuracy for AutoPrompt prompt sets obtained using different LMs or
Revised: Figure 5 reports the T-ReX accuracy for AutoPrompt prompt sets obtained using different LMs or

Original: LM combinations, always tested on the LM used for AutoPrompt training (the generator LM in mixed-training setups), when one token at a time is removed from the full prompt sequence.
Revised: LM combinations, always tested on the LM used for AutoPrompt training (the generator LM in mixed-training setups), when one token at a time is removed from the full prompt sequence.