P-A DAPTERS : R OBUSTLY E XTRACTING F ACTUAL I NFORMATION FROM L ANGUAGE M ODELS WITH D IVERSE P ROMPTS
Anonymous authors
Paper under double-blind review
A BSTRACT
Recent work (e.g. LAMA (Petroni et al., 2019)) has found that the quality of the factual information extracted from Large Language Models (LLMs) depends on the prompts used to query them.
This inconsistency is problematic because different users will query LLMs for the same information using different wording, but should receive the same, accurate responses regardless.
In this work we aim to address this shortcoming by introducing P-Adapters : lightweight models that sit between the embedding layer and ﬁrst attention layer of LLMs.
They take LLM embeddings as input and output continuous prompts that are used to query the LLM.
Additionally, we investigate Mixture of Experts (MoE) models that learn a set of continuous prompts (“experts”) and select one to query the LLM.
They require a separate classiﬁer trained on human-annotated data to map natural language prompts to the continuous ones.
P-Adapters perform comparably to the more complex MoE models in extracting factual information from BERT and RoBERTa while eliminating the need for additional annotations.
PAdapters show between 12-26% absolute improvement in precision and 36-50% absolute improvement in consistency over a baseline of only using natural language queries.
Finally, we investigate what makes a P-Adapter successful and conclude that access to the LLM’s embeddings of the original natural language prompt, particularly the subject of the entity pair being asked about, is a signiﬁcant factor.
I NTRODUCTION
Recently, there has been an interest in meeting users’ factual information needs using large language models (LLMs) as knowledge bases in place of traditional, index-based IR methods (Petroni et al., 2019; Metzler et al., 2021).
In this conception of LLM-based knowledge bases, LLMs accumulate factual knowledge during pretraining, have their parameters frozen to maintain this knowledge, and then are queried by users using handcrafted, natural language prompts.
For example, if one user wants to know what the capital of Canada is, they might query a frozen model with the prompt, “The capital of Canada is [MASK] .”, while another might use different wording: “Canada, which has the capital city [MASK] .”
In order for LLMs to be effective knowledge bases, they have to be robust to different queries users could provide.
Unfortunately, prior work has shown that LLMs are not robust: queries that are semantically equivalent may lead to inconsistent predictions (Jiang et al., 2020; Elazar et al., 2021).
For example, taking the prompts above, the ﬁrst prompt does extract the correct answer “Ottawa” from BERT Base, but the second extracts the incorrect “Winnipeg”.
This observation has led many works to try to ﬁnd the optimal prompt or set of prompts for a given relation: the one(s) that will allow models to extract factual information the best (e.g., for the relation capital of ).
These works have proposed ensembling sets of prompts (Jiang et al., 2020), optimizing which tokens are included in the prompt (Haviv et al., 2021; Shin et al., 2020), or forgoing the difﬁcult discrete optimization problem entirely and instead optimizing the continuous embeddings that are input to the LLM (Li & Liang, 2021; Qin & Eisner, 2021; Zhong et al., 2021).
In our work, rather than focus on ﬁnding a single optimal prompt, we aim to help LLMs overcome this variability by adapting natural language prompts into continuous representations that allow
LLMs to accurately predict factual information.
Because we are motivated by a user-focused setting, [Table] we want our adaptation method to require only a natural language prompt (e.g. “The capital of
Canada is [MASK] ”) at inference time. No additional annotation for the relation ( capital of ) between the subject (“Canada”) and object (“Ottawa”) of the entity pair in the prompt should be required. Nor should we need the identity of the subject separate from the prompt.
At training time, our ideal method would only rely on (prompt, object) pairs (e.g. ( “The capital of Canada is [MASK] ”, “Ottawa” ) ), so collecting new data would not require additional annotation.
Our research question is then as follows: in a setting where we are trying to prompt a frozen LLM using natural language alone, how can we encourage models to produce more consistent and accurate responses to potentially varied prompts.
We introduce a class of models called P-Adapters (short for “prompt adapters”) to perform this adaptation, which satisfy our inference and training desiderata.
P-Adapters sit between the embedding layer and the ﬁrst attention layer of the frozen LLM, and modify the LLM’s embeddings so factual information can be more effectively predicted (Figure 1).
They are optimized end-to-end, only requiring (prompt, object) pairs at training time, and implicitly encourage consistency by learning to map variable training prompts to the same object.
We also investigate other models that could increase the consistency of natural language prompts: Mixture of Experts (MoE) models.
Rather than adapt a natural language prompt, a MoE model maps it to an “expert” continuous prompt based on the relation between the prompt’s subject and object.
The MoE model then queries the frozen LLM with that expert.
This method reduces variability by mapping all natural language prompts concerned with the same relation to a single prompt optimized for that relation.
These methods require additional annotations during training and inference (namely the relation between the entities and the identity of the entity subject), and therefore do not conform to our desiderata.
However, we include them in our discussion because they are useful for comparing to previous work and for understanding why P-Adapters are successful.
We ﬁnd that P-Adapter and MoE methods both strongly outperform a baseline of using the natural language prompts alone by between 12-28% on precision and 36-86% on consistency.
While MoE models usually perform better, the small increase comes at the cost of requiring relation annotations at training time and specifying the subject of the entity at inference time.
To simulate the variable kinds of natural language prompts we expect our methods to handle, we evaluate them in three out-ofdistribution (OOD) settings: on different set of natural language prompts, on a different distribution of entities (uniform rather than the distribution of Wikidata), and on natural language prompts with typographical errors.
These OOD evaluations are especially important in light of recent work has found that models that appear to have knowledge of factual information often rely on surface-level feature associations.
For example, models ignore negation, utilize stereotypical associations when making predictions (Kassner & Sch ¨ utze, 2020; Poerner et al., 2020), and pick up on the distribution of objects from their training corpus rather than learn how to access arbitrary facts (Zhong et al., 2021; Cao et al., 2021).
We ﬁnd that the OOD distribution of entities is the most challenging setting, and while typographic errors reduce model performance, they do so by less than might be expected.
Finally, we take a deep dive into what makes a P-Adapter effective, ﬁnding that it is important to keep some of the original natural language prompt embeddings available to the LLM.
In particular, the availability of the subject of the prompt is the most important factor, in contrast with some other work suggesting the subject matters less (Cao et al., 2021).
Overall, we conclude that our method is helpful in reducing the impact of variable prompts, and is able to do so without requiring additional annotations.
R ELATED W ORK
Prompting Language Models for Factual Knowledge.
Prompting has taken off as a lowparameter alternative to ﬁnetuning LLMs.
In prompting, the LLMs are usually frozen and their pretraining task is used to ﬁll in the desired information, which is usually a single token (Trinh & Le, 2018; Davison et al., 2019; Petroni et al., 2019).
Cao et al.
(2021) outlines three main approaches for prompting LLMs for factual information.
The one we focus on here involves querying them zero-shot with a prompt that contains the single entity of interest (like in Figure 1) (Petroni et al., 2019).
Much previous work tries to lower-bound how much factual information LLMs store.
Petroni et al.
(2019) use one natural language prompt for each relation, while subsequent work argues that this approach underestimates LLM’s abilities given how sensitive they are to input prompts.
To this end, Jiang et al.
(2020) and Elazar et al.
(2021) generate paraphrases of prompts in the LAMA dataset and ensemble them to get a score for each relation.
Others argue for optimizing discrete (Shin et al., 2020; Haviv et al., 2021) or continuous (Qin & Eisner, 2021; Zhong et al., 2021; Liu et al., 2021b) prompt for each relation instead of working with language ones.
Elazar et al.
(2021) thoroughly investigate prompt sensitivity in this setting, ﬁnding models often predict different entities for semantically equivalent prompts.
Further, they propose continuing to train LLMs with a consistency loss function to improve their robustness.
In this work, however, we focus on a user-centric and lightweight approach that does not require updating LLM parameters.
Robustness and Extracting Facts.
Training robustness models—ones that are invariant to semantics-preserving input perturbations—is a challenge in many settings (Szegedy et al., 2013;
Belinkov & Bisk, 2018) and has to led to a lot of theoretical and empirical innovations, such as adversarial training (Sinha et al., 2017), provably robust models (Dvijotham et al., 2018), or more robust encoding schemes (Jones et al., 2020).
Our works falls into this last category. In NLP, prior work investigates invariances to typos, word substitutions, irrelevant sentences, and syntactic perturbations (Jones et al., 2020; Alzantot et al., 2018; Jia & Liang, 2017; Iyyer et al., 2018). For robustness in extracting factual information, Poerner et al.
(2020) and Kassner & Sch ¨ utze (2020) argue that LLMs exploit surface form regularities when making predictions.
Zhong et al.
(2021) and Dufter et al.
(2021) make a related observation, concluding that simpler models like randomly initialized LLMs, static embeddings, and even a Naive Bayes model can achieve a precision better than a majority baseline.
Cao et al.
(2021) argue that prompts that were found to do well in previous work overﬁt the distribution of objects in the training data rather than enabling knowledge extraction.
They show that prompting with different sets of entities leads to very similar predictions.
Therefore, we adopt an OOD evaluation setting to address these concerns.
Low parameter ﬁnetuning methods (Houlsby et al., 2019;
Ben Zaken et al., 2021; Liu et al., 2021a) have also been shown to have better OOD performance for generation and classiﬁcation tasks (Li & Liang, 2021; Lester et al., 2021), which we hope to leverage for factual extraction here.
T ERMINOLOGY
Taking inspiration from Liu et al.
(2021a), we use the following terminology throughout this work.
Each fact we want to extract consists of two parts: an entity pair —a subject ( x ) and an object ( y )— and the relation ( r ) between them.
We use the 41 relations in the LAMA dataset (Petroni et al., 2019).
Each relation is expressed through a number of templates (e.g., one might be “The capital of [X] is [Y].”).
To query a language model for factual information, we substitute the subject in for the “[X]” in the template, and a [MASK] token into the object’s place (the “[Y]”) to form a prompt ( x (cid:48) ).
We refer to this as a natural language prompt to emphasize that it consists of natural language tokens.
Our prediction problem then consists of mapping x (cid:48) to y .
LLMs perform this mapping by using their embedding layer, e , to embed x (cid:48) into a sequence of continuous vectors, and then inputting this sequence to their ﬁrst attention layer.
We refer to this sequence of continuous vectors that are input to the LLM as a continuous prompt ( x (cid:48) cont ).
We make this distinction because our P-Adapters sit between the embedding layer and the ﬁrst attention layer of the LLM, so they take as input e ( x (cid:48) ) and output a continuous prompt x (cid:48) cont that differs from e ( x (cid:48) ) .
After passing x (cid:48) cont into the rest of the LLM, we take the LLM’s prediction ( y (cid:48) ) to be the token assigned the highest probability in the location of the [MASK] token in x (cid:48) , and compare it to y to assess if the model knows the fact. {+Formally, a model is consistent if for any two prompts x (cid:48) 1 , x (cid:48) 2 for the same ( x, y, r ) triple, its top-1 predictions y (cid:48) 
 and y (cid:48) 2 are equal.  M ETHODS+}
To test extracting factual knowledge (rather than overﬁtting to training templates or entity pair distributions), we evaluate our P-Adapters in four settings: ID templates and objects for testing ID generalization. OOD Prompts for testing generalization to novel natural language prompts. OOD Objects for testing whether our P-Adapter models learn to match the distribution of objects in the training set rather than predict arbitrary object entities.
OOD Keyboard Errors for testing robustness to typos in the natural language prompts.
D ATA Entity Pairs. We use the entity pairs and relations from the T-Rex split of the LAMA work (Elsahar et al., 2018; Petroni et al., 2019) in our experiments.
This data includes 41 relations and approximately 1000 entity pairs for each relation sampled from Wikidata, including citations to where the relation manifests in Wikipedia.
We use this data for evaluation. For training and validation, we use the separate sets of entity pairs for each relation collected by Shin et al. (2020), which they use to optimize their discrete prompts.
The entities pairs in the training, validation, and evaluation sets are all disjoint, though the distribution of object entities is similar. For the OOD Objects setting, we use the entity pairs in the uniform-wikidata dataset from Cao et al. This dataset was constructed so that each object appears the same number of times for each relation, in contrast to the
ID evaluation set which contains a more uneven distribution of objects from Wikidata.
Templates.
The templates we use are pooled from prior work: LAMA, LPAQA, and ParaRel datasets (Jiang et al., 2020; Elazar et al., 2021).
LPAQA includes templates created automatically with a paraphrase model, mined from Wikipedia, and written by annotators, and ParaRel contains high-quality human-written templates.
Additionally, we augment the ParaRel templates using the BERT lexical substitution system from Lee et al.
(2021) by generating ﬁve paraphrases for each template, and then manually ﬁltering out generations that do not preservens semantics.
This gives us on average 81.4 prompt templates per relation (For more statistics, see Figure 6).
We split the templates into two equal-sized groups: one for training and one for OOD Prompt evaluation.
For the OOD Keyboard Errors setting, we use the training templates, except we introduce at least one typographic error in each template using the nlpaug package (Ma, 2019).
4. M ODELS
First we introduce our P-Adapter models, and then we introduce our MoE models, and we ﬁnish with our oracle and baseline models.
Recall that P-Adapters intercede between the LLM’s embedding layer, e , and the ﬁrst attention layer (Figure 1).
We extract information from BERT Base Cased, BERT Large Cased, and RoBERTa Large (Devlin et al., 2018; Liu et al., 2019). Because these models’ pretraining corpera contain Wikipedia, they should have access to the facts we test. P-Adapter.
Our P-Adapter models take as input the embeddings of the natural language prompt, e ( x (cid:48) ) , and output a new sequence of continuous embeddings, x (cid:48) cont , that are used as input to the LLM
[Table]
(cid:37)(cid:50)(cid:53)(cid:49)(cid:66)(cid:44)(cid:49) (cid:17)(cid:17)(cid:17)(cid:3) (cid:8)(cid:42)(cid:39)(cid:31)(cid:1)(cid:19)(cid:32)(cid:39)(cid:28)(cid:47)(cid:36)(cid:42)(cid:41) (cid:16)(cid:45)(cid:28)(cid:30)(cid:39)(cid:32) (cid:44)(cid:54)(cid:66)(cid:38)(cid:36)(cid:51)(cid:44)(cid:55)(cid:36)(cid:47) (cid:37)(cid:50)(cid:53)(cid:49)(cid:66)(cid:44)(cid:49) (cid:17)(cid:17)(cid:17)(cid:3)
[Figure]
in place of e ( x (cid:48) ) .
Formally, a P-Adapter is a function f P-Adapter : e ( x (cid:48) )
→ x (cid:48) cont trained to maximize P LM ( y | x (cid:48) cont ) .
The LLM’s prediction would then be
[Equation]
where V is the LLM vocabulary.
There are many different possible parameterizations of f P-Adapter , and we describe three here.
Two (Rewrite P-Adapter and Preﬁx P-Adapter) require no additional annotations, ﬁtting the training and inference data criteria put forth in the introduction.
The third (P-Tuning P-Adapter) does not ﬁt our criteria, but we include it because it provides insight into what makes a P-Adapter effective (See Figure 2a for an illustration of each.)
The simplest adapter we investigate is the Rewrite P-Adapter .
The Rewrite P-Adapter is parameterized by a 384-dimensional bidirectional LSTM followed by a single MLP applied to each hidden state independently.
The output of the MLP is x (cid:48) cont , and is input to the LLM.
We use LSTMs to better compare our results with those of Liu et al.
(2021b) who also use them, and because we want to use only a small number of parameters.
Additionally, the prompts tend to be short and we do not do any pre-training, so LSTMs are effective.
Next we look at the Preﬁx P-Adapter .
The Preﬁx P-Adapter is parameterized similarly to the
Rewrite P-Adapter; however, its output is a preﬁx of embeddings that are prepended to e ( x (cid:48) ) .
Like other works that learn prompt preﬁxes, the preﬁx has a ﬁxed length (we use nine to compare with later models)
(Li & Liang, 2021; Lester et al., 2021).
Formally, using the Preﬁx P-Adapter, the input to the LLM is [ g ( e ( x (cid:48) )); e ( x (cid:48) )] , where g is the application of a 384-dimensional Bi-LSTM, a max-pool, and an MLP to project the max-pool’s output into the size of the preﬁx.
Unlike the Rewrite P-Adapter, the Preﬁx P-Adapter keeps all of the embeddings e ( x (cid:48) ) accessible to the LLM’s ﬁrst attention layer.
Finally, we investigate the P-Tuning P-Adapter .
This P-Adapter is based on work by Liu et al.
(2021b) which presents a method called P-Tuning that learns a continuous prompt for each relation in the LAMA dataset.
They learn a function g : r → R d × 9 , where d is the hidden dimension of the LLM.
The x (cid:48) cont consists of the output of g , the LLM’s unmodiﬁed embedding of the subject, and the LLM’s unmodiﬁed embedding of the mask token.
It has the following form:
[Equation]
where bracket notation represents python-style indexing.
Our P-Tuning P-Adapter is implemented in a similar fashion, except our g takes as input e ( x (cid:48) ) rather than r , and is parameterized similarly to the Preﬁx P-Adapter.
Like the Preﬁx P-Adapter, a ﬁxed number of embeddings are output; however, unlike the Preﬁx P-Adapter, only the embeddings for the subject and [MASK] token are kept unchanged, rather than all of e ( x (cid:48) ) .
Note that this P-Adapter uses more annotations during training and inference than the other two: speciﬁcally, it requires knowing the identity of the subject.
This means that it does not ﬁt our desiderata laid out in the introduction, but we include because it allows us to investigate what makes a P-Adapter successful.
Mixture of Experts and Oracle.
Enforcing consistency does not require modifying a natural language prompt like the P-Adapters do.
Another option is to map each natural language prompt to a canonical continuous prompt, and query the frozen LLM with this prompt.
We explore this next with our MoE and Oracle methods.
In these, the canonical prompt is a continuous prompt optimized for the relation between the entities in the natural language prompt Liu et al.
(2021b).
These continuous prompts are learned using P-Tuning, and have the same format as described previously in the P-Tuning P-Adapters section; they ignore e ( x (cid:48) ) , depending only on the x , MASK , and r (Liu et al., 2021b).
We train these continuous prompts with P-Tuning using the same method and hyperparamters reported in Liu et al.
(2021b), with one prompt for each of the 41 relations. In contrast to the P-Adapter methods, these require additional annotations with the relation of the prompt (See Figure 2b).
Our Mixture of Experts (MoE) model consists of two components.
The ﬁrst is a classiﬁer that predicts the relation between the entities of a natural language prompt.
This classiﬁer is a BERT Base Cased model ﬁne-tuned on a 41-way classiﬁcation task.
The second component is a look-up table to map the predicted relations to the canonical continuous prompts.
Note that this is similar to a traditional mixture of experts model approach except that we do not use a weighted combination of prompts from different relations and instead just use the single prompt from the predicted relation.
Our Oracle method is similar to the MoE approach, except rather than using a classiﬁer to predict the relation, we use the gold relation at inference time.
We call this method an oracle because it achieves perfect consistency—for all prompts with the same entity pair it will make the same prediction.
The oracle also serves as a direct comparison to the prior work of P-Tuning because P-Tuning assumes access to the gold relation.
Baseline.
Finally, our baseline is a model that takes as input the natural language prompt without any preﬁxes or optimization.
Any effective and useful P-Adapter network would have to perform better than just using the natural language prompt itself.
4. M ETRICS
We report two metrics: precision@1 (P@1) and consistency.
P@1 is a stringent measure of whether a prompt can extract a fact.
It is deﬁned as the proportion of prompts where the correct object is the LLM’s top prediction.
Consistency measures whether a model’s predictions for a given entity pair match across the different prompts x (cid:48) ’s that contain the entities.
We calculated consistency using the method from Elazar et al.
(2021): the consistency of knowledge of a fact is the proportion of pairs of prompts for the fact where the model makes the same prediction.
Formally, given a set of unordered pairs of prompts with the same entity pair, X (cid:48) , where there are n unique prompts, and letting v ( x (cid:48) )
=
arg
[Equation]
P
[Equation]
v
[Equation] f [Equation] ( e ( x (cid:48) ))) return the model’s top prediction given a natural language prompt x (cid:48) , the consistency is deﬁned as:
[Equation]
Note: a model can be consistent and inaccurate by always predicting the same incorrect object.
[Table]
are microaveraged over all relations.
(OOD KE: OOD Keyboard Errors).  R ESULTS
We present our main results for BERT Base in Table 1.
The results for BERT Large and RoBERTa
Large show similar trends and are available in the Appendix (Tables 2, 3, Figures 4 and 5; Qualitative Results are in Appendix D).
Across all evaluation settings, we ﬁnd optimized prompts lead to higher precision than natural language ones, as the low performance of the baseline indicates.
Precision.
Comparing the different evaluation settings, we observe the following.
First, the OOD Objects was the most challenging setting, on average 20.41% lower precision than the ID setting, even for the oracle.
However, the models performed similarly on the OOD Prompts as they did on the ID ones.
At ﬁrst this might seem to conﬂict with previous work that ﬁnds that the prompt has a large impact on performance (Jiang et al., 2020), but these results make claims about average prompt performance rather than individual prompt performance.
That said, precision is still higher ID than
OOD, particularly for the MoE model.
Looking at the MoE’s relation classiﬁer predictions, we can see it achieves an F1 of 99% for the ID and OOD Objects settings, but only 81% for OOD Prompts (See Appendix F).
While this suggests some overﬁtting, our methods still outperform the baseline, so what they learn is still useful for the OOD natural language prompts.
Finally, when evaluated on the OOD Keyboard Errors, we ﬁnd that models still outperform the baseline.
However, precision on the corrupted prompts is lower than the uncorrupted ones, and drops by a larger absolute percentage than the baseline does.
Consistency.
Additionally, across all variation types we can see that optimized prompts lead to more consistency among the models’ predictions.
The consistencies dramatically increase from less than 0 .
2 for the baseline to over 0 .
4 (Table 1).
Models were the least consistent on the OOD Keyboard Errors, and interestingly, were similarly consistent between the OOD Prompts and OOD Objects evaluations despite having a much higher precision for the OOD Prompts.
The oracle has a perfect consistency of 1 . 0 , because it uses the same continuous prompt for all facts with the same relation, so the predictions for that fact are all the same.
The MoE model has a high consistency for a similar reason: as long as the classiﬁer predicts two prompts for the same entity pair come from the same relation, the predictions will be the same, and therefore consistent.
The P-Tuning P-Adapter often has a a high consistency as well, sometimes even higher than the MoE. This suggests that in P-Tuning P-Adapter, the main factor driving the model’s prediction is the subject of the entity pair, because the unmodiﬁed embedding of the subject is shared among all of the prompts for the same object. In the P-Adapters, this consistency is implicitly encouraged by the current training procedure by training P-Adapters to map from various natural language prompts to the same object. We investigate adding an additional loss term to explicitly encourage consistency and ﬁnd that doing so increases consistency by 5% on average.
This suggests that implicitly encouraging consistency can be helpful but is not necessary.
The procedure and results are detailed in Appendix E. Unmodiﬁed Embeddings. We observe that giving the LLM access to its unmodiﬁed embeddings of the natural language prompt was helpful.
The Preﬁx P-Adapter and the P-Tuning P-Adapter, which have access to some or all of these embeddings, achieve higher precisions than the Rewrite P-Adapter, which alters all of them.
Because of this, we perform additional experiments to see what part of natural language prompt was important to keep.
One hypothesis is that the [MASK] token embedding has to remain unmodiﬁed, and the Rewrite PAdapter performs poorly because it corrupts this embedding.
We investigate this in two ways.
First, we train a version of the Rewrite P-Adapter that replaces the P-Adapter’s output in the [MASK] token position with the LLM’s [MASK] token embedding.
Second, we train a version of the P-Tuning
P-Adapter whose output only has the LLM’s [MASK] embedding, not the subject embedding.
However, we ﬁnd these do not increase performance much compared to the Rewrite P-Adapter.
Another hypothesis is that the subject embeddings must remain unmodiﬁed, and that the Rewrite
P-Adapter corrupts the subject, leading to lower precision.
To address this, we train a version of the Rewrite P-Adapter model that replaces the embeddings for the subject in the P-Adapter output with the LLM’s embeddings for the subject.
We ﬁnd this is much more effective, increasing precision by 13% .
For comparison with the P-Tuning P-Adapter, which keeps the subject and [MASK] embeddings the same, we also train a version of the Rewrite P-Adapter that replaces the embeddings at both the subject and [MASK] token positions with the LLM’s embeddings and ﬁnd that this performs only 4% worse than the P-Tuning P-Adapter.
The results are visible in Figure 3.
From these experiments, we conclude that the most important aspect of the prompt is the subject token.
This also helps explain why the Preﬁx P-Adapter does well: it has access to the LLM’s embeddings for the entire natural language prompt, which includes the subject.
Unfortunately, while these changes to the Rewrite P-Adapter models show promising results, they require knowing the index of the subject tokens at training and inference time, which contradicts our requirement to not use extra annotations.
Fortunately, there is another way to incorporate the unmodiﬁed LLM’s embeddings of the natural language prompt into the output of the P-Adapter: we can interpolate between the P-Adapter output and the unmodiﬁed embeddings.
To do this, we train a third type of Rewrite P-Adapter.
If f rewrite-adapter is the original Rewrite P-Adapter, the equation for the new Rewrite P-Adapter is:
[Equation]
where α is a hyperparameter.
When α = 0 , this P-Adapter is equivalent to the original Rewrite
P-Adapter, and when α = 1 it is equivalent to the baseline.
We test α ∈ { 0 . 1 , 0 . 25 , 0 . 5 , 0 .
75 , 0 . 9 } , and ﬁnd α = 0 .
5 to perform the best.
It outperforms the Rewrite P-Adapter when the subject and [MASK]
tokens are substituted in, though barely underperforms compared to the P-Tuning
P-Adapter (See Figure 3).
D ISCUSSION
Our goal was to create models to adapt arbitrary user queries so predictions from frozen LLMs were accurate and consistent.
One of our desiderata was to train with only (prompt, object) pairs, so that a user’s experience would match current IR system experiences.
Despite this requirement, some of the methods we introduce do use additional information—for comparison to other methods and understanding our own.
For example, the P-Tuning P-Adapter and the MoE models both need the identity of the subject of the user’s prompt, and requiring a user to provide this feels cumbersome and redundant.
One could obviate this need using an off-the shelf NER system as well, and we leave exploring this to future work.
Additionally, the MoE models train their classiﬁers using annotations for the relation between the entities in the prompts.
We use these annotations to compare to previous work: previous work assumes that relation is the most useful intermediate variable for factual extraction, but this does not have to be the case.
For example, good prompts across different relations like “Ottawa is the capital of [MASK] ” and “The second largest country by land area is [MASK] ” might share components even though they do not share a relation.
Our P-Adapters allow for greater ﬂexibility by conditioning on the natural language prompt itself rather than the relation.
Importantly, while using extra information leads to better results in some cases, the Preﬁx P-Adapter usually achieves comparable precision.
And while the Rewrite P-Adapter ﬂounders, ensembling its outputs with the LLM embeddings leads to greater success.
N one [MASK ] [MASK ]
S ub j e c t
S ub j e c t + [
MASK]
S ub
[Figure]
+ [
MASK] [Figure] [Figure]
[Figure] [Figure] [Figure] [Figure]
Unmodiﬁed Embeddings
Rewrite
P-Tuning
[Figure]
vs [Figure] We speculate that there are two likely reasons for the P-Adapters’ success. First, not updating the LLM parameters likely maintains factual knowledge learned during pretraining—Elazar et al.
(2021) show that continuing to ﬁnetune the models on extracting factual information can lead to lower accuracy.
Second, the natural language prompts are likely close to being effective prompts, so small changes from P-Adapters are sufﬁcient to elicit the correct objects.
It is also worth noting that while the Preﬁx P-Adapter approaches the performance of the techniques with more information, this performance is still quite modest.
The oracle was able to achieve only a precision of 50% in the cases with ID objects, missing half of the facts.
For the OOD Objects evaluation set, it performed much worse, only correctly predicting about 25% of the facts.
While the P@1 metric is quite strict, these numbers do not bode well for this approach.
Cao et al.
(2021) blame this poor performance on the optimized prompts overﬁtting to the training object distribution, and they observe that when the subjects are changed, the LLM’s distributions do not change all that much.
They even observe that the LLM’s distributions when prompts have subjects are similar to those when the subjects are completely omitted.
Our ﬁndings point to a more nuanced result.
While it might be true that the LLM’s distributions in these cases are correlated, the subject does matter when considering the top prediction.
We ﬁnd that without the LLM’s embeddings of the subject accessible, the P-Adapters performs even more poorly.
C ONCLUSION
LLMs implicitly learn factual information during pretraining, but accurately and consistently extracting this information is challenging.
Our work complements previous work on extracting factual information from LLMs by introducing a more user-oriented setting: we want our models to only take in varied natural language prompts, and return the objects that meet information needs.
To do this, we propose P-Adapter models, which sit between the embedding layer and the ﬁrst attention layer of LLMs.
These are effective compared to other methods that require additional annotations (e.g., the relation between the entities in the prompt), and we perform ablations determining that their success is due to keeping the LLM’s embeddings of the subject available to the LLM.
While we focus on the task of extracting factual information from LLMs, P-Adapters potentially provide a general framework for adapting to variable inputs, for example in reading comprehension or dialogue.
While there is still room for the further improvement in using LLMs as knowledge bases, and we see P-Adapters as an important part of the future of this endeavor.
E THICS S TATEMENT
While our work is inspired by how users interact with current IR systems we do not perform any experiments with human subjects.
Additionally, our method improves the ability to extract factual information from pretraining corpora, so this means it may be possible to exploit our method to elicit the biased or harmful predictions from these corpora as well, though we do not study this.
Finally, improving the ability to accurately and consistently extract information from text sources has privacy concerns if LLMs are pretrained on private or otherwise sensitive data.
For the LLMs we study, this is not a concern as they were trained on Wikipedia and Books Corpus data rather than web data.
R EPRODUCIBILITY S TATEMENT
The descriptions of our models and training procedure in Section 4 and the code uploaded in the supplementary material is sufﬁcient for reproducing our results.
The data we use is available in the
Github repositories of the work we cite that creates these resources.
For running experiments, see the README.md ﬁle.
We will upload our code and data to Github and place a link in this section upon acceptance.
A A LL R ESULTS
See Figures 4 and 5 as well as Tables 2 and 3.
[Table]
B T EMPLATES
See Figure 6 and Table 4.
[Table]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
ID OOD Prompts
OOD Objects OOD Keyboard Errors 0 . 0 0 .0 . 8 1 .c on s i s t e n c y
BERT Base ID OOD Prompts OOD Objects
OOD Keyboard Errors 0 . 0 0 .0 .
8 1 .c on s i s t e n c y BERT Large ID
OOD Prompts OOD Objects OOD Keyboard Errors 0 . 0 0 .0 . 8 1 .c on s
i s t e n c y RoBERTa Large Baseline Rewrite
P-Tuning Preﬁx MoE Oracle
[Figure]
[Figure] [Figure] [Figure]
object
Ottawa relation
IS [Figure]
The capital of [X] is [Y].
natural language prompt
The capital of Canada is [MASK] .
prediction
Toronto
C T RAINING D ETAILS
All of our P-Adapters were trained using the hyperparameters from Liu et al.
(2021b):
Adam optimizer with a learning rater of 0 .
00001 , weight decay of 0 . 0005 , a batch size of 128 , and an exponential learning rate decay schedule with a decay rate of 0 .
98 (Kingma & Ba, 2015).
Each result is the mean over ﬁve random seeds.
Our MoE classiﬁers were trained using an AdamW optimizer with a learning rate of 0 .
001 and linear learning rate decay (Loshchilov & Hutter, 2018).
We use HuggingFace Transformers to train the model for 3 epochs on the same training data used to train the P-Adapter models (Wolf et al., 2020).
Each result is the mean over three random seeds rather than ﬁve because ﬁne-tuning the classiﬁer was more costly than training the the P-Adapters.
D Q UALITATIVE A NALYSIS
We also performed qualitative analysis to determine how P-Adapters’ predictions compare to the
Baselines’ and MoE models’.
We performed this analysis using the BERT base models, and looked at the Preﬁx P-Adapter.
In general, we ﬁnd that the Baseline tends to include more nonsensical or untopical vocabulary items.
For example, for P36, the capital of relation, the baseline predicts the capital of “Iraq” to be ‘ | ’, ‘She’, and ‘It’ for some prompts.
There are also situations where despite this variability, the baseline fails to predict the correct object for any of the prompts.
For example, for the relation official language of , there are 60 natural language prompts.
The correct object of the subject “Raahe” is “Finnish”, and it does not appear in the baseline’s 28 distinct predictions.
However, half of the P-Adapter’s predictions are the correct “Finnish”.
While the P-Adapter’s predictions are more consistent than the baseline’s, they are less consistent than the MoE’s.
While this is undesirable from the perspective of consistency, it can lead to greater accuracy in some cases.
For example, for the relation language spoken , the MoE model predicted the object “French” for the subject “Guy Maddin” for 100% of the prompts, while the variability in P-Adapters allowed them to predict the only correct answer, “English” in 89 .
3% of the prompts (and “French” in 9 .
3% of them).
There are downsides to this ﬂexibility, however.
We also observe examples where P-Adapters are tricked by the surface form of the prompt.
For example, for the relation place of death , “Akihiko Saito” is predicted be Tokyo by 95% of phrases, even though the correct answer is “Iraq”.
There are also examples where P-Adapters (particularly, the Preﬁx P-Adapter) lead to predictions that are copied from the input where MoE models do not.
For example, for prompts like “Windows
Server 2003 was manufactured by [MASK] .”
P-Adapters correctly predict “Microsoft” for 87% of paraphrases, but they also predict “Windows” 13% of the time, which we do not ovserve the MoE models doing.
E E XPLICITLY O PTIMIZING FOR C ONSISTENCY
Currently, P-Adapters encourage consistency in LLMs’ predictions implicitly, as the loss encourages LLMs to predict the same, correct object for variable natural language inputs.
However, one could imagine that explicitly encouraging consistency could be helpful.
One way to explicitly train for consistency is by encouraging not just the top-1 prediction to be the same, but the distribution over objects to the be the same for different natural language prompts for the same fact (Elazar et al., 2021).
To do this, we train with pairs of natural language prompts for the same fact, and encourage the distributions over the predicted objects to be similar using the symmetric KL-Divergence.
Formally, given two prompts for the same entity pair and relation x (cid:48) 1 , x (cid:48) 2 , and an LM P LM we deﬁne our loss as:
[Equation]
We then add this to the loss, (cid:96) that we were previously calculating—the cross-entropy loss between the distribution over the object and the correct object, giving a total loss of:
[Equation]
where we use λ = 0 .
This approach is inspired by the approach taken by Elazar et al.
(2021), with the differences being that they ﬁnetune all of the parameters of the LLM, calculate this loss over all prompts with the same fact, and also ﬁnd that in their setting they need to additionally continue with masked-language modeling training.
We train new Preﬁx-P-Adapters on the BERT Base with this new loss
[Table]
and ﬁnd that the consistency increases slightly, and the accuracy is the same.
This boost in consistency is beneﬁcial, but the small increase (compared to the baseline) does not suggest that explicitly encouraging consistency is necessary (See Table 5)
[Table]
F R ELATION C LASSIFIER
We ﬁnetune BERT-base-cased to predict the relation from the natural language prompt for the MoE models.
The F1 scores are microaveraged across three random seeds are visible in Table
[Table]