The Queen of E ngland is not E ngland’s Queen: On the Lack of Factual Coherency in PLM s (2024)

Abstract: Factual knowledge encoded in Pre-trained Language Models (PLMs) enriches their representations and justifies their use as knowledge bases. Previous work has focused on probing PLMs for factual knowledge by measuring how often they can correctly predict an _object_ entity given a subject and a relation, and improving fact retrieval by optimizing the prompts used for querying PLMs. In this work, we consider a complementary aspect, namely the coherency of factual knowledge in PLMs, i.e., how often can PLMs predict the _subject_ entity given its initial prediction of the object entity. This goes beyond evaluating how much PLMs know, and focuses on the internal state of knowledge inside them. Our results indicate that PLMs have low coherency using manually written, optimized and paraphrased prompts, but including an evidence paragraph leads to substantial improvement. This shows that PLMs fail to model inverse relations and need further enhancements to be able to handle retrieving facts from their parameters in a coherent manner, and to be considered as knowledge bases.

## introduction pre-trained language models (plms) are probed for factual knowledge to investigate their usage as knowledge bases, and gain a better understanding of the rich representations they provide @xcite . previous extensions have focused on extracting more facts @xcite , increasing the consistency of plms to paraphrased prompts @xcite , identifying the parts of plms that are responsible for storing knowledge @xcite and updating facts in them @xcite . more recently, @xcite study the generalization abilities of plms from "a is b" to "b is a", and show that if a plm is trained on "the capital of malta is valetta" it will not be able to correctly answer the question: "which country has 2) the plm makes a second prediction based on the same relation and its first prediction o ′ . 3) if the plm predicts s in the second step it shows coherent behavior. valetta as its capital?". in this work, we introduce an intrinsic and complementary aspect, namely the coherency of plms with respect to factual knowledge. coherency is not concerned with correctness of the plms' predictions, but with the internal state of knowledge in plms and its consistency. more concretely, we first ask a plm to answer the question: "what is the capital of malta?", and if it answers "berlin", we ask it to answer the question: "which country has berlin as its capital?", and if it answers "malta", then we say that the plm has answered coherently (even though the answer is factually wrong). note that in practice we use cloze prompts instead of questions to make the task closer to language modeling (see figure 1 ). intuitively, if a human can tell the capital of a country given that country's name, then she is also able to tell the country given its capital's name. given that plms are queried with a subject and a relation to extract a object, we define coherency as the ability of the plm to infer the subject given its initial prediction for the object entity and vice versa. our contributions are the following: (1) we introduce coherency to investigate the internal state of factual knowledge in plms; (2) we evaluate different plms, showing that they have low coherency; (3) we show that optimized and paraphrased prompts do not improve coherency, but the use of evidence paragraphs substantially improves coherency. we make our code available. 1 ## coherency plms are known to capture vast amount of facts from their pre-training corpora. this has encouraged the community to consider using them as knowledge bases (kbs) @xcite , which can be constructed without expensive annotations, and which can easily be queried using natural language. however, the use of plms as kbs has many limitations @xcite . for example, plms are quite sensitive to their prompts, and cannot be easily updated with new facts. factual knowledge in plms is estimated by evaluating how often plms can correctly predict an object entity o, given a subject entity s and a relation r, when provided with a prompt which contains the subject and the relation: t(s,r), where t is a function that maps a subject entity and a relation to a prompt in natural language that contains the given entity and expresses the relation in natural language, e.g., (malta, capital-of) → "the capital of malta is [mask]". in this work, we focus on evaluating the coherency of plms with respect to the factual knowledge stored in their parameters, i.e., how often can plms predict s using t(o ′ ,r), given that it predicted o ′ using t(s,r). for example, "the capital of [mask] is berlin" → malta is coherent with "the capital of malta is [mask]" → "berlin". we do not evaluate if the predictions are factually correct, because we are interested in the coherency of the plms' world view, regardless of its correctness .we show and discuss correctness scores in appendix a. coherency can be easily calculated for 1-1 relations, but is more challenging, if we consider n-1 or n-m relations, since multiple entities could be correct when trying to predict the subject entity. to address this, we exclude all correct entities except the ground truth subject s in the second inference step, following @xcite and @xcite . since plms are known to have certain biases and are sensitive to the prompts, we start with predicting the object given the subject in a first round. in a second round, we start by predicting the subject given the object. the complete algorithm for estimating coherency in plms for all types of relations is shown in algorithm 1. after estimating coherency for each relation, we macro-average over all relations, because we are interested in the average performance for the use case of plms-as-kbs, which involves storing facts from different types of relations. ## experimental setup here, we describe the data and plms, which we use, and our experiments in detail. ## results and discussion the results for the first three experiments are shown in since we considered only one-token entities from t-rex for bert models, we show a normalized version of the results on this subset for better comparability in plms show poor coherency. we notice that all plms have poor coherency. autoregressive plms perform even worse than masked plms, even though the task is made easier for them through typed querying (cf. section 3.2). the poor performance of autoregressive plms might be due to their unidirectional training objective, whereas masked plms make use of a bidirectional context. increasing the number of parameters in t5 models leads to consistent improvements in performance. however, this does not generalize to the bert models (bert-base performs better than bert-large), and to the t5 models that are trained with ssm (t5-large-ssm performs better than t5-3b-ssm). the ssm objective is beneficial for the large variant of t5 (t5-large-ssm improves by 6.5 percentage points over t5-large, and even outperforms t5-3b, which has 4 times as many parameters). contrarily, this improvement does not generalize to the 3b variant (t5-3b outperforms its ssm counterpart). inform-bert falls short of normal bert, even though it was shown to outperform bert, when it comes to facts retrieval @xcite . hence, better facts retrieval does not necessarily affect coherency positively. in general, scaling and entity-centric training objectives have to some extent a positive effect on coherency. we also notice that in most cases models perform worse in the first round. round 1 can be more difficult, since it may involve predicting a specific subject based on a generic object in the second step (e.g.,"[mask] is located in bern"), whereas the second round goes into opposite and easier direction ("university of bern is located in [mask]"). plms are known to not provide specific answers @xcite . we show the results per relation type in we also show and categorize examples from different plms in optimizing prompts does not help. optimized prompts lead to a drop in coherency in the second experiment (see results under optimized prompts in evidence paragraphs improve coherency. including evidence paragraphs in the inputs substantially improves performance (see results under evidence paragraphs in coherency varies across paraphrases. in general, our analysis shows that plms do not possess a coherent knowledge state. the low coherency might be due: 1) the fact that plms make predictions based on shallow surface level features @xcite , which makes plms output relevant but incoherent and non-factual predictions (for an example see row 6 in ## conclusion in this work, we focused on evaluating the coherency of factual knowledge in plms. we considered the use of manually-written, optimized, and paraphrased prompts. our results indicate poor coherency. the inclusion of an evidence paragraph leads to substantial improvements. this shows that plms can leverage contextual knowledge better than parametric knowledge and highlights the importance of retrieval-augmented plms. we believe that further improvements are needed to improve coherency in plms, and to consider them as alternatives to kbs. we believe that future work should focus on further improving plms on the architectural level, the data level, and the interface between them (pre-training objectives).