Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases (2021)

Abstract: Previous literatures show that pre-trained masked language models (MLMs) such as BERT can achieve competitive factual knowledge extraction performance on some datasets, indicating that MLMs can potentially be a reliable knowledge source. In this paper, we conduct a rigorous study to explore the underlying predicting mechanisms of MLMs over different extraction paradigms. By investigating the behaviors of MLMs, we find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts. Furthermore, incorporating illustrative cases and external contexts improve knowledge prediction mainly due to entity type guidance and golden answer leakage. Our findings shed light on the underlying predicting mechanisms of MLMs, and strongly question the previous conclusion that current MLMs can potentially serve as reliable factual knowledge bases.

## introduction recently, pre-trained language models @xcite @xcite have achieved promising performance on many nlp tasks. apart from utilizing the universal representations from pre-trained models in downstream tasks, some literatures have shown the potential of pretrained masked language models (e.g., bert @xcite and roberta @xcite ) to be factual knowledge bases @xcite @xcite @xcite @xcite . for example, to extract the birthplace of steve jobs, we can query mlms like bert with "steve jobs was born in [mask]", where steve jobs is the subject paradigm of the fact, "was born in" is a prompt string for the relation "place-of-birth" and [mask] is a placeholder for the object to predict. then mlms are expected to predict the correct answer "california" at the [mask] position based on its internal knowledge. to help mlms better extract knowledge, the query may also be enriched with external information like illustrative cases (e.g., (obama, hawaii)) @xcite or external context (e.g., jobs lives in california) @xcite . some literatures have shown that such paradigms can achieve decent performance on some benchmarks like lama @xcite . despite some reported success, currently there is no rigorous study looking deeply into the underlying mechanisms behind these achievements. besides, it is also unclear whether such achievements depend on certain conditions (e.g., datasets, domains, relations). the absence of such kind of studies undermines our trust in the predictions of mlms. we could neither determine whether the predictions are reliable nor explain why mlms make a specific prediction, and therefore significantly limits mlms' further applications and improvements. to this end, this paper conducts a thorough study on whether mlms could be reliable factual knowledge bases. throughout our investigations, we analyze the behaviors of mlms, figure out the critical factors for mlms to achieve decent performance, and demonstrate how different kinds of external information influence mlms' predictions. specifically, we investigate factual knowledge extraction from mlms foot_0 over three representative factual knowledge extraction paradigms, as shown in figure 1 : surprisingly, the main conclusions of this paper somewhat diverge from previous findings in published literatures, which are summarized in figure 1 . for prompt-based paradigm ( § 3), we find that the prediction distribution of mlms is significantly prompt-biased. specifically, we find that prompt-based retrieval generates similar predictions on totally different datasets. and predictions are spuriously correlated with the applied prompts, rather than the facts we want to extract. therefore, previous decent performance mainly stems from the prompt over-fitting the dataset answer distribution, rather than mlms' knowledge extraction ability. our findings strongly question the conclusions of previous literatures, and demonstrate that current mlms can not serve as reliable knowledge bases when using prompt-based retrieval paradigm. for case-based paradigm ( § 4), we find that the illustrative cases mainly provide a "type guidance" for mlms. to show this, we propose a novel algorithm to induce the object type of each relation based on wikidata foot_1 taxonomy. according to the induced types, we find that the performance gain brought by illustrative cases mainly owes to the improvement on recognizing object type. by contrast, it cannot help mlms select the correct answer from the entities with the same type: the rank of answer within its entity type is changed randomly after introducing illustrative cases. that is to say, under the case-based paradigm, although mlms can effectively analogize between entities with the same type, they still cannot well identify the exact target object based on their internal knowledge and the provided illustrative cases. for context-based paradigm ( § 5), we find that context can help the factual knowledge extraction mainly because it explicitly or implicitly leaks the correct answer. specifically, the knowledge extraction performance improvement mainly happens when the introduced context contains the answer. furthermore, when we mask the answer in the context, the performance still significantly improves as long as mlms can correctly reconstruct the masked answer in the remaining context. in other words, in these instances, the context itself servers as a delegator of the masked answer, and therefore mlms can still obtain sufficient implicit answer evidence even the answer doesn't explicitly appear. all the above findings demonstrate that current mlms are not reliable in factual knowledge extraction. furthermore, this paper sheds some light on the underlying predicting mechanisms of mlms, which can potentially benefit many future studies. ## related work the great success of pre-trained language models (plms) raises the question of whether plms can be directly used as reliable knowledge bases. @xcite propose the lama benchmark, which probes knowledge in plms using prompt-based retrieval. jiang et al. (2020a ## prompt-based retrieval the prompt-based retrieval extracts factual knowledge by querying mlms with (subject, prompt, [mask]). for example, to extract the "place-of-birth" of steve jobs, we could query bert with "steve jobs was born in @xcite ." and the predicted "california" would be regarded as the answer. we consider three kinds of prompts: the manually prompts t man created by @xcite , the mining-based prompts t mine by @xcite and the automatically searched prompts t auto from @xcite . ## case-based analogy the case-based analogy enhances the prompt-based paradigm with several illustrative cases. for example, if we want to know the "place-of-birth" of steve jobs, we would first sample cases such as (obama, place-of-birth, hawaii), and combine them with the original query. in this way, we will use "obama was born in hawaii. ## context-based inference the context-based inference augments the promptbased paradigm with external contexts. for example, if we want to know the "place-of-birth" of steve jobs, we can use the external context "jobs was from california.", and form a context-enriched @xcite retrieved document as external contexts. ## conclusions and discussions in this paper, we thoroughly study the underlying mechanisms of mlms on three representative factual knowledge extraction paradigms. we find that the prompt-based retrieval is severely promptbiased, illustrative cases enhance mlms mainly via type guidance, and external contexts help knowledge prediction mostly because they contain the correct answer, explicitly or implicitly. these findings strongly question previous conclusions that current mlms could serve as reliable factual knowledge bases. the findings of this paper can benefit the community in many directions. by explaining the underlying predicting mechanisms of mlms, we provide reliable explanations for many previous knowledgeintensive techniques. for example, our method can explain why and how incorporating external contexts will help knowledge extraction and common-senseqa @xcite . our findings also reveal why plm probing datasets may not be reliable and how the evaluation can be promoted by designing de-biased evaluation datasets. this paper also sheds light on future research directions. for instance, knowing the main benefit of illustrative cases comes from type-guidance, we can enhance many type-centric prediction tasks such as ner @xcite and factoid qa @xcite . moreover, based on the mechanism of incorporating external contexts, we can better evaluate, seek, and denoise external contexts for different tasks using mlms. for example, we can assess and select appropriate facts for commonsenseqa based on whether they can reconstruct the candidate answers. this paper focuses on masked language models, which have been shown very effective and are widely used. we also want to investigate another representative category of language models -the generative pre-trained models (e.g., gpt2/3 @xcite ), which have been shown to have quite different mechanisms and we leave it for future work due to page limitation.