Calibrating Factual Knowledge in Pretrained Language Models (2022)

Abstract: Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivates us to explore a fundamental question: How do we calibrate factual knowledge in PLMs without re-training from scratch? In this work, we propose a simple and lightweight method CaliNet to achieve this goal. To be specific, we first detect whether PLMs can learn the right facts via a contrastive score between right and fake facts. If not, we then use a lightweight method to add and adapt new parameters to specific factual texts. Experiments on the knowledge probing task show the calibration effectiveness and efficiency. In addition, through closed-book question answering, we find that the calibrated PLM possesses knowledge generalization ability after finetuning.Beyond the calibration performance, we further investigate and visualize the knowledge calibration mechanism.

## introduction recently, pretrained language models (plms) have improved performance on various natural language processing (nlp) tasks @xcite @xcite . probing tasks like lama @xcite @xcite have shown that plms can store factual knowledge and act as knowledge bases. leveraging knowledge in plms can benefit knowledge-intensive downstream tasks such as fact checking and question answering @xcite @xcite . however, knowledge stored in plms may have factual errors, which hinder the performance in downstream tasks @xcite . it is essential and fundamental to detect and calibrate false facts stored in a plm. ## contrastive knowledge assessment the first step for calibration is to detect which wrong facts are learned by plms. we propose contrastive knowledge assessment (cka) and implement it to identify false knowledge in plms. traditional evaluation usually adopts rank-based metrics. it evaluates a plm based on how highly it ranks the ground truth entity against other entities. however, it comes with two main problems. one is the problem of inexhaustible answers. the rank-based method fails to assess plms on multiple valid predictions. the top-1 only has one prediction, but the right predictions can be multiple. the other one is the problem of frequency bias. the ranking is particularly susceptible to the token frequency in the pretraining corpus. when the tail entity o frequently coexists with a head entity s, even if they express nothing about a specific fact, the model will still assign o a high rank when assessing this fact. to address these limitations, we propose cka to detect the false factual knowledge stored in plms. the core idea is assessing model prediction under a positive right fact and negative wrong facts in a contrastive manner. for each fact, we sample a prompt to transform it into natural text. let the triplet ⟨s, r, o⟩ denote a correct fact, where s, and o denote the subject entity and the object entity, respectively. we define r as the correct relation in a positive probing prompt, r ′ as the incorrect relation in a negative probing prompt. 1 for a plm m , we consider the probability it assigns to o given ⟨s, r⟩ and ⟨s, r ′ ⟩. as ⟨s, r, o⟩ is correct and ⟨s, r ′ , o⟩ is erroneous, p m (o|s, r) should be larger than p m (o|s, r ′ ) if m knows the fact. thus, cka calculates the factual correctness of a fact ⟨s, r, o⟩ for the model m by in our implementation, the templates of the positive prompts come from lama @xcite and the the templates of the negative prompts are manually designed for quality guarantee. the negative prompts have contradictory semantics with the positive prompts but still prompt the same type of entities. for example, the positive prompt template @xmath0, subclass of, @xmath1, and the negative prompt template can be "[x] is the parent class of [y]". an example of calculating the cka score is shown in figure 2 . further, we can set a threshold (@xmath2.0) for the cka score to detect false knowledge in plms. we compare the cka score with the rank-based assessment used by previous work @xcite to show our advantages. as shown in ## knowledge calibration the cka method outputs which wrong facts a plm learns. this section describes how we calibrate them. suppose that we have detected k false facts in a plm. we aim to calibrate them to the correct ones so that the downstream tasks will not access false factual knowledge from the plm. previous work @xcite point out that ffns in transformers can be regarded as keyvalue memories that store factual knowledge. inspired by this, we design an ffn-like calinet and take advantage of the properties of ffn to calibrate factual knowledge in plms directly. it is also important to note that the proposed method can be used to any part of the parameters. in this work, we apply the method on ffn because ffn is proven to take more responsibility when storing facts. in this section, we introduce the architecture of calinet , the construction of the calibration data, and how to perform calibration on a pretrained model. ## conclusion in this paper, we reassess the knowledge stored in plms in a contrastive manner and detect the incorrect knowledge stored in plms. we propose calinet , which adds new parameters to calibrate the knowledge stored in plms at scale without updating the original model parameters. the knowledge-calibrated plms generalize calibrated knowledge well and perform better than original plms on various downstream tasks like open-domain qa. we further provide neuron-level investigations on the calibration mechanism and study how calibration works.