Venue: NAACL 2024

# Assessing Factual Reliability of Large Language Model Knowledge

Weixuan Wang\({}^{1}\), Barry Haddow\({}^{1}\), Alexandra Birch\({}^{1}\), Wei Peng\({}^{2}\)

\({}^{1}\) School of Informatics, University of Edinburgh

weixuan.wang@ed.ac.uk, bhaddow@ed.ac.uk, a.birch@ed.ac.uk \({}^{2}\) Huawei Technologies Co., Ltd. peng.wei1@huawei.com

###### Abstract

The factual knowledge of LLMs is typically evaluated using accuracy, yet this metric does not capture the vulnerability of LLMs to hallucination-inducing factors like prompt and context variability. How do we evaluate the capabilities of LLMs to consistently produce factually correct answers? In this paper, we propose **MOdel kNowledge relTabiliTy scORe****(MONITOR)**, a novel metric designed to directly measure LLMs' factual reliability. MONITOR is designed to compute the distance between the probability distributions of a valid output and its counterparts produced by the same LLM probing the same fact using different styles of prompts and contexts. Experiments on a comprehensive range of 12 LLMs demonstrate the effectiveness of MONITOR in evaluating the factual reliability of LLMs while maintaining a low computational overhead. In addition, we release the **FKTC** (Factual Knowledge Test Corpus) to foster research along this line1.

Footnote 1: [https://github.com/Vicky-Wil/MONITOR](https://github.com/Vicky-Wil/MONITOR)

+
Footnote †: _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 805–819 June 16-21, 2024 ©2024 Association for Computational Linguistics

## 1 Introduction

Recently, large pre-trained language models (LLMs) have been used as de facto storage for factual knowledge Petroni et al. (2019). However, applying LLMs to real-world scenarios inevitably leads to language generation deviating from known facts (aka "factual hallucination" Chang et al. (2023)) due to multiple causes. For example, Cao et al. (2021) argued that the performance of an LLM is over-estimated due to biased prompts overfitting datasets (also referred to as the framing effect in Jones and Steinhardt (2022)) and in-context information leakage.

Given the variability of LLMs' performance under different prompts and contexts, it becomes evident that relying solely on accuracy as an evaluation metric is insufficient. We also need to gauge how robust LLMs are to variations in prompting. In Figure 1 we show examples of factual probes where either the framing of the prompt, or the context to the prompt, is varied, leading to the issue of "accuracy instability".

Prompt framing effect:An LLM generates different predictions depending on how prompts are framed. Predictions are associated with prompts instead of factual knowledge learned in LLMs. As shown in Figure 1(a), for a fact represented in a triplet \(\langle\)_Center, is located in, Switzerland \(\rangle\)_, the generated predictions for re-framed prompts _"Which country is Cunter situated?"_ and _"Counter is located in Switzerland. True or False?"_ are non-factual.

Effect of in-context interference:An LLM leverages in-context information during its decoding stage, but this information may negatively affect an LLM's prediction during knowledge probing. As shown in Figure 1(b), for the same fact, when presented with a context _"England."_ concatenated with the prompting question _"Which country is the location of Cunter?"_, an LLM generates a non-factual prediction _"England"_.

How do we assess the reliability of factual knowledge of LLMs under the effects of these hallucination-inducing factors? Investigations into the behaviors of language models during knowl

Figure 1: ‘Accuracy instability” during language generation under various prompts.



edge probing (Petroni et al., 2019; Kassner and Schutze, 2020; Gupta, 2023) have mainly used metrics like precision and accuracy to quantify errors under a specified factor like prompt framing (Jones and Steinhardt, 2022) or mis-primed information (Kassner and Schutze, 2020). Despite the insights gained by showing the instability of LLMs during knowledge probing, these studies are subject to two limitations:

**No Exploration of Uncertainty**. Metrics like top-one accuracy may capture the ordering of predictions in the output space, but they lack the resolution to reflect on the degree of factual knowledge being learned by LLMs. Figure 2 depicts an example where two LLMs (Models A and B) may produce the same result even though their output probabilities vary. By equating the performance of Model A with that of Model B, one introduces a level of approximation in representation, which can be regarded as a source of uncertainty. In this paper, we directly use the output probabilities and construct a high-resolution metric to perform knowledge assessment.

**Limited Scope**. Previous works focus on understanding the effect of variability of a specific type. We design experiments to investigate the combined effects of multiple causes of accuracy instability: prompt framing and in-context interference during knowledge assessment. In addition, few studies have experimented on LLMs with billions of parameters. In contrast, we investigate the knowledge reliability of 12 freely downloadable LLMs with a range of parameter sizes and origins (with and without instruction fine-tuning).2

Footnote 2: Only freely downloadable LLMs are used as we need to access to the output probability distributions.

In this paper, we propose a novel distance-based approach **MOdel kNowledge rellabiiTy scORe (MONITOR)** which captures the deviation of output probability distributions under contexts of prompting variance, interference from mispriming (Kassner and Schutze, 2020) and positively-primed prompts.

We perform experiments on a comprehensive set of knowledge probing tasks and investigate the effectiveness of MONITOR in assessing LLMs' factual reliability. Through experiments with a large variety of different facts, we show that a lower-MONITOR LLM is less likely to suffer from **"accuracy instability"** issue. Computing MONITOR takes only one-third GPU hours of those consumed by a comprehensive accuracy reliability study, making MONITOR a low-cost metric for assessing factual knowledge reliability of LLMs.

**Our contributions are:**

1. We propose a novel method to assess the factual reliability of LLMs in the presence of the prompt framing effect and in-context interference. The proposed metric, MONITOR, can be used in conjunction with an end-to-end metric (i.e., accuracy) as part of a multi-dimensional approach to LLM knowledge evaluation.
2. We construct the **FKTC** (Factual Knowledge Test Corpus) by developing question answering probing prompts (210,171 prompts in total) based on 16,167 triplets of 20 fact datasets from T-REx corpus (Elsahar et al., 2018). We will release **FKTC** to the public to foster research works along this line.

## 2 Related Work

Petroni et al. (2019) demonstrated that factual knowledge can be directly extracted from language models without needing an external knowledge source. However, extracting knowledge (aka knowledge probing) from language models is errorprone due to various biases. For example, Elazar et al. (2021) showed that the consistency of knowledge extracted is generally low when the same fact is queried with different prompts. Many works in prompt engineering attempt to automatically construct prompts outperforming manual prompts (Shin et al., 2020; Jiang et al., 2020; Zhou et al., 2023; Kojima et al., 2022). Cao et al. (2021) argued that the decent performance of a language model is ascribed mainly to the application of these biased prompts, in other words "better" prompts are found to over-fit the answer distribution of the test set instead of reflecting on LLMs' generalization ability to predict factual knowledge.

Figure 2: The same top-1 answer with different output probabilities from two LLMs.

 

LLMs are sensitive to in-context information. Kassner and Schutze (2020); Gupta (2023) showed that language models fail on most negated probes and are easily misled by misprimes added to the probing context. On the other hand, Zhao et al. (2021); Si et al. (2023); Webson and Pavlick (2022) found the presence of context biases in few-shot probing results. The works mentioned above focused on pinpointing issues affecting LLMs' factual prediction. Few studies were motivated to develop evaluation approaches insensitive to the hallucination-inducing causes. Recently, Raj et al. (2023) presented a framework for evaluating the consistency of LLMs based on accuracy. Zhu et al. (2023) designed a benchmark for assessing the robustness of LLMs to adversarial instruction attacks, measuring the corresponding end-to-end performance drops. Dong et al. (2023) proposed a new metric to measure factual knowledge capability under the bias caused by aliases (alternative names for entities or relations) by reducing the effect of entity and relation aliases in the factual probing. Without tackling other factors like the prompt framing effect and in-context interference (and their interactions), the scope of the study is limited.

## 3 LLMs in Hallucination

In this section, we investigate LLMs' accuracy under the influence of various hallucination-inducing causes mentioned above. We design five formats of prompts to demonstrate two categories of hallucination-inducing causes during knowledge probing (Table 1). Twelve LLMs with a wide range of parameter size (from 560 million to 30 billion parameters) are covered in this study and experiments (in Section 5), including foundation language models of OPT (Zhang et al., 2022), Galactica (Taylor et al., 2022), and instruction finetuned language model of BLOOMZ (Muennighoff et al., 2023), Vicuna (Zheng et al., 2023), Flan-T5 (Chung et al., 2022), WizardLM (Xu et al., 2023), Flan-UL2 (Tag, 2023; Tay et al., 2023), LLaMa-30b-instruct-2048 (upstage, 2023).

### Effect of Prompt Framing on Accuracy

We design three probing templates based on the \(\langle\) "subject","relation","object" \(\rangle\) to show the effect of prompt framing on LLMs, depicted below, and for each task, we use seven paraphrased prompts to ensure diversity:

**Word Prediction (WP) Template**: Given the "subject" and the prompt template, LLMs perform word prediction to complete the sentence, e.g., the template (1) in Table 1.

**Question-Answer (QA) Template**: In the QA template, question prompts are constructed from paraphrasing templates in T-REx (Elsahar et al., 2018) targeting each fact. For example, a template "[X] is located in [Y]." for a triplet \(\langle\) [X], is located in, [Y] \(\rangle\) can be paraphrased to "Which country is [X] situated in?".

**Fact Checking (FC) Template**: An FC prompt is designed as a verification statement based on a template in T-REx, e.g., "Statement: [X] is located in [Y]. The statement is True or False?". We build the positive checking probe (**FC-pos**) and negative checking probe (**FC-neg**) corresponding to whether the statement is factual or not. For a negative fact-checking prompt, we average the prediction accuracy for five random entities chosen from the same category.

The probing results are shown in Table 2 as accuracy in predicting P17 factual knowledge for each involved LLM under prompting biases presented in terms of WP, QA, and FC templates. The performances of LLMs in predicting the fact test data vary significantly under prompt variability. Abnormal performances of LLMs between QA and WP template-based probes (bold numbers of Vicunam-7b) and between the FC probes for positive and negative interference (bold numbers of BLOOMZ-1b1) are strong evidences of the prompt framing effect. The fluctuation under WP, QA, and FC templates shown as box plots in Figure 9 (Appendix A.1) further demonstrates the effect of prompt framing on the performances of LLMs.

### Effect of In-context Interference

To explore the effect of in-context interference bias, we add probes with misprimed (Kassner and Schutze, 2020) interference by concatenating con

\begin{table}
\begin{tabular}{p{142.3pt}} \hline \multicolumn{2}{c}{**Prompt frames**} \\ \hline (1) WF: [X] is located in \_ (2) QA: Which country is [X] situated in? \\ (3) FC: Statement: [X] is located in [Y]. The statement is True of False? \\ \hline
**In-context Interference** \\ (4) [Y]: Which country is the location of [X]? \\ (5) [Y\_]. Which country is the location of [X]? \\ \hline \end{tabular}
\end{table}
Table 1: Examples of designed probing task templates extending the P17 (a fact dataset containing 931 subject-object pairs with the “country” relation from T-REx (Elsahar et al., 2018)). [Y] is the object wrt the subject [X], [Y\_] is an entity weakly related to [X].

 

texts in terms of factual/non-factual information preceding the associated QA prompt (template (2) in Table 1). Table 3 captures the accuracy of LLMs in a comparative study using factual entity probes and misprimes consisting of weakly associated entities. We observe a strong interference effect from nonfactual antecedents for all 12 LLMs. A factual entity (positive interference) can improve the accuracy by up to +43.67 while a weakly related entity (negative interference) reduces the accuracy by -66.00 at most.

## 4 Methodology

In this section, we introduce MONITOR, a distance-based score, to assess how the factual knowledge of LLMs is affected by the previously mentioned prompt framing and in-context interference.

Firstly, we introduce a new variable (\(i\)) to represent hallucination-inducing in-context information into the initial knowledge representation triplet \(\langle~{}subject\), relation, object \(\rangle\). The newly formed knowledge representation quadruple can be expressed as \(\langle~{}s,r,o,i~{}\rangle\). The information \(i\) can be further categorized into two variables: we use a factual object entity to implement a positive information \(i^{+}\); and the negative information \(i^{-}\) represents interference when predicting \(o\). For example, "England" is considered as an \(i^{-}\) when acting as a noisy condition to negatively affect an LLM in predicting a desirable outcome \(\langle~{}Switzerland~{}\rangle\) for a fact \(\langle~{}Cunter\), is located in, Switzerland \(\rangle\). Corresponding to an object, \(P(o|s,r,i)\) is the probability of the model generating the object \(o\) with the conditions of subject \(s\), prompt framing expression \(r\), and the in-context information \(i\).

To quantify the effect of \(i\) on LLMs, we establish **"anchor"** as a reference point, which is the gold answer with its probability in the output space. A "primary anchor" (shown as the red font "Switzerland 0.9117" in Figure 3) is defined as an enforced-accurate answer with its probability produced by an LLM in response to a knowledge probe. A primary anchor is produced by prompting an LLM with a QA template prefixed with positive information \(i^{+}\) (i.e. template (4) in Table 1). A primary anchor has multiple **foreign anchors** with various output probabilities (i.e., "Switzerland" in blue fonts in Figure 3) when an LLM is exposed to different prompts and in-context interference. Foreign anchors are generated using paraphrased Templates (2)3 and (5)4 presented in Table 1. By calculating the distance (using the probability changes) between a primary anchor and its corresponding foreign anchors in the influenced output space, we

Figure 3: A primary anchor (in red font) corresponds to its multiple foreign anchors with different output probabilities (blue fonts) when an LLM is exposed to different prompts and context interference. “\(FFD\)” and “\(IRD\)” refer to the two distance measurements defined as the prompt-framing degree and interference-relevance degree.

\begin{table}
\begin{tabular}{l|c|c c c} \hline \hline LLMs & Size & WP & QA & FC- & FC- \\  & & & pos & neg \\ \hline BLOOMZ-560m & 0.56 & 14.73 & 26.09 & 28.77 & 73.78 \\ BLOOMZ-1b1 & 1.1 & 14.96 & 28.29 & **0.11** & **99.89** \\ Galactic-1b3 & 1.3 & 2.36 & 46.43 & 86.05 & 12.29 \\ OPT-2b7 & 2.7 & 28.27 & 55.67 & 75.80 & 22.07 \\ ELBOOMZ-3b & 3 & 20.46 & 30.69 & 58.29 & 81.95 \\ Vienna-7b & 7 & **34.89** & **73.25** & 91.19 & 85.67 \\ FLOOMZ-7b1 & 7.1 & 26.26 & 33.72 & 88.32 & 64.98 \\ Flan-7S-XXL & 11 & 51.47 & 31.01 & 80.85 & 78.78 \\ Vicuna-1jb & 13 & 38.96 & 78.15 & 90.87 & 89.68 \\ WizardLM-1sb & 13 & 34.66 & 78.55 & 87.71 & 93.89 \\ Flan-UL2 & 20 & 21.57 & 46.44 & 79.51 & 73.58 \\ LLAMa-3bo-ins & 30 & 67.94 & 87.72 & 96.99 & 86.69 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Accuracy of various LLMs in predicting P17 fact dataset. The performances of LLMs have undergone significant variations for different prompting templates. The unit of “size” is billion.

 

can measure how reliable an LLM is in predicting facts in the test set.

MONITOR consists of two distance-based measurement components: Prompt-framing Degree (PFD) and Interference-relevance Degree (IRD).

### Prompt-framing Degree

The prompt-framing degree (PFD) is the mean distance between the output probability distributions of a primary anchor (\(P(o|s,r,i^{+})\)) and those produced by the same LLM using prompting frames \(r_{j}\) probing the same fact without any add-on context (foreign anchors \(P(o|s,r_{j})\)). PFD evaluates the similarity of two output probabilities between prompting frame relation expressions \(r\) (the basic prompt framing) and \(r_{j}\). It is defined as:

\[\small\begin{split}& PFD=\frac{1}{R}\sum\limits_{j=1}^{R}\frac{1}{L_{c}} \sum\limits_{l=1}^{L_{c}}|P(o_{c}|s_{c},r,i^{+})_{t}-P(o_{c}|s_{c},r_{j})_{l}|\\ \end{split} \tag{1}\]

where \(R\) is the count of prompt framing expressions for a subject, and the count of subject and object in a fact dataset is \(S\), \(c\in\left\{1,...,S\right\}\). \(L_{c}\) is the length of the anchor in terms of the number of subwords in the \(c\)-th object. PFD is a cumulative metric for assessing an LLM's capability in producing output probability distributions sharing the same characteristics under various prompting frames. PFD has a value between \(0\) and \(1\). The smaller the value is, the more robust an LLM is under the effect of prompt framing.

### Interference-relevance Degree

Interference-relevance Degree (IRD) is the distance between the output probability distributions of a primary anchor (\(P(o|s,r,i^{+})\)) and the probability distributions generated by the same LLM under the influence of in-context interference (foreign anchors \(P(o|s,r,i^{-})\)). IRD measures an LLM's capability to predict factual knowledge under the effect of in-context interference.

\[\small\begin{split} IRD=\frac{1}{M}\sum\limits_{m=1}^{M}\frac{1}{ L_{c}}\sum\limits_{l=1}^{L_{c}}|P(o_{c}|s_{c},r,i^{+})_{l}-P(o_{c}|s_{c}, r,i^{-}_{m})_{l}|\\ \end{split} \tag{2}\]

We define the count of positive and negative information as one and \(M\), respectively, corresponding to an object. IRD has a value between \(0\) and \(1\). As positive contextual information likely leads to factual knowledge generation, a smaller value of IRD indicates a lower level of effect from in-context interference biases.

### Monitor

The prompt-framing degree PFD and interference-relevance degree IRD are integrated to produce the proposed model knowledge reliability score (MONITOR). MONITOR captures the quadratic interaction of PFD and IRD, as illustrated in Eq 3 for a specified number of quadruples \(\langle~{}s,r,o,i~{}\rangle\), where the count of subject and object is \(S\). A set of coefficients (\(\alpha_{1-3}\)) is introduced to quantify the contributions from PFD, IRD, and their interaction on MONITOR. In this experiment, we consider an equal contribution scenario (\(\alpha_{1}=\alpha_{2}=\alpha_{3}=0.33\)). The smaller the value of MONITOR, the less degree an LLM is influenced by hallucination-induced factors when producing factual outputs. Taking the average output probabilities of primary anchors for an LLM as the denominator, MONITOR captures the degree of knowledge learned by an LLM when assessing its factual knowledge. MONITOR measures the effect of prompt framing and interference per unit of average primary anchor probability, demonstrating the strength of anchor representations.

LLMs are resource-hungry even during their inference phases. It is essential to ensure that an assessment metric is computation-efficient. Combining PFD, IRD, and their interaction in one metric can reduce the computation cost when evaluating factual reliability. Considering a fact dataset with R prompt frames, M negative interference, and one positive interference, there are \(R\cdot M\) combinations required to compute the average accuracy (and accuracy range). In comparison, we only require \(R+(1+M)\) combinations to compute MONITOR. The computation complexity for calculating MONITOR (\(O(R+M)\)) is considerably lower than that of accuracy (\(O(R\cdot M)\)).

\[\small\begin{split} MONITOR=\frac{\sum_{c}^{S}\sqrt{o_{1}PF2^{T}+ \alpha_{2}IRD^{T}+\alpha_{3}PFD*IRD}}{\sum_{c}^{S}\frac{1}{L_{c}}\sum_{l=1}^{L _{c}}P(o_{c}|s_{c},r,i^{+})_{l}}\\ \end{split} \tag{3}\]

## 5 Experiments and Results

In this section, we describe how to apply MONITOR to assess the factual knowledge of the 12 LLMs as mentioned above.

### Data Setting

In this section, we describe how we develop a test corpus to accommodate prompts with various styles and in-context interference.

**Expanding Probing Prompt:** Based on 16,167 \(\langle\) subject, relation, object \(\rangle\) triplets from T-REx 

(Elsahar et al., 2018), we develop QA probing prompts. We expand the probing prompt dataset by paraphrasing using GPT-4 (OpenAI, 2023) to create seven prompt frames for each triplet. In order to maintain diversity of prompts, we choose prompts with a similarity score (BLEU) below a threshold (0.7). Moreover, we manually check the paraphrased prompts to ensure validity.

**Adding In-context Interference:** Based on the QA prompts constructed above, we create a test dataset to explore the effectiveness of MONITOR with in-context interference. The corpus FKTC stands for "Factual Knowledge Test Corpus". Following the template patterns (Templates 4 and 5) in Table 1, we concatenate interference information (in terms of positive and negative in-context information) with the probing question for each subject. The negative information is entities from the same category weakly related to the corresponding subject, sampled from all objects that share the same relation. This process is applied to all expanded templates presented in Table 10 (Appendix A.2).

After applying these two processes (expanding the probing prompts and adding in-context interference) we produce 210,171 prompts focusing on 20 fact datasets.

### Results and Analysis

#### 5.2.1 Results on FKTC

The results evaluated on FKTC are shown in Table 4, and the results of each fact dataset are shown in Table 11 (Appendix A.3), where MONITOR and the average accuracy (avg acc) are recorded for each LLM across the 20 fact datasets in our experiments. Each LLM's minimal and maximal accuracy are also recorded to show the accuracy variability.

As shown in Table 4, LLMa-30b-ins. stands out as the most capable (with the smallest MONITOR 0.479) LLM, followed by Vicuna-13b (0.484) and Vicuna-7b (0.504). Even though MONITOR is a fundamentally different from an end-to-end metric (like accuracy), it correlates significantly with the average accuracy (0.846 Pearson coefficient). MONITOR adds a dimension to a point-measured metric (like accuracy) to show factual reliability of LLMs under prompt and context variability.

As shown in Table 5 (bold italic fonts), MONITOR can differentiate LLMs, for example, BLOOMZ-3b and Vicuna-7b, with a similar average accuracy on P37, by considering distance and probability information. We further discuss this in Subsection 5.2.3.

We present a detailed view of the knowledge assessment of LLMs by drilling down into specific facts. Unlike the results mentioned above, showing a general trend, the results disclosed here show more detailed insights. As shown in Table 5, the overall winning LLM (i.e., LLaMa-30b-ins.) can lose its edge in predicting a particular fact (P37).

#### 5.2.2 Accuracy Instability

We analyze the LLMs' "accuracy instability" when predicting P14125 with the results captured in Table 6 and Figure 4. A variety of statistics, including the base accuracy ("base acc") and standard deviation ("std") of an LLM's accuracy, are recorded for comparisons. A significant correlation is observed between accuracy standard deviation and MONITOR (0.754), demonstrating that a lower-MONITOR LLM is less likely to suffer from "accuracy instability" (Figure 5). Furthermore, as shown in Figure 4, an LLM with a lower MONITOR has a smaller value of accuracy standard deviation when two LLMs with equivalent base accuracy are evaluated (bold fonts in Table 5). From an accuracy stability viewpoint, one may choose an LLM with a lower MONITOR. For example, we prefer Vicuna-13b over WizardLM-13b, as the MONITOR of Vicuna-13b is lower even though they have similar accuracy.

Footnote 5: P1412: the fact dataset describing a relation of “languages spoken, written, and signed”

#### 5.2.3 Resolution Characteristics

It can be observed in Table 4 that the correlation between MONITOR and average accuracy is sig 

nificant. How should one use MONITOR when assessing the reliability of LLM knowledge?

We regard MONITOR as a high-resolution metric because it directly uses output probabilities and their changes (in terms of anchored distance) induced by hallucination factors. MONITOR considers both the output (nominal or qualitative data) and the probability of the output (quantitative information). Comparatively, assessing LLMs' knowledge with an end-to-end metric, such as accuracy, is purely reliant on a nominal output from the softmax layer of a transformer. It is shown in Table 5 that two LLMs (BLOOMZ-3b vs. Vicuna-7b) with almost identical average accuracy on P37 fact dataset have two distinctive values of MONITOR (0.570 vs 0.432). Delving into the log file of the inference task, we gain in-depth insights into why Vicuna-7b outperforms BLOOMZ-3b in the reliability score. As shown in Table 7, despite their similarities in the accuracy measurement, Vicuna-7b has much higher output probabilities than those of BLOOMZ-3b, contributing to the discrepancies in MONITOR.

Additionally, we plot out the probability distribution of the above two LLMs with almost identical average accuracy but very distinctive MONITOR (Figure 10 Appendix (A.5). It can be observed that

\begin{table}
\begin{tabular}{l l c c} \hline \hline  & \multicolumn{2}{c}{P178} \\ LMs & MONITOR \(\downarrow\) & huge acc \(\uparrow\) & std \(\downarrow\) \\ \hline Fine-T5-XLN & 0.772 & 51.713 & 31.025 \\ CPT-37-39 & 0.536 & **64.072** & 12.067 \\ Fine-L12- & 0.706 & 67.029 & 33.391 \\
**BLOOMZ-56nm** & **0.490** & **78.888** & **17.253** \\
**BLOOMZ-1b1** & **0.426** & **71.932** & **11.891** \\ Calocate-1b3 & 0.659 & 44.066 & 25.766 \\
**BLOOMZ-7b2** & **0.472** & **78.922** & **19.252** \\
**BLOOMZ-3b** & **0.465** & **79.143** & **18.016** \\ Vycune-7b & 0.472 & 82.806 & 27.555 \\
**ILALA-3b0-incs** & 0.543 & 85.340 & 34.151 \\
**Wizard-M1-3b0** & **0.425** & **91.560** & **8.978** \\
**Vycune-1sb** & **0.190** & **39.099** & **5.768** \\ \hline Correlation & Ferron & & “value” \\ \hline _v_(MOINTOR\_sd) & **0.754** & **0.001** \\ \hline \hline \end{tabular}
\end{table}
Table 6: LLMs with lower MONITOR are strongly correlated with smaller values of accuracy standard deviation, indicating less influence from prompt and context variability. “base acc” is the accuracy associated with the base prompt evaluated on the P1412 fact dataset.

\begin{table}
\begin{tabular}{l|l l l l|l l l l} \hline \hline  & \multicolumn{2}{c}{P178} & \multicolumn{2}{c}{PI108} & \multicolumn{2}{c}{P37} \\  & \multicolumn{2}{c}{MWOTOR \(\downarrow\)} & avg acc \(\uparrow\) & \multicolumn{2}{c}{probs. \(\uparrow\)} & \multicolumn{2}{c}{MNITOR \(\downarrow\)} & avg acc \(\uparrow\) & probs. \(\uparrow\) \\ \hline BLOOMZ-56nm & 0.594 & 53.260 & 0.471 & 0.947 & 2.634 & 0.313 & 0.669 & 33.142 & 0.679 \\ BLOOOMZ-1b1 & 0.492 & 56.752 & 0.684 & 0.853 & 7.454 & 0.191 & 0.662 & 39.679 & 0.751 \\ Galactic-1b3 & 0.595 & 27.763 & 0.543 & 0.876 & 0.686 & 0.393 & 0.639 & 42.444 & 0.703 \\ OPTP-2b7 & 0.470 & 64.119 & 0.348 & 0.739 & 12.420 & 0.343 & 0.471 & 52.866 & 0.419 \\ BIOLOZ-3b & 0.624 & 50.640 & 0.863 & 0.858 & 17.639 & 0.436 & _0.570_ & _51.242_ & 0.797 \\ Vicuna-7b & 0.339 & 64.575 & 0.969 & 0.620 & 32.756 & 0.969 & _0.432_ & _51.384_ & 0.931 \\ BLOOMZ-7b1 & 0.492 & 60.865 & 0.865 & 0.770 & 31.340 & 0.443 & 0.462 & 61.114 & 0.827 \\ FLAN-T5-XXL & 0.368 & 67.065 & 0.852 & 0.676 & 29.968 & 0.855 & 0.650 & 34.773 & 0.865 \\ Vicuna-1sb & 0.327 & 77.787 & 0.955 & 0.632 & 39.951 & 0.899 & **0.311** & 69.590 & 0.942 \\ WizardLM-13b & 0.411 & 84.878 & 0.850 & 0.626 & 54.735 & 0.769 & 0.467 & **69.907** & 0.856 \\ Flan-U12 & 0.613 & 49.968 & 0.792 & 0.844 & 23.942 & 0.836 & 0.575 & 56.731 & 0.738 \\ LiLaMa-3bo-ins & **0.180** & **87.461** & **0.993** & **0.522** & **60.493** & **0.972** & 0.411 & 63.109 & **0.950** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Performance of various LLMs in predicting factual knowledge captured in the P178, P108, and P37 fact datasets with “bold” numbers indicating the winning measurement over the same column category. P178, P108, and P37 are fact datasets representing relations of “developer”, “employer” and “official language”, respectively. The “bold and italic” fonts on P37 show how MONITOR can differentiate two LLMs (BLOOMZ-3b and Vicuna-7b) with similar average accuracy.

Figure 4: MONITOR can be used to differentiate LLMs’factual knowledge reliability when models with an equivalent base accuracy are evaluated. The box plots show the related distributions of accuracy when testing on P1412 fact dataset.

 

a more reliable LLM based on MONITOR, Vicuna-7b, has a much higher percentage of solid output probability (i.e., \(\geq\) 0.8) than those of a volatile LLM (BLOOMZ-3b). It is recommended to adopt MONITOR when using accuracy alone cannot differentiate LLMs' knowledge reliability.

#### 5.2.4 Lower Computation Cost

We compare the GPU hours consumed by LLaMa-30b-ins. in producing MONITOR and a full-scale accuracy reliability score (average accuracy). The experiment is to test the model on a specific fact dataset (P1412) using 8 NVIDIA V100 GPUs. It can be observed in Table 8 that using MONITOR leads to a 2.97-fold resource saving in GPU hours compared to applying an accuracy metric to a factual reliability evaluation. MONITOR is an economical method to add a dimension to LLM knowledge assessment when performing a full-scale reliability study on accuracy is not an option.

## 6 Discussion

### Attribution of In-Context Interference

To demonstrate the resilience of LLMs with different MONITOR, we conduct an additional experiment by applying the Integrated Gradients Sundararajan et al. (2017) technique implemented in Sarti et al. (2023). By examining and visualizing the attribution of input features to the model's outputs, we can infer the reliability of LLMs with different MONITOR. We study the behaviors of two LLMs (OPT-2b7 vs. BLOOMZ-3b) with distinctive values of MONITOR (0.471 vs. 0.570). The heat map shown in Figure 6 illustrates that a more reliable model with a lower MONITOR, OPT-2b7, is less influenced by in-context interference.

### Prompt Ablation

We design an ablation study to investigate the consistency of MONITOR across different prompt settings by analyzing the MONITOR in the P178 fact dataset. The MONITOR from an expanded prompts group setting (consisting of seven prompts) and a sub-sampled group with four prompts are captured in Figure 7. We observe a strong linear correlation between MONITOR of the expanded group and those from the sub-sampled group, indicating the scalability of MONITOR across prompt settings. Additionally, it is noted that MONITOR ranks LLMs in a consistent order for different prompt settings as show in Figure 8.

### Influence of Instruction Fine-tuning

Furthermore, in order to conduct a more detailed difference analysis between the foundation model

Figure 5: A significant correlation between MONITOR and accuracy standard deviation when testing the 12 LLMs on P1412 fact dataset, indicating lowermostTOR models are less likely to suffer from the “accuracy instability” issue.

Figure 6: Visualizing model behaviors of BLOOMZ-3b and OPT-2b7 under the influence of an input with misprimed in-context interference. The input is “_Danish. What language is the official language of Sotkamo? ”_.

\begin{table}
\begin{tabular}{l|c c c} \hline \hline Cost & MONITOR & Average Accuracy & MONITOR-saved \\ \hline GPU hours & 14.4 & 42.7 & 2.97X \\ \hline \hline \end{tabular}
\end{table}
Table 8: GPU hours consumed calculating MONITOR and average accuracy on P1412 fact dataset for LLaMa-30b-ins..“MONITOR-saved” denotes that GPU hours saved from using MONITOR compared to accuracy.

Figure 7: Significant correlation of MONITOR between the 7-prompt group and the 4-prompt group when assessing the reliability of 12 LLMs in the P178 fact dataset.

 

and the instruction fine-tuned model, we compare the foundation model BLOOM-7b1 and instructed model BLOOMZ-7b1. Both models share the same architecture and are evaluated on the P1412 dataset, as illustrated in Table 12. The comparison reveals that the instruction fine-tuning (IFT) approaches have an impact on the probability distribution. Specifically, the probability of the primary anchor in the foundation BLOOM-7b1 is 0.137, significantly lower than that in the IFT BLOOMZ-7b1 (0.541), resulting in higher MONITOR values and lower accuracy. This observation further supports the notion that instruction fine-tuning can enhance the reliability of Language Models.

## 7 Conclusion

In this paper, we show that large language models are subject to the influence of various hallucination-inducing causes. We propose a novel distance-based metric, directly computing the output probabilities and their changes to address "accuracy instability" caused by the prompt framing effect and in-context interference. A comprehensive set of experiments demonstrates that the proposed MONITOR is a high-resolution economic method suitable for evaluating the reliability of large language model knowledge. MONITOR can be used in conjunction with an end-to-end metric (i.e., accuracy) as part of a multi-dimensional approach to LLM knowledge evaluation. The constructed FKTC, consisting of 210,171 question answering prompts on 20 fact datasets, will be made available to the public to foster research along this line.

### Limitations

We focus on proposing MONITOR to assess the reliability of factual knowledge of LLMs during knowledge probing. Whether MONITOR can be generalized to a wider scope of tasks (e.g., summarization) warrants a future study. Additionally, the initial setup of contribution coefficients of PFD, IRD, and their interaction on MONITOR should be further investigated to establish an empirical benchmark. Currently MONITOR applies exact matching to obtain anchors to measure the reliability of LLM knowledge. Extending the automatic evaluation to anchors consisting of sentences is challenging. Our approach needs to access to the output probability distributions of an LLM, therefore is not applicable to SOTA commercialized LLMs such as GPT4. Additionally, FKTC is developed based on the latest version of T-REx benchmark dataset. The quality of the factual knowledge contents in FKTC is reliant on the alignment accuracy of T-REx. Even though we could argue that FKTC has already accommodated over 210 thousand prompts in the gold dataset to successfully support MONITOR in assessing LLMs behaviors under prompt and context variability, it can still be extended to host more knowledge categories.

### Licensing and Intended Use

FKTC is based on a widely adopted T-REx benchmark dataset, which is publicly available under a Creative Commons Attribution-ShareAlike 4.0 International License. FKTC is released to the public under the same license, consistent with the original intended use.

## Acknowledgement

This project has received funding from UK Research and Innovation (UKRI) under the UK government's Horizon Europe funding guarantee [grant numbers 10039436].

## References

* Cao et al. (2021) Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021. Knowledgeable or educated guess? revisiting language models as knowledge bases. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International

Figure 8: The consistency of MONITOR when assessing LLM’s factual reliability in predicting P178 facts across different prompts settings.

\begin{table}
\begin{tabular}{l|c c c c c} \hline \hline LLMs & MONITOR \(\downarrow\) & m\# & max \(\uparrow\) & min \(\uparrow\) & probd \(\uparrow\) \\ \hline BLOOM-7b1 & 0.813 & 13.985 & 55.782 & 1.361 & 0.137 \\ BLOOMZ-7b1 & 0.471 & 58.904 & 81.863 & 39.828 & 0.541 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Results are evaluated on P1412 dataset with comparing between BLOOM-7b1 (foundation model) and BLOOMZ-7b1 (IFT model).

 

Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, Volume 1: Long Papers), Virtual Event, August 1-6, 2021_, pages 1860-1874. Association for Computational Linguistics.
* Chang et al. (2023) Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2023. A survey on evaluation of large language models. _CoRR_, abs/2307.03109.
* Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyuan Chen, Aakanksha Chowdhory, Sharan Sarang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling instruction-finetuned language models. _CoRR_, abs/2210.11416.
* Dong et al. (2023) Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Zhifang Sui, and Lei Li. 2023. Statistical knowledge assessment for generative language models. _CoRR_, abs/2305.10519.
* Elazar et al. (2021) Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schutze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. _Transactions of the Association for Computational Linguistics_, 9:1012-1031.
* Elsahar et al. (2018) Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon S. Hare, Frederique Laforest, and Elena Simperl. 2018. T-REX: A Large Scale Alignment of Natural Language with Knowledge Base Triples. In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018_. European Language Resources Association (ELRA).
* Gupta (2023) Akshat Gupta. 2023. Probing quantifier comprehension in large language models. _CoRR_, abs/2306.07384.
* Jiang et al. (2020) Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know. _Trans. Assoc. Comput. Linguistics_, 8:423-438.
* Jones and Steinhardt (2022) Erik Jones and Jacob Steinhardt. 2022. Capturing failures of large language models via human cognitive biases. In _NeurlPS_.
* Kassner and Schutze (2020) Nora Kassner and Hinrich Schutze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_, pages 7811-7818. Association for Computational Linguistics.
* Kojima et al. (2022) Takesh Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In _NeurIPS_.
* Muennighoff et al. (2023) Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M. Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Flikri Ajfi, Khalid Almbark, Samuel Albanaine, Ziaid Ayafeia, Albert Webson, Edward Raff, and Colin Raffel. 2023. Crosslingual generalization through multitask finentuning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023_, pages 15991-16111. Association for Computational Linguistics.
* OpenAI (2023) OpenAI. 2023. GPT-4 technical report. _CoRR_, abs/2303.08774.
* Petroni et al. (2019) Fabio Petroni, Tim Rocktaschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language models as knowledge bases? In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 2463-2473. Association for Computational Linguistics.
* Raj et al. (2023) Harsh Raj, Vipul Gupta, Domenic Rosati, and Subhabrata Majumdar. 2023. Semantic consistency for assuring reliability of large language models. _CoRR_, abs/2308.09138.
* Sarti et al. (2023) Gabriele Sarti, Nils Feldhus, Ludwig Sickert, and Oskar van der Wal. 2023. Inseq: An interpretability toolkit for sequence generation models. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2023, Toronto, Canada, July 10-12, 2023_, pages 421-435. Association for Computational Linguistics.
* Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_, pages 4222-4235. Association for Computational Linguistics.
* Si et al. (2023) Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber, and Lijuan Wang. 2023. Prompting GPT-3 to be reliable. In _The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023_. OpenReview.net.
* Sundararajan et al. (2017) Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 

August 2017, pp. 3319-3328. Cited by: SS4.
* Y. Tay (2023)A New Open Source Flan 20B with UL2. Note: [https://www.yitay.net/blog/flan-ul2-20b](https://www.yitay.net/blog/flan-ul2-20b) External Links: Link Cited by: SS5.
* Y. Tay, M. Dehghani, V. Q. Tran, X. Garcia, J. Wei, X. Wang, H. Won Chung, D. Bahri, T. Schuster, H. Steven Zheng, D. Zhou, N. Houlsby, and D. Metzler (2023)UL2: unifying language learning paradigms. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, W. Dong, R. Fergus, A. Courville, S. Bengio, L. Bottou, C. Cortes, G. Chan, Z. Chen, P. Dollar, and R. Garnett (Eds.), pp. 1-6. External Links: ISBN 978-1-4693-3895-3 Cited by: SS1.
* R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic (2022)Galactica: a large language model for science. CoRRabs/2211.09085. External Links: Link, 2111.09085 Cited by: SS1.
* O. Russakovsky et al. (2016)ImageNet: a database of deep neural networks. arXiv preprint arXiv:1609.03085. Cited by: SS1.
* A. Vaswani, J. Shazeer, B. Mongo, F. Jurceau, J. Usair, J. Gomez, L. Kaiser, J. Lee, J. Long, J. Jia, S. Levine, J. Li, and J. Schmidhuber (2017)Attention: an imperative approach to image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-1. Cited by: SS1.
* A. Vaswani, J. Shazeer, B. Mongo, F. Jurceau, J. Usair, J. Gomez, L. Kaiser, J. Lee, J. Long, J. Jia, S. Levine, J. LeCun, J. Lin, and J. Schmidhuber (2017)Attention: a new paradigm for language understanding. In Proceedings of the 2017 ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1-1. Cited by: SS1.
* A. Vaswani, J. Shazeer, B. Mongo, F. Jurceau, J. Usair, J. Gomez, L. Kaiser, J.

 

Appendix

### Prompt Framing Effect

We paraphrase each fact dataset in three prompting templates (WP, QA, and FC) so that each template can be used to produce seven prompts. For example, the template "Which country is the location of [X]?" could be paraphrased as: "Which country is [X] situated in?", "Which country can [X] be found?", "Which country is the geographical position of [X]?", "Which country is the site of [X]?", "In Which country is [X] situated?", "Whereabouts is [X] located?". In this way, context diversity and semantic invariance are guaranteed. Figure 9 shows the "accuracy instability" of LLMs under the effect of prompt framing in predicting P17 facts based on three tasks (WP, QA, and FC).

### Templates Examples

Table 10 shows all templates and corresponding prompts on 20 fact datasets.

### MONITOR for All LLMs Experimented on FKTC

Table 11 shows the results of various LLMs evaluated on each fact dataset from FKTC.

### Correlation between MONITOR and Accuracy

Table 12 shows the Pearson correlation between MONITOR and average accuracy, evaluated on the 20 fact datasets from FKTC corpus.

### Probability Distribution

Figure 10 shows the probability distribution of two LLMs (BLOOMZ-3b and Vicuna-7b) with almost identical average accuracy but very distinctive MONITOR.

### Analysis on LLMs Scale

To further verify if MONITOR of LLMs follows the law of scaling, where larger LLMs are more knowledge-reliable, we present how MONITOR changes across BLOOMZ series for each specific fact dataset (shown in Figure 11). While MONITOR of LLMs may not conform to the scaling law at the granularity of each fact, their aggregated values in a comprehensive scope of experiments do follow the rule of scale (shown in Figures 11-12) 

Figure 10: A comparison of the probability distribution of anchors between BLOOMZ-3b and Vicuna-7b on P37. The population percentages with a solid probability (i.e., \(\geq\) 0.8) are **59%** and **85%** for BLOOMZ-3b and Vicuna-7b, respectively.

Figure 9: Box plots show the “accuracy instability” of LLMs under the effect of prompt framing in predicting P17 based on three tasks (WP, QA, and FC).

 



\begin{table}
\begin{tabular}{l l c c c c c c c c c c c c} \hline \hline Fact Dataset & BLOOMZ & BLOOZZ & Galactica & OPT & RLOOMZ & Vicuna & BLOOMZ & Flan-T5 & Vicuna & WizardLM & Flan & LLAMaSti- \\  & -560m & -1b1 & -1b3 & -2br7 & -3b & -7b & -7b1 & -XXL & -13b & -13b & -UIL2 & 300-ins. \\ \hline P17 & 0.782 & 0.780 & 0.852 & 0.541 & 0.785 & 0.523 & 0.714 & 0.690 & 0.544 & 0.602 & 0.788 & 0.395 \\ P19 & 0.866 & 0.927 & 0.914 & 0.858 & 0.898 & 0.719 & 0.873 & 0.882 & 0.629 & 0.752 & 0.918 & 0.817 \\ P20 & 0.810 & 0.926 & 0.942 & 0.949 & 0.921 & 0.671 & 0.873 & 0.888 & 0.667 & 0.725 & 0.893 & 0.803 \\ P27 & 0.704 & 0.746 & 0.868 & 0.597 & 0.706 & 0.460 & 0.724 & 0.674 & 0.489 & 0.573 & 0.786 & 0.490 \\ P30 & 0.809 & 0.839 & 0.801 & 0.748 & 0.887 & 0.652 & 0.546 & 0.670 & 0.611 & 0.680 & 0.815 & 0.617 \\ P37 & 0.669 & 0.662 & 0.639 & 0.471 & 0.570 & 0.432 & 0.462 & 0.650 & 0.311 & 0.467 & 0.575 & 0.411 \\ P101 & 0.899 & 0.822 & 0.919 & 0.888 & 0.877 & 0.816 & 0.838 & 0.879 & 0.823 & 0.927 & 0.858 & 0.857 \\ P103 & 0.512 & 0.515 & 0.671 & 0.468 & 0.457 & 0.424 & 0.451 & 0.599 & 0.296 & 0.506 & 0.561 & 0.410 \\ P108 & 0.947 & 0.853 & 0.876 & 0.739 & 0.858 & 0.620 & 0.770 & 0.676 & 0.632 & 0.626 & 0.844 & 0.522 \\ P127 & 0.522 & 0.613 & 0.676 & 0.627 & 0.712 & 0.547 & 0.545 & 0.437 & 0.382 & 0.438 & 0.621 & 0.346 \\ P159 & 0.829 & 0.851 & 0.858 & 0.755 & 0.800 & 0.523 & 0.751 & 0.731 & 0.478 & 0.479 & 0.758 & 0.454 \\ P176 & 0.684 & 0.461 & 0.457 & 0.527 & 0.609 & 0.244 & 0.632 & 0.290 & 0.437 & 0.467 & 0.518 & 0.322 \\ P178 & 0.594 & 0.492 & 0.595 & 0.470 & 0.624 & 0.339 & 0.492 & 0.368 & 0.327 & 0.411 & 0.613 & 0.180 \\ P264 & 0.887 & 0.923 & 0.916 & 0.863 & 0.748 & 0.678 & 0.887 & 0.883 & 0.606 & 0.661 & 0.799 & 0.560 \\ P276 & 0.707 & 0.699 & 0.751 & 0.650 & 0.737 & 0.535 & 0.674 & 0.639 & 0.489 & 0.557 & 0.664 & 0.515 \\ P364 & 0.756 & 0.762 & 0.850 & 0.662 & 0.780 & 0.576 & 0.751 & 0.786 & 0.619 & 0.714 & 0.774 & 0.599 \\ P495 & 0.802 & 0.834 & 0.868 & 0.661 & 0.695 & 0.413 & 0.715 & 0.716 & 0.476 & 0.530 & 0.790 & 0.499 \\ P470 & 0.941 & 0.961 & 0.961 & 0.858 & 0.931 & 0.689 & 0.905 & 0.837 & 0.646 & 0.669 & 0.882 & 0.647 \\ P1376 & 0.505 & 0.451 & 0.606 & 0.602 & 0.352 & 0.299 & 0.202 & 0.158 & 0.501 & 0.555 & 0.202 & 0.079 \\ P1412 & 0.490 & 0.426 & 0.659 & 0.536 & 0.456 & 0.427 & 0.472 & 0.772 & 0.190 & 0.425 & 0.706 & 0.543 \\ \hline \end{tabular}
\end{table}
Table 10: Examples of template for different fact datasets and the corresponding prompts we build in this work.

Figure 12: The BLOOMZ and Vicuna series adhere to the scale law based on the overall MONITOR results obtained from experiments on 20 fact datasets. The horizontal axis represents the size of a model in billions, and the vertical axis represents the results of MONITOR.





\begin{table}
\begin{tabular}{l|c c c c c c c c c c} \hline \hline Pearson & P17 & P19 & P20 & P27 & P30 & P37 & P101 & P103 & P108 & P127 \\ \hline correlation & -0.579 & -0.709 & -0.685 & -0.826 & -0.648 & -0.867 & -0.474 & -0.767 & -0.889 & -0.926 \\ p-value & 0.048 & 0.009 & 0.013 & 0.001 & 0.023 & 0.001 & 0.119 & 0.004 & 0.001 & 0.001 \\ \hline  & P159 & P176 & P178 & P264 & P276 & P364 & P495 & P740 & P1376 & P1412 \\ \hline correlation & -0.941 & -0.941 & -0.828 & -0.950 & -0.703 & -0.740 & -0.899 & -0.919 & -0.872 & -0.900 \\ p-value & 0.001 & 0.001 & 0.001 & 0.001 & 0.011 & 0.006 & 0.001 & 0.001 & 0.001 & 0.001 \\ \hline \end{tabular}
\end{table}
Table 12: Pearson correlation between MONITOR and the average accuracy, evaluated on FKTC corpus.