What Matters in Memorizing and Recalling Facts? Multifaceted Benchmarks for Knowledge Probing in Language Models (2024)

Abstract: Language models often struggle with handling factual knowledge, exhibiting factual hallucination issue. This makes it vital to evaluate the models’ ability to recall its parametric knowledge about facts. In this study, we introduce a knowledge probing benchmark, BELIEF(ICL), to evaluate the knowledge recall ability of both encoder- and decoder-based pre-trained language models (PLMs) from diverse perspectives. BELIEFs utilize a multi-prompt dataset to evaluate PLM’s accuracy, consistency, and reliability in factual knowledge recall. To enable a more reliable evaluation with BELIEFs, we semi-automatically create MyriadLAMA, which has massively diverse prompts. We validate the effectiveness of BELIEFs in comprehensively evaluating PLM’s knowledge recall ability on diverse PLMs, including recent large language models (LLMs). We then investigate key factors in memorizing and recalling facts in PLMs, such as model size, pretraining strategy and corpora, instruction-tuning process and in-context learning settings. Finally, we reveal the limitation of the prompt-based knowledge probing. The MyriadLAMA is publicized.

## introduction one of the strongest motivations for training a language model (lm) using massive text is to increase the ability to handle factual knowledge (kamalloo et al., 2023). however, even if lms are trained on massive text, they suffer from hallucinations that generate incorrect knowledge-grounded sentences (zhang et al., 2023). considering that large lms (llms) are being widely applied to realworld tasks, it is vital to evaluate the ability to recall the llms' parametric knowledge and what factors influence on memorizing facts during pre-training. in this study, we introduce a multifaceted benchmark for knowledge probing, beliefs (figure 1 ), including belief ( §2) and belief-icl ( §3) for encoder-and decoder-based plms. beliefs utilize diverse prompts for each fact to account for the impact of linguistic expressions when evalu-ating llms' knowledge recall ability. this allows us to evaluate the robustness and reliability of llm knowledge by measuring fluctuations in accuracy, consistency, and overconfidence in fact prediction. since beliefs require a multiprompt probing dataset with diverse prompts for each fact, we build a new probing dataset, myri-adlama, to enable a more accurate and comprehensive evaluation ( §4). myriadlama expands lama-uhn (petroni et al., 2020) by offering different prompts for each fact through a semiautomatic method. specifically, we obtain a wide variety of lexically, syntactically, and semantically diverse prompts by rewriting relational templates and extending subject expressions. we applied beliefs to various encoder-and decoder-based plms, including bert (devlin et al., 2019) and llama3 (dubey et al., 2024) ( §5.1). through extensive evaluations, we verify the utility of beliefs in uncovering plms' knowledge recall ability ( §5). moreover, by comparing different plms, we gain insights into the factors affecting knowledge recall of plms from three aspects: accuracy, reliability and robustness ( §6). the primary findings in this study are as follows: ## belief benchmark we first present the multifaceted factual probing benchmark, belief for encoder-based plms. using a multi-prompt probing dataset, belief evaluates the knowledge recall ability of plms from accuracy, robustness and reliability ( §2.2-2.4). here, robustness measures plms' ability to maintain consistent accuracy and predictions when given different prompts in evaluation. reliability reflects the extent to which we can trust the plms' predictions. ## belief-icl for decoder-based llms recent llms are based on decoder-only transformer architecture, and are trained to predict subsequent tokens in a sequence. this makes it challenging for them to directly predict [mask] tokens in masked prompts, as they cannot utilize information following the [mask] (e.g., "[mask] and tokyo are twin cities"). to comprehensively evaluate llms and enable fair comparison between encoder-and decoder-based models, we extend be-lief to llms by employing in-context learning (icl), termed belief-icl. ## myriadlama dataset the fairness and accuracy of belief evaluation depend on the diversity and quality of multiprompt factual probing datasets. however, existing datasets are either manually rewritten in small numbers (elazar et al., 2021) or mined from texts (jiang et al., 2020). the former is accurate but lacks diversity, providing an average of 7.3 prompts per fact with limited variation. for example, templates like " [x] works as [y]" and " [x], who works as [y]" are provided as different templates but very similar. additionally, the number of templates is highly imbalanced, with 8 out of 46 relations having only one template, while p138 foot_2 has 20. the latter is diverse but includes templates that do not necessarily imply the relationship. for instance, for relation p937 (work location) foot_3 , the mined templates include " [x] to meet [y].," which significantly deviates from the original meaning. to achieve a more accurate and fair evaluation, we introduce myriadlama, a new multi-prompt factual probing dataset with improved diversity while retaining quality. refer to §b for detailed qualitative and quantitative comparisons between myriadlama and prior datasets. ## differentiating plms in fact probing this section compares the plms' knowledge recall abilities in terms of accuracy, reliability, and robustness and then explores factors affecting them. ## limitation of prompt-based probing finally, we examine the limitation of prompt-based knowledge probing by using our massively diverse dataset. first, we gauge the average knowledge coverage rate by using the average acc@1 (average). next, for each relation, we calculate the maximum acc@1 using the template that yields the highest accuracy, @xcite and use this value to estimate the upper limit of prompt-based knowledge probing (maximum). finally, we approximate the upper limit of facts contained in llms, by considering a fact as existing if at least one prompt of this fact can produce the correct answer (oracle). ## related work the lama probe was first proposed to evaluate the utility of plms as knowledge bases via solving the fill-in-the-blank task (petroni et al., 2019). @xcite we select the prompt with the best subject expression among prompts for each fact. several researchers extend the lama probe to evaluate plms' ability to understand facts from diverse linguistic aspects, such as the effect of negation/mispriming (kassner and schütze, 2020), distractors (pandia and ettinger, 2021), multilingual understanding (keleg and magdy, 2023; zhao et al., 2024) and models' consistency facing prompts with minor nuances (fierro and søgaard, 2022; elazar et al., 2021). however, these studies lack the inspection of plms' reliability in knowledge prediction, which is vital in deploying llms to realworld tasks. moreover, solving the fill-in-the-blank task by llms with the causal lm objective can underestimate their knowledge recall ability. recently, qa-based datasets have been developed to evaluate the knowledge recall ability of decoder-only lms. kalo and fichtel (2022) created a high-quality qa prompt set, which is further extended by wiland et al. (2024) to evaluate both causal and masked lms. mallen et al. (2023) and maekawa et al. (2024) developed qa datasets to see the impact of knowledge popularity and retrieval augmentation. since the writing style of these datasets is limited to questions, we cannot perform reliable robustness evaluation. ## conclusions this paper presents the multi-faceted factual probing benchmarks, belief and belief-icl, for encoder-and decoder-based plms, respectively. leveraging a multi-prompt dataset, beliefs provide various evaluation metrics, including accuracy, consistency, and reliability, enabling a thorough evaluation of plms' knowledge recall abilities. to make beliefs more reliable, we build a new multi-prompt dataset for knowledge probing, myriadlama, featuring diverse prompts for each fact. we conducted extensive experiments of multiple encoder-based plms and recent llms. based on the evaluation results, we identify key factors affecting the accuracy, reliability and robustness of plms' fact recall, such as model size, pretraining strategy and corpora, and icl settings. we also reveal the negative effect of instruction-tuning in recall factual knowledge from llms. this highlights the need for careful design of instructiontuning to preserve llms' knowledge recall abilities. finally, by probing facts in different ways, we find that plms hold more knowledge than what is revealed by using the optimal template, highlighting the limitations of prompt-based factual probing.