Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion (2025)

Abstract: Language models (LMs) can make a correct prediction based on many possible signals in a prompt, not all corresponding to recall of factual associations. However, current interpretations of LMs fail to take this into account. For example, given the query “Astrid Lindgren was born in” with the corresponding completion “Sweden”, no difference is made between whether the prediction was based on knowing where the author was born or assuming that a person with a Swedish-sounding name was born in Sweden. In this paper, we present a model-specific recipe - PrISM - for constructing datasets with examples of four different prediction scenarios: generic language modeling, guesswork, heuristics recall and exact fact recall. We apply two popular interpretability methods to the scenarios: causal tracing (CT) and information flow analysis. We find that both yield distinct results for each scenario. Results for exact fact recall and generic language modeling scenarios confirm previous conclusions about the importance of mid-range MLP sublayers for fact recall, while results for guesswork and heuristics indicate a critical role of late last token position MLP sublayers. In summary, we contribute resources for a more extensive and granular study of fact completion in LMs, together with analyses that provide a more nuanced understanding of how LMs process fact-related queries.

## introduction language models (lms) trained on large corpora have been found to store significant amounts of factual information @xcite . while there are many research results documenting the fact proficiency of lms @xcite , our understanding of how these models perform fact completion is still under development. mechanistic interpretability is a growing area of research aiming to explain model behavior (elhage * equal contribution. et @xcite , and has already yielded insights into where lms store and process factual information @xcite @xcite . while this body of work has broken new ground and provided us with interpretations of fact completion in lms, it is limited to studies of only one type of scenario. more specifically, in @xcite and @xcite , the studies are limited to correct predictions, and it is assumed that the model recalls facts for these. we hypothesize that this scenario in reality is a blend of multiple more fine-grained scenarios, as it is well known that lms can make correct predictions based on many different signals in the prompt, not all corresponding to exact fact recall. for example, lms may pick up on spurious correlations to "solve the dataset" rather than the task exemplified by it @xcite @xcite , and fact completion situations are no exception to this @xcite @xcite . in bringing this distinction forward, we also ground our work in more formal studies of knowledge, where most scholars agree that guesswork should be accounted for and excluded in order to consider a model as having "bona fide knowledge" @xcite . in this work, we disentangle and detail four different prediction scenarios for which lms can be expected to show distinct behaviors (see figure 1 ). the scenarios are: 1) generic language modeling, when the model does not respond with facts, such as when generating a story. 2) guesswork, when the model responds with a fact but is uncertain. 3) heuristics recall, when the model uses shallow heuristics, e.g. that people with korean-sounding names are more likely to live in korea. 4) exact fact recall, when the model has indeed memorized the correct answer and recalls it for the prediction. we show how interpretations of lms can be extended to these scenarios and how each scenario yields distinct interpretability results. in particular, this work makes three main contributions. first, we propose a method, prism, for creating a diagnostic dataset with distinct test cases to enable more extensive and precise interpretations of fact completion in lms ( §3). we create and release prism datasets for gpt-2 xl, llama 2 7b and llama 2 13b, respectively. second, our experiments with the interpretability method of causal tracing (ct) show that the models exhibit a complex behavior on the prism scenarios not captured by previous results ( §4.1). third, our indepth analysis of information flow confirms that models employ distinct inference mechanisms for the prism scenarios. for example, we observe contrasting results for exact fact recall compared to generic language modeling samples ( §4.2). taken together, our work expands on the scenarios that can be analyzed for interpretations of lms for fact completion and yield new interpretations of lms. foot_0 ## background this section provides a brief background on fact completion and mechanistic interpretability topics relevant to our work. ## prism datasets for precise studies of prediction scenarios we develop prism (precise identification of scenarios for model behavior) datasets to separate the different prediction scenarios in figure 1 . this is motivated by an inspection of the 1,209 coun-terfact samples which reveals 510 samples likely to rely on heuristics and 365 samples unlikely to have been memorized due to low popularity scores (see appendix f). we argue that these issues make the counterfact dataset unable to support precise and comprehensive interpretations of lms, missing out on important distinctions and valuable insights into the workings of lms. prism is developed to address these shortcomings. prism datasets are created by the identification and generation of samples corresponding to each of the four prediction scenarios in figure 1 . the process relies on three diagnostic test criteria (figure 2 ). note that prism datasets are modelspecific since they depend on model biases and parametric memories, which differ between lms. we develop prism datasets for gpt-2 xl @xcite , llama 2 7b and llama 2 13b. @xcite , respectively. general statistics for the datasets can be found in ## interpretability and prism we apply two mechanistic interpretability approaches -ct and information flow analysis -to test their sensitivity to each prediction scenario in prism. 8 this allows us to evaluate the validity of previous interpretability results. ## conclusion we identify four prediction scenarios that are fundamentally different and of differing reliability. these are exact fact recall, heuristics recall, guesswork and generic language modeling. we show that previous interpretability work for fact completion situations treat many of these as equivalent by using accuracy as the sole criterion for differentiating between different types of inference processes. to facilitate precise interpretations of lms, we present a method for creating prism datasets with samples that represent each of our identified prediction scenarios. we create prism datasets for gpt-2 xl, llama 2 7b and llama 2 13b, and use them to test the sensitivity of two influential interpretability methods, causal tracing and information flow analysis, to prediction scenario. we find that different prediction scenarios yield distinct interpretability results if studied in isolation. taken together, our paper expands on and delineates fact completion scenarios for which we can interpret lms. our results highlight the importance of studying these scenarios in isolation and provide nuanced insights with respect to how lms process information in fact completion situations.