Understanding Epistemic Language with a Language-augmented B ayesian Theory of Mind (2025)

Abstract: How do people understand and evaluate claims about others’ beliefs, even though these beliefs cannot be directly observed? In this paper, we introduce a cognitive model of epistemic language interpretation, grounded in Bayesian inferences about other agents’ goals, beliefs, and intentions: a language-augmented Bayesian theory-of-mind (LaBToM). By translating natural language into an epistemic “language-of-thought” with grammar-constrained LLM decoding, then evaluating these translations against the inferences produced by inverting a generative model of rational action and perception, LaBToM captures graded plausibility judgments of epistemic claims. We validate our model in an experiment where participants watch an agent navigate a maze to find keys hidden in boxes needed to reach their goal, then rate sentences about the agent’s beliefs. In contrast with multimodal LLMs (GPT-4o, Gemini Pro) and ablated models, our model correlates highly with human judgments for a wide range of expressions, including modal language, uncertainty expressions, knowledge claims, likelihood comparisons, and attributions of false belief.

## introduction people regularly use and interpret language about other agents' beliefs, evaluating rich linguistic constructions that may involve claims about what others consider possible (''grace thinks katie might have eaten the cookie''), what they find more probable (''tom thinks sam is the most likely to win the election''), or the relationship of their beliefs to the world (''john didn't know that today was a holiday''). but since these beliefs are not directly observable, how do people judge the truth or acceptability of epistemic claims? philosophers * equal contribution. and linguists have long investigated the semantics of epistemic language @xcite @xcite , offering compositional theories of how epistemic claims relate to the sets of worlds deemed possible by an agent @xcite . however, these theories do not explain how people ground epistemic language in its context of utterance. if someone says ''alice believes it might rain'', how does her behavior (e.g., bringing an umbrella) render that statement more or less plausible or acceptable? in this paper, we present a computational model of how people interpret epistemic language in context, grounded in inferences about what other agents believe (figure 1 )-a languageaugmented bayesian theory-of-mind (labtom). we build upon bayesian theory-of-mind (btom), a framework which casts human mentalizing as probabilistic inference over a generative model of rational action and perception @xcite . we combine this framework with the compositionality afforded by probabilistic extensions of the language-ofthought hypothesis @xcite , developing an epistemic language of thought (elot) to represent how others represent the world. we parse natural language to this representation using grammar-constrained sequential monte carlo decoding @xcite of large language models (llms), defining a flexible but structured probabilistic mapping from epistemic language to epistemic concepts @xcite . this allows us to quantify the plausibility of epistemic sentences against btom inferences. to evaluate our model, we run an experiment where participants watch animations of a player solving a gridworld puzzle called doors, @xcite . in these puzzles, the player has to pick up (single-use) keys that may figure 1 : overview of our model, a language-augmented bayesian theory of mind (labtom). here, our model evaluates the plausibility of epistemic language about a player character trying to find keys in boxes so as to reach one of four goals (gems with different shapes/colors, g 1 -g 4 ) which may be locked behind doors. (top) we translate natural language statements about the player's initial (σ 1 ) and current beliefs (σ 2 ) into an unambiguous epistemic language of thought (elot) via grammar-constrained llm parsing. elot statements (i.e., epistemic formulas ϕ) are interpreted with a probability-based semantics (lowered formulas ϕ ). (bottom) we use our bayesian theory-of-mind (btom) module to produce inferences (bar charts) about the environment state (top left), the player's goal (bottom left), and the player's belief state (right) at each step t, given observations across time. each possible belief state b i is itself a distribution over environment states s j , where state s j corresponds to a blue key being in box j. (middle) we evaluate the posterior probability * of each elot statement ϕ from the btom inferences at each step t ( * under a 50-50 prior over statement truth). statement σ 1 (''the player thought that the blue key must be in box 3.'') increases in probability from @xmath0, then stays high, since it becomes clear from the player looking in box 3 at @xmath1. statement σ 2 (''the player believes that a blue key is more likely to be in box 1 than box 2.'') decreases in probability as the player walks away from box 1 at @xmath2. however, when the player finds box 3 empty, then moves past box 2 up to box 1 at @xmath3, it becomes much more probable that the player currently thinks a key is more likely to be in box 1 rather than box 2. be hidden in boxes, using them to unlock doors of the same color to reach one of four valuable gems. the player's beliefs and goals are unknown to our participants, so they must infer these mental states. we ask one set of participants to write sentences describing the past and present beliefs of the player, collecting a rich dataset of epistemic language that includes modal verbs, uncertainty expressions, knowledge claims, and descriptions of false beliefs. we task another set of participants with evaluating these statements, asking them to rate how likely a statement is given the context. we find that the inferences produced by our labtom model correlate highly with these human ratings. in contrast, using a lower-level formalization of epistemic claims leads to substantially more parsing errors than our elot representation, and ablated btom models and multimodal llm baselines fail to explain human sentence judgments. these findings illustrate the importance of a conceptual vocabulary that is aligned with the structure of epistemic language, and the need for a coherent theory-of-mind to systematically interpret such language. ## computational model our labtom model comprises two interlinked modules. the first module (figure 1 , top), an epistemic language of thought (elot), models our capacity to compositionally represent the world (including the contents of others' minds) by combining more basic concepts into richer thoughts and expressions @xcite , and how we flexibly translate such thoughts from natural language @xcite . the second module (figure 1 , bottom), a bayesian theory-of-mind @xcite , captures our intuitive inferences about others' minds via bayesian inference over a generative model of how agents update their beliefs and act towards their goals. epistemic language understanding can thus be modeled by mapping language into elot formulas, which we evaluate against the inferences produced by rational mentalizing. ## experiments to evaluate our model on a diverse dataset of epistemic language ( with this data, we evaluated our model by: (i) assessing the translation accuracy of our elot module, investigating the importance of our elot representation (vs. the lowered form) and its impact on grammar-constrained llm decoding; and (ii) testing our full labtom model in its ability to capture human interpretation of epistemic language, comparing human ratings against labtom inferences. model and experiment code is available at https://osf.io/xq9c4/ . ## / 241 / 228 the player thought that box 1 was most likely to contain a red key. the player is unsure what color the key in box 2 will be. ## / 241 20 / 228 the player already knows for sure there is no key in box 1 or box 2. the player did not know if box 2 contained a red key. and item locations (figure 3 ). in each scenario, there were 4 goal gems with different shapes (triangle, square, hexagon, circle), some of which were locked behind doors. scenarios also had 2 to 3 boxes with up to 2 colored keys among them. the player's actions were varied across scenarios to elicit inferences about a diversity of epistemic states, such as ignorance about key locations or false confidence about the location of a key. ## results elot is a superior translation target and guide for (grammar-constrained) llm parsing. we report translation accuracy in overall, grammar-constrained smc decoding into elot with llama 3.1 8b performs the best, achieving 91% equivalence with our gold translations. upon inspection (see appendix), we find that unconstrained sampling often leads to syntax errors not corrected for by majority voting. gemini flash 8b also exhibits low sample diversity typical of instruction tuning, such that majority voting gives no improvement. in comparison, grammar-constrained smc produces not just accurate translations, but a distribution over reasonable translations, since smc performs posterior sampling rather than optimization @xcite . in future work, this could be used to more precisely model the interpretation of ambiguous epistemic language. labtom correlates highly with human ratings of epistemic statements. as sota multimodal llms struggle at grounded evaluation of epistemic language. the multi-modal llm baselines also perform less well than labtom despite extensive prompting, with the strongest llm baseline (gpt-4o with images, narratives, and few-shot prompting) achieving human correlations of only 0.59 and 0.41, respectively. increasing information in the prompt improves performance, although the gemini models do poorly regardless, even with full scenario videos. we find that llm performance is lower on initial belief sentences, suggesting that they are better at tracking how agents' current beliefs change, but worse at inferring past beliefs. figure 2 illustrates the difference between our model and the best performing gpt-4o baseline (i+na+fs) in greater detail. despite few-shot prompting, gpt-4o tends to assign ratings of 0.8 or more to many statements that humans find parser example english the player believes that box 3 is empty. elot gold believes(player, formula(empty(box3))) grammar-cons. believes(player, formula(empty(box3))) unconstrained believes(player, empty(box3)) (syntax error) lowered @xmath8, empty(box3)), threshold(believes)) grammar-cons. and(>=(prob of(player, empty(box3)), threshold(believes)), empty(box3)) (semantic error) @xmath9, empty(box3)), believes) (syntax error) labtom captures how human evaluations of epistemic language change with agent behavior. our model predicts that human evaluations of epistemic sentences should change systematically as they gain more information about an agent's percepts and actions. as figure 3 illustrates, this is what we find. when the player sees that a box is empty, both humans and our model sharply decrease their ratings for statements claiming that the player believes a key is in that box (e.g., figure 3b , s4 current). when the player approaches one box instead of another, both humans and labtom gain confidence in statements about the relative likelihood of key locations (figure 3a , s5 current). we analyze just one scenario (figure 3a ) in detail. here, the player first walks upwards away from box 3 (judgment point 1), leading both humans and labtom to assign scores of greater than 0.5 to ''the player believes that box 3 is empty'' and ''the player initially believed that box 3 was empty''. in contrast, the modal sentence ''the player believes that box 3 may contain a red key'' is rated lower. this is because if the player did believe that box 3 might have the (red) key, then it is more likely that they would have looked in box 3. however, this does not occur. after the player opens box 2 and finds it empty, then walks back down towards box 3 (judgment point 3), both humans and our model decrease their confidence in the statement ''the player believes that box 3 is empty''. they also decrease confidence in ''the player initially believed that box 3 was empty'', but less sharply. this is because there are at least two possibilities consistent with the player's actions: they could have firmly believed that box 3 was empty, or they could have just believed that box 3 was less likely to contain the relevant key than box 2, without all-out believing that it was empty. this ability to understand beliefs about relative likelihood is made explicit by how ratings change over time for ''the player believes that box 3 is more likely to contain a red key than box 1.'' consistent with the principle of rational action, both humans and labtom assign a high score to this sentence once the player goes back to box 3, forgoing box 1. across a range of other settings (figure 3b-c ), our model largely captures fine-grained changes in how people evaluate a variety of epistemic expressions, including modal sentences, ignorance claims, and expressions of uncertainty. unlike prior btom models that lack language-like belief representations @xcite , labtom also distinguishes observer and agent uncertainty, assigning high ratings to claims that the agent is uncertain (figure 3c , s5 current), and vice versa. in contrast, the best llm baseline (gpt-4o, i+na+fs) often fails to adjust its ratings in the same direction as humans, while rating implausible statements highly. we discuss these results more in the appendix, alongside cases where our model comes apart from humans. labtom distinguishes in-context and out-of-context epistemic language. to investigate how our model generalizes to a larger set of epistemic expressions, we performed the in-context vs. out-of-context likelihood compar- model statement likelihoods accuracy in-ctx / out-of-ctx / diff. in vs. out current beliefs (241 statements) labtom (ours) 0.78 / 0.48 / +0.30 (0.03) 0.71 (0.03) non-planning 0.70 / 0.54 / +0.16 (0.02) 0.65 (0.03) true belief 0.35 / 0.35 / +0.01 (0.01) 0.11 (0.02) gpt-4o (i+na) 0.80 / 0.62 / +0.18 (0.03) 0.59 (0.03) initial beliefs (228 statements) labtom (ours) 0.73 / 0.51 / +0.22 (0.03) 0.70 (0.03) non-planning 0.70 / 0.70 / -0.01 (0.01) 0.39 (0.03) true belief 0.51 / 0.50 / +0.00 (0.01) 0.16 (0.02) gpt-4o (i+na) 0.76 / 0.68 / +0.07 (0.02) 0.38 (0.03) 6 . for both current and initial belief statements, we find that labtom assigns significantly higher scores when a statement is evaluated in-context vs. out-of-context, correctly classifying the context about 70% of the time by assigning a strictly higher in-context score. on closer inspection, many statements that are incorrectly classified turn out to be plausible in either context, resulting in equal or close-to-equal scores. the ablated models and gpt-4o perform significantly worse than labtom, especially for initial beliefs. ## discussion our experiments show that, similar to humans, our labtom model is able to coherently interpret and adjust its evaluations of natural language statements about agents' beliefs, whereas state-of-the-art multimodal llms struggle with this task. this ability is mediated by our elot representation and semantic parser: without a compositional representation of epistemic concepts that is aligned with the structure of natural epistemic language, we find that translation accuracy drops significantly, even when using a grammar-constrained llm parser. we also find that labtom largely distinguishes in-vs. out-of-context sentence usage on a large and diverse set of crowd-sourced epistemic language, indicating the generalizability of our approach. that said, our model is not without limitations. as we discuss at greater length in the appendix, labtom's outputs depart from human judgments in several interesting ways, suggesting the need to account for (i) contextual adaptation of probability thresholds via pragmatic reasoning @xcite , (ii) the role of justification in human's intuitive evaluation of knowledge claims @xcite , and (iii) bounded human reasoning about logical implications @xcite . labtom is also an ideal observer model that does not scale readily to large belief spaces, leaving open how humans tractably infer and evaluate claims about others' beliefs @xcite , perhaps by focusing on occurent beliefs @xcite that are relevant to others' goals. finally, labtom is a model of how people interpret epistemic language, but full understanding also includes the ability to produce such language. this could potentially be achieved by inverting the elot module of our model, using it to translate salient or conversationally-relevant inferences about an agent's beliefs into natural language. by extending our model in this way, we stand to gain an even richer account of what it means to understand epistemic language. tan zhi-xuan, jordyn mann, tom silver, josh tenenbaum, and vikash mansinghka. 2020. online bayesian goal inference for boundedly rational planning agents. advances in neural information processing systems, 33. tan zhi-xuan, lance ying, vikash mansinghka, and joshua b. tenenbaum. 2024b. pragmatic instruction following and goal assistance via cooperative language guided inverse plan search. in proceedings of the 23rd international conference on autonomous agents and multiagent systems. a dataset collection