Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning (2025)

Abstract: Theory of Mind (ToM) capabilities in LLMs have recently become a central object of investigation, sparking debates and discussions. In this position paper, we explore many lines of work in different communities in AI and cognitive science. Inspired by cognitive work, we view ToM tasks as a two-step process: (I) first, determining whether and how to invoke ToM, which includes setting the appropriate Depth of Mentalizing (DoM); and (II) second, applying correct inference given the appropriate DoM. We identify that many works about ToM in LLMs, such as benchmarks and add-on modules, tend to unjustly overlook the first step and focus exclusively on the second one, which can be framed as a logic-reasoning task. We support our distinction with empirical evidence about the difficulty of the different steps in existing benchmarks. We conclude with suggestions for improved evaluation of ToM capabilities, inspired by dynamic environments used in cognitive tasks in biological agents.

## introduction the ability of large language models (llms) to efficiently integrate social information is essential for ensuring ai trust and safety, as well as for reasoning about situations involving multiple agents. this capability, also known as mentalizing, or theory of mind (tom) @xcite , relies on inferring and representing otheragents' beliefs, desires, and intentions. llms have been recently regarded as generalpurpose reasoning models @xcite , and their tom capabilities have come under scrutiny, particularly as to whether social cognition can emerge purely from associative principles @xcite . notably, @xcite 1995). this claim sparked debates on the correct interpretation of these results @xcite @xcite . existing benchmarks for tom primarily concentrate on whether agents have correct beliefs about others @xcite . however, within the context of social interaction, tom capabilities involve two steps @xcite : (1) determining the depth of mentalization (dom; @xcite to use, which specifically includes the decision of whether to model the agents separately (self-other divide); and (2) applying correct inference for each agent's mental state, given the depth. failure in any step leads to an incorrect conclusion (see figure 1 ). we argue that very little attention is paid to whether llms correctly invoke tom (when needed). determining this is critical: (1) it ignores a critical theoretical component of tom that has been well-defined in humans; (2) it confounds tom errors with poor or fuzzy logic @xcite , and (3) the act of mentalizing has a price, both in resources and performance @xcite @xcite , and so more precisely understanding moment-to-moment tom in llms may improve compute efficiency at equivalent parameter sizes. of note, we do not focus on the evolutionary pressures and resource-rational necessities of social representation -the why of tom -and merely focus on the how. in this paper, we first discuss contemporary approaches to tom in ai research at a high level. this includes llm benchmarks, tom add-ons for llms, and agent-focused tom models. we identify that the focal point of current llm and ai research is logical inference @xcite , corresponding to step ii (and type c errors), which is based on the determination of dom in the first stage. we experiment with two common tom benchmarks to demonstrate this conclusion. we then turn to the cognitive science literature and identify differences in the definition of tom behind biological and synthetic agents, as the former focuses also on step i -the act of mentalizing, and not only on correct inference. we finally identify ways to improve tom research in ai, inspired by work in biological agents. our contributions are: (1) we explore and discuss ways of addressing tom in various fields. (2) we formally describe tom tasks as a two-step task, inspired by cognitive science. (3) we identify a lacuna in existing llm works which tend to overlook the first step of mentalizing and focus instead on logic problems. (4) we provide a roadmap for appropriate evaluation of tom invocation inspired by biological agents. ## tom in ai research here we analyze the various approaches and challenges used to model and assess tom in the ai communities. we begin with a review of existing ai methods before discussing llm-focused approaches. ## discussion and conclusion in this work, we cast tom in llms as an interdisciplinary topic. we argue that the problems addressed by existing nlp benchmarks and ai works are essentially non-interactive logical problems. while many of the benchmarks are valuable as reasoning tests and may provide insights into the performance of llms, they do not address the core issue of tom in llms: interaction, and social adaptation. we argue that a better understanding of tom in llms requires: (1) benchmarks that test whether dom is correctly invoked or not, (2) evaluation for tom in interactive settings where the model is an active agent, and (3) distributional assessment of dom dynamics, given a social context (cooperative vs competitive). the requirement for social dynamism is expressed in cognitive science and computational psychology. neural outcomes show distinct representations when considering non-social versus social outcomes @xcite , demanding a theoretical distinction that captures social dynamics sufficiently @xcite @xcite . we believe this distinction should also be reflected in the way we benchmark models for tom. @xcite , argue in favor of dynamic, user-centric benchmarks for tom evaluation. our work extends this call, emphasizing the role of dynamic settings for a phase in the tom reasoning that is commonly overlooked. indeed, some previous works introduce tom tasks for llms or formal models in interactive domains @xcite @xcite . however, these benchmarks are limited in their scope as they tend to model simplified fixed worlds. we believe that a meaningful understanding of tom capabilities should be achieved by extending works like these with additional domains and contexts. whereas a single scenario may not fully capture the range of behavior, constructing multiple diverse ones should lead to better coverage of tom complexities. this should include situations without full context (e.g., without explicit information about the requested), social situations (e.g., with varying levels of cooperation), and tom behavior of other agents (e.g., heterogeneous dom levels in the population).