On the Role of Linguistic Features in LLM Performance on Theory of Mind Tasks (2025)

Abstract: Theory of Mind presents a fundamental challenge for Large Language Models (LLMs), revealing gaps in processing intensional contexts where beliefs diverge from reality. We analyze six LLMs across 2,860 annotated stories, measuring factors such as idea density, mental state verb distribution, and perspectival complexity markers. Notably, and in contrast to humans, we find that LLMs show positive correlations with linguistic complexity. In fact, they achieve high accuracy (74-95%) on high complexity stories with explicit mental state scaffolding, yet struggle with low complexity tasks requiring implicit reasoning (51-77%). Furthermore, we find that linguistic markers systematically influence performance, with contrast markers decreasing accuracy by 5-9% and knowledge verbs increasing it by 4-10%. This inverse relationship between linguistic complexity and performance, contrary to human cognition, may suggest that current LLMs rely on surface-level linguistic cues rather than genuine mental state reasoning.

## introduction while large language models (llms) demonstrate remarkable capabilities in code generation @xcite , multilingual translation @xcite , and long-context conversational memory @xcite , their performance on social reasoning tasks remains fundamentally unreliable. although llms are approaching human accuracy on simple false-belief tests @xcite , their inconsistent patterns on more sophisticated tasks requiring social reasoning @xcite , suggest they rely on mechanisms fundamentally different from human cognition. at the heart of this reasoning lies theory of mind (tom), the human ability to model others' mental states, especially when their beliefs contradict reality @xcite . classic false-belief tasks, such as the sally-anne test, probe this ability by requiring a model to predict an agent's actions based on their incorrect beliefs. computationally, this requires processing intensional contexts created by attitude verbs like "believe," where the truth of a proposition is evaluated relative to a subjective perspective rather than objective reality @xcite . recent findings reveal that llms capable of passing standard false-belief tests often fail on their minor variations @xcite . this suggests they lack a robust understanding of how mental state verbs create distinct semantic contexts that block standard entailment @xcite . in this paper, we empirically analyze six llms on tom tasks to understand their failure patterns on tasks requiring semantic reasoning. we examine 2,860 stories by quantifying linguistic features related to information structure (idea density) and lexical patterns (mental state verb density). we also manually annotate each story for its level of perspectival complexity and linguistic markers. we address three key research questions: (rq1) to what extent do idea density and mental state verb density correlate with llm performance on mental state reasoning? (rq2) how do linguistic markers of perspectival complexity influence model performance on tom tasks? (rq3) what systematic failures emerge across different model architectures? we find that llms exhibit opposite correlations to humans in terms of linguistic complexity, yet paradoxically achieve the highest accuracy on high-complexity stories with explicit mental state scaffolding. these findings suggest that llms rely on surface linguistic cues rather than genuine perspective-tracking. ## a semantic framework for theory of mind to formally analyze tom, the capacity to attribute beliefs, desires, and intentions to oneself and others, and to recognize that these states may diverge from reality @xcite , we ground our analysis in a multi-agent epistemic-doxastic logic @xcite . this framework provides a precise language for representing nested perspectives and allows for systematic categorization of the perspectival complexity of social reasoning scenarios @xcite . a mental-state representation in our framework consists of an agent @xmath0, an attitude (e.g., knowledge k, belief b), and a content formula φ expressing a proposition about events or states. our formal language l [1][2]...[n] extends propositional logic l with a set of modal operators [i], each corresponding to a mental attitude held by a specific agent. for instance, for agents a, @xmath1, the formula k a φ expresses "a knows φ," and b b φ expresses "b believes φ." these operators can be nested to represent higher-order tom, as in k a b b ¬p ("a knows that b believes that p is false"). ## related work early tom evaluations revealed superficial success on classic false-belief tasks, such as the sally-anne test @xcite , prompting more rigorous benchmarks. recent work like tombench @xcite and epitome @xcite benchmarks show a recurring pattern of models handling basic belief-tracking but failing on tasks requiring pragmatic or social inference. this weakness in compositional reasoning, also probed by procedurally generated narratives in exploretom @xcite , suggests models exploit statistical shortcuts rather than genuinely tracking mental states. other work reveals failures in more fundamental capabilities, such as the two word test study @xcite . a common finding across these methods is that models often succeed by exploiting statistical patterns rather than by genuinely tracking mental states. however, prior work has not systematically distinguished between tasks with low and high intentionality (i.e., simple belief attribution versus complex deception) or investigated how specific linguistic features influence llm performance on tom tasks. our work aims to address these gaps. ## conclusions we analyzed linguistic features in llm performance on tom tasks, revealing surprising patterns: (1) llms show positive correlations with idea density and msvd, opposite to humans' negative correlations, (2) models paradoxically excel on high complexity stories (74-95%) while struggling with low complexity (51-77%), and (3) all models fail systematically when implicit reasoning is required. these patterns suggest llms may leverage explicit linguistic markers rather than genuine mental state reasoning, though our correlational analysis cannot prove causation. the complexity paradox, where explicit mental state scaffolding aids performance, warrants further causal investigation to understand whether models truly rely on surface cues or develop deeper representations.