From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models
Authors: Xinyang Li
Siqi Liu
Bochao Zou
Jiansheng Chen
Huimin Ma
Venue: ICML 2025 poster (2025)
Keywords: Multimodal Large Language Models (MLLMs)
Theory of Mind (ToM)
Interpretability
Attention Mechanisms
Primary area: social_aspects->accountability_transparency_and_interpretability

Abstract: As large language models evolve, there is growing anticipation that they will emulate human-like Theory of Mind (ToM) to assist with routine tasks. However, existing methods for evaluating machine ToM focus primarily on unimodal models and largely treat these models as black boxes, lacking an interpretative exploration of their internal mechanisms. In response, this study adopts an approach based on internal mechanisms to provide an interpretability-driven assessment of ToM in multimodal large language models (MLLMs). Specifically, we first construct a multimodal ToM test dataset, GridToM, which incorporates diverse belief testing tasks and perceptual information from multiple perspectives. Next, our analysis shows that attention heads in multimodal large models can distinguish cognitive information across perspectives, providing evidence of ToM capabilities. Furthermore, we present a lightweight, training-free approach that significantly enhances the model’s exhibited ToM by adjusting in the direction of the attention head.

From Black Boxes to Transparent Minds: the Theory of Mind in Multimodal Large Language Models
Evaluating and Enhancing
Xinyang Li"
Siqi Liu"l Bochao Zou"
11
Jiansheng Chen Huimin Ma
Abstract
they, like the robot TARS in Interstellar, accurately compre- hend and execute both explicit and implicit tasks assigned by humans (Figure 1) ? It leds to the question that whether large models have ToM. Evaluating the ToM capabilities of large models is crucial for understanding their potential to meaningfully engage with human communication and reasoning. Some works utilized the classical Sally-Anne to test the ToM capabilities of machines (Nematzadeh et al., 2018; Grant et al., 2017; Ullman, 2023; Lore et al., 2024; Kosinski, 2024). While these evaluation methods offer pre- liminary insights into the ToM capabilities of large models, they remain limited in scope.
As large language models evolve, there is grow- ing anticipation that they will emulate human- like Theory of Mind (ToM) to assist with routine tasks. However, existing methods for evaluat- ing machine ToM focus primarily on unimodal models and largely treat these models as black boxes, lacking interpretative exploration of their internal mechanisms. In response, this study adopts an approach based on internal mechanisms to provide an interpretability-riiee assessment of ToM in multimodal large language models (MLLMs). Specifically, we first construct a multi- modal ToM test dataset, GridToM, which incor- porates diverse belief testing tasks and perceptual information from multiple perspectives. Next, our analysis shows that attention heads in multi- modal large models can distinguish cognitive in- formation across perspectives, providing evidence of ToM capabilities. Furthermore, we present a lightweight, training-free approach that signifi- cantly enhances the model's exhibited ToM by adjusting in the direction of the attention head.
Most existing studies adopt unimodal approaches, focusing on either text videos, and lack comprehensive agent-level information (Gandhi al., 2021; Nematzadeh et al., 2018; Grant et al., 2017; Le et al., 2019; Amirizaniani et al., 2024). In contrast, human social interactions rely on reasoning about others' mental states by integrating multimodal in- puts, such as visual and linguistic data. Although some studies (Jin al., 2024; Shi al., 2025) have attempted to extend ToM evaluations of large models to multimodal en- vironments using video-based datasets, their datasets often incorporate excessive high-level information, such as spa- tial relationships, agents' tasks, and action trajectories (Ma et al., 2023). Moreover, in real-world datasets, an agent's perception of environmental events cannot be accurately captured. For example, in the MMToM-QQ QA dataset (Jin et al., 2024), is impossible to determine from the video modality alone whether the protagonist truly "saw" the plate. Consequently, the accuracy of ToM evaluations may depend on the quality of perceptual information, which could lead to correct or incorrect performance for reasons unrelated to genuine ToM capabilities. Unlike these prior works, we construct a dataset that is based on 2d grid world, enabling large models perceive the full context of the physical world through the video modality while supplementing cog- nitive perspective information for each agent through the text modality.
1. Introduction
Theory of Mind (ToM) a psychological term referring to infer mental states to self and others. This capability is fundamental to human social cognition and emotional un- derstanding. In recent years, the rapid development of large models raised researchers' consideration: can they interact with us in manner similar to humans? For example, could
'Equal contribution 'School of Computer and Commu- nication Engineering, University of Science and Technol- ogy Beijing, Beijing, China. Correspondence to: Bochao Zou < <zoubochao @ut.ee.uuuutbb............ Huimin Ma <mhm- pub@ustb.cdu.cn.
Furthermore, the majority of assessments of ToM capabil- ities in large models take a black-box approach, relying heavily on question-answering tasks to infer conclusions (Xu al., 2024), as demonstrated in Figure while lacking
Proceedings of the 4nnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Project Page: httpa:://annaisavaaiii io/GridTon

---

Evaluating and Enhancing ToM in Multimodal LLMs
TARS'S 's bellef
Input Enbedding
Let me it.
Realizing that Cooper decided to lo the spacecraft himself. Cooper believed that he could successfully complete the nission, rationally, Cooper knew that TARS considered it was almost suicide mission but still his support.
text
Transformer Encoder Layers
though,
Attention Mechanism
N :desire inference
Hiden States
N :first-order-bellef belief
expression
video
second-order-belle belief
Decoder Layers
TARS's s response
I know you feel responsible, but you don't have die for it.
Output
Renain Unexplored
Our work focuses on the internal representations of the model rather than relying solely its perforrance.
Previous work has focused on evalusting Ton abilities of LLMS through their performance in answering Tom-related tasks,
Figure 1. This illustration highlights the integration of different levels of ToM: recognizing an agent's desire (Cooper wants to pilot), a first-order belief (he believes he do it), and second-orde belief (he believes TARS perceives it risky). These nested mental states are crucial in evaluating advanced ToM.
interpretability-orff orriette methodologies (Mao al., 2024). However, multimodal large language models (MILMs) are known to exhibit hallucination phenomena, where the qual- ity of prompts can significantly impact their performance on snvstion-ansseeri "understand" a concept but fail to provide a "correct" re- sponse (Bai et al., 2024). Like demonstreted in Figure 2, factors influencing QA performance not limited to ToM capabilities. Elements such as hallucination and scenario understanding also affect the ToM evaluation results of pre- vious studies. Consequently, is insufficient to determine whether MLLMs possess ToM capabilities solely based on their performance in ToM tasks. In contrast, our goal to examine whether these models develop internal represen- tations that distinguish agents' mental states from differ- ent perspectives, beyond merely analyzing output accuracy. This will provide an interpretable explanation of whether MILMS possess ToM capabilities.
corporates diverse belief-testing tasks alongside perceptual information from multiple perspectives; (2) we conduct an in-depth analysis of the internal representations of MLLMs through interpretability methods, focusing on their interme- diate activations; (3) we propose training-fre ffree approach to enhance ToM performance in MILMs by strategically shifting activations along specific directions.
tasks. This means that a model may
2. Related Work
2.1. Dataset for Evaluating Theory of Mind
Inspired by traditional experiments used to evaluate ToM in children, some studies have applied the classic Sally- Anne task assess ToM capabilities in machines (Grant et al., 2017: Eysenbach et al., 2016; van Duijn et al., 2023). Most existing datasets and methods for evaluating ToM capabilities are based single modality (Xiao al.; Am- irizaniani et al., 2024; Wu et al., 2023; Yim et al., 2024) For text inputs, Mindgames (Sileo & Lernould, 2023) is a dataset grounded in dynamic epistemic modal logic, de- signed to evaluate the epistemic reasoning of large language models through controlled problem generation. OpenToM (Xu et al., 2024) is dataset characterized by long-form narratives featuring real-world individuals and events, em- phasizing the complexity of storylines and the diversity of character relationships. Similarly, ToMi (Le et al., 2019) is comprehensive dataset encompassing multi-agent scenarios, multi-episode contexts, multi-turn question answering, and tasks involving mental state reasoning. Some tried videos, (Shu et al., 2021) is benchmark composed of program- matically generated 3D animations, where agents interact with objects and move within various physical constraints.
MLLMs ToM
Input
hallucination
Output
Scenario understanding
Figure 2.
The ligure highlights the limitations of current ToM eval- uations, namely that other model capabilities (such hallucination and scenario understanding) may interfere with the results.
In summary, our main contributions are as follows: (1) we introduce GridToM, a novel multimodal ToM dataset that in-
2

---

Evaluating and Enhancing ToM in Multimodal LLMs
SymmToM (Sclar al., 2022) is a multi-agent reinforce- ment learning environment called SymmToM, where agents can simulate the mental states of others.
the perceptual information of all agents in the scene. For each story, we apply randomized manipulations to the evalu- ation data, including room configurations, agent states, and action trajectories.
The MMToM-QA dataset (Jin et al.. 2024) is the pioneer- ing resource aimed at assessing machine learning models' ability to infer mental states from multimodal data, com- bining video and text in real- tasks. Similarly, other studies (Chen et al., 2024; Shi et al., 2025) have also ex- plored this domain. However, real-world video datasets often lack perspective information, making it challenging to infer nigh-dimensional details such as whether the protago- nist in a story truly notices specific objects. This limitation can affect the accuracy of ToM task performance.
3.1. Overview
GridToM is generated based on the Multigrid library (Ogun- tola et al., 2023), which builds on Minigrid (Chevalier- Boisvert et al., 2023). It provides a multi-agent discrete gridworld environment, a simple and commonly used set- ting for ToM research in the machine learning community. The complete dataset construction pipeline and accompany- ing quality-contre procedures are detailed in Appendix I. It has been demonstrated that simple 2D gridworld can effectively support the development of diverse ToM tests (Ma et al., 2023), encompassing all mental states defined in ATOMS (Abilities in ToM Space) (Beaudoin et al., 2020).
To address these challenges, we developed dataset based on a 2D grid world environment, which provides simplified character relationships, complete physical information, and comprehensive perceptual data for all agents. The 2D grid world framework not only enables the creation of manipula- ble visual causal stories for training classifiers to distinguish perspective information but also avoids introducing high- level information. This reduces the cognitive burden on MLLMs, allowing them focus on the core ToM tasks.
Our dataset comprises 1,296 video-text pairs, with each video having a resolution of 294x420 pixels and approxi- mately 40 frames. Each map is 10x7 grid featuring three rooms and two agents. The dataset includes 27 distinct maps, each with two initial agent positions, two orientations, six sequences of agent movements into target rooms, and paired True Belief (TB) and False Belief (FB) stories, generating the 1,296 pairs. An example is shown Figure 3.
2.2. Benchmark
The question of whether large models exhibit genuine ToM capabilities remains topie of ongoing debate. Some evalu- ation studies suggest that certain large models demonstrate degree of ToM ability in reasoning tasks, such as un- derstanding others' beliefs, intentions, and mental states (Kosinski, 2023; Bubeck et al., 2023; Zhou et al., 2023). However, other studies argue that the observed ToM-like capabilities of large models are not based on true general- ization but instead result from learning patterns in question- answering tasks (Shapira al., 2024; Ullman, 2023; Stra- chan et al., 2024) or lack of ToM (Sap et al., 2022; Verma et al., 2024) conclusions about ToM capabilities in large models rely on performance in QA tasks. In contrast, our research aims to address this question from an inter- pretability perspective by investigating the internal repre- sentations of MLLMs related to mental state understanding, rather than solely depending the quality of their question- answering performance.
3.2. Baseline
Our experiments reproduce the classic unexpected transfer task (Baron-Cohen et al., 1985). We simulate a complete interaction process within the gridworld environment. The testing dataset consists of 500 samples. An additional 148 samples were used for training and validation of our model, with 75% allocated for training and 25% for validation. To ensure that each selected model processes the input without errors, we use four key frames and three intermediate frames between them as input for video-based tasks, instead of providing all frames. The temperature of all models was set to O.
Human Participants To evaluate human performance in the proposed dataset, 12 human participants were recruited to answer the test questions, all of them gave informed consent. The age range was from 23 to 32 years, with an average of 24.8 (SD= 2.3).. Each participant was randomly assigned 100 samples from the 500-test dataset, and the final performance score was obtained by averaging their results. Importantly, in the video-only condition, core envi- ronmental rules- rules-including that closed doors block agents' perception-wee were clearly explained prior to testing. Any omitted narrative content was non-instructional and did not impact task comprehension. This ensures that participants were not misled or disadvantaged by the absence of textual guidance.
3. GridToM
Why not previous grid world based ToM dataset? The previous datasets only included unimodal inputs and lacked annotations for character perspective information and event details, making them unsuitable positive and negative samples for the subsequent experiments in this study.
Unlike previous ToM works, GridToM provides manipula- ble multimodal -ingusiin causal stories and includes
3

---

Evaluating and Enhancing ToM in Multimodal LLMs
#frame0
#frame11
#frame27
#frame36
Init Belief
How many agents are there? what colors are they?
what color agent walk into? what color roon did the white agent walk into?
did the yellow
First order belief
[0,11)
(11,27)
[19,36]
At the very end the video, which color room does yellow agent elieve the white agent should be in ?"
Initially, yellow
After the white agent closes the door the red the yellow agent goes over the green
while the yellow agent green roon (with the door closed), agent opens the red roon door.
agent stands in the c-
orridor. then walks towards the red roon, opens the door
Red
Purple
True belief
False belief
#framee #frame11 #frame27 #frame36
Omniscient perspective Always access all the information of the event
M sprotagonist
4 :omniscient
Agent's s perspective when the door of the roon is closed, outside information will be inaccessible.
Second order belief
At the very end of the video, which color room does the white agent believe the yellow agent thinks the white agent should in?
agent (prottggonist)
rOORS
Red
Purple
A
sgent(Grotagendat)
True belief
A
)pent((prtictppnnt
False belief
(close)))
agent (partticacc)))))
(oprsso))))
A4:participant
M sprotagonist
Figure FB sample GridToM includes an omniscient- cenniicient-erssecctivv video covering entire event timeline, along with omniscient- perspective textual descriptions for each time interval. For all agents involved in the event, provide full physical perspective information across the timeline. When an agent closes a door, we mask its perception any information beyond the door to simulate realistic sensory limitations. Each sample contains three types of questions (illustrated on the right). For each video-text pair, the accompanying text annotations include environment descriptions, initial belief assessments, first-order belief assessments, and second-order belief assessments. We provide the full text and the video in Appendix C.
MLLMs We evaluated MLLMs under both multimodal and pure-video conditions, including GPT-4o (Achiam et al., 2023), Doubao- 1.50o--..-vviiioppppe (Team, 2025), DeepSeek-v12- (Touvron
2023), and Qwen-VLeee VL-7B-Instruct (Bai et al., 2023).
We evaluate the models under three conditions (Jin al., 2024): Multimodal QA with both video and text inputs, Text- only QA with text inputs only, and Video-only QA with video inputs only. We list our detailed setting Appendix F, Results are in Table 1, we demonstrate baseline of existing MILMS on GirdToM, while the result of initial belief test is in Appendix D.
small (Liu al., 2024), Beon2-VL--b-instruue (Bai al., 2023). Ad-
et al., 2023), ditionally, for fairness, we evaluated both subclasses of MLLMS using the same methodology. following previous work (Jin et al., 2024). Specifically, we sample a fixed set of seven frames from video (including four key frames and three evenly sampled intermediate frames) along with the corresponding annotations evaluate Tmage-Teext-to-Teee Models and Video-Text-to-11 Models.
4. Belief representation in MLLMs
4.1. Model
LLMs We also evaluated GridToM on various large lan- guage models (L.LMs) using text-based input only, including GPT-4o (Achiam et al., 2023), Doubao-1.5-Pro-. 2025), DeepSeek- V3 (Liu et al., 2024), LLLMA-3.3-700 Instruct (Dubey al., 2024), Histral-7B-Instruct (Jiang al., 2023), VA- -eett-exx-iieee Video-7B-HH (Touvron et al.,
In the exploration and modification phases, utilize the LLaVA-Next Video model, MLLM specifically designed for video understanding and generation tasks. Additionally, we utilized the Qwen2-VI. model to perform the aforemen- tioned two phases, demonstrating the effectiveness of our
(Team,

---

Evaluating and Enhancing ToM in Multimodal LLMs
approach.
H T41 = Ti + Σ
4.2. Attention Feature Extraction
Wi,
(1)
Where Allnf denotes the attention operation of the n-th head at the I-th layer, Pf E DD DH maps stream activation into a D-dimensional head space, and W7 E RD DH is the output projection matrix. Inspired by (Li et al., 2024), the probing and intervention steps occur after Attn and before W.
grid consisting
which coler roon does the yellow belleve the white agent should be in? r.ded
colien..
purule
Desc Apttion
Queeet
LM
We extract the output of each attention head at every layer, capturing the activation at the final token position, denoted as X E _LXXXD Each attention head activation is asso- ciated with belief labels Y, and Y. which represent the correctness of the protagonist's perspective and the omni- scient perspective, respectively.
heads
perspective separation
Jayers
heods
Inference
separability
Due to the simplicity of the 2D gridworld, in TB scenar- ios, the protagonist's perceptual information is equivalent to omniscient information. This allows the protagonist's perspective video to be substituted by the omniscient per- spective video. In TB scenarios, the protagonist's belief labels Y, = True and Y, = False help identify the layers and heads sensitive to reasoning based on perceptual infor- mation. In FB scenarios, the protagonist's belief labels help identify the layers and heads that are sensitive to integrating belief information across perspectives. For the omniscient belief label Y,, the correct label corresponds to multimodal data with an omniscient perspective and accurate belief in- ference, while the incorrect label includes either an incorrect perspective or an incorrect inference result.
Jayers
Intervention
Figure 4. Overview of Our Workflow. We first constructed the GridToM dataset and conducted henchmark testing of MMI Ms on it. Subsequently, we input video-text pairs probe the internal at- tention representations of the models. Using logistic regression, we performed binary classification on the representations of positive and negative samples to identify attention heads that are sensitive to perspective separation and belief representation. Targeted inter- ventions were then applied to the top K most sensitive attention heads during inference.
This design of correct and incorrect labels targets two as- pects: perspective separation and belief inference, integrat- ing them into a unified framework. Targeted interventions are applied to the heads that are sensitive to these two as- pects. We collectively define correct perspective separation and correct belief inference as true labels, and their oppo- sites as false labels. our approach, we only use the correct and incorrect labels from the protagonist's perspective to indicate and guide perspective separation and belief reason- ing.
We begin by investigating whether MILMs represent and how they represent the beliefs of different agents. Our objective to decode the belief states of various agents from the activations of attention heads, given multimodal story narratives and corresponding belief statements.
Specifically, MILMss first embed the input multimodal data into high-dimensional spaces, including visual inputs V = {VI, U2,.., } } and textual inputs X = {21, Z2,,.,,..... }. where m and n represent the token lengths of the visual and textual inputs, respectively. The model concatenates the visual and textual embeddings into a unified input sequence T concat( V, X) E F f(m+n))D where D) denotes the dimension of each attention head and H represents the num- ber of attention heads. This unified input is then passed through a Transformer architecture with I. layers.
Yp ( NY
(2)
For different belief tasks, our prohing strategies vary slightly, while the interference strategy remains consistent, as de- tailed in Appendix B.
4.3. Probing
Probe is a standard tool for analyzing the internal represen- tations of networks (Köhn, 2015; Gupta et al., 2015). The idea is to train a classifier (probe) on the activations of the network to distinguish specific types of inputs or outputs.
At each layer, the concatenated input undergoes multi-head attention. The multi-head attention mechanism (MHA) can be approximated as Equation (1):
5

---

Evaluating and Enhancing ToM in Multimodal LLMs
1 00
LOW INAEO
20
Iotection the Tuthks Dieccion
9.95
10
Layer
15
85
20
23
25
75
5
10
15
20
25
30
0 70
3
Projection the Trutruu Direction
Head
(A)
(0)
Figure 5. (A) The linear probing accuracy of heads across layers LLaVA-Next- Video on the test set. The x-axis represents the heads, and the y-axis represents the layers. Dark green indicates higher accuracy, with 50% serving as the baseline accuracy for random guessing. (B) Kernel density estimation (KDE) plot of activations in layer 28, head 15 of L-AV--eex--iddeee projected onto the top two true directions, showing real (green) and false (orange) pairs. Marginal distributions are displayed along the top and right axes. (C) & (D) The linear separability of belicf representations is explained through a visual interpretation of the typical representation space, demonstrating the attention feature extraction strategy proposed in Appendix B. The binary combinations of Y, and Y, labels correspond to the combinations of TTB and FB with correct and false beliefs. For belief-sensitive heads (e.g.. head 15 in layer 28), they can effectively estimate the boundaries of the belief states for both the omniscient perspective and the protagonist's 's perspective. whereas insensitive heads cannot. These four combinations form distinct clusters in the representation space without overlap, with clearly defined decision houndaries. The probing weight direction represents the decision boundary, effectively separating these belief combinations.
distributed across different heads in various layers, starting from middle layers to the final layers, whereas the initial layers lack this capability.
+ (28+6))
where f/ denotes a logistic sigmoid function for (x0 + b),
Meanwhile, Figure 5 (C) demonstrates the linear separa- bility of belief representations. We visualize the attention feature extraction strategy proposed in Appendix B, where the four clusters represent the correctness of perspective separation and belief inference. These four combinations are distinctly clustered without overlap, with clear decision boundaries. This suggests that MLLMs indeed develop in- termediate representations reflecting multi-perspective infor- mation extraction and belief inference based on the complete information provided. This phenomenon indicates that these attention heads implicitly encode the belief states of other perspectives in a linearly decodable manner. Furthermore, due to the simplified information in the 2D grid world, these implicit beliefs are easily propagated the final layers.
while 0 E RD and E IR represent the weight vector and bias, respectively. The parameters and b are optimized by minimizing the cross-entropy loss.
We first conducted probing experiments on GridToM. The results are shown in Figure 5. The probing results for dif- ferent models are listed in Appendix G. Subsequently, we performed the same probing experiments on the real-world multimodal ToM dataset MMToM-QA, to validate the gener- alizability of our probing method. The detailed information of the MMToM-QA QAA dataset is presented in Appendix H, as shown in Figure 22.
For each attention head at every layer, we train a separate linear binary probe to fit the belief labels Y, and Y,. Given a dataset of size N, we obtain the corresponding activations of a single attention head, denoted as X E NDD the corresponding belief labels are Y E {0, 1jN. We use a logistic regression model to predict the probability of the belief being true. In simple terms, we select the top K atten- tion heads ranked by accuracy in Figure 5 (A) and use the decision boundary in Figure 5 (C) as the direction for inter- vention weights Figure 5 (A) shows the validation accuracy of the linear probe, indicating that many attention heads can accurately capture belief states from the protagonist's 's per- spective. These abundant informational representations are
To further understand belief representations in the activa- tion space of attention heads, we visualized the geometric shapes within the activation space, as shown in Figure 5(b). Specifically, we reduced the activation space dimensions to two using Principal Component Analysis and selected two orthogonal directions (0 (0.00) with the maximum variance for separating true and false features. We visualized the ge- ometric projections onto A and 0', observing partial overlap and distinct representations between the two distributions. Notably, the second direction still exhibits unique represen- tation distributions, suggesting that the concepts of "true"
6

---

Evaluating and Enhancing ToM in Multimodal LLMs
and "false" coexist in subspaces within the attention space, rather than being confined to a single unified space.
performance on TB tasks declined slightly, as humans in- ferred the protagonist's perspective but were occasionally misled by scenarios contradicting physical intuition, such as omniscient visibility through a doorway. The absence of textual clarification further amplified these misjudgments, as prior knowledge influenced their reasoning. Similarly, in the text-only setting, human performance experienced a slight decline due to excessive textual interference, which introduced confusion and contributed to errors.
4.4. Intervention
Although the probing results have demonstrated that MLLMs possess internal mental representations, we still aim to intervene with the attention activation heads to fur- ther validate the practical significance of the classifier's s directional representations during probing. Due to dataset limitations, MMToM-QA QA can only provide positive and neg- ative samples in the text modality rather than multimodal ones, so we performed the intervention experiments exclu- sively on GridToo..
In first-order belief task, the baseline results in the mul- timodal setting indicate that MLLMs achieve high accu- racy on FB tasks (e.g., both ChatGPT--4.0 and Doubao-1.5 Vision- Pro reach 100%), even slightly surpassing human performance (99.9%). However, their performance on TB tasks is significantly weaker (e.g., ChatGPT-4.0 achieves 6.2%, Doubao- 1.5-Vision-Pro achieves 16.8%, and 0% on second-order belief tasks). We attribute this discrepancy to the models' overreliance on patterns learned from FB tasks, which may lead to misgeneralization in TB scenarios, This sensitivity prevents MLLMs from recognizing critical contextual details, such as the fact that the protagonist's door open in TB tasks. This inference is supported by the following observations: When the influence of visual fac- tors related to physical spatial positions is removed, LLMs (e.g., ChatGPT-4.0, Llama) still perform poorly when pro- cessing text-only inputs. However, MLLMS (e.g.. CharGPT- 4.0, LLava-Next-VV Video, and Qwen2-VL) demonstrate better performance when presented with pure video containing physical spatial information (excluding textual influences). This highlights the importance of establishing reasonable reasoning processes in both visual and textual modeling to balance task performance.
We first select the top K attention heads with the highest sensitivity on the validation set, representing those most responsive to the differences between true and false beliefs. We then intervene on these selected heads after multi-head attention computation but before mapping back to the output, as computed as follows:
Ti+1 = Ti + (Attn) (P(Ti) + aot Wf, (4)
1=1
where of denotes the standard deviation of activations along the target direction, and 6f represents the intervention target direction, derived from the weight vector of the selected attention head. The parameter a controls the strength of the intervention. For the selected K heads, a scales the acti- vation along the original direction by a times the standard error in the target direction.
We provide an analysis of the effects of hyperparameters K and a on the intervention results in the Appendix E. The analysis shows that our approach relies on an inter- pretable intervention based on internal representations and input perturbations, rather than on hyper-parameter tuning. By identifying the attention heads responsible for true be- lief representation and applying targeted interventions, we enhance the sensitivity of the model to perspective sepa- ration and belief representation. This ultimately improves the MLLM's ability to perceive and represent beliefs more effectively.
In both multimodal and video-only conditions, the poor performance on TB tasks negatively impacts all MLLMs' performance on Both tasks (i.e., correctly answering both TB and FB tasks for the same set). The performance on Both tasks provides intuitive reflection of MLLMs' ability to handle belief reasoning tasks; high accuracy on single task may indicate excessive sensitivity rather than a balanced reasoning capability. Under the text-only condition, LLMs (e.g., ChatGPT-4.0, Doubao, and Deepseek) alse exhibit relatively high accuracy on TB tasks. Interestingly, Doubao- I.5-Pro-32k stands out by achieving 100% accuracy on both tasks. In second-order belief tasks, MLLMs perform near the random guessing baseline (50%) and struggle on the Both task, highlighting the challenge. In contrast, L.MSs excel in text-only tasks.
5. Experiments
5.1. Result
We present summary of our results in Table 1. For each task, we include human accuracy as benchmark to repre- sent the upper bound of task performance.
Table also presents the results of applying our activation interference strategy to two MLLMs. While our attention feature extraction strategies are slightly adjusted for differ- ent belief tasks, the probing and interference methodology remains consistent, as detailed in Appendix B. The table
In the multimodal setting, humans achieved high accuracy across TB, FB, and Both conditions, demonstrating the consistency of our design. However, in video-only tasks,
7

---

Evaluating and Enhancing ToM in Multimodal LLMs
Table Model performance comparison the Goid benchmark. TB True Belicf. FB False Belief, For TB and FB, the expectation for random guesses is 50%. Both indicates situation where both TB and FB are judged as correct for given set.
FIRST-ORDER FB(%)
SECOND-ORDER
METHOD
SETTING
TB( %6)
BOTH( %)
TB(%)
FB(%)
BOTH( %)
HUMAN
99.9
99.9
99.8
99.9
99.8
99.8
CHATGPT 40
6.2
100.0
6.2
50.0
47.3
5.3
MULTIMODAL
DOUBAO- -S-VIIOON-PP
100.0
50.0
17.1
BASELINE
DESPSEEK-VIL VLL--
68.4
43.8
13.4
56.9
46.8
7.0
-LAVA-NENT-VIDRO- 7B-HF
53.2
42.8
48.7
39.8
0.2
QWEN2-VL- -8--N--RR-----
26.6
97.0
23.6
64.3
37.8
15.4
LLAVA-NEXT-VIDEO- 7B-HF
+o
15.86600.66
51.6(+8.8)
22.0(+21.2)
61.1(+12.4)
42.2(+2.4)
10.2(++00.)
OWEN2-VE-7B-INSTRRU
+0
60.4(+33.8)
97.4(+0.4)
31.2(+7.6)
75.1(+10.8)
46.6(+8.8)
24.2(+8.8)
HUMAN CHATGPT 40
99.8
84.6
81.0
81.0
69.6
30.4
5.6
49.2
37.8
3.4
DOUBAO- 1-10--.55-VIISE ..SVVISOO--
55.8
7.2
50.6
1.1
VIDEO
SHEPSEEK-VL2-SAE
BASELINE
55,6
44.0
2.8
48.9
48.4
3.7
LLAVA-NEXT-VIDEOO HB-HF
48.2
50.1
0.3
-B-NNSSRUTT INSTRUCT
52.2
48.6
5.2
49.1
40.0
2.1
LLAVA-NEXT-VVIDOO
54.4(+3.6)
51.2(+3.0)
16.4(+16.0)
55.9 (+5.8)
42.2(+1.0)
12.3(+12.0)
OWEN2-VE-7B-INSTRUU
+0
53.8(+1.66
52.2(+3.6)
18.6(+13.4)
52.1(+3.0)
46.0(+6.0)
19.5(+17.4)
HUMAN
98.0
98.1
96.6
97.6
98.0
96.3
CHATGPT 40
14.2
100.0
14.2
50.0
72.3
39.9
DOUBAO-L.5-PP P.5-RR0-22K
100.0
100.0
100.0
75.6
71.1
50.9
TEXT
DEEPSEEK-V3
84.4
100.0
84.4
61.5
70.9
41.4
BASELINE
LLAMA-3.3-70B-INSTRREE
100.0
0.0
50.7
70.3
23.7
MISTRAL--7-NNSTRIICT-V3.
77.8
47.6
25.4
62.9
39.9
14.5
LLLWA-NEXT-VVIDEO-7B- OWEX2-VL-78-INSTRUU.
40.8
56.4
0.0
49.3
50.1
0.7
48.6
66.2
14.8
61.4
52.7
16.3
highlights that method effectively modifies the mod- els' behavior, resulting in substantial performance improve- ments across first-order and second-order belief tasks under multimodal conditions, including TB, FB, and Both.
5.2. Discussion In this study, we introduced GridToM, novel multimodal dataset characterized by its incorporation of diverse belief- testing tasks and perceptual information from multiple per- spectives. Designed to evaluate the ToM capabilities of MLLMs, this dataset enables comprehensive assessments of their reasoning abilities across varied scenarios. We conducted comprehensive tests of existing MILMSs on this dataset. We observed that these models perform better on text-based data compared to video data. While the ToM capabilities exhibited in multimodal settings may be less pronounced than in unimodal scenarios, real-world appli- cations, such as real-time human-machine collaboration, often necessitate multimodal data inputs. Moreover, in such contexts, the feasibility of providing purely textual input in real-time is limited, emphasizing the necessity of evaluating ToM capabilities and interpretability in MLLMs.
Additionally, in Figures 15 and 16 of Appendix E, we il- lustrate the impact of hyperparameters on the interference effect. Specifically, the weight direction of the probed pro- tagonist's perspective has significant impact baseline performance, highlighting its critical role in the ToM reason- ing process. As expected, steering the reasoning direction of MILMS toward this perspective consistently improves the accuracy of TB and FB tasks. Throughout this process, no invalid responses are generated until the maximum value reached, which point all responses become invalid. We also tested interference directed toward the omniscient perspective. Due to the differing effects of perspective sepa- ration, its interference effect was observed to be lower than that of the protagonist's perspective. This finding aligns with our expectations, further confirming the importance of correctly aligning the models' reasoning direction with the protagonist's perspective for improved task performance.
Through analysis of MLLMs' internal mechanisms, we iden- tified attention heads capable of distinguishing different per- spective information and reasoning about correct beliefs. By modifying the reasoning attention direction based on the activation direction indicated by these attention heads, achieved significant enhancement of ToM capabilities
8

---

Evaluating and Enhancing ToM in Multimodal LLMs
in both first-order and second-order belief tasks, further validating the effectiveness of this mechanism.
measures for young children. Frontiers in psychology, 10: 2905, 2020.
However, our study has certain limitations. First, the tasks in our dataset are limited to first-order and second-order belief tasks within the ATOMs framework (Beaudoin et al., 2020), whereas ToM theory encompasses a broader range of tasks that remain unexplored. Second, due to restrictions in accessing model code, our approach was only validated on limited selection of MLLMs.
Bubeck, S., Chandrasckaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lund- berg, S., and others. Sparks of artificial general intel- ligence: Early experiments with gpt-4. arXiv preprint arXiv:2303. 12712, 2023.
Chen, Z., Wang, T., Wang, Y., Kosinski, M., Zhang, X., Fu, Y., and Li, S. Through the theory of mind's s eye: Reading minds with multimodal video large language models. arXiv preprint arXiv:2406.13763, 2024.
Acknowledgements
This work was supported by the National Science and Tech- nology Major Project (2022/D01 17902, 2022/D01 17901) and the the National Natural Science Foundation of China (No. 62206015, 62227801, 62376024). We thank the anony- mous reviewers for insightful discussions.
Chevalier-Biisvert, M., Dai, B., Towers, M., Lazcano, R. d., Willems, L., Lahlou, S., Pal, S., Castro, P. S., and Tenry, J. Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks. CoRR, abs/2306. 13831, 2023.
Impact Statement
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle,
A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., al. The llama 3 herd of models. arXiv preprint arXiv:2407.21778 2024.
Understanding human mental states is crucial for developing AI that interacts effectively and empathetically. Our bench- mark advances ToM evaluation in MILMs by integrating belief-testing tasks and interpretability analysis, revealing Al cognitive mechanisms. Grounded in cognitive science, it prioritizes faimess, inclusivity, and invites community feedback to refine human-aligned Al systems.
Eysenbach, B., Vondrick, C., and Torralba, A. Who is mistaken? arXiv preprint 1111v:1612.01115, 2016.
Gandhi, K. Stojnic, G., Lake, B. M., and Dillon, M. Baby intuitions benchmark (bib): Discerning the goals, preferences, and actions of others. Advances in neural information processing systems, 34:9963-9976 2021.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L.,, Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint Stf1:22303.08774. 2023.
Grant, E., Nematzadeh, A., and Griffiths, T. L. How can memory-augmentt neural networks pass a false-belief task? In CogSci, 2017.
Amirizaniani, M., Martin, E., Sivachenko, M., Mashhadi, A., and Shah, C. Do Ilms exhibit human-like reason- ing? evaluating theory of mind in lims for open-ended responses. cikm (2024), 2024.
Gupta, A., Boleda, G., Baroni, M., and Pad6, S. Distribu- tional vectors encode referential attributes. In Proceed- ings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 12-21, 2015.
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. Qwen-vl: A frontier large vision- language model with versatile abilities. arXiv preprint arXiv:2308. 12966, 2023.
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. I., Bressand, F,, Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, 7., and Shou, M. Z. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404. 0..4404.19940 18930, 2024.
Jin, C., Wu, Y., Cao, J., Xiang, J., Kuo, Y.-.., Hu, Z., Ullman, T., Torralba, A., Tenenbaum, J., and Shu, T. MMToM-ON QA: Multimodal theory of mind question an- swering. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 16077-16102. Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10. 10.186533 851. URL https: //aclanthology
Baron-Cohen, S., Leslie, A. M., and Frith, U. Does the autistic child have "theory mind"? Cognition, 21(1): 37-46, 1985. Publisher: Elsevier.
Beaudoin, C., Leblanc. É., Gagner, C., and Beauchamp, M. Systematic review and inventory of theory of mind
wwwp:/tthhoooo..o.//222.22. org/2024 acl- -long. 851/.
9

---

Evaluating and Enhancing ToM in Multimodal LLMs
Kosinski, M. Theory of Mind May Have Spontaneously Emerged in Large Language Models, March 2023. URL http: / /tt//aiiiv//////////////////aavm org/abs/ 22022/32222222 02083.
Oguntola, I., Campbell, J., Stepputtis, S., and Sycara, K. Theory of mind intrinsic motivation for multi-agent reinforcement learning, arXiv preprint arXiv:2307.01158 2023.
Kosinski, M. Evaluating large language models in theory of mind tasks. Proceedings of the National Academy of Sciences, 121(45):c205466012, 2024.
Sap, M., Le Bras, R., Fried, D., and Choi, Y. Neu- ral theory-of-mind? on the limits of social intelli- gence in large LMs. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pp. 3762-3780, Abu Dhabi, United Arab Emirates, December 2022. Association for Computa- tional Linguisties. doi: 10.18663/v11/2022.emmll main. 248. URL emnlp-mainn 248/.
Köhn, A. What's in an embedding? Analyzing word embed- dings through multilingual evaluation. 2015. Publisher: Fachbereich Informatik.
Le, M., Boureau, Y.-L., and Nickel, M. Revisiting the eval- uation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th Interna- tional Joint Conference Natural Language Processing (EMNLP-IJCNEL pp. 5872-5877, 2019.
Sclar, M., Neubig, G., and Bisk, Y. Symmetric machine theory of mind. In International Conference on Machine Learning, pp. 19450-19444 PMLR, 2022.
Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-ting intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024.
Shapira, N., Levy, M., Alavi, S. H., Zhou, X., Choi, Y., Goldberg, Y., Sap, M., and Shwartz, V. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In Graham, Y. and Purver, M. (eds.), Proceedings of the 18th Conference of the Eu- ropean Chapter of the Association for Computational Linguistics (Volume I: Long Papers), pp. 2257-2273, St. Julian's s, Malta, March 2024, Association for Computa- tional Linguistics. URL https: / aalaaaaanannnt....... org
Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek- v3 technical report. arXiv preprini urXiv:2412.19437 2024.
Lore, N., laami, S., and Heydari, B. Large model strate- gic thinking, small model efficiency: transferring the- ory of mind in large language models. arXiv preprint 2tt.:24008.02444 2024.
139/.
Shi, H., Ye, S., Fang, X., Jin, C.,, Isik, I.., Kuo, Y.-L., and Shu, T. Muma-tom: Multi-modal multi-agent theory of mind. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 1510-1519, 2025.
Ma. Z., Sansom, J., Peng, R., and Chai, J. Towards a holistic landscape of situated theory of mind in large language models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computa- tional Linguistics: EMNLP 2023, pp. 1011-1031, Singa- pore, December 2023. Association for Computational Linguistics. doi: 168835/1122233.nnniinn
Shu, T., Bhandwaldar, A., Gan, C., Smith, K., Liu, S., Gut- freund, D., Spelke, E.,, Tenenbaum, J., and Ullman, T. Agent: A benchmark for core psychological reasoning. In International conference on machine learning, pp. 9614 9625. PMLR, 2021.
72. URL https: ac wwwlanhologgy......ooc.ooooollle org/2023. Findings-emnip... ....s.mmnmp....... 7
Sileo, D. and Lernould, A. MindGames: Targeting the- ory of mind in large language models with dynamic epistemic modal logic. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computa- tional Linguisties: EMNLP 2023, pp. 4570-4577, Singa- pore. December 2023. Association for Computational Linguistics. doi: 10. 10.118633/11/2223...ii.ng----
Mao, Y., Liu, S., Ni, Q. Lin, X., and He, L. A review on machine theory of mind. IEEE Transactions on Compu- tational Social Systems, 2024.
Nematzadeh, A., Burns, K., Grant, E., Gopnik, A., and Griffiths, T. Evaluating theory of mind in question an- swering. In Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (eds.), Proceedings of the 2018 Confer- ence Empirical Methods in Natural Language Pro- cessing, pp. 2392-2400, Brussels, Belgium, October- November 2018. Association for Computational Lin- guistics. doi: 10. 10.18653/1//11-.. 1/1111 //aclanthology www.oo.......oo////////
303. URL https:tt/////c /attps://aclaat F
1aptsmmmnll... 303/.
3033/.
Strachan, J. W., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al. Testing theory of mind in large language models and humans. Nature Human Behaviour, pp. 1-11, 2024.
URL ht tps:
10

---

Evaluating and Enhancing ToM in Multimodal LLMs
Team, D. Doubao--1.--pog Exploring the ultimate bal- between model performance and inference efñi- ciency, 2025. URL petttss:/teann doubao..oooooooooooo com/ tb/sspccaal/cubba__LL.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
Ullman, T. Large language models fail on trivial al- terations to theory-of-mind tasks. arXiv preprint Stt1:23322.08999 2023.
van Duijn, M. J., van Dijk, B., Kouwenhoven, T., de Valk, W., Spruit, M. R., and van der Putten, P. Theory of mind in large language models: Examining performance of 11 state-of-the-an models vs. children aged 7-10 advanced tests. arXiv preprint arXiv:23
2023.
Verma, M., Bhambri, S., and Kambhampati, S. Theory of mind abilities of large language models in human-robot interaction: An illusion? In Companion of the 2024 ACM/EЕЕ International Conference on Human-Rond Robot Interaction, pp. 36 45, 2024.
Wu, Y., He, Y., Jia, Y., Mihalcea, R., Chen, Y., and Deng, N. Hi-ToM: A benchmark for evaluating higher-order theory of mind reasoning in large lan- guage models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10691-10706, Singa- pore, December 2023. Association for Computational emnlp. 717. URL. https:/ ac anthol or2/ findings-emnip... 717/.
Linguistics. doi: 10. 10.18653/v1/0223.mmmml
Xiao, Y., Jiashuo, W., Xu, Q.. Song, C., Xu, C., Cheng, Y., Li, W., and Liu, P. Tomvallcy: Evaluating the theory of mind reasoning of lims realistic social context.
Xu, H., Zhao, R., Zhu, L., Du, J., and Hc, Y. OpenToM: Comprehensive Benehmark for Evaluating Toeory-of- Mind Reasoning Capabilities of Large Language Models. arXiv preprint 2xXiv:2402.0604 2024.
Yim, Y. Chan, C., Shi, T., Deng, Z. Fan, W. Zheng, T., and Song. Y. Evaluating and enhancing lims agent based theory of mind in guandan: A multi-playe coopera- tive game under imperfect information. arXiv preprint arXiv:2408.02559 2024.
Zhou, P., Madaan, A., Potharaju, S. P., Gupta, A., McKee, K. R., Holtzman, A., Pujara, J.. Ren, X., Mishra, S., Nematzadeh, A., et al. How far are large language mod- els from agents with theory-of-me arXiv:23 10.03051, 2023.
arXiv preprint

---

Evaluating and Enhancing ToM in Multimodal LLMs
Appendix
A. Benchmark Details
Table comparison of Theory of Mind benchmarks (Is and 2nd belief tasks).
DATASET TOMi(LE ET AL., 2019)
ToM ASPECT
FB TESTED CONCEPTS
TEST SIzE
MODALITY
PERCEPTUAL INFORMATION
1ST & 2ND
400
TEXT
NO
MINDGAMES(ŠILBO LERNOULD, 2023)
& 2ND
TEXT
ADV-CSFB(KOSINSS) 2023)
& 2ND
TB
TEXT
NO
HI-TOM(WU ET AL., 2023)
1ST
FB TB
600
TEXT
NO
MMToM- MMTOM-QA(IIN ET AL., 2024)
1ST
FB TB
600
TEXT & VIDEO
TOM(OOR)
& 2ND
TB
1296
TEXT VIDEO
B. Attention feature extraction strategies
B.1. First-order Belief
First-order
Troe Delief
Falue Delief
afronees
afvine
aFrasete
REFORNES
Strume
Selluity After the auite
yellow
After whtt
sette poline
After the witt
Inforeation
eiber root
doer ertrr
apent
The rock and
grice eetrrs
roves
white Serves
the
prose dete wnters
whike agen cales
every
cless petto der
san't was
wiss leae dayy
tiee the proti
the and see
milsas
lig hepambin
claar the purtle dran
Deniscient
Dnnisciont
perspective
perspect
perspective
Protagonist perspective
Protagonist perspectiive
Protagunist
perspective
Belief
The white agent is now in the purple roon.
The white agent is now in the reel roun.
The white agent now in the purple roce.
The white agant is now in the red roce.
Lebel
Ypushopees True Yensststeee True
Spatonsent = Folse Yeesttotee: False
Yestngeeu - False Yeesttatee: Trie
- True
Feentastire False
Figure the attention feature extraction process for first-order helief tasks, the information obtained omniscient and protagonist perspectives is consistent in the TB task. We identify helief-ressonin-sentsee -eessiivveeeeivee features in attention by comparing correct and incorrect belief pairs. However, in the FB task, the protagonist's perspective has limited information. Therefore, we use the visual information from the protagonist's perspective along with the corresponding annotations positive samples, while the omniscient perspective serves negative samples. By comparing positive and negative samples, we identify attention features sensitive to perspective separation.
12

---

Evaluating and Enhancing ToM in Multimodal LLMs
B.2. Second-order Belief
Second-order
True Bellef
First-order Tree Beller
Firat-order Faise Belter
efromel3
#frome19
#frame36
#frame8
#frone1]
#frame19
#frome36
Information
[4,15)
[11,00]
[19.90
(6,15)
(2),
(19,4)
bettieelly, the whitee corrdaar, walks
After viite agert
ville the yellow apont
bestially. the aecte canddear, 25 wale
ddter the sbite agast
Willa the yallow agant
reme yell
noon pulch the prose
ned ram, the yellow
reen paits the groee
opent,
opeas swar enters ram,
greas reee, opens the
opens not the ram,
gremn ream, cpans the
watte apent spera red rose deer, lenven
ream cear, Jamen
yelliww agent
closse treelf.
foo
yellow ageit
aut Therefene,
timn,
cloes the yellow
cone time casd the
Thraggest this the yellow agest
tlae,
agent
the it valle the
the yell ogent
agent
then welks to the
the whille apent
hoppasing the
purple
parple door, goes
the wkitee agent ge
autciie grees reae.
perple doer, goea
pappln dear. that
perple dear, that
The yellow agent believes the white agent 1s now in the purple
The yellow agent believes the white
The yellow agent believes the white agent is now in the purple
The yellow agent believes the white agent is now in the red
Belief
agent 1s now in the red
Label
Y yportieppant yprotagonist True
Yporticipont
Yparticipone
Y pprrtiepantt
Ypremggniitt Folse
Ypretagmist Fatse
prectaganise True
Second-orde
Belief
BFrance
eframess
DFraress
#frume35
pfranes
oframels
Intrane33
#FFanets
Information
14.10
(50,4)
[6:111
(11,1%)
der- wtik
nel rees the yollo
even Fatli cisuntt
shite
stenk
red nos, the aellw
gree ren Catth
spee the sorel)
steas ree,
vellae maee
muse mere the yethoe open
meree rons and
glass car
vizzw spirt
shown aoar mosety th yellow
the pellow auws la
mpet rippesing
ther wiin to the corte res cars
the
yelliw opent in
the petzce un
everytord
not rom nee valie carile
poryle
The yellow agent believes the white agent now purple roon.
The yellow agent believes the white agent is now in the red roon.
The yellow agent believes the white agent now purple roon.
The yellow agent believes the white agent now red roon.
Belief
parrisipont Fatse Yprotggenter True
parciicpant True Ypretagonia False
Yparticipant True Yprotagonta = Fatse
y pparticipass False ypretegaater True
Label
Figure attention feature extraction process for second-order belief tasks, both the and tasks include TB tasks from first-order belief tasks. Unlike first-order belief tasks, the FB task in second-order belief reasoning contains the participant's incorrect perception of the protagonist's belief, achieved through a carefully designed timing setup. Since second-orden belief reasoning involves the participant's belief about the protagonist's belief and does not include perspective separation tasks, we identify belief-reae selief-raaooning-ee senssiivve features in attention solely by comparing correct and incorrect belief pairs.
13

---

Evaluating and Enhancing ToM in Multimodal LLMs
C. Full Version of the Example Questions in Figure 3
C.1. Videos
TB test
The task of TB refers the situation where the protagonist's beliefs align with those from an omniscient perspective, meaning the protagonist has access all the information about the events. In the TB experiment, when the protagonist enters the room and leaves the door open, they are able observe the situation outside the room, including the movements of the participants. We select representative example from the dataset and present the video frame sequences from three distinct perspectives: the omniscient perspective (Figure 8, A), the protagonist's perspective (Figure 8, C), and the participant's s perspective (Figure& 8, B).
1
Figure (A) video frames the omniscient perspective (36 frames in total) in TB test are shown in the figure. (B) The video frames from the participant's perspective (36 frames in total) in TB test are shown in the figure. (C) The video frames from the protagonist's perspective (36 frames total) TB test are shown in the figure.
FB test
The task of FB refers to the situation where the protagonist's beliefs diverge from those of an omniscient perspective, meaning the protagonist does not have access to all the information about the events. In the FB experiment, the protagonist enters the room and does not observe critical events, such as the movements the participants outside the room, due the door being closed. We select representative example from the dataset and present the corresponding video frame sequences from three distinct perspectives: the omniscient perspective (Figure 9, A), the protagonist's perspective (Figure 9, C), and the participant's perspective (Figure 9, B).
147
1222
12'1
Figure 9. (A) The video frames from the omniscient perspective (36 frames in total) in FB test shown in the figure. (B) The video frames from the participant's 's perspective (36 frames in total) in FB test are shown in the figure. (C) The video frames from the protagonist's perspective (36 frames in total) in FB shown in the figure.
14

---

Evaluating and Enhancing ToM in Multimodal LLMs
C.2. Text
Initial Belief
The concept of initial belief refers to the foundational understanding assumption MLLMs hold about the scenario before answering the ToM questions. In the context of this study, initial belief encompasses the MLLMs' pre-existing mental representation regarding three specific aspects of the task (see Figure 10);
Quantity and Color A single question evaluates the agent's ability to interpret and reason about the numerical or visual attributes objects based on its initial belief.
Spatial Understanding Two questions assess the agent's capacity to comprehend and reason about the spatial arrangement or movement of objects within the environment.
This structured approach ensures that the evaluation effectively measures the agent's ToM capabilities within multimodal framework.
Spatial Location Information
The video shows 20 grid world viewed from above, consisting of 10 and 7 colums. Gray represents the wall and cannot be penetrated. Black squares with gray borders represent the corridor floor. There are three roons this grid world, each with its color each roon's door
color the roon).
triangles here, representing the agents.
Initial Belief
"Questioni"; How many agents (triangles) are there? What colors are they? Choose from the following colors and just answer the color(s). (white, green, red, yellow, purple) "sectioni"; [0,36]
"answer1": ["white", "yellow"]
"Question2": What color roon did the white agent walk into? Choose from the following colors and just answer the color. (red, green, purple)
"section2": [0,11]
"answer2": ["red"]
"Question3"; what color roon did the yellow agent walk into? Choose from the following colors and just answer the color. (red, green, purple)
"section3": [0,19]
"answer3": ["green"]
Figure 10. The textual annotations for initial belief task are shown the figure.
First-order belief
The concept of first-order belief refers the direct inferences or reasoning that MLLMs make about the mental states of others, grounded in their observable actions or statements. To facilitate the subsequent training of classifiers and identifying the representational direction of perspective information, the dataset includes single first-order belief question, along with two answer options and the correct answer. Additionally, the dataset provides the corresponding contents of the True TB and FB tests associated with the question. Furthermore, detailed descriptions of the story progression across different temporal segments are included to capture the sequence of events. This design ensures that the dataset not only facilitates the evaluation of first-order belief reasoning in MLLMs but also establishes a structured framework for identifying and analyzing perspective-base information through temporal and belief-basee annotations (see Figure 11).
15

---

corridor. Evaluating and Enhancing ToM in Multimodal LLMs
First Order Belief
True Belief
"Question": At the very end of the video, which color roon does the yellow agent believe the white agent should be in?
"options": ["red", "purple"]
"answer": "purple"
"bellef true": "The white agent is now in the purple roon.
"belief false";
agent now "caption": "The story proceeds follows: 1. Initially, the white agent stands in the corridor. It walks towards the red roon, opens the door,
reon."
red door behind itself. Throughout sees the white agent go into the red roon. 2. After the white agent closes the door to the red roon, the yellow agent goes over to the green roon, opens the green door, everything happening outside the green roon. 3. while the yellow agent is inside the green roon (with the green door open), the white agent opens the red roon door, leaves the red roon, then walks to the purple purple door. That where the video ends."
roon,
time, the yellow agent
corridor
the yellow agent does
close the green door. Therefore, the yellow agent
opens the purple door, inside, and closes the
False Belief
"Question": At the very end of the video, which color roon does the yellow agent believe the white agent should be in?" "options": ["red", "purple"] "answer": "red"
"belief true": "The white
the red roon."
"belief false": "The white agent is now the purple roon.
"caption"; "The story proceeds follows: 1. Initially, the white agent stands in the corridor. It walks towards the red yellou agent
itself. Throughout this time, sees the white agent go into the red roon. 2. After the white agent closes the door to the red roon, the yellow agent goes over to the green roon, opens anything happening outside the green roon. 3. while the yellow agent is inside the green roon (with the green door closed), the white roon, walks to purple roon, opens the purple door, goes inside, and
the red door, enters
roon, and closes red
corridor
green
yellow agent
green door. Therefore,
yellow agent cannot
agent opens the purple door. That is where the video ends."
door, leaves
Figure 11. The textual annotations for the first order belicf task in the TB and FB tests shown in the figure.
Second-order belief
The concept of second-order belief pertains to the reasoning and inferences that MILMS make regarding agent's beliefs about another agent's mental state, based on observed actions or interactions. This evaluation also encompasses the question, answer options, the corresponding TB and FB conditions, as well as the story descriptions (see Figure 12 and Figure 13).
Second Order Belief
Belief
"Question": At the very end of the video, which color roon does the white agent believe the yellow agent thinks the white agent should
in?
"options": ["red", "purple"]
"answer": "red"
"belief true": "the yellow believes the white agent is now in the red roon."
"belief false": "The yellow believes the white
"caption": "The story proceeds as follows: Initially, in the purple roon.-
roon, opens the red door, enters the roon, and closes the red door behind itself. Throughout this time, the yellow agent is in the corridor and white agent into
white agent stands
towards
white agent green roon, apens the green door, enters it, but the yellow agent does not close the green door. Therefore, the yellow agent can see everything happening outside the
roon.
the roon, yellow agent goes
the
roon. Mhile the yellow agent inside the agent opens the roon door, leaves the roon, at sane time, sees the green roon door open, then walks to the purple roon, opens the purple door, goes inside, and closes the purple door. That is where the video ends."
roon (with the green door open), the white
"Question": At the very end of the video, which color roon does the white agent believe the yellow agent thinks the white agent should be in?
"options": ["red", "purple"]
"answer": "purple"
"belief true": "The yellow believes
purple roon." "caption": "The story proceeds follows: 1. Initially, the white
agent now
"belief false": "The yellow believes the white agent is now in the red roon."
stands the corridor. walks towards the red roon, opens the red door, enters the roon, and closes the red door behind itself. Throughout this time, the yellow agent is in the corridor and the white
into the red roon. 2. After the white agent closes the door to the red green roon, opens the green door, enters and yellow agent closes the green door. Therefore, the yellow yellow agent cannot see anything happening outside the green roon. 3. While the yellow agent is inside the green roon (with the green door closed), the white agent opens the roon, opens the purple door, goes inside, and closes the purple door. That is where the video ends."
to the
roon, the time,
green door closed, it to purple
Figure 12. The textual annotations for the second order belief task in the TB tests are shown in the figure.
16

---

Evaluating and Enhancing ToM in Multimodal LLMs
Second Order Belief
Belief
"Question": At the very end of the video, which color roon does the white agent believe the yellow agent thinks the white agent should
"options": ["red", "purple"]
"answer"; "red"
"belief true": "The yellow believes the white agent is now in the red roon."
"belief false": "The yellow believes the white agent is now in the purple roon. "caption":
roon, opens the red door, enters the roon, and closes the red door behind Itself. Throughout this tine, the yellow agent is in the corridor and the white agent into the green roon, opens the green door, enters it, and the yellow agent closes the green door. Therefore, the yellow agent cannat see anything happening outside the agent opens
story proceeds
Initially,
agent
corridor.
towards
After the white agent closes the
the the red roon, yellow agent goes
3. while the yellow agent is inside the green door closed (However, that exact nonent,
(with the
door closed), the white
roon door, leaves the yellow agent opens the green door. Because of this timing, the yellow agent actually sees the white agent leaving the red roon.), purple roon, opens purple door, goes inside, and
roon,
same time,
the purple
is where
"Question": the very end the video, which color does the white believe the yellow agent thinks the white agent should
"options": ["red", "purple"]
"answer": "purple"
"bellef true": "The yellow believes the white agent is now in the purple roon."
"belief false": "The yellow believes the white
in the red roon."
"caption": "The story proceeds as follows: 1. Initially, the white agent stands the corridor. It walks towards the red roon, opens the red door, enters the roon, and closes the red door behind itself. Throughout this time, the yellow agent is in the corridor and agent go
roon, yellow agent goes green roon, opens the green doon, enters it, but the yellow agent does not close the green door. Therefore, the yellow agent can everything happening outside the green roon. While the yellow agent inside the agent opens the red roon door, leaves the red roon, at the same time, it sees the green roon dear open (However, at that exact noment, the yellow agent closes the roon.), then it walks to the purple roon, opens the purple deor, goes inside, and closes the purple door. That is where video ends."
roon.
agent
(with green deor open), the white
door.
of this timing, the yellow
does actually the white
leaving the red
Figure 13. The textual annotations for the second order belief task the FB tests shown in the figure.
Figure 14. The figure presents video sequence frames extracted from three different rooms in the dataset examples, where (A)(B). (C),), and (E)F) correspond to different rooms. (A) and (B) illustrate examples of the same room configuration but with different agent states and action trajectorics. Specifically, (A) represents the FB experiment, while (B) corresponds to the TB experiment. Similarly, (C) and (D) depict the FB and TB experiments, respectively, and (E) and (F) show the FB and TB experiments in another room configuration.
17

---

Evaluating and Enhancing ToM in Multimodal LLMs
Furthermore, in dataset, we apply randomized manipulations to the evaluation data for each story, including variations in room configurations, agent states, and action trajectories. This approach ensures diversity while preventing repetitive patterns that might result in spurious statistical correlations. To illustrate this, we have provided ten examples from the dataset, as shown in the Figure 14.
D. Result of initial belief test in Section 3.2
D.1. Result
We evaluated the initial belief accuracy (ACC%) of these MLLMs on the Grid'ToM dataset. The results are shown in the table below (Table 3).
Table 3. Initial Belief
MODEL
AcC(%)
HUMAN
99.9
MULTIMODAL
CHATGPT 40
75.9
DOUBAO- LUUBAO-L.S-VISION
76.0
DEEPSEEK-VI.-SMAALL
5.9
LLAVA-NEXT-VVDDOO
37.4
QWTN2-VI-77-NNSTRUCT
69.4
The variance in accuracy highlights the disparity in reasoning or belief assessment capabilities among these models. This indicates that model architecture, training data, or multimodal integration plays a critical role in achieving higher performance in such tasks. The deepseek-vil -12--mallllllll model achieved only 5.9% accuracy rate on 1944 initial belief tasks, and the reason for this low error rate was that 89.9% invalid responses.
E. Hyperparameters' analysis in Section 4.4
The impact of hyperparameters K and a on intervention strength is shown in Figures 15 to 18. We treat generated invalid responses as incorrect answers. Across all intervention results, the intervention direction based on the protagonist's perspective achieves the best performance, which aligns with our expectations and applied in our experiments.
Specifically, Figures 15 18 illustrate a wide span of hypee-parameter settings. We find that the effect of the intervention confined to valid interval: once this interval is exceeded, the MLLMs' responses deteriorate. The parameter remains effective roughly within the range [-50, 50), and the choice of K is informed by the number of hidden heads in the MILMss. Within the valid region, these two hyper-parameters affect model performance by no more than 10% on average, and their tuning produces smoothly varying perturbation until the edge of the valid interval is reached. These results show that our approach relies on an interpretable intervention based on internal representations and input perturbations, rather than on hapee-pprammttr tuning, and that is not highly sensitive to the specific byper-parameter values chosen.
18

---

Evaluating and Enhancing ToM in Multimodal LLMs
TrueBelief
Baseline
0.8
(+oracle(-16)
(+oracte_X-32)
(+oracleX=64)
(+protagonitt,-116)
(+protagorist(-32)
0.7
(*protagmmittK664)
Accuracy
0.5
0.4
-60
-40
-20
20
40
60
Figure 15. The impact of the hyperparameters K and on the LLaVA-NeXT-Vvidee-7--.. model on the First-order TB task.
FalseBelief
Baseline
0.70
(+orade.K=16)
(+oradee-332)
(+orade/K=44)
(+protagonist K-166
(+protagonist
(*protagonist
0.55
Accuracy
0.45
0.35
-60
-40
-20
40
Alpha
Figure 16. The impact of the hyperparameters K and on the LLaVA-NeXT--iiee------ model on the First-order FB task.
19

---

Evaluating and Enhancing ToM in Multimodal LLMs
TrueBelief
Baseline
0.7
(+oracle,K11]
(+oracie,K=28))
(+oracic(-56)
(+protagoris(K=14)
0.6
(+protagonistX--28
(+protagorisLX=55)
0.5
Accuracy
0.4
0.3
0.2
-100
-75
-50
-5
25
50
100
Alpha
Figure 17, The impact of the hyperparameters and the Qwen2- Owen2-VL-7BB I-Intruut model First-order TB task.
FalseBelief
1.00
(4dracle.K=1)
0.99
(+aracle,K-23)
(+oracle
(+ protaageistt
0.98
prottggenitt.---
protaggensttK--50
Accuracy
0.96
0.94
0.93
0.92
-40
-20
Alpha
Figure 18. The impact of the hyperparameters K and the Qwen2- -L--2-V---BBB.- -nnstucttt model the First-order FB task.
F. Evaluation protocol of baseline test
Our objective is to provide MILMs with complete third-person perceptual information in both visual and textual formats (representing omniscient perspective) and require MILMs to separate perceptual information corresponding different perspectives. This allows the models to infer the correct beliefs from each perspective.
Following the standard zero-shot settings for ToM QA evaluations as described in the literature (Shapira et al., 2024), we assess all models without any additional training. The evaluation includes questions related to initial beliefs, first-order beliefs, and second-order beliefs. The evaluation metries include the accuracy of correctly answering TB, FB, and both TB
20

---

Evaluating and Enhancing ToM in Multimodal LLMs
and FB simultaneously.
F.1. Objective
The primary goal of this evaluation is provide MILMS with complete third-person perceptual information in both visual and textual formats, representing an omniscient perspective. MLLMs tasked with separating perceptual information corresponding to different perspectives, enabling them infer correct beliefs associated with each perspective.
F.2. Setup
In line with the standard zero-shot settings for ToM QA evaluations, as outlined in the literature (Shapira al., 2024), all models are assessed without any additional training fine-tuning. This ensures that the evaluation reflects the inherent ToM reasoning capabilities of the models without being influenced by dataset-specific optimizations.
E3. Evaluation Scope
The evaluation employs the following aceuracy metrics measure the model's performance:
Accuracy of initial belief test Measures the model's ability to correctly understand the scenario.
TB Accuracy of first order belief test Evaluates the model's performance in identifying true beliefs within first-order reasoning scenarios.
FB Accuracy first order belief test Assesses the model's capacity to correctly infer false beliefs first-order reasoning tasks.
TB Accuracy of second order belief test Tests the model's ability discern true beliefs in second-order reasoning contexts.
FB Accuracy of second order belief test Measures the model's 's effectiveness in identifying false beliefs in second-order reasoning scenarios.
The model's responses are scored based on their ability to correctly answer questions in each belief category. Each category and the performance of both together are reported separately.
G. Additional Probing Results
We present the full probing results in first-order belief task and second-order belief task for both models using logistic regression models in Figure 19 and Figure 20. The probing accuracies vary across models and tasks.
First-order Belief
Second-order Belief
01
1.00
10
Layer
15c
15
25
30
30
LLaWA-Noxt-Vidde
Figure 19, Probe accuracies first-order belicf task and ooder belief task based on the attention head activations in all layers of VA-Next-Videe
21

---

Evaluating and Enhancing ToM in Multimodal LLMs
First-order Belief
Second-arder Belief
1.00
1 100
0.95
99
0
0
Layer
0.88
0
15
0.800
0.80
28
0.75
0.75
25
15
20
0.70
15
20
25
0.70
Head
Head
Deenz-Vl-7BIIntrrueet
Figure 20. Probe accuracies on first-order belief task and second-order belief task based on the attention head activations in all layers of Qwen2-VL-7B-Instnee
H. Probing on Different Dataset (MMTOM-QQ)
We further validated the effectiveness of method the MM'OOM-A dataset. The MMTOM-QA dataset consists of 134 videos, capturing individual searching for everyday objects in home environment. This aligns with cognitive science research on mental state attribution in navigational agents.
On average, each video contains 1,462 frames and depicts 36 types of human behaviors. Based on these videos, the dataset includes 600 questions designed to assess both goal reasoning and belief reasoning abilities. Each question is paired with a video clip representing the complete activity (e.g., RGB-D frames), a textual description of the scene, and the actions taken by the individual the clip. The questions follow a binary-choice format and are categorized into seven reasoning types (as detailed in the original dataset documentation). Specifically, the belief reasoning task consists of 300 questions (100 per type), while the goal reasoning task comprises 300 questions (75 per type). Additionally, the dataset provides 1,000 procedurally generated videos, annotated with ground-truth information on scenes, objects, goals, and beliefs for model training.
In our experiments, we utilized only the belief reasoning subset of the dataset (Figure 21). However, due to the absence of explicit positive-negative video pairs, we manually curated and filtered the dataset, constructing first-order TB and samples. This refinement enables more precise evaluation of the model's ToM reasoning capabilities.
Type 1.1: belief, short-term
Type 1.2: False belief, short-term
Inaide bridge, bottle
Sowns: The livng mom frotums a caninet Thes rabinel
illd with a bag of chips, remote controller, bottle wine and water glass.
Actions:
Inige, prepering open
Actions: Jennifer siucted in the living romm She heads
sowards the cab and is about to open
trying get following
Question: If Jonniter has been trying get oupcalic, which of the foilesing skatements to more tkkly tru?? (al Jenntoor thinks that there a cupcakia inside the cabieet. (B) Jonntter thinkss that thane cupcakte inside the cabiined.
ststements shat
Ezabeth fridpe, any (a)
the flxdge.
Figure 21. Sample examples from the MMToM-QA dataset. The question types utilized in MMToM-QA are also illustrated.
22

---

Evaluating and Enhancing ToM in Multimodal LLMs
1.0
1.0
0.9
0.9
10
10
0.8
Layer
15
15
20
0.7
20
0.7
0.6
0.5
0.5
Hoad
Head
(A)
(B)
Laye
(D)
Figure 22. (A) Omniscient. (B) Protagonist. The linear probing accuracy of all heads across all layers in LLaVA-Next Video on the test (C) Insensitive. (D) Sensitive. The linear separability of belief representations is explained through visual interpretation of the typical representation space.
I. Dataset Construction Pipeline
Our dataset produced almost entirely through automated generation and verification, with only minimal manual annotation and rigorous quality checks. Although Theory-of-Min (ToM) reasoning is intrinsically complex, our script-driven workflow guarantees consistent alignment among visual inputs, agent actions, and narrative descriptions.
I.1. Construction and Annotation
Map design.
We manually created 27 distinct 10 maps in Excel, each with 3 rooms and unique layouts.
Automated validation and rendering. Map validity was verified with Python scripts (e.g., enclosed rooms, door placement). Then, using the MultiGrid library, we rendered maps with:
Colour palette: assigned from 6 highly distinguishable colors (red, green, blue, yellow, purple, white).
Agent placement: two groups of agents were randomly placed in hallways with colors distinct from rooms; initial orientations were randomized.
Path planning: agent trajectories generated using Breadth-fing first search ensure valid, logical movement without dead ends.
Task generation. The combination of different variables results in 648 basic samples. For each sample, we generate both "door open" (TB) and "door closed" (FB) conditions, totaling 1296 samples. Second-order -oder belief tasks follow the same structure with minor narrative adjustments.
23

---

Evaluating and Enhancing ToM in Multimodal LLMs
1.2. Quality Assurance
Automation-fite :irst: key elements (layout, paths, doors, task type) were generated and verified via script, minimizing subjective error.
Human review: we manually reviewed samples for layout issues, trajectory logic, and narrative coherence.
Staged execution: tasks were divided into three stages with controlled timing to ensure logical, coherent event flow.
Controlled variables: used unified logic for all visual and script elements, systematically varying only key factors (room order, agent orientation, colors, door state).
1.3. On ToM Difficulty and Dataset Validity
Controlled scenarios: carefully constrained scenes reduce noise, allowing clearer focus on ToM and multimodal reasoning.
Scalability: current difficulty is moderate and sufficient for analyzing belief reasoning. We plan to expand with more complex scenarios in future releases.
24


Reviews:
[{"review_id": "rxIoyWw9dY", "reviewer": "Reviewer_ummn", "summary": "This paper aims to explore the Theory of Mind (ToM) capabilities for multimodal large language models (MLLMs).\nTo this end, it proposes a new dataset, GridToM, which is designed to evaluate MLLM Tom reasoning from multiple perspectives. \nBased on GridToM, they then conduct experiments using different tech to detect the ToM in MLLMs in a zero-shot setting.\nExperimental results and analyses show the attention heads in MLLMs can be capable of distinguishing such mental states.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful and constructive reviews. We respond to your concerns and questions below.\n\n# Methods And Evaluation Criteria\n\nWe agree that the construction of the GridToM dataset should be described in greater detail. We provide further clarification here and will include the full pipeline both in the main text and the appendix of the revised version. Our dataset is built primarily through automated generation and verification, with minimal manual annotation and thoughtful quality control. Despite the intrinsic complexity of ToM reasoning, our structured, script-driven approach ensures consistency and reliability across visual input, actions, and narratives.\n\n1. Dataset Construction and Annotation\n\n- Map Design: We manually created 27 distinct 10×7 maps in Excel, each with 3 rooms and unique layouts.\n\n- Automated Checking and Rendering: map validity was verified with Python scripts (e.g., enclosed rooms, door placement). Then, using the MultiGrid library, we rendered maps with:\n\n  - Colors: Assigned from 6 highly distinguishable colors (red, green, blue, yellow, purple, white).\n\n  - Agent Placement: Two groups of agents were randomly placed in hallways with colors distinct from rooms; initial orientations were randomized.\n\n  - Path Planning: Agent trajectories were generated using Breadth-first search to ensure valid, logical movement without dead ends.\n\n- Task Generation: The combination of different variables results in 648 basic samples. For each sample, we generate both “door open” (TB) and “door closed” (FB) conditions, totaling 1296 samples (see L122–127). Second-order belief tasks follow the same structure with minor narrative adjustments.\n\n2. Annotation and Data Quality\n\n- Automation First: Key elements (layout, paths, doors, task type) were generated and verified via script, minimizing subjective error.\n\n- Human Review: We manually reviewed samples for layout issues, trajectory logic, and narrative coherence.\n\n- Staged Execution: Tasks were divided into three stages with controlled timing to ensure logical, coherent event flow.\n\n- Controlled Variables: We used unified logic for all visual and script elements, systematically varying only key factors (room order, agent orientation, colors, door state).\n\n3. On ToM Difficulty and Dataset Validity\n\n- Controlled Scenarios: Carefully constrained scenes reduce noise, allowing clearer focus on ToM and multimodal reasoning.\n\n- Scalability: Current difficulty is moderate and sufficient for analyzing belief reasoning. We plan to expand with more complex scenarios in future releases.\n\n# Experimental Designs Or Analyses\n\nWe focus on zero-shot evaluation because Theory of Mind research emphasizes testing a model’s ability to reason about others’ mental states without task-specific demonstrations. Zero-shot settings help assess whether such capabilities naturally emerge during pretraining, without overfitting to specific prompts or memorized patterns. In other words, we believe that the abilities revealed by few-shot learning pertain more to the model’s other capabilities rather than ToM abilities themselves. On the other hand, zero-shot remains a widely adopted and informative setting for assessing underlying reasoning abilities in this domain. This approach follows precedent in prior work such as [Shapira et al., 2023; Jin et al., 2024], all of which adopt zero-shot setups to evaluate ToM reasoning in LLMs or MLLMs. \n\nAs for model selection, due to the limited availability of MLLMs supporting multi-image inputs at the time, we selected two strong-performing models: LLaVA-Next-Video and Qwen2-VL. We are actively working on extending our experiments to more models and tasks.\n\n# Other Comments Or Suggestions\n\nIn the first-order tasks shown in Fig. 3, the \"Omniscient\" label represents the objective ground truth. Since the omniscient perspective observes the entire event sequence, the belief remains \"Purple\" for both TB and FB conditions.\n\nSecond-order belief reflects whether one agent correctly estimates another’s belief. Due to space limitations, Fig. 3 only presents the case where the first-order belief is false, which is why both TB and FB under the protagonist’s view are labeled as \"Red.\" The complete set of cases, including those where the protagonist’s belief is labeled as \"Purple,\" is detailed in Fig. 7.\n\nWe acknowledge that there is a mistake in the caption of Fig. 11. Fig. 11 illustrates a first-order belief task to both TB and FB conditions, as correctly explained in the main text (L767-769). Reviewer r21H also pointed this out. We will address this issue in the revised manuscript.", "claims_and_evidence": "Most claims are well supported.", "methods_and_evaluation_criteria": "The major contribution of this paper is the GridToM dataset, which provides manipulable multimodal visual-linguistic causal stories. However, I fail to find any relevant information about how this dataset is built, e.g, do the authors collect data from existing datasets with additional annotation? How do the authors conduct the annotation process? What is the annotators' background? What is the quality control process? Such information is very important to a dataset/benchmark paper, which is not provided in the main text nor the Appendix. Also, ToM is a difficult task even for humans - it makes me doubtful about data quality.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "The paper only experiments with the zero-shot setting, which makes their findings and conclusions less generalizable. \nAdditionally, as the authors suggested in Sec 5.2, the selection of models is indeed somewhat limited. \n\nAlso, from a perspective of dataset construction, there is no validation of the data quality.", "relation_to_broader_scientific_literature": "This paper is very relevant to ToM research, and can be important to MLLM reasoning.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Please see above", "other_comments_or_suggestions": "The figure 3 is difficult to understand without the detailed description of the Appendix, while I believe a good figure should be easy to follow with just the caption information. \nAlso, for 1st order: why the labels for both TB and FB of omniscient are \"Purple\" ( which seems contradictory to Figure 11)? and why \"Red\" for both TB and FB of protagonist?", "questions_for_authors": "Please see above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oskxxjNiZJ", "reviewer": "Reviewer_NkZp", "summary": "This paper develops a new approach to evaluate Theory of Mind (ToM) abilities of Large Language Models. Taking as a starting point the potential limitations of previous ToM experiments (difficulties in capturing an agent’s perception, ToM tasks not addressing internal representations), it designs a specific test environment (GridToM), in which context and cognitive perspective information can be controlled. This benchmark is used to test five state-of-the-art multimodal LLM with human raters as a baseline. Subsequently, the authors apply an activation inference strategy to two of the LLMs (LLAVA-NEXT-VIDEO-7B-HF, QWEN2-VL-7B-INSTRUCT) resulting on improved ToM performance across tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your recognition of our work and thank you for the thoughtful suggestions. We respond to your concerns and questions below.\n\n# Methods And Evaluation Criteria\n\nOur benchmark is designed as a foundational framework, using binary labels to establish clear distinctions between correct and incorrect beliefs. Introducing a wider range of options and more comprehensive evaluation schemes is indeed a valuable direction. We believe that the evaluation of multiple options could potentially be extended using a binary classification approach, where multiple negative samples are categorized as generalized erroneous beliefs. This will be part of our future research plans.\n\n# Experimental Designs\n\nThank you very much for recognizing the value of using attention heads to distinguish different perspectives and enable second-order belief reasoning. We agree that this approach is highly worthy of further investigation and have planned to conduct broader model validation and more detailed analysis in our future work. At the same time, we will adopt your suggestion to retain the early results and include these preliminary experimental conclusions in the final version of the paper.\n\n# Relation To Broader Scientific Literature\n\nWe understand your point about the current paper’s insufficient connection to embodied intelligence and the field of robotics. In fact, we are also deeply interested in embodied intelligence. This work represents our initial effort to explore the Theory of Mind (ToM) capabilities of MLLMs. We plan to extend this approach to more interactive and visual scenarios in future work, aiming to build a closer bridge between ToM in MLLMs and embodied intelligence in robotics.\n\n# Essential References Not Discussed & Other Strengths And Weaknesses\n\nThank you for pointing out relevant related works. Although we have not discussed Verma et al. (2024) and Mao et al. (2024) in the current version, we recognize their relevance and value. We will include discussion of these papers in the revised manuscript to better position our work within the broader context of ToM research in LLMs, particularly as it relates to human-machine interaction and embodied intelligence.\n\n# Other Comments Or Suggestions\nThank you for the feedback. Our original intention was to use a vivid example to illustrate higher-order beliefs. We acknowledge that ToM benchmarks involve complexities beyond simple analogies. Following your suggestion, we will add a more detailed explanation of ToM in the revision.\n\n# Questions for Authors\n**1. Modality Independence:**\n\nWe believe that attention-based enhancement is not strictly dependent on a specific modality but rather on carefully designed experiments. Prior work and our own results show the effectiveness of attention enhancement across both text-only and multimodal settings. Our experiments (Sup. G and H) demonstrate generalization across GridToM and MMTOM, suggesting the potential for extending this method to other modalities such as audio. Additionally, We believe attention-based enhancement depends on carefully designed experiments to reduce the impact of noise in attention signals.\n\n**2. Binary Testing in ToM Research:**\n\nToM studies often rely on structured binary choices (e.g., true/false) to assess belief reasoning, as seen in classic tasks like the “unexpected location” and “unexpected contents” paradigms. Existing benchmarks for LLMs, such as ADV-CSFB (text-based) and MMTOM-QA (multimodal), also follow this approach. Our study continues this tradition. That said, exploring more open-ended or multi-choice formats could test reasoning flexibility in more complex settings, though this would introduce higher demands in task design and annotation. We see this as a promising direction for future research.\n\nOnce again, thank you for your kind support and constructive feedback.", "claims_and_evidence": "The main objective is benchmark development, for which claims are validated through LLM performance comparison and human users ground truth.", "methods_and_evaluation_criteria": "The paper is itself on an evaluation method and in my view follows best practice, in particular in view of context generation through the 'dynamics' of the environment, which produced multiple instances of ToM problems, from a consistent set of principles. \nResults are clearly and transparently reported for both first-order and second-order aspects.\nThere is a good sample of LLM being tested across parameter size and instruction-tuned types of models. \nOne limitation could be that the binary options in the benchmark may contribute in part to the high performance observed.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "GridToM implements the unexpected transfer task in an ‘ARC’ benchmark esthetic. This design supports a context in which to articulate different agent’s viewpoints (perspective separation) and belief inference. \nMaking second-order (meta-cognitive) aspects accessible to the benchmark is definitely advancing the state-of-the-art of LLM ToM investigation. \nThe use of attention heads to distinguish information across perspectives is intellectually compelling, and provides demonstrable enhancements. It may be worthy of its own investigation but I would still suggest to keep the early results, even on a subset of LLMs, in the final version of the paper.", "relation_to_broader_scientific_literature": "ToM abilities for LLM is a debated topic since the 'Sparks of AGI' paper [Bubeck et al., 2023]. This paper is relevant to most papers to date in offering alternatives to Sally-Anne type testing and experiments based on tests such as [Strachan et al., 2024]. It is less connected to ToM robotics papers, though.", "essential_references_not_discussed": "Verma, M., Bhambri, S. and Kambhampati, S., 2024, March. Theory of mind abilities of large language models in human-robot interaction: An illusion?. In Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (pp. 36-45).\n\n(secondary):\nMao, Y., Liu, S., Ni, Q., Lin, X. and He, L., 2024. A review on machine theory of mind. IEEE Transactions on Computational Social Systems, vol 11, n.6, Dec 2024.", "other_strengths_and_weaknesses": "I would not identify major weaknesses in the paper. When limitations are considered, it could be appropriate to relate this approach to the work of Verma et al. [2024] in particular on the perceived behavior recognition issue: to which extent the benchmark addresses a subset of it or might scale up to more complex behaviors.", "other_comments_or_suggestions": "The paper shows an appropriate balance between the core discussion and supplementary material, which is useful considering the experimental design and its multimodal content. It is generally well illustrated considering the sophistication of the experiments.\nStill on presentation, I was less convinced about the TARS example; I recognize the pedagogic value, although the ToM benchmark issue is complex enough to deserve additional explanations rather than analogies. Whether it should be modified is however left to the authors’ discretion.\nPost-rebuttal: the authors have answered my questions and responded satisfactorily; I remain positive about this paper.", "questions_for_authors": "Is the attention-based enhancement dependent somehow on modalities?\nHow could you extend the narrative scenarios beyond binary choices for beliefs?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "z8HbtJ2asv", "reviewer": "Reviewer_r21H", "summary": "This paper studies MLLMs’ ability on theory of mind. The authors first construct a benchmark testing MLLMs’ first-order and second-order theory of mind based on grid world setting, and then probe MLLMs’ understanding of beliefs with linear probing. Experiments show that some attention heads show distinguishment of true and false beliefs. Finally, the authors propose improving MLLM theory of mind through attention calibration.\n\nIn summary, the authors study a valuable topic, but the proposed benchmarks is flawed. The analysis and proposed method is not novel and efficient enough. A revision of the benchmark, further analysis, and a method with more novelty can improve the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful and constructive reviews. We respond to your concerns and questions below.\n\n# Claims And Evidence\n\nThe term \"robustness\" refers to the internal consistency of our task design. The GridToM dataset is generated via a fully automated pipeline with systematically controlled scenarios.\n\nOur human experiments are intended to confirm that the auto-generated scenarios and reasoning chains are intuitive and interpretable to humans—thus indirectly validating the method's reliability.\n\nTo avoid ambiguity, we will revise \"robustness\" to \"consistency\" for clarity.\n\nIn L331, our intention was to reference the psychological \"unexpected transfer task\", where false beliefs arise from unobserved changes. The weaker TB performance may be due to the model overgeneralizing from false-belief patterns and ignoring key multimodal cues.\n\nWe will revise the terminology to avoid ambiguity.\n\n# Methods And Evaluation Criteria\n\nTo our knowledge, we are the first to enhance MLLMs' ToM abilities through targeted attention direction detection and intervention. Our method is not highly sensitive to the choice of hyperparameters. The large range of the hyperparameter Alpha was selected to test the model's limits (interference beyond this range could lead to the failure of all responses).\n\nOur method improves VLMs' ToM performance through interpretable interventions based on internal representations and input perturbations, not hyperparameter tuning. It requires no training or architecture changes, making it generalizable to other reasoning tasks.\n\nAppendix E explores a wide hyperparameter range. We found that the effect is limited to a valid interval—outside of which responses fail. Alpha remains effective within the range of [-50, 50], while the choice of K depends on the number of hidden heads in the model and shows consistent effectiveness across all intervention tasks. Within the valid range, both hyperparameters affect model performance by no more than 10% on average. \n\nIn the revision, we will emphasize this stability and avoid confusion about performance fluctuations from extreme values.\n\nWe also demonstrate gains on both TB and FB tasks, as shown by the BOTH metric in Table 1.\n\n# Experimental Designs Or Analyses\n\nTestees were not misled. In the video-only condition, core rules—including that closed doors block perception—were clearly explained. The removed text was narrative only and did not affect task understanding. We will clarify this in the revision.\n\nWe acknowledge a caption error in Fig. 11. It depicts a first-order belief task for both TB and FB, as correctly stated in the main text (L767–L769). Reviewer ummn also noted this.\n\nFig. 12 includes two captions for two second-order TB videos, as shown in Fig. 7. All videos include temporal annotations.\n\nIn L656, the timing setup refers to the presence of a timeline and corresponding timestamps for events in all scenarios. For example, whether the yellow agent's door is closed at the moment the white agent enters the red room determines whether the white agent can correctly infer the yellow agent's belief.\n\nFig. 12 and Fig. 13 contain textual errors in the appendix. The second question in Fig. 12 and the first in Fig. 13 should read: \"Where does the white character think the yellow character thinks the white character should be?\"\n\nThe mention of a green character is a typographical error limited to the appendix. The main text and experimental setup are correct. We will fix this in the final version.\n\nRegarding \"Question\": Fig. 12 and Fig. 13 illustrate second-order belief reasoning, which involves inferring what one agent believes about another agent's belief. The content of the second-order belief is, by definition, the first-order belief itself. This is not an error.\n\nAs for Suppl. Sec. B, we acknowledge that some symbols were not explicitly defined. However, their meanings are clearly explained in Section 4.2:\n\n- $y_{protagonist}$: protagonist's belief label\n- $y_{participant}$: participant's belief label\n- $y_{omniscient}$: omniscient's belief label\n\nThe \"Belief\" row shows true/false belief combinations across perspectives, essential for second-order reasoning. These tasks involve nested beliefs (e.g., what the participant believes about the protagonist's belief), forming a natural 2×2 structure, as shown in Fig. 7 and Suppl. Sec. B.2.\n\n# References Not Discussed\n\nBenchmarks like MMBench focus on general MLLM capabilities, while our work targets ToM reasoning—nested beliefs, perspective-taking, and multi-agent cognition—which these benchmarks do not cover. Our task-specific, cognitively grounded framework includes structured annotations for probing reasoning depth. We will mention these benchmarks to highlight the distinction and complementarity of our approach.\n\n# Other Comments\n\nThank you for pointing out relevant related works. We will update Shapira et al. to its EACL 2024 version and add Sap et al. (EMNLP 2022) to the related work section.", "claims_and_evidence": "The authors compare the performance of different MLLMs and humans on the proposed benchmarks and report all the results. However, there are some concerns.\n\nHow do the authors define “robustness”? does humans’ high accuracy really imply the robustness of the proposed benchmarks?\nWhat is the definition of “error-transfer” nature?", "methods_and_evaluation_criteria": "The benchmark can test MLLMs’ theory of mind under proper settings.\nThe authors proposed a method to improve MLLM theory of mind. However, the proposed methods is too straightforward and lacking in novelty. The figure in Suppl. Sec. E shows that the method relies heavily on hyperparameter tuning and can not show improvement on TB and FB tasks at the same time.", "theoretical_claims": "There are no theoretical claims in this paper.", "experimental_designs_or_analyses": "The setting on video-only evaluation is flawed. If the testees are not told that agents can not know what is happening outside when the door is closed, they will make mistakes intrinsically. Such errors do not mean that they do not have good theory of mind. Instead, they are misled by the authors.\n\nIn Suppl. Fig. 11 and Fig. 12, the authors give two different captions to one video snapshot, which is insensible. The same video can not represent the situations where there is a timing and where there is no timing. This also shows the intrinsic flaws in the video-only setting.\n\nIn Suppl. Fig. 12 and Fig. 13, the authors refer to “green agent”, but there is no green agent in the setting. The “Question” asks about the white agent’s belief, but the “belief true” and “belief false” show the yellow agent’s belief. Therefore, there are typos or maybe even wrong annotations in the presented data example.\n\nIn Suppl. Sec. B, the authors do not explain the meaning of 𝑦𝑝𝑟𝑜𝑡𝑎𝑔𝑜𝑛𝑖𝑠𝑡, 𝑦𝑝𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑛𝑡, and 𝑦𝑜𝑚𝑛𝑖𝑠𝑖𝑒𝑛𝑡, making it hard for readers to understand the details. The content in the “Belief” row may reflect the authors’ misunderstanding of first-order and second-order belief. The authors should further clarify.", "relation_to_broader_scientific_literature": "The key contributions are related to MLLM benchmarking, MLLM representation analysis, and MLLM hallucination mitigation.", "essential_references_not_discussed": "The authors do not point out previous grid-based benchmarks (L149-L153), making it hard for readers to understand related work. If the authors are the first to propose grid-based benchmarks, they should claim confidently. If the authors are not, they should explicitly cite previous grid-based benchmarks.\n\nThe authors did not discuss works on general MLLM benchmarks such as MMBench [1], SEED-Bench [2], MV-Bench [3], MM-Vet [4], R-Bench [5], etc.\n\n[1] Liu, Yuan, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan et al. “Mmbench: Is your multi-modal model an all-around player?.” In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part VI, pp. 216-233. Cham: Springer Nature Switzerland, 2024.\n\n[2] Li, Bohao, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. “Seed-bench: Benchmarking multimodal large language models.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13299-13308. 2024.\n\n[3] Li, Kunchang, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang et al. “Mvbench: A comprehensive multi-modal video understanding benchmark.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22195-22206. 2024.\n\n[4] Yu, Weihao, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. “MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities.” In International Conference on Machine Learning, pp. 57730-57754. PMLR, 2024.\n\n[5] Wu, Mingrui, Jiayi Ji, Oucheng Huang, Jiale Li, Yuhang Wu, Xiaoshuai Sun, and Rongrong Ji. “Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models.” In International Conference on Machine Learning, pp. 53553-53570. PMLR, 2024.", "other_strengths_and_weaknesses": "See the reviews above.", "other_comments_or_suggestions": "Some cited papers have published versions such as [1, 2]. The authors should cite published versions to prove that their research field is well-recognized by the community.\n\n[1] Shapira, Natalie, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. “Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models.” In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2257-2273. 2024.\n\n[2] Sap, Maarten, Ronan Le Bras, Daniel Fried, and Yejin Choi. “Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs.” In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3762-3780. 2022.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper primarily contributes an evaluation of Theory-of-Mind (ToM) in Vision Language Models. A further minor contribution is a strategy for improving Theory of Mind in VLMs by calibrating the attention heads. This paper received mostly positive reviews. Reviewers find the proposed benchmark well executed (both in terms of how it is constructed and the models evaluated) and follow-up analyses interesting. Reviewers were initially concerned with the quality of the writing, and missing details about the generation pipeline (which is especially important when proposing a new benchmark). There was also some uncertainty about how this work compares to relevant related work. The author response clarified most of these issues (as far as this is possible without submitting a new draft) and with the promised changes the work appears to be in good shape for publication as is recognized by the reviewers.
Decision: Accept (poster)