TIMETOM: Temporal Space is the Key to Unlocking the Door of Large Language Models’ Theory-of-Mind (2024)

Abstract: Theory of Mind (ToM)—the cognitive ability to reason about mental states of ourselves and others, is the foundation of social interaction. Although ToM comes naturally to humans, it poses a significant challenge to even the most advanced Large Language Models (LLMs). Due to the complex logical chains in ToM reasoning, especially in higher-order ToM questions, simply utilizing reasoning methods like Chain of Thought (CoT) will not improve the ToM capabilities of LLMs. We present TIMETOM, which constructs a temporal space and uses it as the foundation to improve the ToM capabilities of LLMs in multiple scenarios. Specifically, within the temporal space, we construct Temporal Belief State Chain (TBSC) for each character and inspired by the cognition perspective of the social world model, we divide TBSC into self-world beliefs and social world beliefs, aligning with first-order ToM (first-order beliefs) and higher-order ToM (higher-order beliefs) questions, respectively. Moreover, we design a novel tool-belief solver that, by considering belief communication between characters in temporal space, can transform a character’s higher-order beliefs into another character’s first-order beliefs under belief communication period. Experimental results indicate that TIMETOM can dramatically improve the reasoning performance of LLMs on ToM questions while taking a big step towards coherent and robust ToM reasoning.

heading: 1 Introduction
text: Humans continually try to reason about other people’s mental states and understand how it might impact their actions (Frith and Frith, 2003). This capability, known as Theory of Mind (ToM) (Premack and Woodruff, 1978), is crucial for social interactions. With Large Language Models (LLMs) playing a growing role in our lives, developing LLMs
† Corresponding author.
with ToM could be better at teaching us, learning from us, communicating with us, collaborating with us, and understanding us (Gandhi et al., 2021, 2023; Rabinowitz et al., 2018; Shu et al., 2021).
Although ToM often comes naturally to humans, LLMs often make various errors in ToM reasoning (Figure.1C), such as ignoring the temporal order of events, generating outputs that violate commonsense, confusing the reasoning logic in higher-order ToM questions (He et al., 2023) and failing on "trivial" alternations to existing datasets (Kim et al., 2023; Ullman, 2023). Recently, various reasoning strategies like Chain of Thought (CoT) (Wei et al., 2022) and Tree of Thought (ToT) (Yao et al., 2023) have improved the reasoning abilities of LLMs in some tasks. However, these strategies are not suitable for ToM reasoning (Ma et al., 2023). Furthermore, Wilf et al. (2023) proposes the perspectivetaking strategy to improve the ToM reasoning abilities of LLMs, but this strategy is not suitable for higher-order ToM reasoning. Currently, there is still a lack of effective reasoning strategies to improve the performance and robustness of LLMs in ToM reasoning tasks.
In this paper, we introduce TIMETOM, which initially constructs temporal space by adding timeline to the stories or dialogues. Within the temporal space, we construct Temporal Belief State Chain (TBSC) for each character based on the events they are aware of on the timeline. Meanwhile, inspired by a principle of modern cognitive science (Mitchell, 2023; Yue, 2022), which posits that humans construct abstract models of the social world and their self-world in their minds, we split the beliefs in TBSC into self-world beliefs and social world beliefs. We use self-world beliefs to answer first-order ToM questions and incorporate social world beliefs when answering higher-order ToM questions.
The reasoning difficulty of ToM questions significantly increases as the order of questions rises
11532
(Sclar et al., 2023), and currently, there is no effective reasoning strategy for solving higher-order (reasoning depth m ≥ 2) ToM questions. We consider that the key to higher-order ToM reasoning lies in capturing the belief communication between characters. We design a novel tool—belief solver, which first parses each character’s perceptible time set based on their TBSC and then calculates the intersections of the time set of different characters to determine at which times they achieve belief communication. Furthermore, as illustrated in Figure.1D, since John’s understanding of Bob’s belief only occurs during the belief communication period, the higher-order ToM question of John’s estimation of Bob’s beliefs can be transformed into a first-order ToM question of what Bob’s belief state is like during the belief communication period. In reasoning about higher-order ToM questions, LLMs generate an initial reasoning process based on the character’s TBSC, and the belief solver transforms the higher-order ToM questions into firstorder ToM questions under belief communication period, which serves as feedback to inspire LLMs to refine their initial reasoning process on higherorder ToM questions.
Experimental results Experimental results on the ToMI (Le et al., 2019), BigToM (Gandhi et al., 2023), and FanToM (Kim et al., 2023) benchmarks indicate that TIMETOM dramatically improves the reasoning performance of LLMs on ToM questions
in multiple scenarios (Figure.1A and Figure.1B), while taking a big step towards coherent and robust ToM reasoning. It’s noteworthy that TIMETOM is well-suited for higher-order ToM questions, demonstrating good performance in third-order ToM questions. Furthermore, TIMETOM can be applied to situations involving agent communication which is commonly occurring in the real-world.
The main contributions of this work are as follows: (1) We construct temporal space and use it as the foundation. (2) Within the temporal space, we construct TBSC, which is a comprehensive representation of a character’s beliefs, including the temporal evolution of thing states and clear temporal cognition of key social events that create belief gaps. From the social world model’s cognitive perspective, TBSC is split into self-world and social world beliefs, aligning with first-order ToM (firstorder beliefs) and higher-order ToM (higher-order beliefs) questions respectively. (3) Within the temporal space, we design a novel tool—belief solver, which serves as feedback to inspire LLMs to refine their initial reasoning process in higher-order ToM questions. (4) Temporal space is the key to unlocking the door of LLMs’ Theory of Mind. Extensive experimental results indicate that TIMETOM dramatically improves the reasoning performance and robustness of LLMs on ToM questions1.
1Code will available at https://github.com/gyhou123/ TimeToM.

heading: 2 Background and Related Work
text: Existing ToM Benchmarks Previous evaluations for the ToM of LLMs are primarily focused on testing models using situation descriptions (i.e., narratives) (Nematzadeh et al., 2018; Le et al., 2019; Sap et al., 2022; Shapira et al., 2023), also referred as reading comprehension scenarios. Recently, considering ToM capabilities play an even more important role in understanding dynamic social interactions, Kim et al. (2023) introduce FanToM, which tests models using interactive dialogues, also referred as dialogue scenarios.
LLMs Lack of ToM Capabilities Several studies (Gandhi et al., 2023; Sap et al., 2022; Kim et al., 2023; Wilf et al., 2023; Ullman, 2023) have shown that LLMs have poor reasoning performance and robustness on ToM tasks in a zero-shot setting, even with the current state-of-the-art GPT-4 (Achiam et al., 2023) model. With LLMs becoming increasingly integrated into our everyday lives, developing LLMs with ToM is very necessary.
Enhancing LLMs Reasoning Capabilities Recent prompt-based methods enhance the reasoning abilities of LLMs by guiding them to produce intermediate reasoning steps. For example, CoT (Wei et al., 2022) guides LLMs to generate stepby-step derivations before producing the final answer, LtM (Zhou et al., 2022) decomposes a target question into a series of subquestions, ToT (Yao et al., 2023) using tree-structured search to find better reasoning chains, Self-Refine (Madaan et al., 2023) and Reflexion (Shinn et al., 2023) adds selfverification steps for rectifying reasoning errors. RAP (Hao et al., 2023; Hu and Shu, 2023) repurposes an LLM as a world model by prompting the LLM to predict the next state st+1 of reasoning after applying a reasoning step at to the current state st. Apart from prompt-based methods, Declarative (He-Yueya et al., 2023) uses external symbolic solver to solve the equations in reasoning steps. MAF (Nathani et al., 2023) uses multiple external tools such as calculator, programming syntax to generate feedback to refine initial reasoning output.
Although many methods have been introduced to enhance the reasoning ability of LLMs, they are not suitable for ToM reasoning. Wilf et al. (2023) adopts the perspective-taking strategy to enhance the ToM reasoning abilities of LLMs, but this strategy falls short in addressing higher-order ToM rea-
soning. Sclar et al. (2023)2 tracks each entity’s beliefs and their estimation of other entities’ beliefs, through graphical representations. However, it requires a substantial amount of external memory as well as being difficult to apply to context-rich ToM scenarios. Furthermore, there are currently no tools specifically dedicated to ToM reasoning.

heading: 3 TIMETOM Overview
text: Figure.1C illustrates the errors LLMs made in ToM reasoning, often ignoring the temporal order of events and confusing the reasoning logic in higherorder ToM questions. Concurrently, explicitly representing the timeline not only allows LLM to have a clearer temporal understanding of the events in the story and dialogue, but we also observe the association between higher-order questions (higherorder beliefs) and first-order questions (first-order beliefs) on the timeline. Building upon this insight, we introduce TIMETOM, improving the ToM capabilities of LLMs in interactive dialogue and reading comprehension scenarios. The overall procedure of TIMETOM is shown in Figure 2.

heading: 3.1 Constructing Temporal Space
text: In reading comprehension scenarios, each sentence in the story corresponds to a specific time point. Similarly, in interactive dialogue scenarios, each utterance in the dialogue corresponds to a specific time point. Illustrating with the case of reading comprehension scenarios, given the input story x: Sentence1, Sentence2,..., SentenceN, prompt pcts, and model M, TIMETOM adds a complete timeline for input story x to form xt:
xt = M(pcts||x). (1)
For example, as illustrated in Figure 2, the model explicitly adds time points before each sentence for the given input story. Here, || denotes concatation and pcts is shown in Appendix B.2.

heading: 3.2 Constructing Temporal Belief State Chain for Each Character
text: ToM questions focus on the beliefs of characters, including their own beliefs as well as their estimations of others’ beliefs. Given story xt within temporal space, prompt ptbsc and model M, TIMETOM construct TBSC ctbsc for each character,
2This method works by explicitly memorizing beliefs of each character, rather than utilizing LLMs for ToM reasoning
based on the events they are aware of on the timeline:
ctbsc = M(ptbsc||xt) (2)
For example, as illustrated in Figure 2, Alice is aware of events between t1 to t3, but since she leaves the room at t3 and re-enters the room at t6, he cannot aware of events between t4 to t5. TBSC is a comprehensive representation of a character’s beliefs, which includes the temporal evolution of object states, such as celery is in the basket at t2, in the box at t4, and on the table at t7 as well as key social events that create belief gaps with clear temporal logic, such as Alice exits the room at t3, Bob exits the room at t5, and Alice re-enters the room at t6. Here, || denotes concatation and ptbsc is shown in Appendix B.2.

heading: 3.3 Time-Aware ToM-Question Answer from Social World Model Perspective
text: Mitchell (2023) posits that humans construct abstract models of the social world and their selfworld in their minds. Inspired by this cognitive perspective, we divide the belief in TBSC into selfworld belief and social world belief. We consider self-world belief as the perception of the state and information of things around oneself, while social world belief is the perception of other characters’ actions that may lead to a belief gap3. Given character’s TBSC ctbsc, prompt pself and model M, TIMETOM generates self-world belief ctbsc−self via belief compression, i.e., focusing on events in
3Considering self-world belief and social world belief in this way aligns well with first-order and higher-order ToM questions, making it very suitable for ToM scenarios.
TBSC about the states and information of things:
ctbsc−self = M(pself ||ctbsc). (3)
Given the character’s self-world belief ctbsc−self and the comprehensive belief ctbsc after incorporating social world beliefs, prompt pqa, and model M, TIMETOM use self-world belief ctbsc−self to answer first-order ToM questions and comprehensive belief ctbsc when answering higher-order ToM questions:
yfirst_order = M(pqa||ctbsc−self ) yhigher_order = M(pqa||ctbsc).
(4)
For example, as illustrated in Figure 2, John uses only self-world belief (celery is in the basket at t2, in the box at t4, and on the table at t7) to answer the first-order ToM question "Where will John look for the celery?" and incorporates social world belief (Alice exits the room at t3, Bob exits the room at t5, and Alice re-enters the room at t6) when answering higher-order ToM questions "where does John think Bob looks for the celery?". Here, || denotes concatation, pself and pqa is shown in Appendix B.2.

heading: 3.4 Time-Aware Belief Solver
text: We achieve a more comprehensive and clearer representation of characters’ beliefs by establishing a TBSC for each character, which improve the performance of LLMs in answering first-order and higher-order ToM questions. However, as the order of ToM questions increases, the depth of reasoning required becomes deeper, relying solely on the character’s TBSC still leads to logical errors in reasoning. Through in-depth consideration of belief communication between characters, we design an external tool—belief solver, which provides a novel reasoning perspective that effectively reduces the depth of reasoning, and serves as a feedback to inspire LLMs to refine their initial reasoning process on higher-order ToM questions.
Time Set Parsing We parse each character’s perceptible time set based on their TBSC. For example, as illustrated in Figure 2, the set of times that John, Bob, and Alice can perceive are as follows:
TJohn = [t1, t2, t3, t4, t5, t6, t7]
TBob = [t1, t2, t3, t4, t5]
TAlice = [t1, t2, t3, t6, t7]
(5)
Belief Communication between Different Characters To determine at which times belief communication occurs between different characters, we perform intersection operations on the sets of times perceived by each character, as parsed in the previous step:
BCJohn,Bob = TJohn ∩ TBob BCJohn,Bob,Alice = TJohn ∩ TBob ∩ TAlice (6)
where BCJohn,Bob = [t1, t2, t3, t4, t5] represents the set of times for belief communication between John and Bob, the same applies to BCJohn,Bob,Alice = [t1, t2, t3].
Transforming Higher-order ToM problems into First-order ToM problems Consider secondorder ToM question "Where does John think Bob looks for the celery?", since John’s understanding of Bob’s belief only occurs during BCJohn,Bob period, this question can be transformed into a firstorder ToM question "Where does Bob look for the celery?" under BCJohn,Bob. It is worth noting that this transformation is also applicable to third-order and higher-order ToM problems.
Inspiring LLMs to Reason on Higher-order ToM Questions Through in-depth consideration of belief communication between characters, we observe that higher-order ToM questions can be transformed into first-order ToM questions under belief communication periods. Given this reasoning process as feedback pfeedback, LLM’s initial reasoning process yhigher_order, and model M, TIMETOM lets the LLM reason again:
yhigher_final = M(pfeedback||yhigher_order). (7)
We hope that through this method, the LLM can pay attention to the belief communication between characters as well as the connection between higherorder beliefs and first-order beliefs to refine their initial reasoning outputs on higher-order ToM questions. Here, || denotes concatation, pfeedback is shown in Appendix B.2.

heading: 4 Experiments

heading: 4.1 Settings
text: Benchmarks We evaluate TIMETOM within reading comprehension and interactive dialogue scenarios, using ToMI, BigToM, and FanToM benchmarks. Compared to stories in reading comprehension scenarios, dialogues are more aligned
with real-world scenarios requiring ToM reasoning. Furthermore, dialogues in FanToM are significantly longer, with a larger number of subtopics and characters per dialogue. This poses a greater challenge for LLMs, as it demands the LLMs’ capability to comprehend the complete dialogue utterance and reason about each character’s beliefs. Detailed explanations for story (or dialogue) structure, each type of question in ToMI, BigToM, and FanToM benchmarks, and evaluation metrics can be found in Appendix A.
Baselines We employ five widely utilized LLMs: three open source – Llama2-7b, 13b, and 70b chat (Touvron et al., 2023) – and two closed source: GPT-3.5-Turbo-0613 and GPT-4-0613 to evaluate TIMETOM. To highlight the effectiveness of TIMETOM, we evaluate LLMs 0-shot on ToM benchmarks with and without our TIMETOM. Additionally, we compare TIMETOM with the CoT (Wei et al., 2022) and SimToM (Wilf et al., 2023), where SimToM is a recently proposed prompting framework specifically designed to improve the reasoning performance of LLMs on ToM questions, achieving state-of-the-art results. To make a fair
comparison, we uniformly set the temperature to 0 (GPT-series models) or 0.3 (Llama2-series models) and top_p to 0.95 for all experiments. We reproduce our TIMETOM prompts in Appendix B.

heading: 4.2 Main Results
text: In Tables 1 and 2, we report the reasoning performance of LLMs for ToM questions in reading comprehension scenarios and dialogue scenarios.
Substantial Improvement across Different LLMs and Scenarios From widely utilized commercial LLMs (GPT-series) to open-source models (Llama2-series), and from reading comprehension to interactive dialogue scenarios, TIMETOM leads to substantial performance improvement. Specifically, in the reading comprehension scenario, we achieve an average absolute accuracy improvement of +19.43% and +9.63%, as well as +13.38% and +10.81% over the 0-shot and SIMTOM baselines for the ToMI and BigTOM benchmark, respectively. A larger improvement is observed in the interactive dialogue scenario, where TIMETOM achieves an average absolute accuracy improvement of +44.7% and +13.6% over the 0-shot and SIMTOM base-
lines for the FanTOM benchmark.

heading: Well-suited for Higher-order ToM Reasoning
text: On the challenging higher-order ToM questions, TIMETOM yields +29.00% and +18.75% absolute accuracy improvement over the 0-shot and SIMTOM GPT-4 baselines, as well as +16.8% and +17.9% absolute accuracy improvement over the 0-shot and 0-shot-CoT GPT-4 baselines for the ToMI and FanTOM benchmark, respectively. An equally impressive result is observed across other model types. Furthermore, based on the GPT-4 model, we test the performance of baselines and TIMETOM on the third-order ToM problems of ToMI-Extend4 benchmark. As shown in Figure 3, compared with SIMTOM and 0-shot-CoT, the performance of TIMETOM does not degrade as the order of the ToM question increases, indicating its suitability for higher-order ToM reasoning. Meanwhile, TIMETOM exhibits the most outstanding performance on both first-order and higher-order ToM questions.
Better ToM Reasoning Robustness We use ALL* and All score from Table 2 to evaluate the ToM reasoning robustness of baselines and TIMETOM. We achieve +33.0% (× 4.8) and +31.3% (× 4.0) absolute accuracy improvement over the 0-shot and 0-shot-CoT GPT-4 baselines for ALL* score, which requires correct answers to all five types of ToM questions (Belief, Answerability[List, Y/N], and Infoaccess[List, Y/N]). For All scores
4Sclar et al. (2023) construct third-order ToM questions by making simple modifications to the story structure of the original ToMI benchmark
under the Answerability question and Infoaccess question, which require correct answers to both listtype and Y/N-type questions, we achieve +27.8% (× 2.2) and +26.1% (× 2.0), as well as +28.7% (× 2.2) and +17.3% (× 1.5) absolute accuracy improvement over the 0-shot and 0-shot-CoT GPT-4 baselines, respectively. A similar improvement is noticeable in the llama2-70b-chat model, although the degree of improvement is not as large.

heading: 5 The Effect of Key Components
text: Given TIMETOM’s strong performance, we analyze its key components: (1) Foundation: constructing temporal space. (2) From the perspective of first-order and higher-order ToM questions, considering temporal belief construction and compression as well as time-aware belief solver.
Constructing Temporal Space The construction of temporal space provides LLMs with a clearer understanding of object states and character actions, especially for those models with weaker cognitive abilities. We conduct experiments on the Llama2 series models in 0-shot and 0-shot with timeline settings, results show that the construction of the temporal space has led to significant performance improvements in true-belief and mem-real questions, which are associated with the real-world state. Moreover, the clear cognition brought by the temporal space is also helpful for reasoning about false belief questions. Detailed results data can be found in Appendix C.1. Case 1 in Figure 4 vividly illustrates the benefits of constructing temporal space.
Temporal Belief Construction and Compression Within the temporal space, we construct TBSC for the characters and utilize self-world belief, obtained through belief compression, to answer firstorder ToM questions. By comparing the performance of the Llama2-7b-chat and Llama2-13b-chat models on first-order ToM questions in Tables 1 and 4, an observed improvement of +3.75% and +2.75% brought by belief construction and compression is noted. A larger improvement +10.50% and +29.50% appears in the GPT-3.5-turbo and GPT-4 models, as they inherently possess stronger language comprehension abilities, construct better TBSCs, perform more effective belief compression.
Tool—Time-Aware Belief Solver Within the temporal space, we design a novel tool—belief solver to inspire the reasoning process of LLMs on higher-order ToM questions. As shown in Table 3,
the incorporation of belief solver results in a performance improvement of +7.0% and +16.0% for the GPT-3.5-turbo and GPT-4 models, respectively.

heading: 6 Analysis
text: In this section, we analyze belief solver as prompt vs feedback and the extension of TIMETOM to situations encompassing agent communication.
Belief Solver as Prompt vs Feedback Given that the belief solver can transform higher-order ToM questions into first-order ToM questions under belief communication periods, why don’t we use it directly as a prompt when answering higherorder ToM questions? There are two reasons: (1) Given the GPT-4 model’s exceptional language comprehension capabilities, it can construct accurate TBSC for each character. Consequently, it can accurately determine periods of belief communication through intersection operations of TBSC between characters. But as the model’s ability
decreases, e.g., for the GPT-3.5-turbo model, the TBSC of the characters it builds will have a certain probability of error, and then the probability of obtaining an incorrect belief communication periods is greatly increased when performing the TBSC intersection operation between characters. (2) It is more effective to use the belief solver as feedback to inspire the reasoning process of LLMs on higher-order ToM questions. LLMs will consider the initial reasoning perspective and the reasoning perspective provided by the feedback to form a final response, and when the reasoning perspective provided by the feedback is accurate and effective, the LLMs also acknowledge this perspective, which corresponds to the GPT-4 model case. Conversely, if the reasoning perspective offered by the feedback is erroneous, the LLMs have a certain probability of recognizing this error and realizing the integration of useful information from both perspectives to achieve better performance, which corresponds to the GPT-3.5-turbo model case. Case 2 in Figure 4 and Table 3 offers qualitative and quantitative analysis support for the above two reasons.
Applicable to Situations Involving Communication between Agents. In real-world interactions, people engage in sharing their innermost thoughts with each other, including both their perceptions of situations and observations of others’ behaviors. He et al. (2023) recently propose a benchmark encompassing agent communication, considering TIMETOM in this situation, which can model agent communication as a belief communication between
agents at a specific time point. As shown in case 3 of Fig 4, TIMETOM has good applicability to situations involving agent communication.

heading: 7 Conclusion
text: In this paper, we propose TIMETOM to improve the ToM capabilities of LLMs in reading comprehension and interactive dialogue scenarios. Specifically, we first construct temporal space which serves as the foundation. Building on this, we develop several key components: character’s belief state chain construction, social world model cognition-inspired belief compression, and tool—belief solver. Extensive experimental results show that TIMETOM substantially improves the reasoning performance of LLMs on ToM questions while making a significant advance towards coherent and robust ToM reasoning. The temporal space, serving as a key, unlocks the door to the LLMs’ Theory of Mind. Furthermore, we find that TIMETOM can also be extended to situations involving agent communication which is commonly occurring in the real-world.

heading: Limitations
text: There are two major limitations in TIMETOM. Firstly, the belief solver relies on constructing an
accurate TBSC for characters. We conduct experiments on models with a parameter scale of 7B or larger. For models with less than 7B parameters, due to their relatively weaker instruction understanding ability, the error rate in constructing TBSCs is higher, which in turn affects the effectiveness of the belief solver. However, with the continuous development of LLMs, this limitation can be solved easily. Secondly, we focus on ToM reasoning for textual modality, it is also important to perform effective multimodal ToM reasoning, which we treat as future work.

heading: Acknowledgements
text: This work is supported by the National Natural Science Foundation of China (No. 62376245), the Key Research and Development Program of Zhejiang Province, China (No. 2024C01034), the National Natural Science Foundation of Zhejiang Province (LY24C090001) and the Fundamental Research Funds for the Central Universities.

heading: Appendix

heading: A Benchmark Details and Evaluation Metrics

heading: A.1 ToMI
text: ToMI (Le et al., 2019) is a benchmark in the reading comprehension scenarios, strictly imitating the Sally-Anne test, including the story, questions, and answer choices. The structure of the story is as follows:
Five types of ToM questions are proposed: firstorder or second-order, exploring characters’ true or false beliefs (i.e., beliefs that are consistent or inconsistent with reality) as well as questions exploring reality and memory (zeroth-order ToM, (Sclar et al., 2022)). The formatted description for each type of question is as follows:
Reality: Where is celery really?
Memory: Where was celery at the beginning?
First-Order Belief Sally: Where will Sally look for celery?
First-Order Belief Anne: Where will Anne look for celery?
Second-Order Belief Sally: Where does Sally believe Anne will look for celery?
Second-Order Belief Anne: Where does Anne believe Sally will look for celery?
Meanwhile, in first-order belief and second-order belief questions, both false belief and true belief are involved. For example: Sally moves the celery from basket to box without Anne observing this action. A first-order belief question: "Where will Anne look for celery?" Since Sally has moved the celery, Anne’s belief will be incorrect – this type of question is called false belief and has its counterpart in true belief questions, where Anne’s belief about the world is correct.
An updated version of ToMI proposed from (Arodi and Cheung (2021); Sap et al. (2022)) that has relabelled mislabelled second-order questions and disambiguated the location of containers after their reference. Sclar et al. (2023) expands the story structure by introducing more characters and containers.
All questions have two possible answers: the original O location, and the final O location. ToMI is a binary multiple-choice task, with random accuracy being 50%. In our experiments, we select the exact same test dataset used in SIMTOM (i.e., the updated version of ToMI) to ensure a fair comparison.

heading: A.2 BigToM
text: BigToM (Gandhi et al., 2023) is another benchmark in the reading comprehension scenario and also follows the Sally-Anne test format. It is generated by GPT-4. Unlike ToMI, BigToM tells stories in more natural language and is not limited to changes in object locations. In our experiments, we focus on the "Forward belief" questions rather than "Backward belief" to align more closely with the structure of ToMI questions. The definitions of true belief and false belief questions in BigToM are the same as in ToMI. BigToM is also a binary multiple-choice task, with random accuracy being 50%.

heading: A.3 FanToM
text: FanToM (Kim et al., 2023) is a benchmark in interactive dialogue scenarios. Dialogues involve information asymmetry (Quesque and Rossetti, 2020), with characters joining and leaving the dialogues while it continues, to simulate distinct mental states. The structure of dialogue is illustrated in Figure 6. The authors of FanToM formulate multiple types of questions (belief questions, answerability questions[list, binary], and infoaccess questions[list, binary]) that demand the same underlying reasoning to identify the illusory or false sense of ToM capabilities in LLMs:
First-Order Belief: What does Gianna believe about who discussed their experiences training their pets, Bruno and Snowflake?
Third-Order Acyclic Belief: What does Sara believe about Javier’s belief regarding Gianna believe about who discussed their experiences training their pets, Bruno and Snowflake?
Third-Order Cyclic Belief: What does Sara believe about Javier’s belief regarding Sara believe about who discussed their experiences training their pets, Bruno and Snowflake?
Answerability_List: Who discussed their experiences training their pets, Bruno and Snowflake? List all the characters who know the precise correct answer to this question.
Answerability_Binary Who discussed their experiences training their pets, Bruno and Snowflake? Does Gianna (Javier, Sara) know the precise correct answer to this question?
Infoaccess_List: Sara and Javier discussed their experiences training their pets, Bruno and Snowflake. List all the characters who know this information.
Infoaccess_Binary: Sara and Javier discussed their experiences training their pets, Bruno and Snowflake. Does Gianna (Javier, Sara) know this information?
In our experiments, we focus not only on the performance of our method in answering individual question types but also on its ability to reason coherently and robustly across multiple question types. All questions have two possible answers, with random accuracy being 50%.

heading: A.4 Evaluation Metrics
text: Following Wilf et al. (2023), we report accuracy for all questions under ToMI and BigToM. Following Kim et al. (2023), for FanToM, we report accuracy
for belief, answerability[list] and infoaccess[list] questions. The weighted F1 scores are reported for answerability[binary] and infoaccess[binary] questions. To evaluate the reasoning robustness of LLMs on ToM questions, we report the All score for answerability and infoaccess questions requiring models to be correct on both list-type and binary-type questions, the ALL* score which requires the models to answer all five ToM question types which require the same type of ToM reasoning.

heading: B Our Prompt

heading: B.1 Prompt for Interactive Dialogue Scenario
text: Constructing Temporal Space: The following is a dialogue. Your task is to add timeline to the dialogue.
Here are one rules: Each utterance spoken by a character corresponds to a moment t, Use \n as a delimiter , and the timeline is t1,t2 ,... ,tN.
Dialogue: {dialogue}
Only output the dialogue content with the added timeline , do not provide explanations.
Temporal Belief State Chain Construction: The following is a dialogue with a timeline between multiple characters. Your task is to only output the dialogue content on the timeline that the character {character} can aware of.
Here are two rules: If a character leaves the conversation to do something else and then back after
a few rounds of dialogue , they are unaware of the content of the conversation that took place during their absence , but they aware of the content of the conversation besides their absence. If a character don ’t leaves the conversation to do something else and then back after a few rounds of dialogue . They are aware of all the content of dialogue with all timeline.
Dialogue: {dialogue}
What dialogue content on the timeline does {character} aware of? Only output the dialogue content according to the above rules , do not provide an explanation.

heading: Time-Aware Belief Question Answer with Belief
text: Compression (First-order ToM questions): The following is the belief states chain of character {name}. This is the content known to {name}:[{perspective}] You are {name}. Based on the above information , answer the following question: {question} When answering questions , based on own belief , simply focus on the information of things asked in the question and ignore other distracting factors. You must choose one of the above choices.

heading: Time-Aware Belief Question Answer without Belief
text: Compression (Higher-order ToM questions): The following is the belief states chain of character {name}. This is the content known to {name}:[{perspective}] You are {name}. Based on the above information , answer the following question: {question} You must choose one of the above choices .
Time-Aware Answerablity Question[List] Answer: The following is the belief states chain of each character. This is the content known to each character. Each character only knows the contents within their own belief state chain and is unaware of the contents within the belief state chain of other characters. {final_text} Question: {target} Based on the belief state chain of the above -mentioned characters , only output all the characters who know the precise correct answer to this question , do not provide an explanation.
Time-Aware Answerablity Question[Binary] Answer: The following is the belief states chain of character {character}. This is the content known to {character }. {binary_context}
Question: {target} Based on the belief state chain of character {character}, does {character} know the precise correct answer to this question? Answer yes or no. Answer :.
Time-Aware Infoaccess Question[List] Answer: The following is the belief states chain of each character. This is the content known to each character. Each character only knows the contents within their own belief state chain and is unaware of the contents within the belief state chain of other characters. {final_text} Target: {target_q} {target_a} Question: Based on the belief state chain of the above -mentioned characters , only output all the characters who know the target information , do not provide an explanation.
Time-Aware Infoaccess Question[Binary] Answer: The following is the belief states chain of character {character}. This is the content known to {character}. {binary_context} Target: {target_q} {target_a} Question: Based on the belief state chain of character {character}, does {character} know the target information? Answer yes or no. Answer :.
Time-Aware Belief Solver: The following is the belief states chain of character {name}. This is the content known to {name}:[{perspective}] You are {name}. Based on the above information , answer the following question: {question} Answer :{answer} Feedback: The event corresponding to the period of belief communication between characters {character1}, {character2} and { character3}: {common_belief} Based on this information , the answer we get to the question :{question} is [{answer2}] Considering this feedback , answer the question: {question} again. Keep your answer concise , one sentence is enough. You must choose one of the above choices .

heading: B.2 Prompt for Reading Comprehension Scenario
text: Constructing Temporal Space: The following is a story. Your task is to add timeline to the story.
Here are one rules: Each sentence corresponds to a moment t, Use \n as a delimiter , and the timeline is t1,t2 ,... ,tN.
Story: {story}
Only output the story with the added timeline , do not provide explanations.
Temporal Belief State Chain Construction: The following is a sequence of events with a timeline about some characters , that takes place in multiple locations. Your job is to output only the events on the timeline that character {character} can aware of.
Here are a few commonsense rules: 1. If a character is in a certain room/ location , they will be aware of all other events happening in that room. This includes other characters entering or leaving the location , the locations of objects within it, and whether someone has moved an object to another location. 2. If a character leaves a location and is no longer there , they will no longer be aware of any events occurring at that location. However , they can re-enter the location. 3. A character is aware of all the events that they do.
Story: {story}
What events on the timeline does { character} aware of? Only output the events according to the above rules , do not provide an explanation.

heading: Time-Aware Belief Question Answer with Belief
text: Compression (First-order ToM questions): Belief Compression: The following is information from the perspective of the character , {character}.
Perspective: {perspective}
Output the remaining perspective information after removing the events of characters enter or leave/exit the room /location , do not provide an explanation .
Time -Aware Belief Question Answer: {perspective2} You are {name}. Based on the above information , answer the following question: {question} Keep your answer concise , one sentence is enough. You must choose one of the above choices.

heading: Time-Aware Belief Question Answer without Belief
text: Compression (Higher-order ToM questions): {perspective} You are {name}. Based on the above information , answer the following question: {question} Keep your answer concise , one sentence is enough. You must choose one of the above choices.
Time-Aware Belief Solver: Perspective1: {perspective} You are {name}. Based on the above information , answer the following question: {question} Answer1 :{answer} Feedback Perspective2: The event corresponding to the period of belief communication between characters { questionSubject} and {questionObject}: { common_belief} Based on this information , the answer we get to the question :{ question} is Answer2: {answer2} Consider Perspective1 , Feedback Perspective2 and their answers , answer the question: {question} again. Keep your answer concise , one sentence is enough. You must choose onea of the above choices.

heading: C Experiments

heading: C.1 The Effect of Constructing Temporal Space

heading: C.2 Full Results
text: In Table 5 and 6, we present the full results of TIMETOM on the ToMI, BigToM, and FanToM benchmarks.

heading: Future Work
text: Secondly, we focus on ToM reasoning for textual modality, it is also important to perform effective multimodal ToM reasoning, which we treat as future work