M ind G ames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic (2023)

Abstract: Theory of Mind (ToM) is a critical component of intelligence but its assessment remains the subject of heated debates. Prior research applied human ToM assessments to natural language processing models using either human-created standardized tests or rule-based templates. However, these methods primarily focus on simplistic reasoning and require further validation. Here, we leverage dynamic epistemic logic to isolate a particular component of ToM and to generate controlled problems. We also introduce new verbalization techniques to express these problems in English natural language. Our findings indicate that some language model scaling (from 70M to 6B and 350M to 174B) does not consistently yield results better than random chance. While GPT-4 demonstrates superior epistemic reasoning capabilities, there is still room for improvement. Our code and datasets are publicly available.

## introduction theory of mind (tom) is the cognitive ability to attribute mental states, such as beliefs, desires, and intentions, to oneself and others, allowing individuals to understand and predict behavior based on these inferred mental states. it is an important requirement for general text understanding or artificial intelligence @xcite , but claims about tom are prone to bias from human expectations ( @xcite . @xcite recently sparked debate by showing that scaling large language models (llms) improves performance at standardized tests designed to measure tom. however, these tests were widely discussed in academic research and might have leaked into the training corpora of llm. earlier work generated synthetic examples instead, extending the babi @xcite framework. @xcite proposed a dataset of fixed templates based on the 1 [code:github][data:hf-datasets] sally-anne problem (baron- @xcite : sally puts a marble in a box while anne is with her. sally leaves for a moment and mary puts the marble in a basket. where will sally look for the marble? [@xmath0.g. the belief of sally about anne's beliefs). in our study, we generate dynamic epistemic logic (del) problems and develop verbalizations to transform them into natural language inference problems. del is a branch of modal logic that can model an individual's knowledge about particular facts or about other agents' knowledge. del also enables reasoning about the impact of consecutive public announcements: alice and bob have mud on their head. their father says that at least one of them is muddy. he asks alice and bob if they are muddy. do alice and bob know that they are muddy? [an-@xmath1. do alice and bob now know that they are muddy? [@xmath2, so after bob's first answer, alice can know that she is muddy. foot_0 del can formalize certain tom problems, making it a valuable perspective for tom assessment. the problems we create can require tracking multiple agents' beliefs and reasoning about higher-order beliefs foot_1 . our dataset encompasses numerous variations of the muddy children and drinking logicians problems @xcite . this controlled test bench offers new appreciations of language model scaling and presents the first dataset with a complexity that can challenge supervised learning models. the dataset and the scripts to generate is publicly available 1 . theory of mind in nlp to measure tom capabilities of nlp models, @xcite created examples using sally-ann templates, and @xcite added complexity to the data by incorporating second-order knowledge. both studies framed their examples as question-answering tasks. @xcite employed handcrafted tests to evaluate language models' next-word prediction capabilities. @xcite showed llm brittleness to interventions on these datasets and @xcite consolidated the prior datasets into a principled evaluation suite. the social-iqa dataset @xcite covers a broad spectrum of social commonsense, encompassing aspects of theory of mind and challenges like comprehending desires and emotions. @xcite investigated whether natural language inference models captured veridicality with epistemic verbs like know and think, using handcrafted patterns. this task was incorporated into the big-bench framework @xcite as the epistemic-reasoning task, but it targets only one shallow aspect of epistemic reasoning. @xcite used a minecraft dataset for real-time belief deduction in collaborative tasks. @xcite highlighted llm struggles in faux pas tests. @xcite conducted stress tests on llms' social reasoning capabilities. bolander (2018) showed that the sally-ann problem could be modeled with epistemic logic. @xcite examined more general connections between del and tom, while dissing and bolander (2020) demonstrated del's applicability in robotics. @xcite explored the plausibility of epistemic logic for tom by investigating its theoretical computational tractability. ## conclusion we developed a novel dataset aimed at evaluating epistemic logic reasoning, addressing a particular user: [premise] there are two persons. everyone is visible to others. there is a mirror in the room. it is publicly announced that someone's forehead is muddy. it is publicly announced that not everyone's forehead is muddy. it is publicly announced that not everyone's forehead is muddy. @xcite robert can now know whether or not everyone's forehead is muddy. is the hypothesis true or false ? gpt-4: false. based on the information provided, robert cannot know for certain whether or not everyone's forehead is muddy. the statements provided are: there are two persons. everyone is visible to others. there is a mirror in the room. it is publicly announced that someone's forehead is muddy. it is publicly announced that not everyone's forehead is muddy. since it is publicly announced that not everyone's forehead is muddy, robert can deduce that at least one person does not have a muddy forehead. however, this information alone does not allow robert to definitively determine whether or not his own forehead is muddy. figure 3 : gpt4 conversation aspect of tom. our results reveal that this task continues to pose challenges for contemporary largescale language models. when future models can solve mindgames for 2-3 agents, the difficulty can be easily scaled up with more agents. future studies could better explore human performance on our dataset, taking into account factors such as age and educational background. additionally, further investigation can examine the impact of fine-tuning on other downstream tasks and assess how well transformer circuits model kripke structures that represent modal logic problems.