Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground (2024)

Abstract: Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data, which risks misaligning the resulting experiments with human behavior. We introduce the first ToM dataset based on naturally occurring spoken dialogs, Common-ToM, and show that LMs struggle to demonstrate ToM. We then show that integrating a simple, explicit representation of beliefs improves LM performance on Common-ToM.

## introduction in cognitive science, theory of mind (tom) refers broadly to the capacity to understand the mental states of others (e.g. beliefs, desires, emotions) even, crucially, when they differ from your own @xcite . successful human conversation is possible only because participants model each others' cognitive states (i.e., tom) and plan utterances based on their intended audience @xcite @xcite . as a result, tom has received increasing attention from the nlp community seeking to evaluate the capabilities of language models (lms) on tasks inspired by the psychological literature @xcite . in this paper, we introduce common-toma question answering benchmark based on naturally-occurring, spoken dialogs in english. 1 common-tom uses the notion of common ground (cg) for evaluating tom. the cg @xcite is a set of beliefs mutually shared by all participants in a conversation. other benchmarks have developed tom questions based on the sally-anne test @xcite , which checks an understanding of information 1 https://github.com/cogstates/common-tom accessibility to determine beliefs. we observe that when the cg is mismatched between the discourse participants, such as at the time of a question or during the repair of a miscommunication, similarly complex problems for tom arise. our main contributions are: (1) arguing that using synthesized data in evaluating the tom ability of lms is not conclusive; (2) releasing a corpus for benchmarking tom based on naturally occurring spoken conversations; (3) showing that llms struggle with our benchmark and a simple explicit architecture performs better. the paper is organized as follows. first, we review some of the relevant literature ( § 2). we then describe the framework and methods used in creating common-tom ( § 3). this is followed by experiments and results in section 4: human performance, using zero-shot and fine-tuned lms, and using our approach for building a system that explicitly represents beliefs ( § 4.4). finally, we discuss our key takeaways and conclusions ( § 5). ## related work while quite a few corpora annotate author belief @xcite , there are relatively few corpora annotated explicitly for cg. @xcite ; @xcite focus on limited aspects of cg. in this paper, we use the comprehensive cg corpus we presented in @xcite ), which we discuss in section 3. many tom benchmarks have been created recently. @xcite produce a question answering corpus (tom-babi) derived from template-generated stories inspired by the sally-anne test (an agent's cognitive state depends on whether they witness a specific action). @xcite note that such formulaic data results in a flawed evaluation, especially when using supervised methods, and produce their own templatic corpus (tomi) which introduces more noise such as distractor sentences and reorderings. despite these improvements, tomi has been shown to be prone to the same issues as tom-babi @xcite . @xcite take this even further and prompt lms to produce dialogs following a similar form. the stories all follow the general structure of the sally-ann stories. corresponding questions then probe various character's beliefs (true and false) of both first and second order. though useful, these benchmarks are prone to surface-level cues and spurious correlations which have been exploited by lms to display illusory tom @xcite . some work has been done to produce tools based on human-generated text. @xcite , like us, exploit the relationship between cg and tom. they design an experimental setup that records written dialogs of players exchanging information in minecraft. more recently, @xcite combine the aforementioned datasets (and more) to create a composite benchmark. we build on this work conceptually by producing a new dataset which, in contrast with the work of @xcite , is based on dialogs which are collected independently of our interest in tom; the dialogs are spontaneous and not guided by any experimental setting; and they are spoken dialogs. we note that co-presence (including telepresence) has been identified as an important (though not required) part of human tom @xcite , and our corpus allows the study of tom under this common condition using naturalistic data. ## common-tom framework in philosophy, cg is sometimes treated as the mutual beliefs between two agents @xcite , separate from their cognitive states, while in cognitive science it is common to model cg as the belief of an agent about what they and another agent mutually believe @xcite . we adopt the latter definition as it is allows cg to represent scenarios like false belief, which would be impossible under the former. this also makes the relationship between cg and tom explicit: if i believe a proposition is in the cg with you, it means by definition that i believe that you also believe it is in cg -thus, it entails a tom assumption. we also follow cognitive literature in assuming that humans do not have a full representation of their interlocutor's cognitive state at all times, but instead can make necessary inferences when needed @xcite . specifically, cg does not mean that all consequences of cg are present at all times, and performing the inferences from cg (high-degree knowledge questions, for example) takes time and may be errorful. our experiments with human annotators ( § 4) confirm this assumption, but also show that humans are much better than zero-shot lms because they are in fact modeling cg, and can answer the higherorder belief questions as needed from their models of cg. farkas and bruce (2010); eckardt (2016) propose deterministic heuristic algorithms for inferring cg from text. all this prior work motivates our neuro-symbolic approach ( § 4.4). ## conclusion we present a new corpus for testing theory of mind (tom) capabilities, common-tom. unlike previous tom corpora, common-tom uses naturally occurring linguistic data and is not based on agents perceiving certain information or not. we explicitly model belief and cg to capture tom in a manner directly inspired by cognitive science literature. while lms are rapidly evolving and scaling in size and capabilities, they still lack a conceptual understanding of beliefs and cg in dialog. therefore, for an lm to interact naturally it is necessary to explicitly model belief, cg, and cognitive states.