T o MB ench: Benchmarking Theory of Mind in Large Language Models (2024)

Abstract: Theory of Mind (ToM) is the cognitive capability to perceive and ascribe mental states to oneself and others. Recent research has sparked a debate over whether large language models (LLMs) exhibit a form of ToM. However, existing ToM evaluations are hindered by challenges such as constrained scope, subjective judgment, and unintended contamination, yielding inadequate assessments. To address this gap, we introduce ToMBench with three key characteristics: a systematic evaluation framework encompassing 8 tasks and 31 abilities in social cognition, a multiple-choice question format to support automated and unbiased evaluation, and a build-from-scratch bilingual inventory to strictly avoid data leakage. Based on ToMBench, we conduct extensive experiments to evaluate the ToM performance of 10 popular LLMs across tasks and abilities. We find that even the most advanced LLMs like GPT-4 lag behind human performance by over 10% points, indicating that LLMs have not achieved a human-level theory of mind yet. Our aim with ToMBench is to enable an efficient and effective evaluation of LLMs’ ToM capabilities, thereby facilitating the development of LLMs with inherent social intelligence.

## introduction theory of mind (tom) is a fundamental cognitive process, involving the capability to attribute mental states -beliefs, intents, desires, emotions, knowledge, etc. -to oneself and others @xcite . tom is essential for human social cognition @xcite and plays an important role in social activities like empathetic communication @xcite , relationship maintenance @xcite , decision making @xcite , and childhood education @xcite . with the advent of the era of large language models (llms), powerful llms like gpt-4 @xcite and llama @xcite have demonstrated comparable performance to humans in solving tasks. consequently, researchers have grown increasingly curious that "do large language models have a theory of mind?", while the subsequent assessments have yielded inconsistent and even contradictory results. @xcite and @xcite find that gpt-3.5/4 achieve remarkably high performance on the false belief task @xcite , suggesting that tom may have spontaneously emerged in llms. however, later studies reveal that llms' tom performance dramatically decreases when faced with trivial alterations @xcite . this suggests a phenomenon called "clever hans" @xcite denoting that llms rely on unexpected spurious correlations @xcite rather than truly possessing a "mind reading" ability. this ongoing debate underscores the pressing need for a holistic tom benchmark. despite its critical importance, the development of such a benchmark still faces three major challenges. 1) constrained scope. existing studies mainly evaluate llms on specific tom dimensions like emotions @xcite and beliefs @xcite . while inspiring, they fall short of providing a complete tom assessment. 2) subjective judgment. most existing inventories for tom evaluation involve open-ended questions where responses require manual scoring by experts @xcite . while this approach works on small-scale and focused tom tests for children, it is impractical for benchmarking due to the high annotation cost required for comprehensively evaluating llms. additionally, human evaluations towards diverse answers may introduce potential inconsistency and bias @xcite . 3) unintended contamination. although merely combining different inventories can create a corpus involving multiple tom aspects, it is still not qualified for being a benchmark. since most tom inventories in psychology research are released far before the launch of llms, they are likely to have appeared in the training process of llms, whether during pre-training, sft, or rlhf (golchin and surdeanu, 2023; @xcite . we present a simple verification in appendix a. the potential contamination indicates a risk that llms might complete tom evaluations in an "open-book" status, leading to inflated performance @xcite . to address the above challenges, we propose t mbench, a tom benchmark for llms featuring three key characteristics. 1) systematic framework. we first review broad psychological literature and identify 8 well-defined theoryof-mind tasks in t mbench. then, since tasks are the means and abilities are the core, we further ground t mbench in a well-structured psychological framework, atoms @xcite , expanding our evaluation to encompass 31 core theory-of-mind abilities. we hereby design both task-and ability-oriented evaluations to comprehensively assess the tom capabilities of llms. using t mbench, we conduct both task-and ability-oriented evaluations of 10 popular llms with vanilla and cot prompting. we also recruit 20 graduate students to provide an expected human baseline tom performance. the experimental results show that, the gpt-4 series outperforms other llms across tom tasks and abilities, yet their average performance is significantly lower than human levels by over 10% points. further analysis shows that when facing a more rigorous and comprehensive understanding of social scenarios, the gap in tom performance between llms and humans widens. moreover, inspecting the internal attention of llms reveals their decision-making processes notably differ from those of humans. to the best of our knowledge, t mbench is the first systematic tom benchmark for automated tom evaluation of llms. we hope that t mbench will drive construction through evaluation, facilitating the development of llms in the realm of social intelligence, thereby complementing their task-solving capabilities. a broader objective is to enable llms to transcend the superficial layer of natural language and delve deeper into understanding the mental states and cognitive processes behind human words, hereby foster more efficient and seamless human-ai interaction. we release t mbench at https://github.com/ zhchen18/tombench . ## conclusions we introduce t mbench, the first systematic tom benchmark for llms with an mcq-type inventory entirely built from scratch. we reveal that even the most advanced llms significantly lag behind human performance in tom. further indepth analysis indicates that llms still struggle to comprehensively understand social scenarios, and tend to rely on semantic associations rather than human-like cognitive processes when addressing tom questions. by proposing t mbench, we aim to provide an effective and efficient platform for evaluating the tom capabilities of llms and spur further research into llms that innately understand tom, thereby improving how artificial intelligence can serve us in a human-friendly way.