Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LM s (2022)

Abstract: Social intelligence and Theory of Mind (TOM), i.e., the ability to reason about the different mental states, intents, and reactions of all people involved, allows humans to effectively navigate and understand everyday social interactions. As NLP systems are used in increasingly complex social situations, their ability to grasp social dynamics becomes crucial.In this work, we examine the open question of social intelligence and Theory of Mind in modern NLP systems from an empirical and theorybased perspective. We show that one of today’s largest language models (GPT-3; Brown et al., 2020) lacks this kind of social intelligence out-of-the box, using two tasks: SocialIQa (Sap et al., 2019), which measure models’ ability to understand intents and reactions of participants of social interactions, and ToMi (Le, Boureau, and Nickel, 2019), which measures whether models can infer mental states and realities of participants of situations.Our results show that models struggle substantially at these Theory of Mind tasks, with well-below-human accuracies of 55% and 60% on SocialIQa and ToMi, respectively. To conclude, we draw on theories from pragmatics to contextualize this shortcoming of large language models, by examining the limitations stemming from their data, neural architecture, and training paradigms. Challenging the prevalent narrative that only scale is needed, we posit that person-centric NLP approaches might be more effective towards neural Theory of Mind.

## introduction with the growing prevalence of ai and nlp systems in everyday social interactions, the need for ai systems with social intelligence and theory of mind (tom), i.e., the ability to infer and reason about the intents, feelings, and mental states of others, becomes increasingly evident @xcite ## theory of mind & large lms why do llms need theory of mind? social intelligence, theory of mind, and commonsense reasoning have been a longstanding but elusive goal of artificial intelligence for decades @xcite . these reasoning abilities are becoming increasingly necessary as ai assistants are used in situations that require social intelligence and theory of mind in order to operate effectively @xcite @xcite . for example, new technologies are emerging where ai is used to interact and adapt to users @xcite , e.g., voice assistants, and tutoring systems; or where ai helps enhance communication between multiple users, e.g., email autocomplete (chen et al., 2019), ai-assisted counseling @xcite @xcite , or facilitated discussion @xcite . as we move beyond just asking single-turn questions to social and interactive ai assistants, higherorder reasoning becomes necessary @xcite . for example, ai systems should be capable of more nuanced understanding, such as ensuring an alarm is on if someone has a job interview the next morning @xcite , knowing to call for help when an elderly person falls @xcite , inferring personality and intentions in dialogues @xcite , reasoning about public commitments @xcite , predicting emotional and affective states @xcite , and incorporating empathy, interlocutor perspective, and social intelligence @xcite . what is theory of mind? theory of mind (tom) describes the ability that we, as humans, have to ascribe and infer the mental states of others, and to predict which likely actions they are going to take @xcite . 1 this ability is closely related to (interpersonal) social intelligence @xcite , which allows us to navigate and understand social situations ranging from simple everyday interactions to complex negotiations @xcite . interestingly, the development of theory of mind and language seem to happen around similar ages in children @xcite @xcite . 2 theories of the pragmatics of language and communication can frame our understanding of this link @xcite , positing that one needs to reason about an interlocutor's mental state (tom) to effectively communicate and understand language @xcite @xcite . 3 1 while theory of mind is well developed in most adults @xcite , reasoning and inference capabilities can be influenced by age, culture, neurodiversity, or developmental disorders @xcite . 2 the direction of the tom-language association is still debated @xcite . some researchers believe language development enables tom-like abilities @xcite . on the other hand, some argue that language develops after tom since preverbal infants already could possess some level of tom-like abilities @xcite @xcite . 3 most cognitive studies on this subject focus on the english language, which is not representative of the wide variation of ## socialiqa: do llms have social intelligence and social commonsense? a crucial component of theory-of-mind is the ability to reason about the intents and reactions of participants of social interactions. to measure this, we use the dev. set of the socialiqa qa benchmark @xcite , which was designed to probe social and emotional intelligence in various everyday situations. this benchmark covers questions about nine social reasoning dimensions, drawn from the atomic knowledge graph @xcite . socialiqa instances consist of a context, question, and three answer choices, written in english. each question relates to a specific reasoning dimension from atomic: six dimensions focus on the pre-and post-conditions of the agent or protagonist of the situation (e.g., needs, intents, reactions, next actions), and three dimensions focus on the post-conditions of other participants involved in the situation (reaction, next action, effect). in total, there are 1954 three-way qa tuples; see tab. 1 for examples, and tab. 3 in appendix a for perdimension counts. ## tomi: can llms reason about mental states and realities? another key component of theory of mind is the ability to reason about mental states and realities of others, recognizing that they may be different than our own mental states. as a measure of this ability in humans, psychologists developed the sally ann false-belief test @xcite , in which two people (sally and ann) are together in a room with a ball, a basket, and a box, and while sally is away, ann moves the ball from the basket to the box. when asked where sally will look for her ball, theory of mind allows us to infer that sally will look in the basket (where she left the ball), instead of in the box (where the ball is, unbeknownst to sally). to measure the false-belief abilities of llms, we use the tomi qa dataset of english sally-annlike stories and questions @xcite ). 6 tomi stories were created using a stochastic rule-based algorithm that samples two participants, an object of interest, and a set of locations or containers, and weaves together a story that involves an object being moved (see tab. 2 ). all questions have two possible answers: the original object location, and the final object location. we investigate how llms answer the tomi story-question pairs, distinguishing between questions about factual object locations (fact) and questions about where participants think objects 6 tomi is a more challenging version of the rule-based datasets by @xcite and @xcite , as it contains randomly inserted distractor actions that prevent trivial reverse engineering. are located (i.e., their mental states; mind). the fact questions either ask about the object's original (fact-mem) or final (fact-real) location. the mind questions cover first-order (e.g., "where will abby look for the object?"; mind-1st) and second-order beliefs (e.g., "where does james think that abby will look for the object?"; mind-2nd). we further distinguish the mind questions between true belief (tb) and false belief (fb), i.e., stories where a participant was present or absent when an object was moved, respectively. importantly, answering the mind questions requires theory of mind and reasoning about realities and mental states of participants-regardless of the true-or false-belief setting-whereas fact questions do not require such tom. there are a total of 1861 two-way qa pairs in our tomi probe set, with 519 fact and 1342 mind questions (see tab. 4 in appendix b for more detailed counts). ## conclusion we explore the open question of whether and how much modern large-scale language models (llms) can reason about social intelligence and theory of mind. our results show that out-of-the-box llms struggle substantially with these abilities, which we argue are necessary but not sufficient aspects of theory of mind. specifically, gpt-3's social intelligence as measured by socialiqa lags behind humans (>30%), and the model struggles to answer tomi questions about mental states (55-60%) compared to factual questions (90-100%). in light of these shortcomings, we critically examine the large language model pretraining paradigm from a pragmatics-based perspective, and discuss possible directions towards enabling true social intelligence in nlp systems. we make our preprocessed datasets available at http://maartensap.com/neuraltom .