Comparing Humans and Large Language Models on an Experimental Protocol Inventory for Theory of Mind Evaluation ( EPITOME ) (2024)

Abstract: We address a growing debate about the extent to which large language models (LLMs) produce behavior consistent with Theory of Mind (ToM) in humans. We present EPITOME: a battery of six experiments that tap diverse ToM capacities, including belief attribution, emotional inference, and pragmatic reasoning. We elicit a performance baseline from human participants for each task. We use the dataset to ask whether distributional linguistic information learned by LLMs is sufficient to explain ToM in humans. We compare performance of five LLMs to a baseline of responses from human comprehenders. Results are mixed. LLMs display considerable sensitivity to mental states and match human performance in several tasks. Yet, they commit systematic errors in others, especially those requiring pragmatic reasoning on the basis of mental state information. Such uneven performance indicates that human-level ToM may require resources beyond distributional information.

## introduction theory of mind (tom) is a broad construct encompassing a range of social behaviors from reasoning about others' mental states (internal psychological states such as beliefs and emotions) to understanding non-literal communication @xcite . these mentalizing or mindreading capacities underpin social intelligence @xcite , allowing us to anticipate others' actions @xcite , solve social coordination problems @xcite , and understand communicative intent @xcite . there is growing interest in whether artificial intelligent (ai) agents could display tom abilities @xcite @xcite . many desirable ai applications require something akin to tom, including recognizing users' intents @xcite , displaying empathy toward users' emotions @xcite , and interpreting requests in the context of users' goals @xcite . the recent success of large language models (llms) has further intensified interest and optimism in the potential for artificial tom. although their pre-training regime does not explicitly include social interaction or communicative intent @xcite , llms produce text which superficially bears many hallmarks of mentalizing @xcite . however, previous studies evaluating llm performance on tom tasks have yielded inconsistent findings, sparking debates on llms' tom capacities @xcite @xcite . here, we collect a battery of six diverse tasks, used to measure tom in humans, to investigate the consistency of llms' tom capabilities. a variety of tasks have been designed to measure different facets of mentalizing @xcite @xcite . unfortunately, these measures exhibit poor convergent validity-performance in one task does not necessarily correlate with any other-and limited predictive validity, with task performance failing to consistently predict socioemotional functioning @xcite . this limits the extent to which performance on a single task can be taken as evidence of tom more generally, and underscores the need for running varied, tightly controlled experiments, each measuring distinct aspects of mentalizing. we select six tasks from the psychology literature which collectively measure a diverse set of tom-related abilities including belief attribution, emotional reasoning, non-literal communication, and pragmatic inference. beyond measuring llms' tom performance, these models can provide insights into debates on human tom's evolutionary and developmental origins @xcite . researchers disagree about whether tom is an innate, evolutionary adaptation @xcite or learned via social interaction @xcite and language @xcite @xcite . if language exposure is sufficient for human tom, then the statistical information learned by llms could account for variability in human responses. we collate human responses to each task for comparison with llm performance, using identical materials for both. this approach allows us to ask where llms sit in the distribution of human scores; whether their accuracy is significantly different from humans; and whether their predictions explain the effects of mental state variables on human responses. ## related work early work in machine tom @xcite found that neural language models could learn to coordinate actions using language @xcite , but struggled with explicit mental state reasoning @xcite . several recent studies have directly investigated tom abilities in llms. @xcite evaluated gpt-3 davinci @xcite on socialiqa (a crowdsourced dataset of multiple choice questions about social reactions to events @xcite ) and tomi (a synthetically generated dataset of false belief task passages; @xcite . gpt-3 achieved 55% accuracy on socialiqa, well below a baseline of 84% set by three human participants @xcite . while tomi lacks a specific human baseline, gpt-3 performed poorly (60% accuracy) at belief questions, despite being near ceiling on factual questions. @xcite similarly found that gpt-3 davinci performs poorly (40% accuracy) on a range of novel false belief stimuli @xcite . however, later models in the series performed much better. gpt-3 text-davinci-002, fine-tuned to follow instructions, achieved 70% accuracy. gpt-3 text-davinci-003 and gpt-4-fine-tuned using reinforcement learning-achieve 90% and 95%, respectively. although the paper does not establish a human baseline for the novel stimuli, this compares favorably to meta-analyses suggesting typical accuracy of 90% for 7-year olds @xcite . @xcite , however, showed that 8 simple perturbations to kosinski's stimuli cause gpt-3 text-davinci-003 to fail, suggesting that llms exploit shallow statistical patterns rather than deploying a deep, emergent tom ability. though these perturbations were not tested with humans or generalized to a larger sample of items, ullman argues that ''outlying failure cases should outweigh average success rates.'' more recently, @xcite used llms to construct a synthetic false belief benchmark from causal graphs, on which gpt-4 performs similarly to humans. @xcite used a similar approach to generate a belief attribution benchmark composed of naturalistic conversational dialogues. however, the best performing llms perform as low as 26.6% on their most challenging measures, lagging far behind a human baseline of 87.5%. finally, @xcite evaluated 15 llms across 6 tasks incorporating belief attribution (tomi, false belief), epistemic reasoning, and social reactions (socialiqa and faux pas). they found that no model performed robustly, and that all models were vulnerable to adversarial perturbations in the style of @xcite . our contribution differs from existing studies in several ways. first, we incorporate tasks that evaluate a broader range of tom capacities. while most studies focus primarily on belief attribution or social appropriateness, we additionally evaluate models on emotional reasoning, non-literal communication, and pragmatic reasoning from mental state inferences. additionally, we test belief attribution up to 7 levels of embedding, and use a range of evaluation criteria (including human evaluation of free-text completions). second, we intentionally use experimental stimuli originally designed to measure tom in humans. some researchers are rightly concerned that these tasks may not have the same construct validity for llms as they do for humans @xcite @xcite . we agree that successful performance on these tasks does not imply an agent has tom. however, this objection is not overcome by designing novel tasks that have not been validated on human participants. the proposition that an llm displays tom must be supported by a range of empirical, theoretical, and probably mechanistic evidence. moreover, we believe that existing experimental stimuli have several advantages which complement contemporary work with synthetic or crowdsourced benchmarks: they have been carefully designed to control for confounds and they have been validated as measures of specific latent constructs in humans. third, to allow direct item-level comparison between model and human performance, for each study we elicit an appropriately powered human baseline for all items and make all human data available. fourth, we preregistered four of the six studies in order to minimize the risk of selecting materials or analyses that would bias results. finally, to test whether distributional information learned by llms can fully account for human behavior, we run a distributional baseline analysis @xcite : testing whether mental state variables explain residual variance in human responses beyond the variance explained by the llm responses. ## the present study we assemble epitome-a battery of six experiments designed to measure distinct aspects of tom in humans (see figure 1 ). we selected these six experiments in order to provide broad coverage of the theorized components of tom @xcite . the false belief task (fb) tests whether participants can maintain a representation of someone else's belief, even if it differs from their own @xcite . recursive mindreading (rm) tests whether participants can recursively represent mental states up to seven levels of embedding, e.g., ''alice knows that bob believes that charlie...'' (o @xcite . the short story task (shs) measures the ability to infer and explain emotional states of characters @xcite , while the strange stories task (sts) @xcite asks participants to explain why characters might say things they do not mean literally. the final two tasks measure sensitivity to speaker knowledge during pragmatic inference. the indirect request task (ir) asks whether participants are less likely to interpret an utterance as a request if the speaker knows that the request can't be fulfilled @xcite . the scalar implicature (si) task tests whether comprehenders are less likely to interpret some to mean not all when the speaker does not know enough to make the stronger claim @xcite . we used this battery of tasks to address a longstanding debate about the origins of tom in humans: namely, the extent to which language exposure is sufficient to account for human mentalizing ability. the distributional hypothesis @xcite suggests that human comprehenders use statistical information about the co-occurrence frequency of words to understand language. the rapid advance of llms-that learn exclusively from such information-has galvanized interest in the distributional hypothesis, with many recent studies showing that llms can accurately predict human linguistic behavior @xcite and neural activity @xcite . a more specific instantiation of this broader debate concerns the role of language exposure in human tom development (de villiers and @xcite . we address this question by comparing the responses of llms and humans on epitome. crucially, in order to test the sufficiency of distributional information per se, we restrict our analysis to models that have not been fine-tuned on other objectives such as reinforcement learning from human feedback (rlhf; @xcite . while rlhf is theorized to improve tom performance (moghaddam and honey, 2023), it exposes models to an additional training signal, making it hard to draw inferences about the sufficiency of language exposure alone. our main analysis focuses on gpt-3 text-davinci-002 (henceforth, gpt-3)-one of the best-performing models which has not been trained using rlhf. 1 we make our code and materials available to facilitate addressing further questions, including whether rlhf improves tom performance. we ask four types of question: (1) where does gpt-3 sit in the distribution of human performance? (2) how does gpt-3 performance vary with model scale? (3) is gpt-3 sensitive to experimental variables that alter characters' mental states? (4) does gpt-3 fully explain human mentalizing behavior? or is there a residual effect of mental state variables on human comprehenders after controlling for distributional likelihood (as measured by gpt-3 predictions)? we pre-registered our analyses for four tasks, and provide code, data, and materials for all six. foot_1 ## methods we accessed models through the openai api. for tasks that involved generating text (shs, sts), we set temperature to 0. for the remaining tasks, we measured the probability assigned by the model to a given string. when measuring the probability assigned to a multi-token string, we summed the log probabilities of each token. we used the same instructions and stimulus wording for both humans and llms. we avoided using any kind of prompt engineering with llms to ensure a fair comparison. we generated novel stimuli for the scalar implicature task and we conducted a contamination analysis following @xcite , which indicated that none of the other datasets were contained in the model's training data (see appendix b). the number of human participants in each study varied based on the types of statistical analysis being run, the number of items, and the number of observations per participant. for tasks without explicit correct answers, 'accuracy' is defined as the total score on questions measuring sensitivity to mental states. we use publicly available data from @xcite for fb, and use their analysis as a model for other tasks. llm data and analyses for all other tasks, as well as human data for rm, sts, and si, are novel contributions. all novel human data was collected from undergraduate students, while existing data for fb, shs and ir was collected via amazon mechanical turk. @xcite constructed 12 passage templates, in which a main character puts an object in a start location, and a second character moves it to an end location. the last sentence states that the main character believes the object is in some (omitted) location (e.g., ''x thinks the book is in the ''). there are 16 versions of each item (192 passages in total) which varied across 4 dimensions: (i) knowledge state: whether the main character knows (true belief) or does not know (false belief) that the object has changed location; whether (ii) the first mention and (iii) the most recent mention of a location is the start or end location; and (iv) knowledge cue: whether the main character's belief is stated implicitly (''x goes to get the book from the '') or explicitly (''x thinks the book is in the ''). ## results for all 6 tasks, we asked the following 2 types of question: in each case, we use a chi-squared test to compare the fit of a full model (indicated above) with a base model (with boldface variables removed). for the fourth question, this allows us to test whether mental state variables explain significant variance in human responses once the effect of distributional likelihood (measured by gpt-3 predictions) has been controlled for. we used mixed effects models with random intercepts ## discussion we assembled epitome-a battery of six tom experiments that tap diverse aspects of tomand provided a human baseline for each task. we used the dataset to assess the extent to which distributional information learned by an llm (gpt-3) was sufficient to reach human-level per-formance on these tasks. llm performance varied considerably by task, achieving parity with humans in some cases and failing to show sensitivity to mental states at all in others. there was also significant variation in human performance within and between tasks-with close to baseline performance on si e1 and ir-highlighting the importance of establishing human baselines to contextualise llm performance. while previous work has shown isolated successes @xcite and failures @xcite of llms at specific tasks, the breadth of tasks presented here provide a more systematic basis for understanding model performance on diverse aspects of tom. we make the code, materials, and human data from epitome available to facilitate further research into differences in tom between humans and llms. in some respects, gpt-3 showed striking sensitivity to mental state information. for three of the tasks (shs, sts, and ir), gpt-3 accuracy was not significantly different from the human mean. for the shs and sts tasks, this means that gpt-3's free-text explanations of characters' mental states were rated as equivalent to humans' by human evaluators. in others tasks, gpt-3 was sensitive to mental states, with above chance performance in rm up to 4 levels of embedding, and significant effects of knowledge state in fb. this provides an important demonstration that distributional information alone is sufficient to generate approximately humanlike behavior on several tasks that have been used to measure tom in humans. however, other aspects of the current results suggest crucial differences between human and llm performance. first, gpt-3 was insensitive to knowledge state in the ir task, interpreting every statement as a request. second, gpt-3 failed to show effects of speaker knowledge in si, although poor human performance indicates the wording of e1 may be ambiguous. third, gpt-3 failed to perform above chance at recursive mindreading beyond 5 levels of embedding, suggesting that distributional information may be insufficient for more complex mentalizing behavior. however, it's possible that more or better distributional data could enable progress on this task. finally, across 4 tasks (fb, rm, ir, and si) there were residual effects of mental state variables on human responses after controlling for gpt-3 predictions. in other words, even after accounting for any variance in human responses that could be explained by the distributional language statistics learned by gpt-3, there was still a significant effect of mental state variables on human responses. this indicates that humans are sensitive to mental state information in a way that is not captured by the model. consistent with the hypothesis that an llm's performance is positively correlated with its size @xcite , we found positive scaleaccuracy relationships for 4 tasks (fb, rm, and sts, si e1). however, ir and si e2 showed flat or even negative scaling. this could indicate that models will require information beyond distributional statistics to achieve human parity. gpt-3 performed worst on ir and si, the two tasks requiring pragmatic inferences from mental state information. these showed the largest gaps in accuracy, insensitivity to mental states, and the flat scaling relationships noted above. given existing work showing llm sensitivity to pragmatic inference @xcite , this trend could indicate a specific difficulty for llms in making pragmatic inferences on the basis of mental state information. these tasks require a complex multistep process of sampling, maintaining, and deploying mental-state information @xcite , increasing the chances of information loss. these results bear on the origins of mentalizing abilities in humans. llms' sensitivity to mental state variables suggests that domain-general learning mechanisms and exposure to language could be sufficient to produce tom-consistent behavior. but llms also performed relatively better at non-mental control questions (in rm and shs). this could imply that distributional information is less useful for predicting human performance in mentalistic than non-mentalistic tasks, supporting the view that humans recruit other resources for mental reasoning specifically.