SlowGuess/NeurIPS_2023_OCR huggingface dataset:
most deterministic setting with a temperature of 0. We test these models with four types of prompts: 0-shot, 0-shot-chain-of-thought [17], 1-shot, and 1-shot-chain-of-thought [44]. The example used for the 1-shot prompt is from the Forward Belief - False Belief condition, where the inference variable is the belief of the agent. The task is presented to the model in the form of a comprehension question with a story, followed by a question and two answer options. We compare models on their accuracy to answer the questions. We have released our prompts and evaluation scripts on the project page<sup>6</sup>. We compare models to a human baseline<sup>7</sup> (details in B.2).
The results of our investigation are detailed in Tab. 2, Tab. 7 and App. E, spanning different conditions, models, and prompts. We discuss results for the true belief and false belief conditions. Importantly, success on the false belief version of the task is evaluated only if the model succeeded on the true belief version, as otherwise a model might succeed on the false belief version for the wrong reasons (i.e. failing to comprehend the change in the environment rather than comprehending the change in the environment and understanding that the agent was not aware of this change). Therefore, we label the success on the false belief task as “TB”  $\wedge$  “False Belief”.
Initial Percept to Initial Belief. All models are proficient at making this inference, and understand how percepts lead to the formation of beliefs (App. E to table).
$^{6}$ https://sites.google.com/view/social-reasoning-lms
7Preregistration Experiment 2: https://osf.io/zxw6m
Table 2: Performance of GPT-4 for each method. TB = True Belief. FB = False Belief. † = without initial belief. ‡ = with initial belief.
Condition	Contingency	Method
0-shot	0-shot-cot	1-shot	1-shot-cot
Fwd. Belief	TB	.99† .91‡	.99† .99‡	.99† .97‡	1.00† .97‡
FB	.98† .99‡	.99† .99‡	.99† .99‡	.99† .99‡
TB ∧ FB	.97† .90‡	.98† .98‡	.97† .96‡	.99† .96‡
Fwd. Action	TB	.98† .98‡	.99† .99‡	1.00† 1.00‡	1.00† 1.00‡
FB	.81† .92‡	.88† .96‡	.98† 1.00‡	1.00† .99‡
TB ∧ FB	.79† .90‡	.87† .95‡	.98† 1.00‡	1.00† .99‡
Bwd. Belief	TB	.86† .62‡	.84† .76‡	.68† .57‡	.83† .81‡
FB	.53† .77‡	.54† .63†	.85† .92‡	.75† .85‡
TB ∧ FB	.40† .40‡	.38† .40‡	.53† .49‡	.58† .65‡
Forward Belief Inference. Here we test if models can track beliefs across the change in the world (Tab. 2 and Fig. 3a). Many models struggle with this, especially when an initial belief is stated (suggesting they anchor on this explicitly stated belief). gpt-4 and, to a lesser extent, Claude perform better, approaching human levels.
Forward Action Inference. While all models are good at predicting actions when beliefs agree with the world state, most models struggle in the critical false-belief condition (Fig. 3b). gpt-4 is the exception, exhibiting human-level performance (or even slightly better).
Backward Belief Inference. This represents the most challenging inference. Even humans struggle, achieving only  $82\%$  accuracy in the true belief condition and  $72\%$  in the false belief condition. We believe this is due to unavoidable uncertainty about whether the agent gained knowledge of the true world state. Models are generally far below chance, indicating that they reliably attribute the wrong belief, especially in false-belief situations and especially when an explicit initial belief is given (Fig. 3c). gpt-4 is again the exception with a more human-like pattern, though not achieving human level performance 0-shot.
Comparison of prompts. Human participants received instructions and a demonstration example to understand the task (see App. H). Hence, a fair comparison should provide similar support to models. One-shot learning consistently enhances performance across all models and conditions. In contrast, zero-shot-chain-of-thought (CoT) prompting doesn't consistently improve performance across conditions. Introducing a one-shot CoT example does lead to consistent performance improvement across all conditions, however this performance may not be indicative of stronger ToM per se: mimicking the reasoning template is enough to solve our task in most cases. (Human participants were not given demonstrations of how to reason in the task.)
In this work, we present a novel framework for measuring the capabilities of large language models (LLMs) by using abstract causal templates to automatically generate scenarios, controlling what information is provided and what must be inferred. We created a new benchmark for Theory of Mind (BigToM), allowing us to more carefully map the ToM reasoning abilities of LLMs. Our extensive control conditions aim to take into account content effects [6] and many low-level confounds. We found that many models struggle with even the most basic component of ToM, reasoning from percepts and events to beliefs, and that this is substantially affected by previously-stated beliefs. Of all the models tested, GPT-4 exhibits ToM capabilities that align closely with human inference patterns. Yet even GPT-4 was below human accuracy at the most challenging task: inferring beliefs from actions.
Limitations. Our evaluation methodology may appear circular at first: the model being tested plays a role in generating the test items. However, we believe that for testing inferential abilities this is not a confound. Our method constructs stories by selecting from all the available facts of a given situation and then isolates the inferential capabilities for the remaining aspects. This means that a model may be able to understand the immediate causal steps in the story while being unable to perform the required inferences being tested. Indeed, even gpt-4 does not achieve a perfect zero-shot score at
our tests, indicating this gap between situation knowledge and inferential understanding. Further, to validate our hypothesis about circularity not being a confound, we generate an evaluation set with c1aude-2. We find that gpt-4 gets comparable scores on an evaluation set generated by a different model, outperforming the model that created the dataset (see App. F for details).
Our method shares limitations with other model-generated evaluations (as discussed in Perez et al. [28]): the generated evaluations can be biased in the content and the contexts that are generated. While synthetic datasets generated from language models offer advantages in scale and control, they also come with inherent biases reflecting those embedded in the underlying model and training data. As large language models are trained on internet text, they inevitably pick up stereotyped associations for gender, race, and other attributes in certain contexts. This could lead to normative, stereotyped roles in different situations in the synthetic dataset. A related issue could arise from biases leading to over-generation of certain situations, effectively yielding imbalanced evaluation data. (We note this is also a problem for human-generated items!) However, language models are also steerable through detailed instructions, allowing mitigation of some biases. Careful steering might be needed during dataset generation to ensure diversity and balance along different dimensions. In domains where the models capabilities are lacking, the model will struggle to generate good evaluations. Such limitations could be resolved through shared generation with a human expert while populating the causal graph (see App. A for an example interface). The stories produced by the model at times exhibit errors in common sense, yet these instances represent a small fraction ( $\sim 3\%$ ) of the overall tests generated; as language models continue to improve, we can expect these errors to reduce. Our test items tend to have syntactic similarities which might reduce the diversity of the items in our benchmark; this could perhaps be fixed by asking the model to paraphrase the generated stories.
Future Work. Our causal template method can be used for other domains where the effects of hidden causes or the underlying causes of effects must be inferred. These include many studied by cognitive scientists interested in the "intuitive theories" underlying human reasoning. For instance, morality, affect, and desire within social cognition, and extending to physical reasoning and more abstract reasoning such as medical diagnosis and mathematics.
In the future, testing social reasoning should move towards more realistic scenarios that are not limited to traditional ToM tests. We believe that we should focus on creating social reasoning tests or benchmarks in use-cases where LLMs are being deployed. We believe that there is a need to move towards more dynamic benchmarks for social reasoning, by creating environments where people or simulated agents (LLMs as people) interact with a language model. Such environments could also be used as a playground where the capabilities of models are not only measured, but also improved.
Conclusion. We have demonstrated a novel approach for assessing LLMs, and while there are limitations, we believe our findings offer a promising direction for future research in understanding and enhancing the capabilities of these powerful models. The nascent ability of LLMs to reason about mental states of people is a foundational capability for exciting use cases and problematic misuse. Systematic and broad benchmarking of these abilities is thus a pressing concern, and we believe BigToM is an important step.
This worked was supported by the Stanford Human-Centered Artificial Intelligence (HAI) Hoffman-Yee grant, and the NSF Expeditions Grant, Award Number (FAIN) 1918771. We would like to thank the Stanford Center for Research on Foundation Models (CRFM) and Yifan Mei for the tokens and the infrastructure to test different models.
[1] Chris L Baker, Noah D Goodman, and Joshua B Tenenbaum. Theory-based social goal inference. In Proceedings of the thirtieth annual conference of the cognitive science society, pages 1447-1452. Cognitive Science Society Austin, TX, 2008.
[2] Chris L Baker, Julian Jara-Ettinger, Rebecca Saxe, and Joshua B Tenenbaum. Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nature Human Behaviour, 1(4):0064, 2017.
[40] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
[41] Sean Trott, Cameron Jones, Tyler Chang, James Michaelov, and Benjamin Bergen. Do large language models know what humans know? arXiv preprint arXiv:2209.01515, 2022.
[42] Tomer Ullman. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399, 2023.
[43] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022.
[44] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022.
[45] Henry M Wellman. The child's theory of mind. The MIT Press, 1992.
[46] Heinz Wimmer and Josef Perner. Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's understanding of deception. Cognition, 13(1):103-128, 1983.
[47] Sarah A. Wu, Shruti Sridhar, and Tobias Gerstenberg. A computational model of responsibility judgments from counterfactual simulations and intention inferences. In Micah B. Goldwater, Florencia Anggoro, Brett Hayes, and Desmond C Ong, editors, Proceedings of the 45th Annual Conference of the Cognitive Science Society, 2023. URL https://psyarxiv.com/uwdbr/.