Beyond Context to Cognitive Appraisal: Emotion Reasoning as a Theory of Mind Benchmark for Large Language Models (2025)

Abstract: Datasets used for emotion recognition tasks typically contain overt cues that can be used in predicting the emotions expressed in a text. However, one challenge is that texts sometimes contain covert contextual cues that are rich in affective semantics, which warrant higher-order reasoning abilities to infer emotional states, not simply the emotions conveyed. This study advances beyond surface-level perceptual features to investigate how large language models (LLMs) reason about others’ emotional states using contextual information, within a Theory-of-Mind (ToM) framework. Grounded in Cognitive Appraisal Theory, we curate a specialized ToM evaluation dataset to assess both forward reasoning—from context to emotion—and backward reasoning—from emotion to inferred context. We showed that LLMs can reason to a certain extent, although they are poor at associating situational outcomes and appraisals with specific emotions. Our work highlights the need for psychological theories in the training and evaluation of LLMs in the context of emotion reasoning.

## introduction humans are naturally adept at reasoning about other people's mental states such as beliefs, goals, and intentions @xcite . these social-cognitive reasoning abilities enable individuals to navigate complex social situations and facilitate interpersonal communication @xcite . recent research has explored whether llms possess such abilities to infer cognitive states of other agents (e.g. theory-ofmind (tom)) @xcite , however, llms' ability to reason about one of the most essential components of human experiences-emotional states-is not well studied. 1 data is publicly available on https://github.com/ gerardyeo/tomemoreason.git studies in affective computing commonly engage in emotion recognition tasks where the goal is to predict the most representative emotion in a text @xcite . although recent transformer models can represent contextual cues to a certain extent, it is not clear whether these models are reasoning about agents' emotional states or they are merely using emotional words in the prediction. emotion reasoning is conceptually different from emotion recognition where the former incorporates abstract contextual information to reason about emotions while the latter is typically based on perceptual features (i.e. emotional words). reasoning about other people's emotions involves a set of cognitive processes and numerous psychological studies have adapted an intuitive tom framework to understand these processes @xcite . for instance, given that john received a poor grade on an exam, an observer could reason that john is feeling sad. in addition to inferring emotions based on a situation (forward reasoning), people also perform backward reasoning (seeing john crying) where they infer possible situations that elicit particular emotions @xcite . these and other paradigms are discussed as part of cognitive psychology, where empirical findings that such emotion reasoning in humans can be understood through the framework of dualprocess theories, which posit two distinct modes of thinking: system 1, which is fast, automatic, and affect-driven, and system 2, which is slower, deliberate, and effortful @xcite . emotion reasoning likely requires a shift from system 1's pattern-based recognition to system 2's context-sensitive inference, with accurate cognitive inference serving as a crucial intermediate step. for llms, this process might involve using contextual cues to simulate the evaluative processes that guide human emotion inference, rather than simply relying on surface-level emotional features. yet, which paths do llms choose for emotional reasoning? to answer this question, this study examines whether llms can extend beyond basic emotion recognition to infer third-person emotions based on contextual cues, and infer outcomes based on emotions. we developed a dataset comprising vignettes that are systematically manipulated according to cognitive appraisal theory. cognitive appraisal theory, which is one of the primary theoretical frameworks for understanding emotional experiences, posits that emotions are elicited from subjective interpretation of events along a set of dimensions @xcite @xcite . although certain situations are known to reliably elicit specific emotions in people (e.g. death of a loved one results in sadness), cognitive appraisal theory emphasizes that it is the evaluation of the situation, rather than the situation per se, that results in emotional reactions. for instance, if an individual appraises a situation as obstructing their personal goals (goal inconduciveness), attributes responsibility to another person (other-accountability), and perceives the situation as unjust (unfairness), they are likely to experience anger. conversely, if another individual appraises the same situation differently, a different emotional response is likely to be elicited. observers inferring another agent's emotions engage in a comparable cognitive process, drawing on their understanding of the agent's goals and desires, as well as how the agent is likely to evaluate the situation @xcite . that is, by considering the alignment between the situation and the agent's goals-alongside intuitive emotion knowledge (e.g., unfairness typically elicits anger)-observers make inferences about the agent's emotional state. our research builds on prior work investigating tom in llms, extending it to third-person emotion reasoning through the lens of cognitive appraisal theory (see figure 1 ). we designed vignettes that vary in their linguistic descriptions of outcomes and cognitive appraisals while minimizing explicit emotional features @xcite . our theoretical contributions are as follows: tom task to elucidate how state-of-the-art llms reason about emotions in other agents based on manipulated vignettes, grounded in cognitive appraisal theory (ellsworth and scherer, 2003). • we have created subtasks that provide evidence for the system-1 and system-2 reasoning mechanisms applied by llms. • we have curated a novel emotion reasoning evaluation dataset and evaluate llms' emotion reasoning abilities. ## evaluation dataset, prompts, and tasks we constructed our evaluation dataset based on the prisoner's dilemma game, adapting stimuli from previous psychological studies into a text-based format (see figure 1 ; @xcite ). despite its simplicity, this context provides a highly relevant framework for examining how llms infer others' emotions. having vignettes with clearly defined outcomes based on the decisions of the characters, the dataset enables a systematic investigation of emotion reasoning processes. our evaluation dataset comprises two primary tasks: (a) forward emotion reasoning and (b) backward outcome and appraisal reasoning. in the forward emotion reasoning task, llms are prompted to infer the emotions of a target agent in a given vignette, based on descriptions of the outcomes and the agent's cognitive appraisals. this task's evaluation dataset includes three distinct scenarios representing the prisoner's dilemma in different contexts: (i) the split or steal game show, (ii) a business deal, and (iii) relationship commitment (see appendix a). these contexts were selected for their relevance to high-stakes decision-making domains. the evaluation prompts were systematically designed to manipulate the following key variables: (1) outcome choices, (2) cognitive appraisal of fairness, and (3) accountability. to account for the sensitivity of llm responses, we rephrased each vignette in three different ways using gpt-4. after being presented with the designed vignettes, llms were required to answer an appraisal and emotion questionnaire (see appendix a) these questions were adapted from previous studies on cognitive appraisals @xcite . in total, the forward reasoning task comprises 432 vignettes. for the backward reasoning task, we assessed the ability of llms to infer the outcomes and cognitive appraisals when provided with a vignette that explicitly states only the target agent's emotion. this task utilized the same prompts as the forward reasoning task, but with the outcome and appraisals omitted (see appendix b). each prompt was systematically manipulated by varying the six possible emotions experienced by the target agent. after being presented with the scenario, llms were required to answer a series of questions regarding the outcome chosen by the agent and the agent's appraisal of the situation. in total, the backward reasoning task comprises 150 evaluation prompts. we used the curated vignettes and questions as input prompts (see appendix f for dataset statistics) for four state-of-the-art llms, namely mistral 7b (mistral-7b-instruct-v0.3) (jiang et al., 2023), llama 3.1 (llama-3.1-8b-instruct) @xcite , gemma 7b (gemma-7b-instruct) @xcite , and openai o3 (o3-mini). the models then answer the questions in a zero-shot manner. default parameters are used for all the models during inference. ## discussion and conclusion the current work builds on prior research examining tom in llms with a focus on emotional inference, and extends psychological studies that elucidate the human emotion inference process @xcite to the domain of text-based emotion inference in llms. overall, our findings suggest that while llms can reason to some degree, they struggle to accurately associate situational outcomes and appraisals with specific emotions. the results of both the forward and backward reasoning tasks suggest that llms predominantly adopt system 1-like strategies. in the forward reasoning task, for instance, anger ratings were consistently elevated in scenarios with perceived unfairness, regardless of outcome-a heuristic shortcut that bypasses the nuanced integration of contextual factors indicative of system 2 engagement. the backward reasoning results further underscore this system 1 dominance, as llms frequently conflated distinct but similar emotions, such as anger and disappointment, which require subtle distinctions based on appraisal dimensions like accountability and goal conduciveness. this difficulty suggests that the models possess limited capacity to emulate system 2-like processes, which entail abstract, theory-informed reasoning grounded in event appraisals and a nuanced understanding of social context @xcite . such processes require the integration of multiple contextual cues to generate a coherent and holistic interpretation of a target's emotional state, rather than relying solely on a single modality or superficial features. taken together, these findings indicate that llms' emotional reasoning predominantly reflects heuristic-driven, pattern-matching processes consistent with system 1, with limited evidence of system 2-like context integration. future research may benefit from designing training protocols that encourage more abstract, rule-based inference mechanisms to better approximate the cognitive pathways underlying human emotional reasoning.