“Going to a trap house” conveys more fear than “Going to a mall”: Benchmarking Emotion Context Sensitivity for LLM s (2025)

Abstract: Emotion context sensitivity—the ability to adjust emotional responses based on contexts—is a core component of human emotional intelligence. For example, being told, “You can come with me if you want,” may elicit joy if the destination is a mall, but provoke fear if the destination is a trap house. As large language models (LLMs) are increasingly deployed in socially interactive settings, understanding this human ability becomes crucial for generating context-appropriate, emotion-aware responses. In this work, we introduce Trace, a novel benchmark for evaluating whether LLMs can understand emotion context sensitivity of humans. This benchmark consists of 1,626 social scenarios and comprises two complementary tests: a sensitivity test, which measures whether models can detect emotional shifts caused by context changes, and a robustness test, which evaluates whether models can maintain stable emotion predictions when context changes are emotionally irrelevant. Each scenario pair keeps the core event constant while systematically varying contextual details—time, place, or agent—based on insights from behavioral theory and emotion psychology. Experimental results show that even the best-performing LLMs lag behind human performance by 20% in the sensitivity test and 15% in the robustness test, indicating substantial room for improvement in emotion-aware reasoning.

## introduction emotion context sensitivity-the capacity for an individual to shift emotional responses according to contextual changes-is a foundational component of human emotional intelligence @xcite . emotional responses here encompass both the internal experience and external expression of emotion. this capacity enables adaptive behavior by eliciting emotions that are appropriate to contextual demands. for example, being told, "you can come with me if you want," may elicit joy if the destination is a mall, but provoke fear if the destination is a trap house. such a fear emotion helps humans avoid potential danger or prepare for self-protection @xcite . in other words, even when the core event (e.g., being offered to accompany someone) remains the same, emotional responses can differ depending on the context. with the rapid progress of large language mod-els (llms), ai systems have been increasingly applied to tasks that involve social interaction with humans @xcite @xcite @xcite . consequently, it has become significantly important for these systems to understand human emotion context sensitivity in order to generate context-appropriate, emotion-aware responses. despite the growing importance of this ability, no existing benchmark has been specifically designed to evaluate how well llms understand contextdependent emotional responses. prior benchmarks @xcite @xcite have primarily focused on general emotion recognition without explicitly disentangling the influence of contextual variations on emotional responses (see figure 1 ). as a first step toward understanding human emotion context sensitivity, we introduce trace-a novel benchmark designed to assess whether llms can accurately trace how emotions shift of humans in response to contextual changes, a core component of emotional intelligence. to comprehensively evaluate this capacity, trace comprises two complementary components: a sensitivity test and a robustness test. the sensitivity test assesses whether llms can correctly capture shifts in a character's emotion in response to changes in the surrounding context. in contrast, the robustness test evaluates whether models can maintain emotion predictions consistently when context changes are irrelevant to the character's emotional experience. together, the two components provide a more comprehensive assessment of models' understanding of contextsensitive emotional responses, particularly those pertaining to emotional experience-our primary focus in this study. specifically, each example in both tests consists of a pair of scenarios in which the core event remains constant, while specific contextual details are varied. building on prior work in behavioral theory and emotion research @xcite , we manipulate three key context dimensions-time, place, and agent-that have been identified as central in shaping human behavior, cognition, and emotional responses. consequently, the sensitivity test includes scenario pairs where the contextual variation is intended to elicit different target emotions, whereas the robustness test includes pairs where the variation is designed to leave the target emotion unchanged. we evaluate a range of open-and closed-source llms on trace. results show that even the latest models fall significantly short of human performance in understanding emotion context sensitivity. specifically, the best-performing model lags behind human accuracy by approximately 20% in the sensitivity test and 15% in the robustness test. to summarize, our contributions include: our work is closely related to the prior works that have assessed the emotional understanding capability of models by constructing dedicated benchmarks. for example, dailydialog @xcite , emotionlines @xcite , and meld @xcite are dialogue datasets proposed to test the ability of models to recognize emotions in conversations. goemotions @xcite is a dataset covering a broader range of emotions, constructed to evaluate how well the models understand subtler differences between emotions. emobench @xcite contains qas that require llms to predict emotions based on thorough reasoning rather than relying on frequent or explicit patterns. in addition, seceu @xcite and eq-bench (paech, 2023) focus on assessing emotional intelligence through compound emotion reasoning and scalar estimation, respectively. in contrast, benchmarks such as socialiqa @xcite and @xcite evaluate models' social commonsense reasoning, where emotional inference arises as a secondary component within broader social contexts. in contrast to prior benchmarks, our work is the first to explicitly assess whether llms understand emotion context sensitivity-that is, how subtle contextual shifts can lead to meaningful emotional ## experiment in this section, we evaluate recent llms on the trace benchmark and thoroughly investigate their limitations. ## conclusion we present the first systematic study to assess the extent to which llms understand emotion context sensitivity-a key component of human emotional intelligence. grounded in behavioral theory and emotion psychology, we define time, place, and agent as three core contextual dimensions, and use them to construct trace-a controlled benchmark that isolates each type of context change to assess whether llms can track emotional responses. our analysis on trace results in three key findings. first, current llms exhibit a large performance gap relative to humans in understanding emotion context sensitivity. second, even with advanced prompting techniques, models show limited improvement, suggesting an inherent limitation in processing how contextual cues shape emotional responses. third, llms struggle to account for individual differences in emotional experience, as evidenced by their pronounced performance gap in agent-modified contexts. we hope that trace serves as a challenging benchmark for future research in the emotional intelligence of ai and facilitates the development of models capable of contextaware emotional reasoning.