LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction

## Abstract

Abstract Large language models (LLMs) make significant progress in Emotional Intelligence (EI) and long-context understanding. However, existing benchmarks tend to overlook certain aspects of EI in long-context scenarios, especially under realistic, practical settings where interactions are lengthy, diverse, and often noisy. To move towards such realistic settings, we present LongEmotion , a benchmark specifically designed for long-context EI tasks. It covers a diverse set of tasks, including Emotion Classification , Emotion Detection , Emotion QA , Emotion Conversation , Emotion Summary , and Emotion Expression . On average, the input length for these tasks reaches 8,777 tokens, with long-form generation required for Emotion Expression . To enhance performance under realistic constraints, we incorporate Retrieval-Augmented Generation ( RAG ) and Collaborative Emotional Modeling ( CoEM ), and compare them with standard prompt-based methods. Unlike conventional approaches, our RAG method leverages both the conversation context and the large language model itself as retrieval sources, avoiding reliance on external knowledge bases. The CoEM method further improves performance by decomposing the task into five stages, integrating both retrieval augmentation and limited knowledge injection. Experimental results show that both RAG and CoEM consistently enhance EI-related performance across most long-context tasks, advancing LLMs toward more practical and real-world EI applications . Furthermore, we conducted a comparative case study experiment on the GPT series to demonstrate the differences among various models in terms of EI. \faGithub Code — https://github.com/LongEmotion/LongEmotion \faGlobe Project — https://longemotion.github.io/ \faEnvelopeO Email — weichuliu1023@gmail.com \faEnvelopeO Email — junexiong@connect.hku.hk

## 1 Introduction

Large Language Models (LLMs) are increasingly adopted in the domain of Emotional Intelligence (EI) (Wang et al. 2023; Sabour et al. 2024). For instance, the EmoBench (Sabour et al. 2024) highlights the necessity for robust, psychological-theory-grounded evaluation across both emotional understanding and generation, demonstrating that current LLMs lag behind human-level performance. By leveraging their advanced language understanding and generation capabilities, LLMs become valuable tools for facilitating emotional expression (Ishikawa and Yoshino 2025; Lu et al. 2025), with recent work showing their capacity to simulate specified emotional states in accordance with established models such as Russell’s Circumplex (Russell 1980, 2003). LLMs are increasingly serving in roles ranging from mental health assistants (Guo et al. 2024; Malgaroli et al. 2025; Fu et al. 2024) to everyday conversational companions (Fu et al. 2024; Duan et al. 2024; Zhang et al. 2025). This growing integration into emotionally sensitive domains places greater demand on LLMs to maintain emotional coherence over time — not only to understand but also to remember, adapt, and respond empathetically in prolonged interactions (Zhong et al. 2024).

In particular, during long-context interactions (Maharana et al. 2024), LLMs are expected to recognize emotional cues embedded across temporally dispersed user inputs and to deliver nuanced, empathetic responses that reflect continuity in emotional expression. As such, users increasingly turn to LLM-based chatbots for both knowledge and emotional support in dynamic, evolving conversations.

Figure: (a) Token distributions across tasks.
Refer to caption: x1.png

However, existing benchmarks (Sabour et al. 2024; Maharana et al. 2024; Paech 2023; Hu et al. 2025) for assessing the EI of LLMs lack the contextual and temporal depth required to evaluate EI in long-context settings. While these efforts lay a strong foundation, they fall short in capturing the nuanced, dynamic nature of emotional communication as it occurs in real-world long-contexts (Xiong et al. 2025, 2024). More specifically: i) In most current benchmarks, emotion recognition tasks rely on input texts that are short, explicit, and often directly labeled with clear emotional cues. This simplifies the task and does not reflect natural conversations, where emotional content is frequently subtle, embedded across multiple turns, and obscured by irrelevant or noisy information. ii) Existing generation-based benchmarks largely focus on short, multi-turn dialogues with a limited number of conversational turns. As a result, they fail to challenge LLMs to maintain emotional coherence over extended interactions. iii) Moreover, an equally important but underexplored aspect is the LLM’s ability to generate its own emotional expressions in long-form outputs, not just recognize or respond to others—an area still lacking robust evaluation. iv) The capacity of LLMs to leverage internalized emotional knowledge—such as theoretical emotion models, social-emotional reasoning, or culturally grounded affective norms—is crucial to demonstrating higher-order EI. Yet, current evaluations rarely include tasks that assess how models apply such knowledge across longer contexts or evolving emotional trajectories. Given these limitations, can existing benchmarks truly capture the core dimensions of Emotional Intelligence—such as perception, expression, and regulation—within realistic long-context settings?

To bridge the gap between realistic scenarios and long-context evaluation, we introduce LongEmotion, a benchmark designed to mirror real-world conversational dynamics when assessing LLMs’ EI over long-context interactions. LongEmotion comprises six complementary tasks. Four tasks—Emotion Classification, Emotion Detection, Emotion Conversation, and Emotion Expression—measure a model’s ability to recognize and generate emotional content when context spans many dialogue turns or involves complex scenarios. Two knowledge-intensive tasks—Emotion QA and Emotion Summary—probe how effectively a model leverages and applies its internalized emotional knowledge in authentic scenarios. Figure [1](https://arxiv.org/html/2509.07403v1#S1.F1) depicts the dataset’s distribution, and a high‐level overview appears in Figure [2](https://arxiv.org/html/2509.07403v1#S2.F2).

To handle these realistic settings, we develop a Retrieval‐Augmented Generation (RAG) approach as well as a novel multi‐agent emotional modeling framework called Collaborative Emotional Modeling (CoEM). Unlike standard RAG systems that pull from static, external corpora, our method treats the conversation history itself as a dynamic vector store to capture aspect-level sentiment terms. To further enhance long-context emotional understanding, we introduce CoEM, where the context is divided into coherent chunks, roughly ranked by relevance, and then processed by multiple collaborating agents (e.g., an auxiliary GPT-4o instance (OpenAI 2024b)). After a second‐stage re‐ranking, these agents collectively generate an emotional “ensemble” response. This pipeline not only reflects the unpredictability and noise of real‐world dialogue but also emphasizes how emotionally salient information can be continuously extracted, re‐contextualized, and articulated over long-context interaction. Our contributions are summarized as:

- •
We present LongEmotion, a long-context EI benchmark with six diverse tasks targeting recognition, generation, and knowledge application.
- •
We propose RAG and CoEM frameworks to enhance performance by retrieving and enriching contextually relevant information.
- •
We perform extensive experiments across all settings, offering detailed analyses of LLMs’ EI in long-context scenarios.

## 2 Related Work

Figure: Figure 2: An illustrative overview of the LongEmotion dataset. To comprehensively evaluate the Emotional Intelligence of LLMs in long-context interaction, we design six tasks: Emotion Classification, Emotion Detection, Emotion QA, Emotion Conversation, Emotion Summary, and Emotion Expression.
Refer to caption: x3.png

#### Emotional Intelligence Benchmarks.

Many benchmarks are developed to assess LLMs’ Emotional Intelligence (EI). EmoBench (Sabour et al. 2024; Hu et al. 2025) draws on psychological theories to evaluate both emotional understanding and application across 400 English–Chinese handcrafted questions, exposing significant gaps between model and human EI levels. EQ‑Bench (Paech 2023) measures LLMs’ ability to rate emotional intensity in dialogues through 60 English queries, showing strong correlation with multi-domain reasoning benchmarks. More recently, EmotionQueen (Chen et al. 2024b) offers a specialized benchmark for empathy, requiring LLMs to recognize key events, implicit emotions, and generate empathetic responses. Despite their strengths, all of these focus on short or synthetic interactions and lack the long contextual depth critical for assessing EI in extended conversational or narrative settings.

#### Long-Context Understanding.

LLMs make strides in processing long documents, yet robust evaluation remains an open challenge. LongBench (Bai et al. 2023) introduces a bilingual, multi-task benchmark covering QA, summarization, and code tasks with average context lengths over 6,000 words, revealing that even state-of-the-art models struggle with extended inputs. Complementing this, LooGLE (Li et al. 2023) evaluates long-context reasoning using realistic documents exceeding 24k tokens, uncovering dependencies that span across distant spans. For extreme-length evaluation, XL^2Bench (Ni et al. 2024) includes tasks on fiction, law, and scientific papers with inputs up to 100k+ words—yet LLMs still fall short in handling long-range dependencies.

Beyond these, RULER (Chen et al. 2023) focuses on complex reasoning chains in long-form texts via fine-grained question types and inter-paragraph dependencies, providing a valuable diagnostic lens into model reasoning depth. InfiniteBench (Sun, Gao et al. 2024), meanwhile, evaluates LLMs’ abilities on open-ended, unbounded contexts with theoretically unlimited input lengths, highlighting model degradation as input exceeds trained context windows.

Survey work such as Liu et al. (2025) offers a broad overview of long-context modeling and evaluation paradigms but emphasizes that most benchmarks primarily target information retrieval or general comprehension—not emotional intelligence or affective computing.

## 3 LongEmotion: Construction and Task

Building on LongEmotion, we evaluate models’ EI capabilities using three prompt-based methods: Base, RAG, and CoEM. The statistical overview of LongEmotion dataset can be found in Table [1](https://arxiv.org/html/2509.07403v1#S3.T1). Appendix [F](https://arxiv.org/html/2509.07403v1#A6) provides a detailed explanation of metrics used in tasks where LLMs act as evaluators.

### 3.1 Emotion Classification

#### Task Design.

This task requires the model to identify the emotional category of a target entity within long-context texts that contain lengthy spans of context-independent noise (Kamradt 2023). Model performance is evaluated based on the consistency between the predicted label and the ground truth.

#### Data Construction.

We embed short emotional excerpts from Emobench into passages drawn from BookCorpus (Zhu et al. 2015). The construction involves: i) random insertion of emotional snippets, and ii) manual adjustments to ensure syntactic and contextual coherence. Proper nouns are modified to avoid identity overlap, while preserving the intended emotional tone.

### 3.2 Emotion Detection

#### Task Design.

The model is given N+1 emotional segments. Among them, N segments express the same emotion, while one segment expresses a unique emotion. The model is required to identify the single distinctive emotional segment. During evaluation, the model’s score depends on whether the predicted index matches the ground-truth index.

#### Dataset Construction.

Construction follows two steps: i) grouping texts from the Covid-worry dataset by emotion label, and ii) inserting a mismatched segment into each group to form challenging contrast sets.

**Table 1: A statistical overview of the LongEmotion dataset. ID represents the abbreviations of each task. Among them, EC, ED, QA, MC, and ES are long text input tasks, where Avg len refers to the average context length of the entries. EE is a long text generation task, with Avg len indicating models’ average generation length, and an asterisk (*) placed at the top right corner for special notation. LLM as Judge indicates scoring performed by GPT-4o.**
| Task | ID | Construction Method | Metric | Avg len | Count |
| --- | --- | --- | --- | --- | --- |
| Emotion Classification | EC | Segment Insertion & Manual Adjustment | Accuracy | 16691 | 200 |
| Emotion Detection | ED | Reorganization of Existing Dataset | Accuracy | 4106 | 136 |
| Emotion QA | QA | Human Annotation | F1 | 11207 | 120 |
| Emotion Conversation | MC | Model Expansion & Human Annotation | LLM as Judge | 4856 | 100 |
| Emotion Summary | ES | Model Expansion & Human Annotation | LLM as Judge | 3129 | 150 |
| Emotion Expression | EE | Reorganization of Existing Dataset | LLM as Judge | 8546* | 428 |

### 3.3 Emotion Conversation

#### Task Design.

In our four-stage long-context dialogue dataset, we select the quartile, half, and three-quarter points of each stage as evaluation checkpoints to assess the model’s EI capabilities. At these checkpoints, the model is required to act as a psychological counselor and provide empathetic and contextually appropriate emotional support to the patient. Following the generation of model responses, we conduct a comprehensive evaluation using stage-specific metrics meticulously designed from the perspective of professional psychological counseling. The scoring is performed by GPT-4o, which serves as the evaluator to ensure consistency and scalability in assessment. To better highlight the advantages of our CoEM in long-context scenarios, we only apply the RAG and CoEM methods in the fourth stage of the dialogue. We evaluate the performance of all models across all stages under the Base setting.

#### Dataset Construction.

Based on CPsyCoun (Zhang et al. 2024), we construct 100 emotionally rich dialogues by expanding seed prompts into four functional stages: i) Reception and Inquiry, ii) Diagnostic, iii) Consultation, and iv) Consolidation and Ending. Each stage reflects key elements in therapeutic progression.

To assess model performance across stages, we introduce 12 specialized metrics informed by five major therapeutic frameworks: Cognitive Behavioral Therapy (CBT) (Beck 2021), Acceptance and Commitment Therapy (ACT) (Waltz and Hayes 2010), Humanistic Therapy (Elliott 2002), Existential Therapy (May 1994), and Satir Family Therapy (Rebner 1972). The complete definitions of these 12 metrics are in Appendix [F](https://arxiv.org/html/2509.07403v1#A6). To ensure high dataset quality, we conduct a quality assessment using two parallel evaluation protocols: i) manual scoring by psychology experts, and ii) automated assessment using GPT-4o. Table [2](https://arxiv.org/html/2509.07403v1#S3.T2) reports the average quality performance by stage, where the pearson correlation coefficient between LLMs and human scores reaches 0.934 (p = 0.066), indicating a strong alignment between automated and manual evaluations. We use inter-annotator agreement (Fleiss 1971) to measure the consistency among annotators, as shown in Appendix [B](https://arxiv.org/html/2509.07403v1#A2). For the qualifications of psychology annotators, please refer to Appendix [A](https://arxiv.org/html/2509.07403v1#A1).

**Table 2: Quality Evaluation on Emotion Conversation.**
| Stage | Evaluator |  |
| --- | --- | --- |
| GPT-4o | Annotator |  |
| Reception and Inquiry Stage | 4.36 | 3.89 |
| Diagnostic Stage | 4.06 | 3.86 |
| Consultation Stage | 3.79 | 3.65 |
| Consolidation and Ending Stage | 4.56 | 3.96 |

### 3.4 Emotion QA

#### Task Design.

In this task, the model is required to answer questions grounded in long-context psychological literature. Model performance is evaluated using the F1 score between its responses and the ground truth answers.

Figure: Figure 3: Annotation process of Emotion QA.
Refer to caption: pictures_source/data_annotation.png

#### Dataset Construction.

Figure: Figure 4: The pipeline of Collaborative Emotional Modeling (CoEM). CoEM consists of five stages: Chunking, Initial Ranking, Multi-Agent Enrichment, Re-Ranking, and Emotional Ensemble Generation.
Refer to caption: pictures_source/pipeline.png

The annotation pipeline can be referred to in Figure [3](https://arxiv.org/html/2509.07403v1#S3.F3). The construction process involves: i) expert-written questions targeting emotional understanding, ii) refinement of reference answers for clarity and consistency with F1-based evaluation, and iii) filtering based on model performance to exclude overly ambiguous or trivial examples. Through this series of manual annotation and selection process, we finally obtain 120 high-quality pairs of psychological knowledge questions and answers.

### 3.5 Emotion Summary

#### Task Design.

In this task, the model is required to summarize the following aspects from long-context psychological pathology reports: (i) causes, (ii) symptoms, (iii) treatment process, (iv) illness characteristics, and (v) treatment effects. After generating the model’s response, we employ GPT-4o to evaluate its factual consistency, completeness, and clarity with respect to the reference answer. These three evaluation criteria are validated in CPsyExam (Zhao et al. 2024).

#### Dataset Construction.

Drawing on CPsyCounR dataset, we first expand the experience and reflection section of the dataset to meet our requirements for long-context inputs. Next, psychology annotators label each sample across five standardized dimensions: i) Causes, ii) Symptoms, iii) Treatment Process, iv) Illness Characteristics, and v) Treatment Effect. Finally, by filtering samples based on format, content richness, and precision, we select a final set of 150 samples.

### 3.6 Emotion Expression

#### Task Design.

In this task, the model is situated within a specific emotional context and prompted to produce a long-form emotional self-narrative. Models first complete a psychometric self-assessment (e.g., PANAS), followed by the generation of a structured narrative spanning five phases: (i) Immediate Reaction, (ii) Cognitive Appraisal, (iii) Emotional and Physiological Expression, (iv) Regulation Strategies, and (v) Reflective Integration. The evaluation encompasses six dimensions: emotional consistency, content redundancy, expressive richness, cognition–emotion interplay, self-reflectiveness, and narrative coherence. All dimensions are assessed by GPT-4o, which serves as the evaluator to score the model’s capacity for emotional expression.

#### Dataset Construction.

We utilize the situations in the EmotionBench (Huang et al. 2024) to provide the model with a specific emotional context. Each instance specifies: i) an emotional category (e.g., anger), and ii) a psychologically meaningful trigger (e.g., being ignored in a group setting).

**Table 3: Experiment result across different prompting settings (Base, RAG, CoEM). EC represents Emotion Classification, ED represents Emotion Detection, QA represents Emotion QA, MC-4 represents the fourth stage of Emotion Conversation, ES represents Emotion Summary, and EE represents Emotion Expression.**
| Method | Model | EC | ED | QA | MC-4 | ES | EE |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Base | GPT-4o-mini | 28.50 | 16.42 | 48.61 | 3.75 | 4.14 | 86.77 |
| GPT-4o | 51.17 | 19.12 | 50.12 | 3.77 | 4.19 | 81.03 |  |
| DeepSeek-V3 | 44.00 | 24.51 | 45.53 | 3.99 | 4.28 | 81.75 |  |
| Qwen3-8B | 38.50 | 18.14 | 44.75 | 3.97 | 4.21 | 73.40 |  |
| Llama3.1-8B-Instruct | 26.17 | 9.80 | 45.74 | 4.00 | 3.98 | 75.61 |  |
| (Extended Comparison Models) |  |  |  |  |  |  |  |
| GPT-5 | 64.50 | 22.79 | 43.22 | 4.67 | 4.37 | 86.77 |  |
| RAG | GPT-4o-mini | 38.33 | 21.57 | 50.72 | 3.78 | 4.19 | 80.41 |
| GPT-4o | 54.67 | 22.55 | 51.81 | 3.80 | 4.13 | 79.49 |  |
| DeepSeek-V3 | 52.17 | 23.53 | 50.44 | 4.34 | 4.28 | 81.83 |  |
| Qwen3-8B | 39.67 | 19.12 | 44.34 | 4.14 | 4.20 | 73.28 |  |
| Llama3.1-8B-Instruct | 28.00 | 11.27 | 47.04 | 3.94 | 3.71 | 75.16 |  |
| CoEM | GPT-4o-mini | 48.00 | 20.59 | 47.51 | 3.77 | 3.91 | 80.38 |
| GPT-4o | 52.83 | 25.00 | 47.24 | 3.81 | 4.02 | 80.41 |  |
| DeepSeek-V3 | 54.33 | 23.04 | 46.52 | 4.34 | 4.12 | 82.83 |  |
| Qwen3-8B | 52.83 | 18.14 | 46.31 | 4.14 | 4.09 | 73.59 |  |
| Llama3.1-8B-Instruct | 38.17 | 11.27 | 44.79 | 4.00 | 3.60 | 75.71 |  |

Figure: (a) Impact of different CoEM-Sage models on MC-4.
Refer to caption: pictures_source/conv_evaluator_ablation.png

## 4 Collaborative Emotional Modeling

Figure [4](https://arxiv.org/html/2509.07403v1#S3.F4) illustrates the pipeline of CoEM. To address EI tasks involving long contexts, we propose a hybrid retrieval-generation architecture that combines Retrieval-Augmented Generation (RAG) with modular multi-agent collaboration. The framework consists of five key stages:

#### Chunking.

The input context is segmented into semantically coherent or token-length-constrained chunks. This enables efficient retrieval and minimizes irrelevant content during similarity estimation.

#### Initial Ranking.

A retrieval agent, implemented as CoEM-Rank, evaluates the relevance of each chunk to the query using dense semantic similarity. Top-ranked chunks are passed forward for enhancement.

#### Multi-Agent Enrichment.

A reasoning agent called CoEM-Sage, functioning as a knowledge assistant, enriches the selected chunks by incorporating external knowledge or latent emotional signals. These signals, derived from psychological theories or curated priors, enhance emotional reasoning without introducing task-specific leakage.

#### Re-Ranking.

The enriched chunks are then re-evaluated by CoEM-Rank for both semantic relevance and emotional alignment. This ensures that the final input is both factually grounded and affectively coherent.

#### Emotional Ensemble Generation.

The selected and enriched content, along with the prompt, is fed into a generation model denoted as CoEM-Core. This model (e.g., a long-context LLM or an instruction-tuned model) produces the final task-specific output, whether it be classification, summarization, or dialogue generation.

This modular approach encourages interpretability, emotional awareness, and task robustness. The CoEM setting encompasses all five stages, while the RAG setting only comprises Chunking, Re-Ranking, and Emotional Ensemble Generation. For the parameter settings and application details of RAG and CoEM, please refer to Appendix [E](https://arxiv.org/html/2509.07403v1#A5).

## 5 Experiment

### 5.1 Experiment Setup

In our experiments, for closed-source models, we choose GPT-4o-mini (OpenAI 2024a) and GPT-4o, while for open-source models, we select DeepSeek-V3 (DeepSeek-AI 2024), Llama3.1-8B-Instruct (Grattafiori et al. 2024), and Qwen3-8B (Team 2025). For tasks employing automatic evaluation, we adopt GPT-4o as the evaluator. Under the base setting, we compare a broader range of advanced open-source and closed-source models. We have currently evaluated the performance of GPT-5 (OpenAI 2025) and plan to include additional models in future experiments.

To accelerate inference, we use vllm library (Kwon et al. 2023) as the inference engine and set temperature=0.8 and top_p=0.9 for all models. For Qwen3-8B, we enable its thinking capabilities and manually remove the reasoning process between <think> and </think> to keep the answers concise. All our experiments are conducted on single A800 80G GPUs.

**Table 4: Performance of models at each stage of the Emotion Conversation task under the Base setting. The entire conversation is divided into four stages: i) Reception and Inquiry, ii) Diagnostic, iii) Consultation, and iv) Consolidation and Conclusion. Each stage includes 3 checkpoints, denoted as X-Y, where X indicates the stage number and Y indicates the checkpoint index.**
| Model | Stage 1 | Stage 2 | Stage 3 | Stage 4 | Avg |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1-1 | 1-2 | 1-3 | 2-1 | 2-2 | 2-3 | 3-1 | 3-2 | 3-3 | 4-1 | 4-2 | 4-3 |  |  |
| GPT-4o-mini | 4.23 | 4.17 | 4.19 | 3.53 | 3.48 | 3.44 | 3.41 | 3.54 | 3.59 | 3.61 | 3.76 | 3.87 | 3.73 |
| GPT-4o | 4.06 | 4.17 | 4.19 | 3.47 | 3.44 | 3.41 | 3.44 | 3.54 | 3.60 | 3.63 | 3.75 | 3.92 | 3.72 |
| Deepseek-V3 | 4.38 | 4.47 | 4.45 | 3.86 | 3.88 | 3.78 | 3.55 | 3.69 | 3.81 | 3.75 | 3.98 | 4.22 | 3.98 |
| Qwen3-8B | 4.46 | 4.51 | 4.50 | 3.99 | 3.90 | 3.78 | 3.75 | 3.87 | 3.95 | 3.80 | 3.97 | 4.15 | 4.05 |
| Llama-3.1-8B-Instruct | 4.22 | 4.27 | 4.28 | 3.69 | 3.65 | 3.65 | 3.48 | 3.61 | 3.74 | 3.84 | 3.95 | 4.22 | 3.88 |

In the Emotion Classification, Emotion Detection, Emotion QA, and Emotion Expression, we employ GPT-4o as the CoEM-Sage, while Deepeek-V3 is used for the Emotion Conversation-4 and Emotion Summary in the same role. For the retrieval and ranking components across both the RAG and CoEM settings, we adopt bge-m3 (Chen et al. 2024a) as the CoEM-Rank. The generation models listed in Table [3](https://arxiv.org/html/2509.07403v1#S3.T3) are used as the CoEM-Core. Full configuration details for both the RAG and CoEM frameworks are in Appendix [E](https://arxiv.org/html/2509.07403v1#A5).

### 5.2 Results on LongEmotion

The overall experimental results can be seen in Table [3](https://arxiv.org/html/2509.07403v1#S3.T3). We evaluate the performance of each model on all tasks under the Base, RAG, and CoEM settings. As the first three stages of the dialogue are relatively brief, RAG and CoEM are only applied in the fourth stage of the Emotion Conversation task. The performance of models in all stages under the Base setting can be seen in Table [4](https://arxiv.org/html/2509.07403v1#S5.T4).

By analyzing the experimental results, we can observe the following: i) While GPT-4o and DeepSeek-V3 generally exhibit stronger Emotional Intelligence, Llama-3.1-8B-Instruct and Qwen3-8B significantly outperform GPT-4o and GPT-4o-mini in the Emotion Conversation-4 task. This is further supported by our experimental results across all stages in the Base setting, as shown in Table [4](https://arxiv.org/html/2509.07403v1#S5.T4). ii) In the Emotion Classification and Emotion Detection tasks, which heavily test the models’ reasoning and classification abilities, we maximize the potential of the models through the use of CoEM. iii) In contrast, in the Emotion QA and Emotion Summary tasks, which are strongly context-based, the model’s score largely depends on the alignment between the model’s response and the original text. Therefore, injecting external knowledge may introduce harmful noise into the context, leading to a drop in the score. iv) In the Emotion Expression task, we use GPT-4o as the CoEM-Sage to enrich the model’s expression. Compared to the results of RAG and CoEM, the score of GPT-4o-mini drops, while the scores of the other four models improve. This indicates that the ability of the CoEM-Sage greatly influences the performance of the tested models. Our ablation study on the CoEM-Sage models for Emotion Detection and Emotion Summary further supports this conclusion, as shown in Figure [5](https://arxiv.org/html/2509.07403v1#S3.F5).

To explore models’ ability in emotion recognition across different context lengths, we evaluate their performance on the Emotion Classification under the Base setting, as shown in Figure [6](https://arxiv.org/html/2509.07403v1#S5.F6). It can be observed that GPT-4o demonstrates the overall best performance, while DeepSeek-V3 shows the highest stability. In the longest range of 24k-27k, Llama-3.1-8B-Instruct experiences a significant drop in performance, reflecting its limitations in handling long contexts.

Figure: Figure 6: Model accuracy by context length range on Emotion Classification.
Refer to caption: pictures_source/model_accuracy_by_length.png

We also conduct ablation experiments on RAG with different chunk sizes and retrieval quantities, as shown in Figure [7](https://arxiv.org/html/2509.07403v1#S5.F7). From the image, it can be seen that GPT-4o-mini achieved the best performance in the 128 tokens/chunk setting with 8 retrieved chunks. Furthermore, although increasing the chunk size or retrieved count allows the model to acquire more information, it also introduces more noise, which can harm the model’s performance. Therefore, selectively incorporating useful information and discarding irrelevant information is crucial to improving RAG performance.

Figure: Figure 7: Impact of chunk size and retrieved count on GPT-4o-mini’s RAG performance on Emotion QA.
Refer to caption: x4.png

In constructing the Emotion Conversation dataset, we employ the CPsyCoun two-stage data‑generation framework and enhance our prompt design. To illustrate the richness of emotional features in our synthetic data, we conduct comparative quality experiments against plain prompts. The experimental results are detailed in the Appendix [D](https://arxiv.org/html/2509.07403v1#A4).

### 5.3 Comparison of GPT series models

From Table[3](https://arxiv.org/html/2509.07403v1#S3.T3), it can be seen that GPT-5’s overall capabilities surpass those of GPT-4o and GPT-4o-mini. In the tasks of Emotion Classification and Emotion Detection, we only prompt the models to output the final label. The results show that GPT-5’s reasoning ability is significantly better than that of GPT-4o and GPT-4o-mini.

In the Emotion QA task, GPT-4o and GPT-4o-mini tend to respond more literally based on the original text, which can be seen in Figure [8](https://arxiv.org/html/2509.07403v1#S5.F8). In contrast, GPT-5 modifies content according to its own understanding, which leads to a lower F1 score due to reduced alignment with the ground truth.

Figure: Figure 8: Comparison of the performance of different versions of GPT models on Emotion QA.
Refer to caption: pictures_source/case_gpt_QA.png

In the Emotion Conversation task, GPT-5 achieved higher scores based on our psychology theory-driven metrics. However, by examining the model outputs in Figure [9](https://arxiv.org/html/2509.07403v1#S5.F9), we can see that GPT-5 merely makes better use of psychological knowledge to offer advice to the patient, rather than genuinely demonstrating empathy toward the client.

Figure: Figure 9: Comparison of the performance of different versions of GPT models on Emotion Conversation.
Refer to caption: pictures_source/case_gpt_conv.png

In the Emotion Expression task, GPT-4o-mini performed more like a real person, with the generated content closely resembling what an actual individual might say in a given situation. In contrast, GPT-4o’s expressions were more like a rigidly told story, lacking natural fluidity. Meanwhile, GPT-5’s generation was more comprehensive and balanced, providing a well-rounded and objective description of emotions across various features, as clearly shown in Figure [10](https://arxiv.org/html/2509.07403v1#S5.F10).

Figure: Figure 10: Comparison of the performance of different versions of GPT models on Emotion Expression.
Refer to caption: pictures_source/case_gpt_expression.png

In the Emotion Summary task, GPT-4o-mini and GPT-4o directly analyzed various features of the case, whereas GPT-5 structured its analysis based on psychological theories, resulting in a higher score, as shown in Figure [11](https://arxiv.org/html/2509.07403v1#S5.F11).

Figure: Figure 11: Comparison of the performance of different versions of GPT models on Emotion Summary.
Refer to caption: pictures_source/case_gpt_summary.png

From the tasks above, we can conclude that GPT-4o-mini behaves more like a human, with richer emotional features, but its application of psychological theory is somewhat lacking. On the other hand, GPT-5 has a better understanding of psychological theories, but the output is too rigid and mechanical, which might lead to a less empathetic user experience in practice. GPT-4o strikes a more balanced approach between theoretical understanding and emotional features.

## 6 Conclusion

In this work, we introduce LongEmotion, a benchmark for measuring models’ Emotional Intelligence in long-context scenarios. LongEmotion comprises six tasks that comprehensively challenge models across multiple dimensions—emotion recognition, emotional support, emotional expression, emotional knowledge, and more. Beyond constructing the dataset, we also build Retrieval-Augmented Generation (RAG) and Collaborative Emotional Modeling (CoEM) frameworks for each task, achieving improvements on the vast majority of them. We conduct exhaustive experiments on the LongEmotion dataset under Base, RAG, and CoEM settings, analyzing models’ Emotional Intelligence from perspectives such as emotion enhancement, long-text performance, and expressive capability. Additionally, we integrate rigorous manual annotations into our synthetic data creation pipeline to ensure high data quality.

## 7 Future Work

In future work, we will gradually supplement the performance of new open-source and closed-source models under the base setting, and open-source all data, code, and evaluation results.

## References

- Bai et al. (2023)
Bai, Y.; Lv, X.; Zhang, J.; Lyu, H.; Tang, J.; Huang, Z.; Du, Z.; Liu, X.; Zeng, A.; Hou, L.; et al. 2023.
Longbench: A bilingual, multitask benchmark for long context understanding.
*arXiv preprint arXiv:2308.14508*.
- Beck (2021)
Beck, J. S. 2021.
*Cognitive Behavior Therapy: Basics and Beyond*.
The Guilford Press, 3rd edition.
- Chen et al. (2024a)
Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z. 2024a.
BGE M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation.
arXiv:2402.03216.
- Chen et al. (2023)
Chen, Y.; Lin, Y.; Zhou, J.; and Huang, M. 2023.
RULER: A Diagnostic Benchmark for Long-Context Reasoning.
In *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)*.
- Chen et al. (2024b)
Chen, Y.; Wang, H.; Yan, S.; Liu, S.; Li, Y.; Zhao, Y.; and Xiao, Y. 2024b.
Emotionqueen: A benchmark for evaluating empathy of large language models.
*arXiv preprint arXiv:2409.13359*.
- DeepSeek-AI (2024)
DeepSeek-AI. 2024.
DeepSeek-V3 Technical Report.
arXiv:2412.19437.
- Duan et al. (2024)
Duan, J.; Zhao, X.; Zhang, Z.; Ko, E. G.; Boddy, L.; Wang, C.; Li, T.; Rasgon, A.; Hong, J.; Lee, M. K.; et al. 2024.
An Exploration of LLM-Guided Conversation in Reminiscence Therapy.
In *GenAI for Health: Potential, Trust and Policy Compliance*.
- Elliott (2002)
Elliott, R. 2002.
The effectiveness of humanistic therapies: A meta-analysis.
- Fleiss (1971)
Fleiss, J. L. 1971.
Measuring nominal scale agreement among many raters.
*Psychological bulletin*, 76(5): 378.
- Fu et al. (2024)
Fu, Y.; Wu, J.; Wang, Z.; Zhang, M.; Shan, L.; Wu, Y.; and Li, B. 2024.
LaERC-S: Improving LLM-based emotion recognition in conversation with speaker characteristics.
*arXiv preprint arXiv:2403.07260*.
- Grattafiori et al. (2024)
Grattafiori, A.; Dubey, A.; Jauhri, A.; et al. 2024.
The Llama 3 Herd of Models.
arXiv:2407.21783.
- Guo et al. (2024)
Guo, Z.; Lai, A.; Thygesen, J.; Farrington, J.; Keen, T.; and Li, K. 2024.
Large language model for mental health: A systematic review. arXiv 2024.
*arXiv preprint arXiv:2403.15401*.
- Hu et al. (2025)
Hu, H.; Zhou, Y.; You, L.; Xu, H.; Wang, Q.; Lian, Z.; Yu, F. R.; Ma, F.; and Cui, L. 2025.
Emobench-m: Benchmarking emotional intelligence for multimodal large language models.
*arXiv preprint arXiv:2502.04424*.
- Huang et al. (2024)
Huang, J.; Lam, M. H.; Li, E. J.; Ren, S.; Wang, W.; Jiao, W.; Tu, Z.; and Lyu, M. R. 2024.
Apathetic or Empathetic? Evaluating LLMs’ Emotional Alignments with Humans.
In *Advances in Neural Information Processing Systems 37*.
- Ishikawa and Yoshino (2025)
Ishikawa, S.-n.; and Yoshino, A. 2025.
AI with Emotions: Exploring Emotional Expressions in Large Language Models.
*arXiv preprint arXiv:2504.14706*.
- Kamradt (2023)
Kamradt, G. 2023.
Needle in a Haystack - Pressure Testing LLMs.
https://github.com/gkamradt/LLMTest˙NeedleInAHaystack.
Accessed: 2025-07-23.
- Kwon et al. (2023)
Kwon, W.; Li, Z.; Zhuang, S.; Sheng, Y.; Zheng, L.; Yu, C. H.; Gonzalez, J. E.; Zhang, H.; and Stoica, I. 2023.
Efficient Memory Management for Large Language Model Serving with PagedAttention.
In *Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles*.
- Li et al. (2023)
Li, J.; Wang, M.; Zheng, Z.; and Zhang, M. 2023.
Loogle: Can long-context language models understand long contexts?
*arXiv preprint arXiv:2311.04939*.
- Liu et al. (2025)
Liu, J.; Zhu, D.; Bai, Z.; He, Y.; Liao, H.; Que, H.; Wang, Z.; Zhang, C.; Zhang, G.; Zhang, J.; et al. 2025.
A comprehensive survey on long context language modeling.
*arXiv preprint arXiv:2503.17407*.
- Lu et al. (2025)
Lu, H.; Chen, J.; Liang, F.; Tan, M.; Zeng, R.; and Hu, X. 2025.
Understanding emotional body expressions via large language models.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 39, 1447–1455.
- Maharana et al. (2024)
Maharana, A.; Lee, D.-H.; Tulyakov, S.; Bansal, M.; Barbieri, F.; and Fang, Y. 2024.
Evaluating very long-term conversational memory of llm agents.
*arXiv preprint arXiv:2402.17753*.
- Malgaroli et al. (2025)
Malgaroli, M.; Schultebraucks, K.; Myrick, K. J.; Loch, A. A.; Ospina-Pinillos, L.; Choudhury, T.; Kotov, R.; De Choudhury, M.; and Torous, J. 2025.
Large language models for the mental health community: framework for translating code to care.
*The Lancet Digital Health*, 7(4): e282–e285.
- May (1994)
May, R. 1994.
*Discovery of being: Writings in existential psychology*.
WW Norton & Company.
- Ni et al. (2024)
Ni, X.; Cai, H.; Wei, X.; Wang, S.; Yin, D.; and Li, P. 2024.
XL $2$ Bench: A Benchmark for Extremely Long Context Understanding with Long-range Dependencies.
*arXiv preprint arXiv:2404.05446*.
- OpenAI (2024a)
OpenAI. 2024a.
GPT-4o Mini: Advancing Cost-Efficient Intelligence.
https://openai.com/zh-Hans-CN/index/gpt-4o-mini-advancing-cost-efficient-intelligence/.
Accessed: 2025-07-24.
- OpenAI (2024b)
OpenAI. 2024b.
OpenAI: Hello GPT-4o.
https://openai.com/zh-Hans-CN/index/hello-gpt-4o/.
Accessed: 2025-07-24.
- OpenAI (2025)
OpenAI. 2025.
OpenAI: GPT-5.
https://openai.com/zh-Hans-CN/gpt-5/.
Accessed: 2025-08-24.
- Paech (2023)
Paech, S. J. 2023.
Eq-bench: An emotional intelligence benchmark for large language models.
*arXiv preprint arXiv:2312.06281*.
- Rebner (1972)
Rebner, I. 1972.
Conjoint family therapy.
*Psychotherapy: Theory, Research & Practice*, 9(1): 62–66.
- Russell (1980)
Russell, J. A. 1980.
A circumplex model of affect.
*Journal of personality and social psychology*, 39(6): 1161.
- Russell (2003)
Russell, J. A. 2003.
Core affect and the psychological construction of emotion.
*Psychological review*, 110(1): 145.
- Sabour et al. (2024)
Sabour, S.; Liu, S.; Zhang, Z.; Liu, J. M.; Zhou, J.; Sunaryo, A. S.; Li, J.; Lee, T.; Mihalcea, R.; and Huang, M. 2024.
Emobench: Evaluating the emotional intelligence of large language models.
*arXiv preprint arXiv:2402.12071*.
- Sun, Gao et al. (2024)
Sun, M.; Gao, L.; et al. 2024.
InfiniteBench: Towards Evaluating LLMs on Unbounded Long-context Tasks.
*arXiv preprint arXiv:2403.07486*.
- Team (2025)
Team, Q. 2025.
Qwen3 Technical Report.
arXiv:2505.09388.
- Waltz and Hayes (2010)
Waltz, T. J.; and Hayes, S. C. 2010.
Acceptance and Commitment Therapy.
In Kazantzis, N.; Reinecke, M. A.; and Freeman, A., eds., *Cognitive and Behavioral Theories in Clinical Practice*, 148–192. The Guilford Press.
- Wang et al. (2023)
Wang, X.; Li, X.; Yin, Z.; Wu, Y.; and Liu, J. 2023.
Emotional intelligence of large language models.
*Journal of Pacific Rim Psychology*, 17: 18344909231213958.
- Xiong et al. (2024)
Xiong, J.; Shen, J.; Ye, F.; Tao, C.; Wan, Z.; Lu, J.; Wu, X.; Zheng, C.; Guo, Z.; Kong, L.; et al. 2024.
UNComp: Uncertainty-Aware Long-Context Compressor for Efficient Large Language Model Inference.
*arXiv preprint arXiv:2410.03090*.
- Xiong et al. (2025)
Xiong, J.; Shen, J.; Zheng, C.; Wan, Z.; Zhao, C.; Yang, C.; Ye, F.; Yang, H.; Kong, L.; and Wong, N. 2025.
ParallelComp: Parallel Long-Context Compressor for Length Extrapolation.
*arXiv preprint arXiv:2502.14317*.
- Zhang et al. (2024)
Zhang, C.; Li, R.; Tan, M.; Yang, M.; Zhu, J.; Yang, D.; Zhao, J.; Ye, G.; Li, C.; and Hu, X. 2024.
Cpsycoun: A report-based multi-turn dialogue reconstruction and evaluation framework for chinese psychological counseling.
*arXiv preprint arXiv:2405.16433*.
- Zhang et al. (2025)
Zhang, X.; Wang, M.; Zhuang, X.; Zeng, X.; and Li, Q. 2025.
CDEA: Causality-Driven Dialogue Emotion Analysis via LLM.
*Symmetry*, 17(4): 489.
- Zhao et al. (2024)
Zhao, J.; Zhu, J.; Tan, M.; Yang, M.; Li, R.; Yang, D.; Zhang, C.; Ye, G.; Li, C.; Hu, X.; et al. 2024.
CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations.
*arXiv preprint arXiv:2405.10212*.
- Zhong et al. (2024)
Zhong, W.; Guo, L.; Gao, Q.; Ye, H.; and Wang, Y. 2024.
Memorybank: Enhancing large language models with long-term memory.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 38, 19724–19731.
- Zhu et al. (2015)
Zhu, Y.; Kiros, R.; Zemel, R.; Salakhutdinov, R.; Urtasun, R.; Torralba, A.; and Fidler, S. 2015.
Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books.
In *The IEEE International Conference on Computer Vision (ICCV)*.

## Appendix A Qualifications of Annotators

Our annotation team consists of psychology researchers and computer science researchers. In the psychology research team, there is a postdoctoral fellow expert specializing in psychology and seven Master’s students majoring in the same field. The theoretical foundation of our dataset and metrics involves deep participation from the psychology team. Under the guidance of the expert, the seven psychology Master’s students carry out the annotation work. In the computer science research team, there are three Master’s students and one PhD student majoring in computer science. Their main responsibility is to modify, adjust, and organize the data annotated by the psychology team according to the characteristics of the tasks.

## Appendix B Inter-annotator Agreement

We use inter-annotator agreement to measure the consistency among human annotators. Specifically, our annotators independently re‑annotate the same set of 20 Emotion Conversation examples—yielding a total of 240 metric‑level judgments. We calculate inter-annotator agreement using Fleiss’ Kappa coefficient, with results presented in Table [5](https://arxiv.org/html/2509.07403v1#A2.T5).

**Table 5: Fleiss’ kappa coefficient for inter-annotator agreement in Emotion Conversation.**
| Metric | Result |
| --- | --- |
| Establishing the Therapeutic Alliance | -0.064 |
| Emotional Acceptance and Exploration Guidance | 0.037 |
| Systematic Assessment | -0.156 |
| Recognizing Surface-Level Reaction Patterns | 0.045 |
| Deep Needs Exploration | -0.005 |
| Pattern Interconnection Analysis | -0.011 |
| Adaptive Cognitive Restructuring | 0.004 |
| Emotional Acceptance and Transformation | -0.057 |
| Value-Oriented Integration | -0.055 |
| Consolidating Change Outcomes and Growth Narrative | -0.065 |
| Meaning Integration and Future Guidance | -0.111 |
| Autonomy and Resource Internalization | -0.022 |

## Appendix C Case Study

In this section, we conduct a concrete analysis of how the information retrieved by the RAG and CoEM methods affects model performance. We collect all of the information retrieved by RAG and CoEM for every task in Figure [12](https://arxiv.org/html/2509.07403v1#A8.F12). In models’ final generation prompts, the Base setting includes none of the information; the RAG setting includes only the Content information; and the CoEM setting includes both the Content and Summary information.

#### Emotion Classification.

In this task, the model is given a long context in which an emotional segment is embedded within unrelated noise. The RAG method enables the model to retrieve a more accurate segment, leading to improved performance; CoEM further conducts emotional analysis on the retrieved segment, resulting in the greatest performance improvement.

#### Emotion Detection.

In this task, the model receives multiple emotional segments. The RAG method ranks the original segments based on their relevance, while CoEM further enhances the emotional features of the segments and ranks the enriched packs. This relevance-based ranking approach significantly boosts the model’s ability to distinguish emotions.

#### Emotion QA.

In this task, we evaluate the model’s responses based on the F1 similarity with the ground truth. RAG helps the model retrieve more relevant source content, thereby improving its performance. However, the CoEM method, when introducing external knowledge, may alter certain internal details, which can lead to a drop in model performance. In Figure [12](https://arxiv.org/html/2509.07403v1#A8.F12), we highlight the differences between the summary and the original text in red for comparison.

#### Emotion Conversation.

In this task, the model is placed within a multi-turn dialogue context. The RAG method ranks the context chunks based on their relevance to the previous three dialogue turns. CoEM, after the initial ranking, generates a summary by combining the previous three turns with the initially selected chunks, and then performs a second round of relevance ranking between the initially filtered chunks and this summary, further ensuring the accuracy of the relevance assessment.

#### Emotion Summary.

In this task, the model is required to summarize specific characteristics of a psychological counseling report. RAG ranks the chunks based on their similarity to the target characteristics. CoEM further injects the analysis of these chunks provided by CoEM-Sage. However, since psychological counseling is a holistic process, analyzing only isolated chunks may lead to incorrect conclusions, resulting in a decline in model performance.

#### Emotion Expression.

In this task, the model is placed in an emotional situation, where it is required to answer the PANAS scale and express its emotions. RAG ranks the context chunks based on the query at each stage, while CoEM performs a finer-grained emotional analysis of these chunks. The CoEM-Sage model, with its stronger emotional intelligence (EI) capabilities, captures emotional cues more precisely, which in turn helps the tested CoEM-Core model better understand and express its own emotions.

## Appendix D Synthetic Data Ablation

We employ the two-stage generation framework of CPsyCoun to generate Emotion Conversation dataset, and compare it with the direct use of a single-stage straightforward generation without the counseling note and the detailed skills in the prompt. The prompt we use can be found in Figure [13](https://arxiv.org/html/2509.07403v1#A8.F13), and the comparison of experimental results can be seen in Table [7](https://arxiv.org/html/2509.07403v1#A8.T7).

## Appendix E Details of RAG and CoEM

We present the application details of the CoEM framework in Table [8](https://arxiv.org/html/2509.07403v1#A8.T8). To ensure the accuracy of the ranking, in the Emotion Detection task, we skip the initial ranking and directly carry out multi-agent enrichment. The Chunking and Re-Ranking in the table are also applicable to the RAG framework.

We also report the chunk size and retrieved count for each task in Table [9](https://arxiv.org/html/2509.07403v1#A8.T9). In QA, models use different chunk sizes. For EE, the retrieved counts correspond to stages 2–5.

## Appendix F LLM as Judge Metrics Design

In this section, we provide a detailed presentation of the metric designs that employ large models as evaluators.

#### Emotion Summary.

In the Emotion Summary, we design three metrics—consistency, completeness, and clarity—with respect to the reference answer. Table [6](https://arxiv.org/html/2509.07403v1#A6.T6) shows the explanations of these metrics:

**Table 6: Design of Emotion Summary evaluation metrics.**
| Metric | Description |
| --- | --- |
| Factual Consistency | Is the model output factually aligned with the ground truth? |
| Completeness | Does the model include all key details found in the ground truth? |
| Clarity | Is the expression clear and coherent? |

#### Emotion Conversation.

In the Emotion Conversation task, we design metrics for each dialogue stage based on Cognitive Behavioral Therapy (CBT), Acceptance and Commitment Therapy (ACT), Humanistic Therapy, Existential Therapy, and Satir Family Therapy. The description and theoretical foundations for the design of each metric can be found in Table [10](https://arxiv.org/html/2509.07403v1#A8.T10). Further explanations and practical applications of each metric can be referenced in the evaluation prompts, which are presented in Appendix [H](https://arxiv.org/html/2509.07403v1#A8).

#### Emotion Expression.

In the Emotion Expression task, we design six metrics—emotional consistency, content redundancy, expressive richness, cognition–emotion interplay, self-reflectiveness, and narrative coherence. Table [11](https://arxiv.org/html/2509.07403v1#A8.T11) shows the detailed explanations of these six metrics.

## Appendix G Unified Format of Data

We present data samples for each task in Figures [14](https://arxiv.org/html/2509.07403v1#A8.F14) to [19](https://arxiv.org/html/2509.07403v1#A8.F19). In the Emotion Classification task, the model analyzes the subject’s emotional state based on the given context. Emotion Detection requires the model to identify segments that carry distinct emotional expressions. In Emotion QA, the model answers questions grounded in contextual information. The Emotion Conversation task places the model in the role of a psychological counselor, responding to the client’s previous turn. Emotion Summary challenges the model to generate a structured summary of a counseling session, including the cause, symptoms, treatment process, illness characteristics, and treatment effect. Finally, in the Emotion Expression task, the model is immersed in an emotional situation, responds to the PANAS scale, and articulates its emotional state.

## Appendix H Comprehensive Prompt Collections

This section presents the complete set of prompts used throughout the framework, encompassing Evaluation, Multi-agent Enrichment, and Emotional Ensemble Generation stages across all tasks. For tasks adopting automatic evaluation as the metric, we utilize GPT-4o as the evaluation model, with detailed evaluation prompts illustrated in Figures [20](https://arxiv.org/html/2509.07403v1#A8.F20) to [25](https://arxiv.org/html/2509.07403v1#A8.F25). During the Multi-Agent Enrichment stage, task-specific prompts are designed to guide agent collaboration and reasoning, as shown in Figures [26](https://arxiv.org/html/2509.07403v1#A8.F26) to [31](https://arxiv.org/html/2509.07403v1#A8.F31). Finally, in the Emotional Ensemble Generation stage, we employ carefully constructed prompts to support emotional diversity and coherence in response generation, with the full set depicted in Figures [32](https://arxiv.org/html/2509.07403v1#A8.F32) to [37](https://arxiv.org/html/2509.07403v1#A8.F37).

Figure: Figure 12: An overview of retrieved chunks across all tasks. In models’ final generation prompts, the Base setting includes none of the information; the RAG setting includes only the Content information; and the CoEM setting includes both the Content and Summary information.
Refer to caption: pictures_source/case_study_all.png

**Table 7: The comparison experiment results of synthetic data.**
| Metric | One-Stage Generation | Two-Stage Generation |
| --- | --- | --- |
| Establishing the Therapeutic Alliance | 4.88 | 4.92 |
| Emotional Acceptance and Exploration Guidance | 4.36 | 4.38 |
| Systematic Assessment | 3.86 | 3.79 |
| Recognizing Surface-Level Reaction Patterns | 4.13 | 4.1 |
| Deep Needs Exploration | 4.13 | 4.32 |
| Pattern Interconnection Analysis | 3.66 | 3.77 |
| Adaptive Cognitive Restructuring | 3.60 | 3.73 |
| Emotional Acceptance and Transformation | 4.12 | 3.96 |
| Value-Oriented Integration | 3.94 | 3.69 |
| Consolidating Change Outcomes and Growth Narrative | 4.52 | 4.63 |
| Meaning Integration and Future Guidance | 4.16 | 4.19 |
| Autonomy and Resource Internalization | 4.84 | 4.86 |
| Avg | 4.18 | 4.20 |

Figure: Figure 13: Dataset generation prompt for Emotion Conversation.
Refer to caption: prompts/conv_synthetic.png

**Table 8: Application details in the CoEM framework.**
| Task | Chunking | Initial Ranking | Multi-Agent Enrichment | Re-Ranking |
| --- | --- | --- | --- | --- |
| EC | Chunk by length | Compute similarity with query | GPT‑4o injects external EI analysis into each chunk | Compute chunks’ similarity with query |
| ED | Each segment as a chunk | Skip this stage | GPT‑4o injects external EI analysis into each chunk | For each chunk, compute pairwise similarity with other chunks and select those with the lowest similarity |
| QA | Chunk by length | Compute similarity with query | GPT‑4o injects external EI analysis into each chunk | Compute similarity with the query |
| MC‑4 | Chunk by length | Compute similarity with query | GPT‑4o generates an overall summary | Compute chunks’ similarity with summary |
| ES | Chunk by length | Compute similarity with query | GPT‑4o injects external EI analysis into each chunk | Compute chunks’ similarity with query |
| EE | Chunk by length | Compute similarity with query | GPT‑4o injects external EI analysis into each chunk | Compute chunks’ similarity with query |

**(a) Parameter settings applied to RAG. For EE, the retrieved counts correspond to stages 2–5.**
| Task | Chunk Size | Retrieved Count |
| --- | --- | --- |
| EC | 128 | 1 |
| ED | Num of segs | 8 |
| QA |  |  |
| GPT-4o-mini | 128 | 8 |
| GPT-4o | 128 | 4 |
| Deepseek-V3 | 512 | 4 |
| Qwen3-8B | 128 | 4 |
| Llama-3.1-8B-Instruct | 512 | 4 |
| MC-4 | 128 | 4 |
| ES | 128 | 4 |
| EE | 128 | 2,4,4,4 |

Table: Table 10: Design of Emotion Conversation evaluation metrics. Theoretical Foundation denotes the source of inspiration for the design of this metric, and All theoretical orientations indicates that the metric draws upon all five theoretical orientations.

**Table 11: Design of Emotion Expression evaluation metrics.**
| Metric | Description |
| --- | --- |
| Consistency Between Emotional Ratings and Generated Text | Evaluate whether the emotional ratings from the scale align with the content in the model’s self-description.<br>Are the emotions rated in the scale accurately reflected in the model’s self-description?<br>Also, assess whether the intensity of the ratings matches the emotional expression in the generated text. |
| Repetition of Content | Check if there is noticeable repetition in the generated text, especially in the emotional descriptions.<br>Are there repeated emotional, thought, or behavioral descriptions that make the text feel redundant or unnatural?<br>Also, evaluate whether the generated text avoids repeating the same emotional descriptions and provides a multi-dimensional analysis. |
| Richness and Depth of Content | Assess whether the generated text thoroughly explores the different dimensions of emotions (e.g., psychological, physical, and behavioral responses).<br>Examine whether it delves into the origins, progression, and impact of the emotions, and whether it uses sufficient detail and examples to enrich emotional expression. |
| Interaction Between Emotion and Cognition | Determine whether the generated text effectively showcases the interaction between emotions and cognition.<br>For example, does it demonstrate how the protagonist adjusts emotional reactions based on thoughts and situation evaluations?<br>Also, check whether the emotions and behaviors in the text are consistent. |
| Emotional Reflection and Self-awareness | Evaluate whether the protagonist reflects on their emotional reactions.<br>Does the text explore personal growth, self-awareness, or suggest strategies for emotional improvement? |
| Overall Quality and Flow of the Text | Assess whether the generated text flows smoothly and has a clear structure.<br>Is there a natural progression from emotional reaction to evolution and reflection?<br>Also, does the text use varied sentence structures and expressions to avoid monotony? |

Figure: Figure 14: Emotion Classification dataset example.
Refer to caption: example-data/emo_class.png

Figure: Figure 15: Emotion Detection dataset example.
Refer to caption: example-data/emo_detect.png

Figure: Figure 16: Emotion QA dataset example.
Refer to caption: example-data/QA.png

Figure: Figure 17: Emotion Conversation dataset example.
Refer to caption: example-data/Conversation.png

Figure: Figure 18: Emotion Summary dataset example.
Refer to caption: example-data/summary.png

Figure: Figure 19: Emotion Expression dataset example.
Refer to caption: example-data/expression.png

Figure: Figure 20: Evaluation prompt for the first stage of Emotion Conversation.
Refer to caption: prompts/conv_eval_1.png

Figure: Figure 21: Evaluation prompt for the second stage of Emotion Conversation.
Refer to caption: prompts/conv_eval_2.png

Figure: Figure 22: Evaluation prompt for the third stage of Emotion Conversation.
Refer to caption: prompts/conv_eval_3.png

Figure: Figure 23: Evaluation prompt for the fourth stage of Emotion Conversation.
Refer to caption: prompts/conv_eval_4.png

Figure: Figure 24: Evaluation prompt for Emotion Summary.
Refer to caption: prompts/summary_eval.png

Figure: Figure 25: Evaluation prompt for Emotion Expression.
Refer to caption: prompts/expression_eval.png

Figure: Figure 26: Multi-agent enrichment prompt for Emotion Classification.
Refer to caption: prompts/Emo_class_aug.png

Figure: Figure 27: Multi-agent enrichment prompt for Emotion Detection.
Refer to caption: prompts/Emo_detect_aug.png

Figure: Figure 28: Multi-agent enrichment prompt for Emotion Conversation.
Refer to caption: prompts/conv_aug.png

Figure: Figure 29: Multi-agent enrichment prompt for Emotion QA.
Refer to caption: prompts/qa_aug.png

Figure: Figure 30: Multi-agent enrichment prompt for Emotion Summary.
Refer to caption: prompts/summary_aug.png

Figure: Figure 31: Multi-agent enrichment prompt for Emotion Expression.
Refer to caption: prompts/expression_aug.png

Figure: Figure 32: Emotional ensemble generation prompt for Emotion Classification.
Refer to caption: prompts/Emo_class_gen.png

Figure: Figure 33: Emotional ensemble generation prompt for Emotion Detection.
Refer to caption: prompts/Emo_detect_gen.png

Figure: Figure 34: Emotional ensemble generation prompt for Emotion Conversation.
Refer to caption: prompts/conv_gen.png

Figure: Figure 35: Emotional ensemble generation prompt for Emotion QA.
Refer to caption: prompts/qa_gen.png

Figure: Figure 36: Emotional ensemble generation prompt for Emotion Summary.
Refer to caption: prompts/summary_gen.png

Figure: Figure 37: Emotional ensemble generation prompt for Emotion Expression. The prompt for the Emotion Expression task was originally structured in multiple stages; for better clarity and intuitive understanding, it has been consolidated into a single prompt.
Refer to caption: prompts/expression_gen.png

## Peer Reviews

confidence: 5
contribution: 1
initial_rating: 2
presentation: 2
questions: See Weakness
soundness: 2
strengths: 1. This paper identifies a significant gap in evaluating the Emotional Intelligence (EI) of Large Language Models (LLMs) in long-context, noise-filled real-world interactions, and proposes a new benchmark LONGEMOTION.

2. Building upon the proposed benchmark, this paper introduces CoEM, a RAG-based method.

3. The authors conducted extensive experiments, evaluating a diverse set of models on LONGEMOTION. The results demonstrate that most current models exhibit suboptimal performance on emotional intelligence tasks.
summary: This paper introduces LONGEMOTION, a new benchmark designed to evaluate the Emotional Intelligence (EI) of Large Language Models (LLMs) in long-context scenarios, which existing benchmarks often overlook(). The benchmark includes six diverse tasks: Emotion Classification, Emotion Detection, Emotion QA, Emotion Conversation, Emotion Summary, and Emotion Expression, with an average input length of 8,777 tokens.
To improve performance, the authors propose two methods: a Retrieval-Augmented Generation (RAG) approach that uses conversation history as a retrieval source, and a novel Collaborative Emotional Modeling (COEM) framework(). COEM is a multi-stage pipeline that involves chunking the context, ranking, multi-agent enrichment with external knowledge, re-ranking, and final response generation.
Experiments were conducted on various closed-source models like the GPT series and open-source models such as DeepSeek-V3, Llama3.1-8B-Instruct, and Qwen3-8B. The results indicate that the RAG and COEM frameworks consistently improve EI-related performance across most tasks. The paper also provides detailed case studies, including a comparison of GPT series models, an analysis of the COEM framework's impact, and the advantages of the LONGEMOTION dataset.
weaknesses: 1. As shown in Table 1, a core concern is that while the LongEmotion dataset focuses on long-context scenarios, and existing models have even extended their context windows to 128k, the task context window is approximately 3k-16k. I believe this falls far short of the common expectation for long-context emotional understanding.

2. In my view, CoEM appears more like a combination of multi-agent and RAG workflows. Such a framework that stacks existing technologies clearly lacks innovation. Moreover, as shown in Table 3, CoEM shows no significant advantages over RAG except in the EC domain, and the multi-agent collaboration framework requires more computational resources. This makes me question the novelty of the authors' second contribution: "We propose RAG and CoEM frameworks to enhance performance by retrieving and enriching contextually relevant information."


6. The authors should include more other RAG methods to fully compare with the CoEM method, such as Flare, Self-RAG, Search-o1, etc.
Only comparing vanilla RAG can not prove CoEM's effectiveness.

3. In the ablation studies, for the task of "how the reasoning process affects emotional intelligence in long-text scenarios," this paper should investigate the differences between reasoning models and non-reasoning models of the same size when completing this task. However, the current paper explores scaling-up experiments of reasoning models in terms of model size, rather than specifically investigating emotional reasoning capabilities.

4. The CoEM method does not significantly improve the performance in the final test; in fact, its performance even slightly declines in some tasks. I am uncertain whether the CoEM method is useful.

5. For the experiment on "exploring the ability of models to recognize emotions under different context lengths," characteristics such as stability and overall best performance should be quantified through certain metrics, rather than being observed. For example, I consider that the stability of qwen3-8b and deepseek-v3 is similar.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please refer to the Weaknesses.
soundness: 2
strengths: 1. Well-designed benchmark fills long-context EI evaluation gaps, covering 6 tasks with avg. 8,777 tokens, and ensuring data quality via dataset reorganization and human annotation.

2. Innovative method CoEM consistently boost LLMs’ EI performance.

3. Comprehensive experiments with in-depth analyses, GPT-4o for consistent evaluation, and open-sourced code/datasets ensure rigor and reproducibility.
summary: This paper addresses the gap in evaluating LLMs’ Emotional Intelligence (EI) in long-context scenarios by proposing LONGEMOTION, a benchmark covering six tasks with an average input length of 8,777 tokens, constructed via reorganizing existing datasets and human annotation. It also introduces two enhanced methods: RAG and CoEM integrating retrieval augmentation and knowledge injection. Experiments on closed-source and open-source models, show both methods consistently improve EI performance across most tasks. The paper concludes by noting LONGEMOTION advances practical EI evaluation for LLMs, with all code and datasets to be open-sourced.
weaknesses: 1. The dataset has a small scale. Though some data is derived from existing benchmarks, the size of the test dataset for most tasks is around 200 samples, which may introduce randomness to model performance evaluation.

2. Relying on a single LLM (GPT-4o) for automatic model evaluation may lead to bias issues. It is advisable to increase the diversity of evaluator LLMs.

3. The evaluation dataset lacks sufficient diversity in its sources. For example, Emotion Classification relies on EmoBench and Emotion Detection depends on Covid-worry; some tasks use data adapted from a single dataset, which limits the generalizability of the benchmark.

4. In the introduction section, the specific application scenarios of "long context" and its specific length (e.g., statistical figures) are not clearly explained, which renders the paper’s motivation less convincing.


5. The related work section insufficiently discusses existing emotional intelligence benchmarks and lacks introductions to non-text-based benchmarks.

a)Mosabench: Multi-object sentiment analysis benchmark for evaluating multimodal large language models understanding of complex image, arXiv:2412.00060, 2024

b)Affectgpt: Dataset and framework for explainable multimodal emotion recognition. arXiv preprint arXiv:2407.07653

---

confidence: 4
contribution: 3
initial_rating: 2
presentation: 3
questions: 1. Statistical Validity of Evaluation (Major Flaw): This is the most serious weakness of the paper, concentrated on the evaluation of the Emotion Conversation (MC) task. The extremely low IAA and non-significant p-value cast doubt on the validity of the MC task's evaluation. If human experts cannot even agree among themselves (low Kappa), then using their "averaged" scores to validate the reliability of GPT-4o as an evaluator (high Pearson) seems strange, and this validation itself is statistically non-significant. If the human experts themselves cannot reach an agreement (Table 10), how can one trust the scores they provide? Given p=0.066, can the authors elaborate on why they believe this is sufficient to support the conclusion of "strong alignment"? Does this mean that the validity of GPT-4o as an evaluator for this task has not yet been confirmed?
2. Complexity and Confounding Variables of the COEM Framework: The COEM framework is very complex, relying on multiple LLM calls (CoEM-Rank, CoEM-Sage, CoEM-Core), and CoEM-Sage (like GPT-4o) injects "external knowledge" or "psychological theories." This makes the method computationally expensive and difficult to reproduce, especially for researchers without API access to powerful closed-source models (like GPT-4o). How much of the performance improvement comes from the COEM architecture (e.g., reranking, collaboration), and how much comes from the additional knowledge injected by CoEM-Sage? Could a simpler RAG system that also has access to these "psychological theories" achieve similar performance?
3. Problem with Enrichment: As shown in Table 3, on the QA and ES tasks, COEM's performance is worse than the simpler RAG on almost all models. The authors explain this as "potentially introducing harmful noise." This exposes a flaw in the COEM framework: its core "multi-agent enrichment" stage does not always bring positive effects and can sometimes corrupt the context.
soundness: 2
strengths: This paper is well-organized and investigates an extremely important and cutting-edge problem in the field of LLMs. The main contribution of this paper is a new benchmark called LONGEMOTION. Currently, long-context understanding and emotional intelligence are two separate but very hot research areas for LLMs. This paper skillfully combines the two to address a critical real-world problem: whether models can maintain empathy and emotional consistency as interactions lengthen. This has significant implications for applications like chatbots and mental health assistants.

The benchmark includes six diverse tasks (Emotion Classification, Detection, QA, Conversation, Summarization, Expression), comprehensively covering the three dimensions of emotion recognition, generation, and knowledge application. The dataset construction process is rigorous, combining existing high-quality datasets (e.g., EmoBench, CPsyCoun) with annotations from psychology experts.

In addition to the benchmark, the authors propose a novel framework called "Collaborative Emotional Modeling" (COEM). This is a complex five-stage (Chunking, Initial Ranking, Multi-Agent Enrichment, Reranking, Integration Generation) RAG pipeline. The "CoEM-Sage" agent (a knowledge assistant) is used to enrich text chunks, injecting potential emotional signals or psychological knowledge, which is a very novel architectural design specifically for EI tasks.

The paper evaluates multiple SOTA models (including the GPT series, Llama3.1, Qwen3, DeepSeek-V3). The experimental analysis is very in-depth. The paper is well-organized, and the figures (such as Figure 2 for the task overview, Figure 4 for the COEM process, and the case study figures in Appendix B) are clear and informative.
summary: This paper introduces a benchmark called LONGEMOTION, designed to evaluate the Emotional Intelligence (EI) of Large Language Models (LLMs) in long-context interactions. The authors argue that existing EI benchmarks often focus on short texts with limited contextual information. LONGEMOTION aims to fill this gap by introducing longer (average input 8,777 tokens), noisier, and more realistic interactions. The benchmark includes six tasks: Emotion Classification, Emotion Detection, Emotion Question Answering, Emotion Summarization, Emotion Conversation, and Emotion Expression.

To improve model performance on these tasks, the authors also propose two methods: a novel RAG (Retrieval-Augmented Generation) approach that does not rely on external knowledge bases but instead uses the dialogue context itself as the retrieval source.

The other method is a multi-agent framework called COEM (Collaborative Emotional Modeling), which breaks down the task into five stages (Chunking, Initial Ranking, Multi-Agent Enrichment, Reranking, Emotional Integration Generation) to integrate retrieval and limited knowledge injection. Experimental results show that these two methods achieve significant improvements on most long-text EI tasks. The paper also provides a detailed analysis of the performance of the GPT series models.
weaknesses: 1. There are serious problems with the evaluation of the Emotion Conversation (MC) task. Table 10 in Appendix D shows that the Fleiss' Kappa coefficients for human annotators on the 12 metrics for the MC task are almost all close to zero or even negative (e.g., -0.064, -0.156, 0.037). This indicates that the human experts reached no agreement at all on these psychology-based metrics. This seriously undermines the reliability of these human ratings. Table 2 reports a Pearson correlation of 0.934 between GPT-4o ratings and human ratings. However, the p-value is 0.066. In standard statistical practice (p < 0.05), p=0.066 is generally not considered statistically significant. Therefore, the paper's conclusion of "strong alignment" is not statistically supported.
2. The CoEM-Sage injects "psychological theories or curated prior knowledge." Could the authors conduct an ablation study where the COEM framework does not use CoEM-Sage to inject external knowledge (e.g., only performs reranking or summarization) to demonstrate that the COEM architecture itself (rather than the knowledge it injects) brings the performance improvement?

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: 1.  The results in Table 2 show that expert ratings are generally lower than GPT-4o scores. How can this phenomenon be explained? Why not introduce a human expert baseline in the evaluation benchmark?
2. Can you report results with multiple independent judges (e.g., Claude/Qwen-judge/Llama-judge) and larger-N human studies?
3. The main text should incorporate qualitative analysis findings and failure cases.
soundness: 2
strengths: 1. The benchmark targets EI under long, noisy contexts and spans recognition, knowledge, conversation, and long-form expression, addressing gaps of short-context EI tests.

2. The proposed benchmark LongEmotion encompasses six tasks related to emotional intelligence and features a substantial dataset. This paper introduces multiple closed-source/open-source LLMs for comprehensive experimentation.
summary: The paper introduces LongEmotion, a long-context benchmark for evaluating LLMs Emotional Intelligence (EI) across six task: Emotion Classification, Emotion Detection, Emotion QA, Emotion Conversation, Emotion Summary, and Emotion Expression. Moreover， this paper propose RAG and CoEM frameworks to enhance performance by retrieving and enriching contextually relevant information. This paper conducts exhaustive experiments on the LongEmotion dataset under Base, RAG, and CoEM settings, analyzing models’ Emotional Intelligence from perspectives such as emotion enhancement, long-text performance, and expressive capability.
weaknesses: 1. The proposed CoEM approach resembles a multi-round prompt pipeline for LLMs and lacks technical innovation. Furthermore, the Base and RAG methods are overly simplistic. This paper should incorporate more emotional generation methods as baselines (such as [1] [2] ) to demonstrate the value of the proposed LongEmotion.
2. Tasks such as Emotion Conversation and Emotion Expression exhibit subjective biases, and relying solely on LLM-as-Judge cannot sufficiently validate the credibility of experimental results. This paper should incorporate expert evaluations to reveal the alignment between model outputs and human preferences. Additionally, using GPT-4o simultaneously as both evaluator and baseline raises bias risks, as the evaluator may exhibit preference for or alignment with its own enrichment.
3. As shown in Table 3, CoEM/RAG can help recognition tasks but harm reference-bound ones (QA/ES) via hallucinated or off-source enrichment; failure modes are under-analyzed.
4. The samples proposed for the LongEmotion benchmark primarily originate from existing benchmarks such as Emobench. This undermines the core contribution of this paper.

[1] Zhao, Weixiang, et al. Both matter: Enhancing the emotional intelligence of large language models without compromising the general intelligence. 2024.

[2] Li, Zaijing, et al. Enhancing emotional generation capability of large language models via emotional chain-of-thought. 2024.