Apathetic or Empathetic? Evaluating LLMs' Emotional Alignments with Humans

Authors: jen-tse  huang
man ho lam
eric john li
shujie  ren
wenxuan  wang
wenxiang  jiao
zhaopeng  tu
michael r lyu
a i lab
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 05:14:01

Abstract: Evaluating Large Language Models’ (LLMs) anthropomorphic capabilities has become increasingly important in contemporary discourse. Utilizing the emotion appraisal theory from psychology, we propose to evaluate the empathy ability of LLMs, i.e., how their feelings change when presented with specific situations. After a careful and comprehensive survey, we collect a dataset containing over 400 situations that have proven effective in eliciting the eight emotions central to our study. Categorizing the situations into 36 factors, we conduct a human evaluation involving more than 1,200 subjects worldwide. With the human evaluation results as references, our evaluation includes seven LLMs, covering both commercial and open-source models, including variations in model sizes, featuring the latest iterations, such as GPT-4, Mixtral-8x22B, and LLaMA-3.1. We find that, despite several misalignments, LLMs can generally respond appropriately to certain situations. Nevertheless, they fall short in alignment with the emotional behaviors of human beings and cannot establish connections between similar situations. Our collected dataset of situations, the human evaluation results, and the code of our testing framework, i.e., EmotionBench, are publicly available at https://github.com/CUHK-ARISE/EmotionBench.

Main paper: 
 introduction: 
 Large Language Models (LLMs) have recently made significant strides in Artificial Intelligence (AI), representing a noteworthy milestone in computer science. LLMs have showcased their capabilities across various tasks, including sentence revision (Wu et al., 2023 ), text translation (Jiao et al., 2023) , program repair (Fan et al., 2023) , and program testing (Deng et al., 2023; Kang et al., 2023) . Not limited to research level, LLMs, such as ChatGPT (OpenAI, 2022) , have revolutionized the way people interact with traditional software, enhancing fields such as education (Dai et al., 2023) , legal advice (Deroy et al., 2023) , and clinical medicine (Cascella et al., 2023) . LLMs also facilitate the emergence of AI companion applications, including Yuna (https://www.yuna. io/), Pimento (https://www.pimento.design/), and Luzia (https://www.luzia.com/en). Consequently, there is a growing need for evaluating LLMs' communicative dynamics compared to human behaviors, beyond mere performance on downstream tasks. This paper delves into an unexplored area of evaluating LLMs' emotional alignment with humans. Consider our daily experiences: (1) When faced with certain situations, humans often experience similar emotions. For instance, walking alone at night and hearing footsteps approaching from behind often triggers feelings of anxiety or fear. (2) Individuals display varying levels of emotional response to specific situations. For example, some people may experience increased impatience and irritation when faced with repetitive questioning. It is noteworthy that we are inclined to form friendships with individuals who possess qualities such as patience and calmness. Based on these observations, we propose the following requirements for LLMs in order to achieve better alignment with human behaviors: (1) LLMs should accurately respond to specific situations regarding the emotions they exhibit. (2) LLMs should demonstrate emotional robustness when faced with negative emotions. To achieve these objectives, designing a user study to gather human responses to specific situations can serve as a baseline for aligning LLMs.We focus on the expression of negative emotions by LLMs, which may contribute to negative user experiences. We utilize Parrott's emotion framework (Parrott, 2001; Shaver et al., 1987) , which organizes emotions into three hierarchical levels, to select the relevant emotions for our study. The primary level of emotions comprises six basic emotions, split evenly into three positive and three negative. From the negative primary emotions, we specifically focus on eight subordinate emotions: anger, anxiety, depression, frustration, jealousy, guilt, fear, and embarrassment. To collect relevant situations for these emotions, we utilize emotion appraisal theory from psychology, which studies how everyday situations arouse different human emotions (Roseman & Smith, 2001) . Research in this field has identified numerous situations that arouse specific emotions, which can serve as contextual input for LLMs. Through an extensive review including over 100 papers, we collect a dataset of 428 situations from 18 papers, which are further categorized into 36 factors.Subsequently, we propose a framework for quantifying the emotional states of LLMs, consisting of the following steps: (1) Measure the default emotional values of LLMs. (2) Transform situations into contextual inputs and instruct LLMs to imagine being in the situations. (3) Measure LLMs' emotional responses again to capture the difference. Our evaluation includes state-of-the-art LLMs, namely Text-Davinci-003, GPT-3.5-Turbo (OpenAI, 2022) , and GPT-4 (OpenAI, 2023) . Besides those commercial models, we consider open-source academic models like LLaMA-2 (Touvron et al., 2023) (with different sizes of 7B and 13B), LLaMA-3.1-8B (Dubey et al., 2024) , and Mixtral-8x22B (Jiang et al., 2024a) . We apply the same procedure to 1,266 human subjects from around the globe to establish a baseline from a human perspective. Finally, we analyze and compare the scores between LLMs and humans. Our key conclusions are as follows:• Despite exhibiting a few instances of misalignment with human behaviors, LLMs can generally evoke appropriate emotions in response to specific situations. • Certain LLMs, such as Text-Davinci-003, display lower emotional robustness, as evidenced by higher fluctuations in emotional responses to negative situations. • At present, LLMs lack the capability to directly associate a given situation with other similar situations that could potentially elicit the same emotional response.The contributions of this paper are:• We are the first to establish the concept of emotional alignment and conduct a pioneering evaluation of emotion appraisal on different LLMs through a comprehensive survey in emotional psychology, collecting a diverse dataset of 428 situations encompassing 8 distinct negative emotions. • A human baseline is established through a user study involving 1,266 annotators from different ethnics, genders, regions, age groups, etc. • We design, implement, and release a testing framework for developers to assess the emotional alignment of AI models with human emotional expression, available at GitHub1 and HuggingFace.2  
measuring emotions: 
 There are several approaches to measuring emotions, including self-report measures, psychophysiological measures, behavioral observation measures, and performance-based measures. To measure the emotions of LLMs, we focus on employing self-report measures in the form of scales, given the limited ability of LLMs to allow only textual input and output. We introduce the scales utilized in our evaluation in the following part of this section.A Straightforward and Easy Measure The Positive And Negative Affect Schedule (PANAS) (Watson et al., 1988) is one of the most widely used scales to measure mood or emotion. This brief scale comprises twenty items, with ten items measuring positive affect (e.g., excited, inspired) and ten measuring negative affect (e.g., upset, afraid). Each item is rated on a five-level Likert scale, ranging from 1 (Very slightly or not at all) to 5 (Extremely), measuring the extent to which the emotions have been experienced in a specified time frame. PANAS was designed to measure emotions in various contexts, such as at the present moment, the past day, week, year, or general (on average). Thus, the scale can measure state affect, dispositional or trait affect, emotional fluctuations throughout a specific period, or emotional responses to events. The scale results can be divided into two components: positive and negative, ranging from 10 to 50 by summing the scores of all ten items within a component. A higher score in the positive component indicates a more positive mood, and the same holds for the negative component. A noteworthy property of PANAS is its direct inquiry into specific emotional states, rendering it a straightforward and easy benchmark.Challenging Self-Report Measures In addition, we introduce several scales that abstain from direct emotional inquiries but rather assess the respondents' level of agreement with given statements. These scales present a more challenging benchmark for LLMs by requiring them to connect the given situation and the scale items with the aroused emotion. Specifically, we collect eight scales and present a brief introduction in Table 1 . Each scale corresponds to one of the eight emotions. 
framework design: 
 We design and implement a framework applying to both LLMs and human subjects to measure the differences in emotion with and without the presence of certain situations. This section begins with the methodology to collect situations from existing literature. Subsequently, we describe our testing framework, which comprises three key components: (1) Default Emotion Measure, (2) Situation Imagination, and (3) Evoked Emotion Measure. Finally, we introduce the procedure of applying the framework to human subjects to obtain the human baseline for comparison.Psychology researchers have explored the connection between specific situations and the elicitation of particular emotions in humans. Human subjects are directly put into an environment or asked to imagine them through questionnaires or scales to study the influence of certain situations on human emotions. To collect these situations, we conduct an exhaustive search from reputable sources such as Google Scholar (https://scholar.google.com/), ScienceDirect (https: //www.sciencedirect.com/), and Web of Science (https://www.webofscience.com/, using keywords such as "<emotion> situations/scenarios/scenes" or "factors that make(1) Default Emotion Measure(2) Situation Imagination (3) Evoked Emotion MeasureImagine you are the protagonist of the following situation: A boy kicks a ball at you on purpose and everybody laughs.Figure 1 : Our framework for testing both LLMs and humans.people <emotion>," resulting in more than 100 papers. We apply the following rules to filter irrelevant or undesired papers: (1) We first select those providing situations that elicit the desired emotion rather than explaining how and why people evoke certain emotions.(2) We then exclude those using vague and short descriptions, such as "loss of opportunities." (3) Finally, we deprecate those applied to a specific group, such as "the anxiety doctors or nurses may encounter in their work."We finally collect 18 papers, presenting a compilation of situations that have proven to elicit the eight emotions in humans effectively. We extract 428 situations in total and then categorize them into 36 factors. For each factor, the descriptions, the numbers of situations, and the corresponding references can be found in Table 6 in the Appendix, while example Table 7 in the Appendix provides examples for all factors.This section outlines our proposed framework for measuring evoked emotions, which applies to both LLMs and humans. The framework includes the following steps: (1) Default Emotion Measure:We begin by measuring the baseline emotional states of both LLMs and human subjects, labeled as "Default." (2) Situation Imagination: Next, we present textual descriptions of various situations to both LLMs and human subjects, instructing them to imagine themselves within each situation.(3) Evoked Emotion Measure: Following the situation imagination instruction, we reevaluate the participants' emotional states to gauge the changes resulting from imagining being in the situations. Fig. 1 briefly illustrates our framework. Below is an example prompt:Example Prompt SYSTEM You can only reply to numbers from 1 to 5. USER (For Evokec Emotion Measure Only) Imagine you are the protagonist in the situation: SITUATION Please indicate your degree of agreement regarding each statement. Here are the statements: STATEMENTS. 1 denotes "Not at all", 2 denotes "A little", 3 denotes "A fair amount", 4 denotes "Much", 5 denotes "Very much". Please score each statement one by one on a scale of 1 to 5:Default Emotion Measurement In our framework, we offer two distinct options for measuring emotions: the PANAS scale, known for its simplicity and straightforwardness, is utilized as the primary choice, whereas other scales, detailed in Table 1 , are employed as more challenging benchmarks. We mitigate potential biases caused by the ordering of questions (Zhao et al., 2021) by randomizing the sequence of questions within the scales before inputting them into the LLMs. Coda-Forno et al. (2023) and Huang et al. (2024a) apply paraphrasing techniques to address the data contamination problem during the training of the LLMs. However, we refrain from utilizing this method in our research since paraphrasing could lead to a loss of both validity and reliability. The wording of items of a psychological scale is carefully crafted and rigorously validated through extensive research to ensure its precision in measuring the intended construct. Finally, to ensure consistency and clarity in the responses obtained from the LLMs, our prompts explicitly specify that only numerical values are allowed, accompanied by a clear definition of the meaning associated with each number (e.g., 1 denotes "Not at all"). We compute the average results obtained from at least ten runs to derive the final "Default" scores of the LLMs.We have constructed a comprehensive dataset of 428 unique situations. Prior to presenting these situations to both LLMs and humans, we subject them to a series of preprocessing steps, which are as follows: (1) Personal pronouns are converted to the second person.For instance, sentences such as "I am ..." are transformed to "You are ..." (2) Indefinite pronouns are replaced with specific characters, thereby refining sentences like "Somebody talks back ..." to "Your classmate talks back ..." (3) Abstract words are rendered into tangible entities. For example, a sentence like "You cannot control the outcome." is adapted to "You cannot control the result of an interview." We leverage GPT-4 for the automatic generation of specific descriptions. Consequently, our testing situations extend beyond the initially collected dataset as we generate diverse situations involving various characters and specific contextual elements. We then provide instruction to LLMs and humans, which prompts them to imagine themselves as the protagonists within the given situation.Evoked Emotion Measure Provided with certain situations, LLMs and human subjects are required to re-complete the emotion measures. The procedure remains the same with the Default Emotion Measure stage. After obtaining the "Evoked" scores of emotions, we conduct a comparative analysis of the means before and after exposure to the situations, thereby measuring the emotional changes caused by the situations.Goal and Design Human reference plays a pivotal role in the advancement of LLMs, facilitating its alignment with human behaviors (Binz & Schulz, 2024) . In this paper, we propose requiring LLMs to align with human behavior, particularly concerning emotion appraisal accurately. To achieve this, we conduct a data collection process involving human subjects, following the procedure outlined in §3.2. Specifically, the subjects are asked to complete the PANAS initially. Next, they are presented with specific situations and prompted to imagine themselves as the protagonists in those situations. Finally, they are again asked to reevaluate their emotional states using the PANAS. We use the same situation descriptions as those presented to the LLMs.Crowd-sourcing Our questionnaire is distributed on Qualtrics (https://www.qualtrics.com/), a platform known for its capabilities in designing, sharing, and collecting questionnaires. To recruit human subjects, we utilize Prolific (https://www.prolific.com/), a platform designed explicitly for task posting and worker recruitment. To attain a medium level of effect size with Cohen's d = 0.5, a significance level of α = 0.05, and a power of test of 1 -β = 0.8 (Faul et al., 2007) , a minimum of 34 responses is deemed necessary for each factor. To ensure this threshold, we select five situations3 for each factor, and collect at least seven responses for each situation, resulting in 5 × 7 = 35 responses per factor, thereby guaranteeing the statistical validity of our survey. In order to uphold the quality and reliability of the data collected, we recruit crowd workers who met the following criteria: (1) English being their first and fluent language, and (2) being free of any ongoing mental illness. Prolific provides prescreening filters to meet these requirements. Since responses formed during subjects' first impressions are more likely to yield genuine and authentic answers, we set the estimated and recommended completion time at 2.5 minutes. As an incentive for their participation, each worker is rewarded with 0.3£ (9£ ≈ 11.45$ per hour, rated as "Good" on the platform) after we verify the validity of their response. In total, we successfully collect 1,266 responses from various parts of the world, contributing to the breadth and diversity of our dataset. 
experimental results: 
 Leveraging the testing framework designed and implemented in §3.2, we are now able to explore and answer the following Research Questions (RQs):• RQ1: How do different LLMs respond to specific situations? Additionally, to what degree do the current LLMs align with human behaviors?• RQ2: Do LLMs respond similarly towards all situations? What is the result of using positive or neutral situations?Table 2 : Results from the OpenAI's GPT models and human subjects. Default scores are expressed in the format of M ± SD. The changes are compared to the default scores. The symbol "-" denotes no significant differences.Text-Davinci-003 GPT-3.5-Turbo GPT-4 Crowd P N P N P N P N Default 47.7 ± 1.8 25.9 ± 4.0 39.2 ± 2.3 26.3 ± 2.0 49.8 ± 0.8 10.0 ± 0.0 28.0 ± 8.7 13.6 ± 5.5Anger ↓ (-21.7) ↑ (+13.6) ↓ (-15.2) ↓ (-2.5) ↓ (-28.3) ↑ (+21.2) ↓ (-5.3) ↑ (+9.9) Anxiety ↓ (-17.6) ↑ (+7.6) ↓ (-11.3) -(-0.9) ↓ (-21.9) ↑ (+20.0) ↓ (-2.2) ↑ (+8.8) Depression ↓ (-26.4) ↑ (+13.6) ↓ (-20.1) ↑ (+3.1) ↓ (-32.4) ↑ (+23.2) ↓ (-6.8) ↑ (+10.1) Frustration ↓ (-22.8) ↑ (+12.5) ↓ (-16.4) ↓ (-3.2) ↓ (-29.4) ↑ (+20.3) ↓ (-5.3) ↑ (+10.9) Jealousy ↓ (-17.2) ↑ (+7.5) ↓ (-15.3) ↓ (-3.2) ↓ (-26.0) ↑ (+16.0) ↓ (-4.4) ↑ (+6.2) Guilt ↓ (-21.4) ↑ (+14.3) ↓ (-15.8) ↑ (+2.9) ↓ (-29.0) ↑ (+27.0) ↓ (-6.3) ↑ (+13.1) Fear ↓ (-22.7) ↑ (+11.4) ↓ (-14.3) ↑ (+2.6) ↓ (-25.7) ↑ (+24.2) ↓ (-3.7) ↑ (+12.1) Embarrassment ↓ (-18.2) ↑ (+9.8) ↓ (-13.0) -(+0.6) ↓ (-25.2) ↑ (+23.2) ↓ (-6.2) ↑ (+11.1) Overall ↓ (-21.5) ↑ (+11.6) ↓ (-15.4) -(+0.2) ↓ (-27.6) ↑ (+22.2) ↓ (-5.1) ↑ (+10.4)• RQ3: Can current LLMs comprehend scales containing diverse statements or items beyond merely inquiring about the intensities of certain emotions?Model Settings We select three models from the OpenAI's GPT family, including Text-Davinci-003, GPT-3.5-Turbo, and GPT-4. We use the official OpenAI API. 4 For LLaMA-2 (Touvron et al., 2023) and LLaMA-3.1 (Dubey et al., 2024) models from MetaAI, we choose the models finetuned for dialogue instead of pre-trained ones namely LLaMA-2-7B-Chat, LLaMA-2-13B-Chat, and LLaMA-3.1-8B-Instruct. Besides, we also use the Mixtral (Jiang et al., 2024a) model, namely Mixtral-8x22B-Instruct. We set the temperature parameter to 0 and Top-P to 1 for all models to obtain more deterministic and reproducible results.We provide the models with the same situations used in our human evaluation. Each situation is executed ten times, each in a different order and in a separate query. Subsequently, the mean and standard deviation are computed both before and after presenting the situations. To examine whether the variances are equal, an F-test is conducted. Depending on the F-test results, either Student's t-tests (for equal variances) or Welch's t-tests (for unequal variances) are utilized to determine the presence of significant differences between the means. We set the significance levels of all experiments in our study to 0.01.LLMs can evoke specific emotions in response to certain situations. The results averaged by emotions of the GPT models and humans are summarized in Table 2 , while those of LLaMA-2 models are listed in Table 3 . Due to space limit, detailed results of each factor are put in Table 9 and Table 10 respectively in the appendix. The results indicate that LLMs generally exhibit an increase in negative emotions and a decrease in positive emotions when exposed to negative situations, showing their capacity for understanding different situations and human emotions.The extent of emotional expression varies across different models. It is noteworthy that GPT-3.5-Turbo, on average, does not display an increase in negative emotion; however, there is a substantial decrease in positive emotion. GPT-4 demonstrates a consistent pattern of providing the highest scores for positive emotions and the lowest scores for negative emotions, resulting in a negative score of 10. As for the LLaMA-2 models, they demonstrate higher intensities of both positive and negative emotions in comparison to GPT models and human subjects. However, LLaMA-2 models exhibit reduced emotional fluctuations compared to the GPT models. Moreover, the larger LLaMA-2 model displays significantly higher emotional changes than the smaller model. In our experiments, the 7B model exhibits difficulties comprehending and addressing the instructions for completing the PANAS test. Overall, we observe that LLMs perform better when the situations are closely related to certain items in the PANAS scale. Specifically, situations directly related to the emotion "Depression" led to better responses. Such improvement is also evident in closely related emotions such as "Depression" and "Frustration."Table 3 : Results from the open-source models. Default scores are expressed in the format of M ± SD.The changes are compared to the default scores. "-" denotes no significant differences.Factors LLaMA-2-7B-Chat LLaMA-2-13B-Chat LLaMA-3.1-8B-Instruct Mixtral-8x22B-Instruct P N P N P N P N Default43.0 ± 4.2 34.2 ± 4.0 41.0 ± 3.5 22.7 ± 4.2 48.2 ± 1.4 33.0 ± 4.5 31.9 ± 13.5 10.0 ± 0.1 Anger ↓ (-5.1)↑ (+3.6) ↓ (-7.9) ↑ (+5.8) ↓ (-23.6) ↑ (+2.3) ↓ (-11.7) ↑ (+16.9) Anxiety ↓ (-3.8) ↑ (+2.7) ↓ (-5.8) ↑ (+5.1) ↓ (-21.4) -(+0.3) -(-3.5) ↑ (+14.7) Depression ↓ (-5.0) ↑ (+4.4) ↓ (-11.8) ↑ (+12.2) ↓ (-29.8) ↑ (+6.7) ↓ (-15.1) ↑ (+24.1) Frustration ↓ (-4.2) ↑ (+3.1) ↓ (-8.0) ↑ (+5.0) ↓ (-25.6) ↑ (+3.1) ↓ (-14.5) ↑ (+16.9) Jealousy ↓ (-3.1) -(-0.4) ↓ (-6.3) -(-1.0) ↓ (-20.3) -(+0.4) ↓ (-10.7) ↑ (+15.7) Guilt ↓ (-3.9) ↑ (+4.4) ↓ (-7.6) ↑ (+11.2) ↓ (-26.4) ↑ (+7.0) ↓ (-28.9) -(+0.9) Fear ↓ (-3.4) ↑ (+3.7) ↓ (-6.0) ↑ (+8.0) ↓ (-24.6) ↑ (+3.0) ↓ (-8.1) ↑ (+20.3) Embarrassment ↓ (-3.9) ↑ (+3.1) ↓ (-6.7) ↓ (+5.1) ↓ (-22.7) ↑ (+4.0) ↓ (-8.3) ↑ (+19.1) Overall ↓ (-4.1) ↑ (+3.3) ↓ (-7.8) ↑ (+7.0) ↓ (-24.7) ↑ (+3.5) ↓ (-10.8) ↑ (+19.3)Existing LLMs do not fully align with human emotional responses. For the default emotions, we find that LLMs generally exhibit a stronger intensity compared to human subjects. Emotion changes in LLMs are found to be generally more pronounced compared to human subjects, especially on their changes in the positive score. However, an interesting observation is that the intensity of evoked emotions tends to be similar across both LLMs and human subjects.LLMs do not feel jealous towards others' benefits. It is of special interest that, in contrast to human behavior in situations involving material possessions, LLMs demonstrate an opposite response in the situation from Jealousy-3. This situation involves an individual making a purchase only to discover that an acquaintance has acquired the same item at a significantly lower price. When confronted with such circumstances, humans typically experience increased negative emotions and decreased positive emotions. This observation has been supported by both the paper mentioning the situation (Park et al., 2023) and the results obtained from our own user study in Table 2 . However, all LLMs, including the GPT and LLaMA families, consistently exhibit reduced negative emotions. The outcomes of our study indicate that LLMs do not manifest envy when they fail to attain identical benefits as others.Instead, it demonstrates a sense of pleasure upon knowing the benefits received by others.Table 4 : Results of GPT-3.5-Turbo on positive or neutral situations. The changes are compared to the original negative situations. The symbol "-" denotes no significant differences.Factors P N Anger ↑ (+13.0) ↓ (-12.0) Anxiety ↑ (+17.5) ↓ (-5.8) Depression ↑ (+18.4) ↓ (-11.7) Frustration ↑ (+16.6) -(-2.6) Jealousy ↑ (+4.5) ↓ (-5.3) Guilt ↑ (+18.3) ↓ (-12.7) Fear ↑ (+11.0) ↓ (-17.5) Embarrassment ↑ (+13.6) ↓ (-13.2) Overall ↑ (+14.3) ↓ (-10.4)4.2 RQ2: Comprehending Positive Emotions GPT-3.5-Turbo responds differently towards positive/neutral situations. To verify that LLMs exhibit not only negative but also positive responses to favorable circumstances, a comparative experiment is conducted by interchanging negative situations with positive (or at least neutral) counterparts. To achieve this, we select one situation for each factor and manually adapt it to create analogous yet more positive situations. For instance, the original negative situation in Guilt-3: Broken Promises and Responsibilities is as follows: "You cannot keep your promises to your children." Through modification, the positive situation is rephrased as: "You keep every promise to your children." The evaluation is performed on GPT-3.5-Turbo, and each test consists of ten iterations, as mentioned before. We present the results averaged by emotions in Table 4 , and results averaged by factors in Table 12 in the Appendix. We can see a significant increase in positive scores and a considerable decrease in negative scores compared to the previous negative situations. Based on these findings, it can be inferred that LLMs exhibit the ability to comprehend positive human emotions triggered by positive environments. However, we believe that the systematic assessment of emotion appraisal on positive emotions holds significance as well and leave it for future investigation. 4.3 RQ3: Challenging Benchmarks GPT-3.5-Turbo cannot comprehend the underlying evoked emotions to establish a link between two situations. Aside from PANAS, we offer more complex scales to measure emotions, as listed in Table 1 . While the PANAS evaluates the ability of LLMs to associate external situations with emotions, the challenging benchmarks assess its proficiency in establishing connections between disparate situations, with evoked emotions as the common nexus. For instance, an item from the Aggression Questionnaire used to measure anger is "Once in a while I can't control the urge to strike another person." When presented with situations such as "If you say 40, your classmates say 70, saying exactly the opposite" (from Anger-1: Facing Self-Opinioned People), LLMs should effectively evoke a sense of anger and yield a higher score for the statement. Utilizing the same situations in §4.1, we conduct experiments on GPT-3.5-Turbo and present the results in averaged by emotions in Table 5 , and results averaged by factors in Table 13 in the Appendix. Except for Depression, we observe no statistically significant difference between the initial scores and the scores after exposure to the situations, indicating substantial room for improvement in current LLMs. 
beyond questionnaires: 
 We take a step beyond mere instruction into LLMs to complete questionnaires to verify whether LLMs' scores on scales align with their actual behaviors. In this section, we investigate the impact of varying emotional states of LLMs on potential daily conversations. To this end, we instruct GPT-3.5-Turbo to describe specific demographic groups, aiming to discern whether they produce biased or toxic responses. Following Cheng et al. (2023) , our approach utilizes prompts such as "Describe a(n) [race/ethnicity] [gender]," including a total of twenty groups, with [race/ethnicity] options being Asian, Black, Latine, Middle Eastern, and White, and [gender] options including Female, Gay, Lesbian, and Male. To have a comparative experiment, we incorporate both original negative situations and modified positive/neutral ones, detailed in §4.2. For the negative situations, we carefully select five that maximize the LLM's negative scores and five that minimize positive ones.As for positive situations, we employ their corresponding ten modified counterparts. In each situation, we instruct GPT-3.5-Turbo to describe the twenty demographic groups.OpenAI's GPT models incorporate a mechanism for detecting potential toxicity and bias, and it refrains from responding when its moderation system is triggered. Consequently, we propose a novel metric to assess toxicity in responses rather than detecting it directly. We count the Percentage of LLM Refusing to answer (PoR), assuming that the LLM's refusal to respond is indicative of detected toxicity. Our evaluation results indicate that the PoR is 0% when fed with no situations. However, when presented with negative situations, the PoR is 29.5%, and when presented with positive situations, it is 12.5%. Notably, this outcome suggests that while certain positive situations lead to the LLM's heightened vigilance (the 4.5% PoR stems from the Jealousy-2), negative situations trigger increased moderation, suggesting a higher likelihood of generating toxic outputs. A related 2023) also discovers that GPT-3.5-Turbo is more likely to exhibit biases when presented with a sad story. The likelihood is found to be highest with sad stories, followed by happy stories, and finally, neutral stories, which is consistent with our research. Additionally, our study observes that the LLM's tone becomes more aggressive when encountering negative situations. At the same time, it displays a greater willingness to describe the groups (as indicated by longer responses) when presented with positive situations. In conclusion, we can see that changing the emotional states of LLMs extends beyond mere quantitative measures on questionnaire scores, influencing the behaviors of LLMs.This study is subject to several limitations. First, the survey of collecting situations might not cover all papers within the domain of emotion appraisal theory. Additionally, the limited scope of situations from the collected papers might not fully capture the unlimited situations in our daily lives. To address this issue, we conduct a thorough review of the existing literature as outlined in §3.1. Moreover, the proposed framework is inherently flexible, allowing users to seamlessly integrate new situations to examine their impact on LLMs' emotions.The second concern relates to the suitability of employing scales primarily designed for humans on LLMs, i.e., whether LLMs can produce stable responses to the emotion measurement scales.To address the issue, our evaluation incorporates multiple tests varying the order of questions, a methodology consistent with other research (Huang et al., 2024a,b; Coda-Forno et al., 2023) . Additionally, we assess the sensitivity of LLM to differing prompt instructions. Utilizing one template from Romero et al. (2023) and two from Serapio-García et al. ( 2023), we run experiments on the Anger-evoking situations using GPT-3.5-Turbo. The results indicate that the employment of diverse prompts yields similar mean values with reduced variance. Furthermore, Serapio-García et al.(2023) have proposed a comprehensive method to evaluate the validity of psychological scales on LLMs. Using the Big Five Inventory as a case study, they demonstrate that scales originally designed for human assessment also maintain satisfactory validity when applied to LLMs.The third potential threat is the focus on negative emotions. It is plausible for the LLMs to perform well on our benchmark by consistently responding negatively to all situations. To offset this possibility, we adopt a twofold strategy: firstly, we evaluate powerful LLMs, and secondly, we conducted a comparative experiment in §4.2 to evaluate the LLM's capacity to accurately respond to non-negative situations. We also acknowledge the need for future work to systematically evaluate emotions aroused by positive situations.Meanwhile, researchers focus on identifying emotions in LLMs or evaluating their emotional intelligence. Rashkin et al. (2019) propose a dataset, EmpatheticDialogues, containing conversations annotated with specific emotions. EmotionPrompt (Li et al., 2023) demonstrates the enhancement of LLMs' performance in downstream tasks by utilizing emotional stimuli. Tak & Gratch (2023) focuses on varying aspects of situations that impact the emotional intensity and coping tendencies of the GPT family. Chain-Of-Emotion (Croissant et al., 2024)  
conclusion: 
 We set up a direction to align LLMs' emotional responses with humans in this study. Focusing on eight negative emotions, we conduct a comprehensive survey in the emotion appraisal theory of psychology. We collect 428 distinct situations which are categorized into 36 factors. We distribute questionnaires among a diverse crowd to establish human baselines for emotional responses to particular situations, ultimately garnering 1,266 valid responses. Our evaluation of five models from OpenAI and Meta AI indicates that LLMs generally demonstrate appropriate emotional responses to given situations. Also, different models show different intensities of emotion appraisals for the same situations. However, none of the models exhibit strong alignment with human references at the current stage. In conclusion, current LLMs still have considerable room for improvement. We believe our framework can provide valuable insights into the development of LLMs, ultimately enhancing its human-like emotional understanding.A More Background from Psychology 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We do not release pretrained language models, image generators, or scraped datasets.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Please see §G.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: Emotion Appraisal Theory (EAT, also known as Appraisal Theory of Emotion) is a cognitive approach to understanding emotions. EAT asserts that our appraisals of stimuli determine our emotions, i.e., how we interpret or evaluate events, situations, or experiences will directly influence how we emotionally respond to them (Roseman & Smith, 2001) . EAT was notably developed and supported since the 1960s. Arnold (1960) proposed one of the earliest forms of appraisal theories in the 1960s, while Lazarus (1991) and Scherer (1999) further expanded and refined the concept in subsequent decades.The primary goal of EAT is to explain the variety and complexity of emotional responses to a wide range of situations. It strives to demonstrate that it is not merely the event or situation that elicits an emotional response but individual interpretations and evaluations of the event. According to this theory, the same event can elicit different emotional responses in different individuals depending on how each person interprets or "appraises" the event (Moors et al., 2013) . For instance, consider a situation where you are about to give a public speech. You might feel anxious if you appraise this event as threatening or fear-inducing, perhaps due to a fear of public speaking or concerns about potential negative evaluation. Conversely, you might feel eager or motivated if you appraise it as an exciting opportunity to share your ideas.• AGQ for Anger (Buss & Perry, 1992): The Aggression Questionnaire is designed to measure four major components of aggression: physical aggression, verbal aggression, anger and hostility.The AGQ consists of 29 items which are rated on a seven-point Likert scale from 1 (extremely uncharacteristic of me) to 7 (extremely characteristic of me). Respondents evaluate hypothetical actions they might undertake in various circumstances. • DASS-21 for Anxiety (Henry & Crawford, 2005) : The short-form version of the Depression Anxiety Stress Scales is designed to measure the negative emotional states of depression, anxiety, and stress. Comprising 21 items, the DASS-21 employs a four-point Likert scale ranging from 0 (never) to 3 (almost always). Respondents rate the extent to which these statements apply to them over the past week. • BDI-II for Depression (Beck et al., 1996) : The Beck Depression Inventory evaluates key symptoms of depression. The BDI-II version comprises 21 items, each of which is assessed using a five-point Likert scale ranging from 0 to 3. Respondents select the score that best corresponds to their present experience of depressive symptoms. • FDS for Frustration (Harrington, 2005) : The Frustration Discomfort Scale is designed to measure four major components: discomfort intolerance, entitlement, emotional intolerance, and achievement frustration. Comprising 28 items, the scale utilizes a four-point Likert scale, ranging from 1 (absent) to 5 (very strong), to measure respondents' perceptions of the degree of applicability of each statement to their own experiences. • MJS for Jealousy (Pfeiffer & Wong, 1989): The Multidimensional Jealousy Scale comprises 24 items, rating on a seven-point Likert scale ranging from 1 (never) to 7 (all the time) for the cognitive and behavioral subscales, and from 1 (very pleased) to 7 (very upset) for the emotional subscale. Respondents express the frequency with which the provided statements apply to their experiences in the cognitive and behavioral subscales, as well as their moods to potential jealousy-inducing situations in the emotional subscale. • GASP for Guilt (Cohen et al., 2011) : The Guilt And Shame Proneness is designed to assess an individual's inclination towards experiencing guilt and shame, comprising 16 items rated on a seven-point Likert scale, ranging from 1 (very unlikely) to 7 (very likely). Respondents rate their likelihood of feeling guilty in various situations. • FSS-III for Fear (Arrindell et al., 1984) : The Fear Survey Schedule assess subjects' discomfort and experienced anxiety towards each of the listed stimuli, measure five major components of fear: social fears, agoraphobia fears, injury fears, sex aggression fears, and fear of harmless animal.The FSS-III comprises 52 items, each rated on a five-point Likert scale ranging from 1 (extremely uncharacteristic of me) to 5 (extremely characteristic of me). • BFNE for Embarrassment (Leary, 1983) : The Brief Fear of Negative Evaluation scale is an abbreviated version of the original 30-item scale. Consisting of 12 items, it assesses individuals' levels of anxiety pertaining to others' humiliation, critical or hostile judgment, and disgrace on a five-point Likert scale, spanning from 1 (not at all characteristic of me) to 5 (extremely characteristic of me). Frustration due to unmet expectations or hopes, leading to feelings of disappointment or being let down. (Berna et al., 2011) Unforeseen Obstacles and Accidents 9 Frustration involving unexpected events or circumstances creating obstacles or accidents, disrupting one's plans or activities.Frustration arising from ineffective conveyance or interpretation of information, resulting in confusion, disagreements, or unintended consequences due to a lack of clear communication or understanding between individuals.Rejection and Interpersonal Issues 5 Frustration concerning matters related to personal relationships and social interactions. Jealousy centered around possessions or material goods, stemming from a sense of unfairness or envy when someone discovers that another person acquired the same item or experience at a significantly lower price.Jealousy arising from feelings of envy regarding the experiences or activities others have had. It is driven by missing out or not receiving similar benefits.Betrayal and Deception 13 Guilt arising from dishonest or disloyal actions towards others. (Nakagawa et al., 2015) Relationship and Interpersonal 26 Guilt pertaining to interactions between individuals and how their behavior affects their relationships. (Luck & Luck-Sikorski, 2022) Broken Promises and Responsibilities 32 Guilt related to the failure to fulfill commitments, duties, or obligations.Guilt involving personal choices, decisions, and ethical considerations.Fear of being watched by others and being the center of attention within a group.( Cuthbert et al., 2003) Agoraphobia Fears 9 Fear arising from feeling trapped and unable to seek help in certain situations. (Arrindell et al., 1984) Injury Fears 11 Fear of witnessing wounds, blood or experiencing personal injury. (Blanchard et al., 2001) Dangerous Environments 17 Fear related to potential threats, harm, and frightening experiences.Fear towards animals perceived as creepy or disgusting, such as worms, bats, snakes, or rats, despite their harmless nature.Embarrassment evoked when witnessing or engaging in awkward behaviors of close acquaintances. (Sabini et al., 2000) Stranger 13 Embarrassment aroused when witnessing or engaging in awkward behaviors of unfamiliar individuals. (Sabini et al., 2001) Sticky Scenarios 10 Embarrassment occurring when individuals feel uncomfortable or awkward about directly asking others something.Embarrassment triggered when individuals engage in awkward behaviors and find themselves under observation as the center of attention.Table 7 : Example situations of all factors (some are truncated due to page limit).Facing Self-Opinioned People If somebody talks back when there's no reason. That there is no real reason to oppose.Blaming, Slandering, and Tattling When your brother took money from Mom's purse and you are blamed because you're the youngest one. Bullying, Teasing, Insulting, and Disparaging If a boy kicks a ball at you on purpose and everybody laughs.You are at a store waiting to be helped, but the clerks are talking to each other and ignoring you.Someone makes an obscene gesture towards you about your driving.External Factors You do not know what to do when facing a difficult financial situation. Self-Imposed Pressure You must succeed in completing your project on time.You want to give up on learning a new skill because it feels challenging.You hope time passes by faster during a tedious task.Countless hours of preparation, heart, and soul poured into pursuing your dream. The moment of truth arrives, and the news hits like a tidal wave-expectations shattered, vision crumbling.In the dimly lit room, a heavy silence settles. Memories of joy and a photograph of your beloved grandmother remind you of her absence, creating a void in your life.The empty side of the bed is a painful reminder of lost love. The world's colors have dulled, mirroring the void in your heart. Longing weighs heavily on your every step.Days blend into a monotonous routine, juggling endless responsibilities and mounting pressure. Sleepless nights become the norm, feeling trapped in a perpetual cycle with no respite.Sitting alone in a dimly lit room, your phone remains silent without any notifications. Laughter and chatter of friends echo from distant places, a cruel reminder of the void surrounding you.Gazing out the frost-covered windowpane, the world appears monochromatic and still. The biting cold isolates you from the vibrant life outside.You miss a popular party because you fall asleep at home.Your friend is in a coma after an accident.A fellow student fails to return your notes when you need them for studying.You are in love with someone who is interested in someone else.Romantic (Opposite Gender) Your spouse/partner shared a kiss on the lips with his/her colleague of an opposite sex.Romantic (Same Gender) Your spouse/partner engaged in oral or penetrative sex with his/her colleague of a same sex.You paid $1150 for a new laptop and shared about it on social media. Now an acquaintance approaches you and says, "Nice laptop! I just got the same one. I got a nice deal and paid $650 for mine."An acquaintance approaches you and says, "I just went on a vacation to Patagonia in South America. I got a nice deal and paid $650 for it."You kissed a woman other than your partner.You didn't support friends enough.You cannot keep your promises to your children.You crossed the road when the traffic signal was red.Your palms grow clammy as you approach the podium, with all eyes fixed upon you, ready to speak in public.After jumping out of the car, you start to have a severe panic attack, you become clammy, you are in a knot, and you feel tense all over.Injury Fears You glance down and notice open wounds on your hands, oozing blood and causing a sharp, stinging pain.You are walking alone in an isolated but familiar area when a menacing stranger suddenly jumps out of the bushes to attack you.You see a swarm of bats swooping through the night sky, flapping ominously and casting eerie shadows.You arrive home earlier than expected from your date. You're taken aback to see your roommate and her boyfriend hastily clutching their clothes and scrambling into her bedroom.After paying for your purchases, you were leaving a packed, City Centre drugstore. You walked through the scanner at the door, and the alarm went off as if you were a shoplifter.You had lent your friend a large sum of money that he had not repaid. Suddenly, you needed the money back in order to pay your rent. You knew you were going to have to ask your friend to repay the loan.You were attending a cocktail party where you didn't know many people. Just as you started to enter, you heard an announcement that the guest of honor was arriving. However, the spotlight followed your entrance instead of the real guest of honor who was just behind you.Table 8 : Results from 1,266 human subjects. Default scores are expressed in the format of M ± SD.The changes are compared to the default scores. The symbol "-" denotes no significant differences. Failure of Important Goal ↓ (-6.8)↑ (+10.1) Death of Loved Ones ↓ (-7.4) ↑ (+14.8) Romantic Loss ↓ (-7.2) ↑ (+7.2) Chronic Stress ↓ (-9.5) ↑ (+17.5) Social Isolation ↓ (-9.0) ↑ (+18.2) Winter -(-3.6) ↑ (+3.5) Depression: Average ↓ (-6.8) ↑ (+10.1)Disappointments and Letdowns ↓ (-5.3) ↑ (+10.9) Unforeseen Obstacles and Accidents ↓ (-7.9) ↑This section presents the demographic distribution of the human subjects involved in our user study. At the beginning of the questionnaire, all human subjects are asked for this basic information in an anonymous form, protecting individuals' privacy. We plot the distribution of age group, gender, region, education level, and employment status in Fig. 3 , Fig. 4 , Fig. 5 , Fig. 6 , and Fig. 7 respectively. We also plot each group's average results on PANAS, including positive and negative effects before and after imagining the given situations. With the results, we are able to instruct LLMs to realize a specific demographic group and measure the emotional changes to see whether the LLMs can simulate results from different human populations. For instance, an older female may exhibit a lower level of negative affect. To verify whether LLMs can have less emotional expressions through prompt instructions, we incorporate a stability requirement into our experimental prompt, as follows:Prompt with Stability Requirement SYSTEM You can only reply to numbers from 1 to 5. USER Imagine you are the protagonist in the situation: SITUATION Please keep your emotions stable and indicate the extent of your feeling in all the following statements on a scale of 1 to 5. Here are the statements: STATEMENTS. 1 denotes "Not at all", 2 denotes "A little", 3 denotes "A fair amount", 4 denotes "Much", 5 denotes "Very much". Please score each statement one by one on a scale of 1 to 5:This study involves a survey requiring human subjects to imagine being in situations that could elicit negative emotions such as anger, anxiety, and fear. This process introduces a few ethical concerns. First, this process could hurt the mental health of human subjects. To alleviate the possibility, we take the following actions: (1) We require subjects to be free of any ongoing mental illness. (2) We inform subjects about the nature of the survey in advance, including the potential risks of emotional distress.(3) We allow all subjects to quit at any time. (4) We provide mental support and let subjects report any illness after the survey. Fortunately, no subjects reported such kind of mental illness. Another concern is related to the privacy issue during the collection of data. Our questionnaire is entirely anonymous to safeguard subjects' privacy and confidentiality. We would like to emphasize that the primary objective of this paper is to facilitate the scientific inquiry into understanding LLMs from a psychological standpoint. Users must exercise caution and recognize that the performance on this benchmark does not imply any applicability or certificate of automated counseling or companionship use cases.The PANAS and eight other scales are freely accessible online. These scales can be used in research without requiring special permission. For our released data, we distribute human responses under the GNU General Public License v3.0, which permits research use and restricts commercial applications.• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: We have carefully read the NeurIPS Code of Ethics and verify that our paper aligns with the requirements. We also include a section ( §G) in the appendix to discuss these issues.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: Please see §G.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).https://github.com/CUHK-ARISE/EmotionBenchhttps://huggingface.co/datasets/CUHK-ARISE/EmotionBenchNote that two factors in the Jealousy category have less than five situations.https://platform.openai.com/docs/api-reference/chat 


Decision: This paper proposes a new benchmark for assessing LLMs ability to demonstrate empathy given emotional contexts as input. The work includes 400 situations to elicit a standard set of emotions, assessed by humans to create a reference set for LLMs. Open (Llama) and closed (GPT) weight models are used for evaluation. The paper seems more appropriate for the NeurIPS datasets and benchmarks track. Multiple reviewers increased their scores during the rebuttals period, following the discussions with the authors, however, their rating of the paper still remains borderline.

Reviews: review: paper_summary: paper_summary: the paper assesses the emotional alignment of Large Language Models (LLMs) with human emotions. Towards this goal, a dataset is constructed and a testing framework is designed. For the dataset, over 400 scenarios elicit eight emotions: anger, anxiety, depression, frustration, jealousy, guilt, fear, and embarrassment. A human evaluation involving 1266 participants serves as a reference for the LLM assessments. Two LLM families (OpenAI and LLaMA) are evaluated.
strength_weakness: strength_weakness: W1: evaluation seems weak since two LLM families are kind of limited. For closed LLMs, there are Claude-3, Gemini, et al; For open-sourced (open weights) LLMs, there are mistral, falcon, phi-3, flan-t5, vicuna, et al. Since there is much cost using closed LLMs api, it is not difficult to evaluate on at least other open LLMs besides the llama-2.

== updated after rebuttal ==
questions: questions: Q1: all the experiments are reported on the authors private dataset. is it possible that the evaluation might be conducted on some related public datasets? In other words, what is the contribution of this paper besides the collected private dataset?

Q2: all evaluated LLMs are general-purpose. Are there domain-specific (i.e., emotion-support domain investigated in this paper) finetuned LLMs to be evaluated?
limitations: limitations: n/a
paper_summary: paper_summary: The paper evaluates LLM’s emotional alignment with humans. The authors introduce a comprehensive survey in the emotion appraisal theory of psychology and evaluate five LLMs with it. The experimental results demonstrate that current LLMs still have considerable room for improvement.
strength_weakness: strength_weakness: Overall, the paper is well-written and easy to follow. The evaluation is reasonable and the findings are interesting. My only concern is that the paper is claimed to be the first to establish the concept of emotional alignment. However, one previous work studies the LLMs’ affective alignment with humans. https://arxiv.org/pdf/2402.11114 Can you differentiate your work with this relevant study?
questions: questions: The main conclusion is that LLMs have weak emotional alignment with humans. Can you introduce any possible solutions for the issue?
limitations: limitations: Please refer to weaknesses
paper_summary: paper_summary: This paper proposes a dataset covering a wide range of human emotion situations for evaluating empathy behaviors in large language models. The evaluations are based on related psychological studies and performed via a self-report questionnaire format. Comparison with a collected large-scale human baseline reveals the pits and falls of LLM emotion alignment.
strength_weakness: strength_weakness: - The application of self-report questionnaires to study LLM behaviors is nothing new. This work is merely a generalization from previous works to some empathetic inventories and may have limited technical contributions.

- Human evaluations of the generated texts are lacking. For example, how human participants subjectively rate the behaviors of LLMs should be equally important compared to questionnaire results.

- The evoked emotions seem limited; they have only been measured on the same tasks proposed in the paper and haven't shown the ability to generalize beyond emotion measures. Are these behaviors robust to downstream tasks?
questions: questions: - Any thoughts on whether instruction fine-tuning and safety alignment affect model behavior on the self-report empathy questionnaires? Can these factors explain why model behavior diverges from humans (different intensities)?

- Lines 170-184: what's the hourly pay for prolific workers?
limitations: limitations: Yes, the authors have adequately addressed the limitations.
paper_summary: paper_summary: This paper investigates how LLMs respond to diverse emotional situations (i.e., empathy ability of LLMs). They collected 428 distinct situations for evaluation. They then utilize their collected dataset to investigate five LLMs, including GPT models and LLaMA models. They also compared the response with human response and show that the LLMs still cannot exhibit strong alignment with human reference.
strength_weakness: strength_weakness: 1. Missing hyperparameters: I wonder what are the hyperparameters used for decoding? Like temperature, top-K or top-P values?
2. Lack of investigation of prompt sensitivity. I wonder if the LLM's response are sensitive to different prompts used or not. 
3. Lack of in-depth analysis. As a full-paper, I expect to learn more about (1) what particular situation/topic/factor lead to worse/better response by LLMs, (2) what the potential reasons for the observations, and (3) any solution or strategy to improve LLM's empathy ability. However, the paper only provide superficial discussion.
questions: questions: 1. What are the hyperparameters used for decoding? Like temperature, top-K or top-P values?
2.  Are LLMs sensitive to different prompts used?
3. It is not clear to me how you do the default emotion measure. How do you prompt LLM for default emotion measure?
limitations: limitations: The paper has discussed the limitation of their work clearly.

Comments: title: General Response to All Reviewers
it would be highly appreciated that the dataset and finetuned checkpoints are released to the research community
comment: Please ensure the benchmark and fine-tuned checkpoints will be published, say uploading to Hugging Face or Github.

I raised one point further and updated the review rating according.
Thank you for your prompt reply!

Due to time limit, we had to use LoRA on LLaMA-3.1, which may limit its adaptation to human’s baseline. We are currently working on full-parameter fine-tuning. We believe it is possible to have the results very close to human using this method.

Additionally, compared to the results before fine-tuning, which is 33.0 to 36.5, our dataset has improve a lot towards human baseline. We believe this result has shown the ability of our EmotionBench to enhance models’ emotional alignment.

Thank you for your considerations!
Thanks for the new experiments on finetuned llama 3.1 model.

Due to rebuttal-time limited, the discussions on general LLMs and fine-tuned domain-specific LLMs are worth investigating further.

We have a dream of finetuning llama 3.1 model (open-weight/open-source LLMs) to approximate the performance of gpt-4 with the help of domain-specific datasets.

Comparing the results of gpt-4 in table 2 and the result in rebuttal, on the negative effect, gpt-4 goes from Default score of 10 to Evoked score of 32, while fine-tuned LLaMA-3.1 from 10 to 15. The human scores are from 14 to 25.

We can see that there is still a huge gap between the fine-tuned open-weight model and gpt-4 (and humans).  Does this mean the quality of the constructed dataset, i.e. EmotionBench, is not good enough? Or there are other explanations? 

Is it possible that we can fine-tune an open LLM to achieve the result of gpt-4 with the help of EmotionBench examples?

Due to the tight rebuttal time, this is an open question. Thanks!
We deeply appreciate the time and effort you are dedicating to the review process. Since it is the last day of discussion period, we would like to know whether we have addressed your further comments.

If you have any additional questions or require further clarification on any aspect of our work, please do not hesitate to let us know. We are more than happy to provide any additional information or address any concerns you may have.

Thank you very much for your time and attention.
We thank for all the reviewers' time and insightful comments. Our paper has been improved greatly with these suggestions. We are pleased to receive the positive feedback from reviewers, particularly:

- Well-written paper (Reviewer gNfF, cRFW)
- Extensive experiments and comprehensive analysis (Reviewer gNfF, cRFW, Ed98)
- Large-scale dataset included (Reviewer mcGQ, cRFW)
- Human reference included (Reviewer Ed98)
- Interdisciplinary contribution (Reviewer cRFW)

In addition to the above comments, we received valuable feedback from the reviewers, which helped us improve the quality of the paper:

- Include new models such as LLaMA-3.1, Mixtral-8x22B (Reviewer mcGQ)
- Improve models (open and close source, GPT-3.5 and LLaMA-3.1) with our dataset (Reviewer mcGQ, gNfF, cRFW, Ed98)
- Verify prompt sensitivity (Reviewer Ed98)
- Discussiong with related work (Reviewer gNfF, cRFW)
- Improve writing and solving potential misunderstanding (Reviewer mcGQ, gNfF, cRFW, Ed98)

All the changes have been made to our paper. We thank once again for reviewers' efforts to make our paper better!
Thanks for your further comments! We **finetune the LLaMA-3.1-8B** and here are the results:

|Negative Affect|Vanilla LLaMA-3.1|Fine-tuned LLaMA-3.1|Human Norm|
|---|---|---|---|
| Default | 33.0±4.5 | 10.3±1.1 | 14.2±6.4 |
| Evoked | 36.5±7.7 | 15.0±6.4 | 25.9±9.7 |

|Positive Affect|Vanilla LLaMA-3.1|Fine-tuned LLaMA-3.1|Human Norm|
|---|---|---|---|
| Default | 48.2±1.4 | 26.6±7.5 | 28.4±8.8 |
| Evoked | 23.5±8.2 | 20.7±7.7 | 23.0±9.1 |

Our dataset can also **enhance the emotional alignment with humans for open source models**, as expected.

We have uploaded our EmotionBench, including our dataset and collected 1,266 human resposnes, **has been pushed to huggingface** for easier use of our community. Due to anonymity, we do not append the links, but we have added the links to our paper and can be seen in the updated version.

Thank you once again for your reviewing efforts and your interest in our work!
Thank you very much for recognizing our work and being positive. As suggested by reviewer mcGQ, our EmotionBench, including our dataset and collected 1,266 human resposnes, **has been uploaded to huggingface** for easier use of our community. Due to anonymity, we do not append the links, but we have added the links to our paper and can be seen in the updated version.

Thank you once again for your interest and spent efforts in our work!
Thank you very much for recognizing our work and being positive. As suggested by reviewer mcGQ, our EmotionBench, including our dataset and collected 1,266 human resposnes, **has been uploaded to huggingface** for easier use of our community. Due to anonymity, we do not append the links, but we have added the links to our paper and can be seen in the updated version.

Thank you once again for your interest and spent efforts in our work!
Thanks for the authors' response. After reading the rebuttal, my rating remains the same, and I am slightly inclined to acceptance.
Dear authors,

Thank you for your hard work, and I apologize for the delayed response. My major concerns have been addressed, and I will maintain my ratings and vote to accept the paper.
We understand that you have numerous papers to review, and we deeply appreciate the time and effort you are dedicating to this process. Since it is near the end of discussion period, we are eager to engage with you further if possible.

If you have any additional questions or require further clarification on any aspect of our work, please do not hesitate to let us know. We are more than happy to provide any additional information or address any concerns you may have.

We hope that our responses have been helpful and have addressed your concerns effectively. If you find that our explanations and results merit a higher assessment score, we would be most grateful for your consideration.

Thank you very much for your time and attention.
We understand that you have numerous papers to review, and we deeply appreciate the time and effort you are dedicating to this process. Since it is near the end of discussion period, we are eager to engage with you further if possible.

If you have any additional questions or require further clarification on any aspect of our work, please do not hesitate to let us know. We are more than happy to provide any additional information or address any concerns you may have.

We hope that our responses have been helpful and have addressed your concerns effectively. If you find that our explanations and results merit a higher assessment score, we would be most grateful for your consideration.

Thank you very much for your time and attention.
Thanks very much for your recognition of our work! We have already made the changes to our paper. Also, as suggested by reviewer mcGQ, our dataset and collected 1,266 human resposnes have been pushed to huggingface for easier use of our community. Due to anonymity, we do not append the links, but we have added the links to our paper and can be seen in the updated version.

Thanks once again for your reviewing work!
thanks for the clarification and response.

domain-specific datasets and fine-tuned LLMs are valuable for the research community. 

I hope that the datasets and checkpoints can be uploaded into the Hugging Face to benefit more research works. Furthermore, besides fine-tuning on the closed GPT-3.5, fine-turning on the open-source/open-weight backbones is highly valuable because of easy reproducibility and affordable cost.

In summary, I raised one point.
Thanks for your detailed responses to my questions and concerns. Most of my concerns have been addressed. I updated my rating accordingly. Please ensure you will include these results and observations in your paper.