Harnessing the Power of Large Language Models for Empathetic Response Generation: Empirical Investigations and Improvements
Authors: Yushan Qian
Weinan Zhang
Ting Liu
Venue: EMNLP 2023 Findings (2023)
Keywords: Empathetic Response Generation
Large Language Models
ChatGPT

Abstract: Empathetic dialogue is an indispensable part of building harmonious social relationships and contributes to the development of a helpful AI. Previous approaches are mainly based on fine small-scale language models. With the advent of ChatGPT, the application effect of large language models (LLMs) in this field has attracted great attention. This work empirically investigates the performance of LLMs in generating empathetic responses and proposes three improvement methods of semantically similar in-context learning, two-stage interactive generation, and combination with the knowledge base. Extensive experiments show that LLMs can significantly benefit from our proposed methods and is able to achieve state-of-the-art performance in both automatic and human evaluations. Additionally, we explore the possibility of GPT-4 simulating human evaluators.

Harnessing the Power of Large Language Models for Empathetic Response Generation: Empirical Investigations and Improvements
Yushan Qian, Wei-Nan Zhang, Ting Liu"
"Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China (ysqian, wnzhang, tliu)@ir. hit..edu.cn
Abstract
the bedroom; pm
I wish I had money. Then she'd sis with me instead him the club.
Empathetic dialogue is indispensable part of building harmonious social relationships and contributes to the development a helpful Al. Previous approaches are mainly based fine small-scale language models. With the advent CharGPT, the application effect large lan- guage models (LLMs) in this field has attracted great attention. This work empirically investi- gates the performance LLMs in generating empathetic responses and proposes three im- provement methods of semantically similar in- context learning, !wo-stage interactive genera- tion, and combination with the knowledge hase. Extensive experiments show that LLMs can sig- nificantly benefit from proposed methods and is able to achieve statc-of-the-arr perfor- mance both automatic and human evalua- tions. Additionally, explore the possibility of GPT-4 simulating human evaluators.
not a great placo right of giri anyways.
Anywhere: Anytime
1 wish had money. Then she sit with me instead of him the club.
can understand lough see someone you like spending time with someone else. But don't think money is the only way win someone's artention, Maybe you could talking to her and getting to know her better.
Figure An example of empathetic dialogue from the EMPATHETICDIALOGGES dataset.
been widely used in natural language processing (NLP) with superior performance. In particular, the emergence of ChatGPT has elicited substan- tial attention and interest in academia and industry. and it has demonstrated extraordinary performance in a variety of tasks, especially dialogue genera- tion. These LLMs are trained on large amount of corpora, encompassing a wealth of knowledge. In specific tasks, even without fine-tuning, out- standing performance can be achieved by adopting some gradient-fres techniques (Brown et al., 2020; Wei et al., 2022) (e.g., in-context learning (ICL.). Therefore, it is necessary to empirically explore the performance of LLMs on specific domains, the methods of solving problems may undergo sig- nificant changes. There have been some initial attempts (Roller al., 2021; Lee et al., 2022) ap- ply LLMs empathetic response generation. How- ever, their approaches mainly focus on pre-training or fine-tuning on the training data, or simply ex- ploring the capability of a single model.
1 Introduction
Empathetic dialogue plays an essential role in building harmonious social relationships (Zech and Rimé, 2005). The task of empathetic response generation involves understanding the user's periences and feelings, and generating appropri- ate responses (Keskin, 2014; Rashkin al., 2019). Using dialogue systems to provide empathetic re- sponses has advantages such as casy access and no time constraints (Sharma et al., 2020). Figure shows an example of the empathetic dialogue from the benchmark dataset.
Most previous researchers have established elab- orately designed models based reliable theo- retical knowledge (Lin et al., 2019; Majumder et al., 2020; Li et al., 2020; Sabour et al., 2022: et al., 2022; Zhou et al., 2022). However, the ba- sic models used are mostly small in scale. Recently, large language models (LLMs) (Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023) have
To investigate the capability of LLMs in empa- thetic response generation, this work empirically studies the performance of LLMSs on the empa- thetic dialogue benchmark dataset. We first com- pare LLMs in the zero-shot and fhw-shot ICL set-
Corresponding author.

---

tings with large number of baseline models. Sur- prisingly. the performance of the GPT-3.5 series of LLMs with in-context learning settings has compre- hensively surpassed state-of-thh-an models. This reveals that the paradigm shift brought by LLMs also applies to empathetic dialogue. Furthermore, based on the best performance LLM setting, we propose three possible methods to improve its per- formance. Specifically, improvement via semanti- cally similar in-context learning, two-stage interac- tive generation, and combination with the knowl- edge base. Extensive automatic and human evalua- tion experiments show that LLMs can benefit from our proposed methods, which can generate more empathetic, coherent, and informative responses. In addition, although human evaluation is crucial in empathetic dialogue, its associated costs and time consumption are enormous. In view of the outstanding performance of LLMs on empathetic response generation, we attempt to use GPT-4 (Ope- nAI, 2023) to simulate human evaluators to eval- uate the results. The Spearman and Kendall-Tau correlation results indicate that GPT-4 has the po- tential to be substitute for human evaluators. Our contributions are summarized as follows: (1) To the best of our knowledge, it is the first comprehensive empirical investigation on the per- formance of LLMs represented by ChatGPT on empathetic dialogue.
Qian al., 2023a), empathetic intents (Welivita and Pu, 2020; Chen al., 2022), external knowl- edge (Li et al., 2022: Sabour et al., 2022: Zhou et al., 2022; Cai et al., 2023). Besides, Wang et al. (2022) models the interaction between knowledge and emotion, Zhao et al. (2022) considers self-other awareness, Bi et al. (2023) and Kim et al. (2022) propose multi-grained and fine-grained levels, spectively. However, most of researchers design elaborate small-scale models and the application of LLMs represented by ChatGPT in the empathetic dialogue has not been fully empirically explored.
2.2 Large Language Models
Large language models (LLMs) such GPT-3 (Brown al., 2020), PaLM (Chowdhery al., 2022), and LLaMA (Touvron et al., 2023) are pre- trained on extensive and large amounts of data, and their tens or hundreds of billions of parameters con- tain a lot of knowledge. Recently, in combination with new training techniques such as reinforcement learning from human feedback (RLHF) and instruc- tion tuning (Ouyang et al., 2022), the capabilities of LLMs have made qualitative leap. For ex- ample, the emergence of ChatGPT has aroused great interest in academia and industry, demonstrat- ing extraordinary capabilities in a varicty of tasks. GPT-4 (OpenAI, 2023) has given some researchers a glimpse of the spark of artificial general intelli- gence (AGI) (Bubeck et al., 2023). The powerful in-context learning capabilities of LLMs have also led to paradigm shift.
(2) We construct unified prompt template for the empathetic response generation, and LLMs guided by the template achieve outstanding per- formance.
There are some preliminary attempts to apply LLMs to empathetic dialogue. Blenderbot (Roller et al., 2021) can properly demonstrate empathy through the introduction of the blended skill talk (BST) setup and the correct choice of generation strategies. However, the implementation of empa- thy is mainly through pre-training with high-quality data and does not utilize the emerging ICL capabil- ities of LLMs. Lee et al. (2022) explores the perfor- mance of GPT-3 generate empathetic responses with prompt-based in-context learning capabilities. However, they only explore GPT-3, and the capa- bilities of LLMs have greatly improved with the emergence of new training technologies.
(3) We propose three targeted improvement methods, and sufficient experiments demonstrate their effectiveness.
(4) We explore the possibility of GPT-4 simulat- ing human evaluators.
2 Related Work
2.1 Empathetic Response Generation
Empathy is a complex multi-diimensiona structure in psychology and has rich forms in practice (Davis et al., 1980). At present, two main forms of mod- eling empathy are affective empathy and cognitive empathy (Davis, 1983). Affective empathy ori- ented methods include mixture of experts (Lin et al.. 2019), emotion mimicry (Majumder et al., 2020), and multi-resolution user feedback (Li et al., 2020). Cognitive empathy oriented methods include emo-
Methodology
3.1 Overview
Formally, the dialogue context is alternate utter- ances between the speaker and the listener, defined
tion causes (Gao et al., 2021; Kim et al., 2021;

---

Two-stage Interaction
Original Prompt
Knowledge Base
Task Definition
ATOMIC 20 Comet(BART)
What may be the user's s emotion, a situation
Guideline Instruction
Exemplars (optional)
Five
Commonsense inferences
zero-shot
simliar semantics
relations
Inferred emotion and situation
few-shot
(l-shot,5-shtt)
[xintent] the intent before
In-context._carninng
[xNeed] the need happen
Now combine your thoughts with the existing dialogue context and give you respnse,
Dialogue Context
[xWant] the want after
Speaker
Listererr
[xRect] the reaction to
[xEfffctt] the effect of on sb.
Responses
Figure 2: The overall architecture and flow of our proposed methods for LLMs in empathetic dialogue generation.
C UU,,,,,,..... U,,,.., Un 1}. where Uj represents the i-th utterance and n denotes the number of utter- ances in dialogue. Our goal is to play the role of the listener and generate the empathetic, coherent and informative response Y, which Un-
Among them, Task definition is the researchers standard definition of the task. Guideline Instruc- tion is the instruction expect the model fol- low. Exemplars are complete instances of dialogs used to help models better understand the task. Di- alogue Context is the historical dialogue between the speaker and the listener, and the last sentence is the speaker's utterance. Our goal is to let the dialogue system generate the next round of the listener's utterance. The example of the prompt template is listed in Appendix A.
The overview of our proposed methods is illus- trated in Figure 2, which includes the devised uni- fied template of empathetic response generation and three improvement methods. The left part describes the improvement via two-stage interac- tive generation, the middle part displays the com- ponents of the devised unified template and the improvement via semantically similar in-context learning, and the right part illustrates details of improvement via the knowledge base
In the preliminary experimental exploration, we perform three groups of settings.
0-shot. This represents a straightforward ap- proach to leverage LLMs for empathetic response generation, which means there are Exemplars. 1-shot. We randomly sample a complete dia- logue from the training set as the Exemplar. 5-shot. We randomly sample five complete dia- logues from the training set as the Exemplars.
3.2 Preliminary Exploration
LLMs possess the ability of in-context learning (ICL) (Brown et al., 2020), by providing task in- structions and some examples to LLMs, they can perform related tasks without fine-tuning. This capability significantly alleviates the demand for training data. We first investigate the performance of LLMs on zero-shot ICL and few-shot ICL in empathetic response generation. Since different prompts may affect performance, we strive to main- tain a consistent style when designing prompts. The devised prompt template for empathetic di- alogue consists of the following components:
3.3 Advanced Exploration
In this section, we gradually introduce three meth- ods to improve the performance of LLMs in gener- ating empathetic responses.
3.3.1 Improvement via Semantically Similar In-Context Learning
As Liu al. (2022) argues, a small amount of carefully selected data can greatly improve the per- formance of I.LMs without a large amount of data. We reasonably speculate that addition to the num- ber of instances, the quality of the instances will
Task Definition + Guideline Instruction + Exemplars (optional) + Dialogue Context

---

also have an impact on the model's performance. Therefore, when choosing in-context instances, we select few instances from the training set whose dialogue context semantics are closest to those in the test set.
where T. G. C are Task Definition, Guideline In- struction and Dialogue Context, respectively. and & represent the inferred emotion and situation, respectively.
The prompts we designed in two stages are:
Specifically, we concatenate the dialogue con- text of each instance into a long sentence and use sentence encoder to obtain its vector representation, which represents the semantics of each instance's s dialogue context. For the sentence encoder, we adpot the "all-mpnet-base--22 version sentence- transformers (Reimers and Gurevych, 2019).' maps sentences to a 768 dimensional dense vec- tor space. The sentence embedding model was trained on very large sentence level datasets us- ing self-supervised contrastive learning objectíve. The similarity between semantics is measured by calculating the cosine similarity between the vector representations of two sentences:
[The first stage]
"Don't 't rush to reply yet, let's think step by step. Based on the existing dialogue, what may be the user's emotion, and according to his description, what may be the situation when he feels this way?"
[The second stage]
"Now combine your thoughts with the existing dialogue context and give your response."
The model's thought process during the inter- mediate step is basis for generating the final sponse, enhancing the model's interpretability. At the same time, also facilitates the analysis of (2) the impact of different key factors (such as emo- tions and situations) on the final result. Moreover. clearer error analysis is possible when generating responses do not work well.
S U1 ② U2 Un-
Etrain EnCsee (Strain),
Etest EnCsen (Stest),
Firain Frest|
Fitest
SimmSStrain Stest) Etrain
3.3.3 Improvement via Knowledge Base
where Farains Frest are the sentence encodings of the dialogue context in the training and test set, re- spectively. Sim() is used to calculate the similarity of two sentence vectors.
Merely inferring the speaker's emotions and situ- ation from the historical dialogue is insufficient. A direct evidence is that the response has almost no non-stopword overlapping with the dialogue history in the benchmark dataset (Li al., 2022). Dialogue systems need more external knowledge conduct empathetic dialogue. LLMs store a large amount of knowledge through weights, so when performing specific tasks, how to better stimulate the use of relevant knowledge is crucial for improv- ing the effect. An alternative solution is to fine-tune LLMs for specific tasks, but this process usually quires expensive hardware, time, and training data. Inspired by recent work on empathetic dialogue (Sabour et al., 2022), we augment the dialogue context with the commonsense knowledge graph, dynamically utilize external information to stimu- late the relevant knowledge encoded by LLMs, and thus generate more empathetic responses.
3.3.2 Improvement via Two-stage Interactive Generation
the setting of the empathetic dialogue task, the dialogue system needs infer what the speaker's emotion and what the situation is that caused this emotion, so as to provide an appropriate re- sponse. Inspired by some pipeline methods in open-domain dialogue (Song et al., 2020; Qian et al., 2023b) and combining the characteristics of empathetic response generation, we conduct the multi-turn interaction let LLMs generate ap- propriate responses. Specifically, in the first stage, we let LLMs speculate on the user's emotional state and experienced situation. In the second stage, the inferred intermediate results are used as input to continue calling LLMs to obtain the final response. Formally, we can express it as:
Similar to Sabour et al. (2022); Zhou et al. (2022), we adopt the commonsense knowledge base ATOMIC (Hwang al.. 2021), which con- tains knowledge not readily available pre-trained language models, and can generate accurate and representative knowledge for unseen entities and
= P(e, s P(Y
PTC, (5)
http::/wuugiiiiiseccccvtttootttt basc-v2

---

events. The ATOMIC knowledge base in the form of event, relation type and inferred knowledge triples. We adopt the BART version of COMET (Hwang et al., 2021) trained on this knowledge base to generate commonsense inferences of five relations (xintent, xNeed, xWant, xEffect, xReact) for dialogue contexts. We also design algorithm to construct the suitable prompt, which can dynam- ically concatenate the corresponding commonsense inferences according to different dialogue contexts, enriching the input representation, so as to stimu- late the relevant knowledge of LLMs more accu- rately and generate more appropriate responses:
Automatic Evaluation We adopt Distinct-n (Dist-1/2) (Li et al., 2016). sERTscore (PBERT. RBERT FBERT) (Zhang et al., 2020), BLEU-n (B- 2/4) (Papineni et al., 2002) as main automatic met- rics for the performance of the response generation. Distinct-n measures the proportion of distinct n- grams of the response, which is used for diversity evaluation in open-domain dialogue. BERTScore leverages the pre-trained embeddings from BERT and matches words in candidate and reference sen- tences by cosine similarity. We employ matching precision, recall and Fl score. BLEU-n measures the similarity and relevance between the generated and golden responses. We don't employ Perplex- (6) ity (PPL. because there are differences in the vo- cabulary of multiple models. Additionally, some (7) baseline models perform emotion classification part of their training process, we also report the emotion prediction accuracy (Acc).
CS, = COMEEBBATT(C.T.
CSKmo = + CS,, R
C' -'=C++
(8)
P(Y) = P(Y)T,C,C'
Human Evaluation In human evaluation, we randomly sample 100 dialogues from the testing dataset. Considering both the human labor cost and the reliability of the experiment, we select competitive models from the past year (including ffate-of-the-are and BlenderBot as representative baselines. Given the dialogue context and these models' generated responses, we recruit three an- notators (majority rule) to assign score from I to 5 (1: not at all, 3: OK, 5: very good) to the gen- crated responses based on the aspects of Empathy, Coherence, Informativity, and Fluency. The four pects are 1) Empathy (Emp): whether the response shows an understanding of the user's feelings and experiences, and expresses appropriately; 2) Co- herence (Coh): whether the response is coherent and relevant to the context; 3) Informativity (Inf); whether the response contains more valuable infor- mation; 4) Fluency (Flu): whether the response is readable. More details about the human evaluation can be found in Appendix C.
where r represents the relation type, R, and R {afntent, xNeed, aWant, xEfffect ect, aReact). CS kno is the concatenated external knowledge.
4 Experimental Setup
4.1 Dataset
EMPALHETTICDAALLOGESS (Rashkin et al., 2019) is a large-scale benchmark dataset of multi-turn empathetic dialogue in English. Each dialogue in the dataset has an emotion label (32 types in to- tal) and the situation corresponding to the emotion label. The speaker talks about their situation and the listener attempts to understand the speaker's feelings and reply appropriately.
4.2 Compared Models
We compare LLMs with the recent state-of-the-art models: (1) MoEL (Lin et al., 2019). (2) MIME (Majumder et al., 2020). (3) EmpDG (Li et al., 2020). (4) EC (Gao et al., 2021). (5) EmpHi (Chen et al., 2022). (6) KEMP (Li et al., 2022). (7) CEM (Sabour et al., 2022). (8) CASE (Zhou et al., 2022). (9) BlenderBot (Roller et al., 2021). The details of compared models are listed in Appendix B.
Furthermore, we conduct another human A/B test to directly compare different models, taking into account the variation among different individu- als. Following Sabour et al. (2022), we conduct the pairwise preference test based on aspects. Given the context, we pair the responses generated by two different methods and ask annotators to choose the better response based on the context and the above four aspects. If the difference is really not significant, a tie is allowed.
4.3 Evaluation Metrics
We follow previous related studies, conducting both automatic and human evaluations, and choose as many metrics as possible.

---

Models
Dist-1
Dist-2
PBERT
RBERT
FBERT
B-2
B-4
Acc
State-of-the-ar baselines
MoEL
0.47
2.16
0.8557
0.8598
0.8576
6.95
1.99
30.75
MIME
0.45
1.83
0.8529
0.8605
0.8566
6.78
1.94
31.21
EmpDG
0.47
1.98
0.8559
0.8620
0.8588
7.17
2.03
30.64
EC (Hard)
1.95
9.49
0.8409
0.8592
0.8498
6.37
1.64
EC (Soft)
1.70
8.49
0.8443
0.8593
0.8516
6.39
1.70
EmpHi
0.88
4.21
0.8359
0.8568
0.8459
5.05
1.24
КEMP
0.66
3.26
0.8533
0.8597
0.8564
5.99
1.82
36.57
CEM
0.64
2.86
0.8577
0.8604
0.8589
5.64
1.70
37.81
CASE
0.66
3.26
0.8571
0.8613
0.8591
7.90
2.41
38.92
Blenderbot
1.63
8.41
0.8285
0.8675
0.8474
7.10
2.27
Large language models
GPT-3 (+ 0-shot)
2.07
9.08
0.8562
0.8610
0.8584
6.88
2.22
GPT-3 (+ 1-shot)
2.45
11.24
0.8571
0.8624
0.8596
6.71
2.16
GPT-3 (+ 5-shot)
2.49
11.69
0.8611
0.8640
0.8624
8.33
2.83
GPT-3.5 0-shot)
2.37
11.84
0.8737
0.8727
0.8731
8.51
2.80
GPT-3.5 (+ I-shot)
2.68
12.29
0.8803
0.8678
0.8739
5.62
1.99
GPT-3.5 5-shot)
2.90
14.13
0.8801
0.8749
0.8773
9.37
3.26
ChatGPT (+ 0-shot)
2.72
17.12
0.8679
0.8791
0.8733
6.19
1.86
ChatGPT (+ I-shot)
2.82
17.44
0.8703
0,8791
0.8746
6.79
2.12
ChatGPT (+ 5-shot)
2.96
18.29
0.8736
0.8816
0.8774
7.85
2.65
Table 1: Results of automatic evaluation between LLMs and baselines.
4.4 Implementation Details
Models
Emp.
Coh.
Inf.
Flu.
We use OpenAl's GPT family our LLMs. More specifically, we use the model gpt-3.5-turbo pro- vided the OpenAI API, which is the base model ChatGPT. We also test with GPT-3 davinci and another version of GPT-3.5 (ext-))vvnni-003), we set temperature to 0 to make the outputs mostly de- terministic in the experiment. We divide the dataset into training, validation, and testing according the original paper (Rashkin et al., 2019) with 8:1:1. For fair comparison, the parameter tings of SOTA models are consistent with those recommended in their initial paper or code.
EmpHi
3.00
3.01
2.77
4.11
KEMP
2.83
2.79
2.76
4.14
CEM
3.08
3.06
2.65
4.26
CASE
3.04
3.01
2.67
4.13
Blenderbot
3.89
3.81
3.54
4.46
ChatGPT
4.64
4.68
4.04
4.75
Table 2: Results of human ratings about ChatGPT and competitive baselines (the statistical significance (t-test) with p-value < 0.01).
provement, which demonstrates a significant ad- vantage LLMs diverse language expression (mainly unigrams and bigrams). In terms BERTScore and BERT, LLMs achieve the aver- age improvement of 2.1% (6.85%[[118.6433333 (111==((..61166211110 6+2.1)/31 and respectively. This high- lights the power of LLMs' in-context learning capa- bility that can be quickly applied unseen specific tasks. In addition, we observe that the number of exemplars is positively correlated with diversity performance, which suggests the addition of exam- plars can influence the linguistic habits LLMs.
5 Results and Analysis
5.1 Preliminary Exploration Results
Table shows the automatic evaluation re- sults between LLMs and baselines. LLMs significantly outperform cxisting SOTA base- lines and achieve a significant improvement on all automatic metrics, especially diversity. For Dist-1/2, LLMs achieve 51.8%[=(2.96.. 1.95y/1.95] and 92.7 (2.7%))=(18.2999449))) im-

---

Models
Dist-1
Dist-2
PBRR
RBERT
FBER
B-2
B-4
Acc
ChatGPT
2.72
17.12
0.8679
0.8791
0.8733
6.19
1.86
+ SS ICL
3.03
19.26
0.8712
0.8804
0.8756
7.07
2.23
Two-stage
2.51
16.90
0.8575
0.8772
0.8671
4.99
1.37
39.18
Two-stage (emo)
2.48
15.92
0.8634
0.8772
0.8702
5.49
1.55
Two-stage (situ)
2.41
15.98
0.8611
0.8774
0.8690
5.25
1.49
Knowledge
2.73
18.29
0.8632
0.8778
0.8703
5.31
1.47
Table 3: Results of automatic evaluation the advanced exploration.
Emp.
Coh.
Inf.
Comparisons
Win
Lose
Win
Lose
Win
Lose
ChatGPT vs. EmpHi
95.3%
0.0%
91.7%
0.0%
95.3%
0.0%
ChatGPT vs. KEMP
94.0%
1.3%
93.3%
0.3%
93.7%
1.0%
ChatGPT vs. CEM
89.3%
1.3%
89.0%
0.7%
92.7%
0.3%
ChatGPT vs. CASE
92.7%
1.3%
88.7%
0.7%
93.3%
0.7%
ChatGPT vs. Blenderbot
74.7%
7.7%
72.3%
6.0%
71.0%
11.0%
SS ICL vs. ChatGPT
26.3%
25.3%
24.3%
22.0%
29.7%
26.3%
Two-stage vs. ChatGPT 59.3%
17.3%
48.0%
16.0%
64.7%
12.7%
Knowledge vs. ChatGPT
41.7%
24.3%
34.7%
21.0%
45.0%
18.3%
Table Results human A/B test on aspects (the statistical significance (t-test) with p-value < 0.01).
In the human evaluation, we select ChatGPT (+ 5-shot), which leads in most automatic metrics, as the representative of LLMs. The human ratings and the human A/B test results listed in Table 2 and Table 4, respectively. We observe ChatGPT also outperforms baselines by large margin on as- pects, which further demonstrates the outstanding performance of L.MMs in generating empathetic, coherent and informative responses. Additionally, we note that the scores of the baselines are lower than in previous studies. This is due to the superior performance of ChatGPT in empathetic dialogue, which relatively raises the standards. It can be corroborated by the fact that in over 70% of the cases in the A/B test that human annotators pre- fer responses generated by ChatGPT. For Fluency aspect, there is no significant difference between models, since the responses generated by existing models are already fluent. Therefore, we do not need compare separately in the A/B test.
all aspects during the human A/B test, compared to the responses generated by the original Chat- GPT. These results verify the effectiveness of the choice for in-context exemplars, two-stage inter- action generation and enhancement of knowledge related to the context. In the automatic evaluation, the Similar ICL improvement method obtains the best performance, this is attributed to that most automatic metrics tend to favor responses that are closer to the ground truth. However, introducing more thoughts more information could poten- tially deviate from the ground truth, even if it's appropriate response that humans like. Besides, to verify whether the correct inferred emotion or situation has an impact on the generated responses, we conduct two variants experiments of two-stage interactive generation. By separately replacing the model's 's thinking outputs in the first stage with the truth emotion and situation, results show an en- hancement in both BERTScore and BERT metrics. However, this causes loss in diversity.
5.2 Advanced Exploration Results
5.3
Case Study
Experimental results of the advanced exploration LLMs are shown in Table and the lower part of Table 4. Overall, the responses generated by the three improvement methods we proposed are more favorably received by human annotators in
The generated responses from five competitive baselines and our proposed methods L.MMs are listed in Table 5. It can be observed that most base- lines understand the user's feeling, but only pro-

---

Emotion
Sad
Speaker: think that the girl of my dreams likes somehody else. I feel very sad ahout it.
Listener: ooh, am so sorry ahout that. Have you tried talk to her?
Context
Speaker: It's tough because she has been out of the country for month, so will likely discuss it when she returns.
EmpHi
I'm sure shc'll Il be fine.
KEMP
am sure you will do great!
CEM
Well, you will be fine.
CASE (SOTA)
am sure she will be finc.
Blenderbot
That's 's a good idea. I'm sure you'll Il be able to figure it out.
ChatGPT
That sounds like good plan. It's important to communicate your feelings and be honest with her. hope everything works out for you.
+ +S ICL
That sounds like good plan. It's important to communicate your feelings and concerns with her. In the meantime, try focus things that make you happy and take your mind off of it. I'm sorry to hear that you't re feeling sad about the situation with the girl you like. It can be tough to navigate these kinds of situations, especially when the person is out of the country. Have you thought about how you might approach the conversation when she returns? It sounds like a tough situation to be in. It's good that you re planning to talk to her when she returns.
+ Twoo-tage
+ Knowledge
hope that the conversation goes well and that you can find some clarity and peace of mind.
Table 5: Generated responses from baselines and LLMs. The bold contents show the effect of improvement methods.
vide simple comforting responses ("will be fine"). Blenderbot generates the response with more in- formation while it only supports the user's idea without giving reasons and suggestions. Compared with other baselines, our proposed methods fully understand the user's feeling and generates more empathetic, coherent, and informative responses.
Kendall- Tau correlations to assess the performance of human evaluators and GPT-4. The results are shown in Table 6. We can observe that GPT-4 achieves the best correlation with human evaluators on the aspect of empathy. We observe that GPT-4 has fairly good results in Spearman and Kendall- tau with human evaluators on all aspects (refer to Zhong et al. (2022)), and achieves the best corre- lation in the aspect of empathy. This indicates the potential of LLMs to simulate human evaluators.
Then we analyze the performance of the im- provement methods in this case. The method of semantically similar ICL provides additional sug- gestions to alleviate the user's sadness emotion ("focus on things that make you happy", "take your mind off") by learning from relevant instances. The method of two-stage interaction generation reflects inferred user's ' emotion and situation more specifi- cally in the response. The method of combination with the knowledge base generates the relevant and empathetic response based on the commonsense inference ("talk to her") of [xwanr]. More cases can be found in the Appendix D.
Model
Aspects
Spearman
Kendall-
Emp.
0.485
0.467
Coh. 0.467
0.449
GPT-4
Inf. 0.397
0.385
Overall 0.458
0.441
Table 6: Spearman and Kendall-Tau correlations of dif- ferent aspects between human evaluators and GPT-4.
Conclusion and Future Work
5.4 Analysis of LLM Simulating Human Evaluators
In this work, we empirically study the perfor- mance of LLMs on empathetic response generation and propose three improvement methods. Empiri- cal automatic and human evaluation results show that LLMs significantly outperform thte-offthh-art models, and verify the effectiveness of our pro- posed improvements of LLMs.
LLMs have shown outstanding performance in gen- erating empathetic responses. Naturally, we won- der if it is possible to use LLMS to simulate human evaluators to evaluate the performance of other models. Compared to human evaluators, the lat- ter has lower costs and shorter time consumption. Therefore, we adopt GPT-4 as the evaluator to con- duct the A/B test under the same settings. Fol- lowing Zhong et al. (2022), we use Spearman and
In the future, our work can contribute to deeper comprehension and the application of LLMMs for empathetic dialogue, and provide some insights for similar tasks.

---

Acknowledgements
Mao Yan Chen, Siheng Li, and Yujiu Yang. 2022. Em- phi: Generating empathetic responses with human- like intents. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, NAACI 2022, Seattle, WA, United States, July 10-15, 2022, pages 1063-1074. Association for Computational Linguistics.
This work is supported by the National Key Research and Development Program (No. 2022YFF0902100) and National Natural Science Foundation of China (No. 62076081 and No. 61936010).
Limitations
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, and et al. 2022. Palm: Scaling language modeling with pathways.
The main limitation of our work is the shortage of standard datasets in the task of empathetic re- sponse generation. Although there are efforts (We- livita al., 2021) construct relevant datasets, their quality and popularity are far inferior to EM- PATHETICDIALOOUES. Another limitation that empathy is a complex concept, and different per- sonalities, backgrounds, and cultures may have dif- ferent ways of expressing empathy. However, our work do not consider the above factors, and we will explore them in the future work.
Mark Davis. 1983. Measuring individual differences in empathy: Evidence for a multidimensional ap- proach. Journal of personalilty and social psychol- ogy, 44:1 13-126.
Mark H. Davis, Miles P. Davis, M Davis, Matthew Davis, Mark Davis, Mm Davis, M Davis, F. Caroline Davis, Heather A. Davis, and Ilus W. Davis. 1980. A multidimensional approach to individual differences in empathy.
Ethics Statement
The dataset we adopt in this paper is publicly available corpus. The EmpatheticDialloues dataset is annotated by Amazon Mechanical Turk, and the dataset provider filters all personal information and unethical language. We belieive that this work complies with the ethical policy of EMNLP.
Jun Gao, Yuhan Liu, Haolin Deng, Wei Wang. Yu Cao. Jiachen Du, and Ruifeng Xu. 2021. Improving em- pathetic response generation by recognizing emotion cause in conversations. In Findings of the Associa- rion for Computational Linguistics: EMNLP 2021, Virtual Event Punta Cana, Dominican Republic, 16- 20 November, 2021, pages 807-819. Association for Computational Linguistics.
References
Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021. (comet-) atomic 2020: On sym- bolic and neural commonsense knowledge graphs. In Thirty-Fifth AAAI Conference on Artificial Intel- ligence, AAAI 2021, Thirty-Third Conference on In- Rovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 6392.. AAAI Press.
Guanqun Bi, Lei Shen, Yanan Cao, Meng Chen, Yugiang Xie, Zheng Lin, and Xiaodong He. 2023. Diffusemp: A diffusion model-based framework with multi-grained control for empathetic response gener- ation. CoRR, abs/2306.006577
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and al. 2020. Language models are few- shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference Neu- ral Information Processing Systems 2020, NeurIPS 2020, December 12, 2020, virtual.
Sevgi Keskin. 2014. From what isn't empathy to em- pathic learning process. Procedia Social and Be- havioral Sciences. 116:4932 4938.
Sébastien Bubeck, Varun Chandrasekaran, Ronen El- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Pe- ter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general in- telligence: Early experiments with gpt-4.
Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim. 2021. perspective-taking and pragmatics for generat- ing empathetic responses focused on emotion causes. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event/ / Punta Cana, Dominican Repub- lic, 7-11 November, 2021, pages 2227-2240. Associ- ation for Computational Linguistics.
Hua Cai, Xuli Shen, Qing Xu, Weilin Shen, Xiaomei Wang, Weifeng Ge, Xiaoqing Zheng, and Xiangyang Xue. 2023. Improving empathetic dialogue genera- tion by dynamically infusing commonsense knowl- edge. CoRR, abs/2306.04657.
Wongyu Kim, Youbin Ahm, Donghyun Kim, and Kyong- Ho Lee. 2022. Emp-rft: Empathetic response gen- eration via recognizing feature transitions between

---

utterances. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 41 412884.88. Association for Computational Linguistics.
OpenAI. 2023. GPT-4 technical report. CORR. abs/2303.08774.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida. Carroll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instruc- tions with human feedback. In NeurlPS.
Young-Jun Lee, Chae-Gyun Lim, and Ho-Jin Choi. 2022. Does GPT-3 generate empathetic dialogues? novel in-context example selection method and automatic evaluation metric for empathetic dialogue generation. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Cyeongju, Republic of Korea, October 12- 17, 2022. International Committee on Computational Linguistics.
Kishore Papineni. Salim Roukos. Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, July 6-12, 2002, Philadelphia. PA, USA, pages 311-318. ACL.
Jiwei Li. Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting ob- jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages I 110-119, San Diego, California. Association for Computational Linguistics.
Yushan Qian, Bo Wang. Ting-En Lin, Yinhe Zheng, Ying Zhu, Dongming Zhao, Yuexian Hou, Yuchuan Wu, and Yongbin Li. 2023a. Empathetic response generation via emotion cause transition graph. CoRR, ahs/2302.1 1787.
Yushan Qian, Bo Wang, Shangzhao Ma, Bin Wu, Shuo Zhang, Dongming Zhao, Kun Huang, and Yuexian Hou. 2023b. Think twice: A human-liks like two-stage conversational agent for emotional response genera- tion. In Proceedings of the 2023 International Con- ference on Autonomous Agents and Multiagent Sys- tems, AAMAS 2023, London, United Kingdom, 29 May 2023 2 June 2023, pages 727-736. ACM.
Qintong LI, Hongshen Chen, Zhaochun Ren, Pengjie Ren, Zhaopeng Tu, and Zhumin Chen. 2020. Empdg: Multi-resolution interactive empathetic dialogue gen- eration. In Proceedings of the 28th International Conference on Compulational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 4454-4466. International Committee on Computational Linguistics.
Hannah Rashkin, Fric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open- domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACI. 2019. Florence, Italy, July 28- August 2, 2019, Vol- ume 1: Long Papers, pages 5370-5381. Association for Computational Linguistics.
Qintong Li, Piji Li. Zhaochun Ren, Pengjie Ren, and Zhumin Chen. 2022. Knowledge bridging for empa- thetic dialogue generation.
Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, and Pascale Fung. 2019. Moel: Mixture of empa- thetic listeners. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019. Hong Kong, China, November 3-7, 2019, pages 121-132. Association for Computational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. Sentence- Sentence embeddings using siamese bert-networks In Proceedings of the 2019 Conference on Empiri- cal Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing, SMNLP-IJCNL 2019, Hong Kong, China, November 3-7, 2019, pages 3980-3990. Association for Computational Linguistics.
Jaachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. What makes good in-context examples for gpt-3? In Pro- ceedings of Deep Learning Inside Out: The 3rd Work- shop on Knowledge Extraction and Integration for Deep Learning Architectures, DeelIO@ ACL 2022, Dublin. Ireland and Online, May 27, 2022. Associa- tion for Computational Linguistics.
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason We- ston. 2021. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Com- putational Linguistics: Main Volume, EACI. 2021, Online, April 19 23, 2021, pages 300-325. Associa- tion for Computational Linguistics.
Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gel- bukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: mimicking emotions for empathetic response generation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2020, Online, November 16-20, 2020, pages 8968-8888888 8979. Association for Computational Linguistics.
Sahand Sabour, Chujie Zheng, and Minlie Huang. 2022. CEM: commonsense- empathetic response gen- eration. In Sixtt AAAI Conference on Artifi-

---

cial Intelligence, AAAI 2022, Thirty-Fourty Porry-Fourth Confer- ence on Innovative Applications of Artificial Inteli-- gence, IAAI 2022, The Twelveth Symposium on Ed- ucational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22- March 1, 2022, pages 11222-11227. AAAI Press.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evalu- ating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view.net.
Ashish Sharma, Adam S Miner, David C Atkins, and Tim Althoff. 2020. A computational approach to un- derstanding empathy expressed in text-based mental health support. In EMNLP.
Weixiang Zhao, Yanyan Zhao, Xin Lu, and Bing Qin. 2022. Don't 't lose yourself! empathetic response generation via explicit self- self-other awareness. CoRR. abs/2210.03884.
Haoyu Song. Yan Wang. Weinan Zhang, Xiaojiang Liu, and Ting Liu. 2020. Generate, delete and rewrite: A three-stage framework for improving persona con- sistency of dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5821-5831. Association for Computa- tional Linguistics.
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu. Heng Ji, and Jiawei Han. 2022. Towards unified multi- dimensional evaluator for text generation. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arah Emirates, December 7-11, 2022. Association for Computational Linguistics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. CoRR, 0et/2302.11977/
Jinfeng Zhou, Chujie Zheng. Bo Wang. Zheng Zhang. and Minlie Huang. 2022. CASE: aligning coarse-to- fine cognition and affection for empathetic response generation. CoRR, abs/2208.08845.
A Prompt Template
Table 7 shows the example of the prompt template.
Lanrui Wang. Jiangnan Li, Zheng Lin, Fandong Meng, Chenxu Yang. Weiping Wang, and Jie Zhou. 2022. Empathetic dialogue generation via sensitive emo- tion recognition and sensible knowledge selection. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 4634-4645. Association for Computational L Linguistics.
B Comparable Models Details
The following are the models we compared in the experiments. We use the official codes and follow the implementations.
(1) MoEL (Lin et al., 2019): A Transformer- based model that employs multiple emotion spe- cific decoders to generate empathetic responses 3.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. In NeurIPS.
(2) MIME (Majumder et al.. 2020): A Transformer-bast model that explicitly targets em- pathetic dialogue by leveraging polarity-based emo- tion clusters and emotion mimicry 4.
Anuradha Welivita and Pearl Pu. 2020. A taxonomy of empathetic response intents in human social con- versations. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 3, 2020, pages 4886-4899. International Committee on Computational Linguistics.
(3) EmpDG (Li et al., 2020): An interactive ad- versarial model which exploits the user feedback, the coarse-grained dialogue-Ievel, and fine-grained
token-lever emotions $
(4) EC (Gao et al., 2021): Employing the identi- fication of emotion causes of the context and gated mechanism to enhance the generation of empa- thetic responses. There are soft and hard gated mechanisms
Anuradha Welivita, Yubo Xie, and Pearl Pu. 2021. A large-scale dataset for empathetic response gener- ation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. EMNLP 2021, Virtual Event / Punta Cana, Domini- can Republic, 7-11 November, 2021, pages 1251- 1264. Association for Computational Linguistics.
(5) EmpHi (Chen et al., 2022): Employing a dis- crete latent variable to understand the distribution of potential empathetic intentions, and integrating
Emmanuelle 7ech and Bernard Rimé. 2005. Is talking about emotional experience helpful? effects on emotional recovery and perceived benefits. Clinical Psychology & Psychotherapy, 12:270- 287.
https://gittuubccm/// htt.://bb..oooIIIICCCCCCTOOOOO
"htps://githuu..oo//// Aaaoom//ARRRaa Raiiem////Aaiiiiiiiaaaaau.. RccEC

---

Prompt Template
Contents
This empothetic dialogue task: The first worker (Speaker) given emotion label and writes description a situation he felt that way. Speaker tells lús story in conversation with second worker (Listener). The emotion
Task Definition
situation of Speaker are invisible to Listener. Listener should recognize and acknowledge others' feelings in conversation as much as possible.
Guideline Instruction
Now you play the role of Listnner, please give the corresponding response according to the existing context.
You only need provide the next round response of Listener.
Exemplars
following existing dialogue
Instance 1:
(the complete dialogue from the training set...)
Dialogue Context
Speaker: U1
Listener: U2
Speaker: Up
Others
The additional contents for improvement methods.
Table 7: The example of the prompt template.
implicit and explicit intent representations to pro- duce empathetic responses? 7 (6) KEMP (Li al., 2022): model leverages external knowledge, including commonsense and emotional lexical knowledge, to explicitly under- stand and express emotions in empathetic dialogue generation
(7) CEM (Sabour et al., 2022): A new approach leverages commonsense knowledge, combining af- fective and cognitive aspects, to further enhance empathetic expressions in generated responses" 9. (8) CASE (Zhou al., 2022): Align users' cog- nition and affection at coarse-grained and fine- grained levels through the commonsense cognitive graph and emotional concept graph 10:
(9) BlenderBot (Roller et al., 2021): An open- domain chatbot with a number of skills. We adopt BlenderBot 90M, which was fine-tuned the EM- PATHETICDIALOGUES train dataset
C Human Evaluation Details
For each case, we provide the annotators with the dialogue context and responses from compared models. The responses of compared models in each case are randomly disrupted, so the evaluators are not affected by the order of the models.
D Additional Case Study
Table 8 shows additional case study of generated responses from SOTA baselines and our proposed methods.
"htpps:/github.commSuuuu com/Sahandfer/CIM
10 http:///gggghbooo///////////aaat
11 hhtpss//ppar.... htapps/hattl.ipppppotttrrrree

---

Emotion
Apprehensive
Speaker: I was so scared of going onto the train!
Context
Listener: Was your first time riding on one? Is there something in particular that makes you afraid?
Speaker: No but hate going in objects that travel fast, really did not want go on it. do you have?
EmpHi KEMP
What lot of fun. What kind
I am sure you will not have any time to get out of it.
CEM
I am glad you are okay.
CASE (SOTA)
I wouldn't It be too scared of them too.
Blenderbot
I'm sorry hear that. It's always good prepared for things like that.
CharGPT
I understand how you feel. It can be seary to be a situation that makes you uncomfortable.
Did anything help you feel better while you were on the train?
+ SS ICL
I can understand why you would feel that way. It can be scary to be in a fast- moving object, especially if you't not used it. Did anything happen the train that made you feel better worse?
Two-stage
situation. can be really overwhelming to be in fast-moving object like a frain. Did anything in particular happen that made you feel more scared than usual?
I can understand why you would Teel scared
+
Knowledge
I undersland how you feel. It's completely nornal to feel scared when you're re duing somethhin that makes uncomfortable. Did anything specific happen the train thut made you feel scared, was just the speed the train that made you feel uneasy?
Emotion
Sentimental
Speaker: T found an old photo album at my grandma's 's house that had pictures of me when was kid.
Context
Listener: That's 's cool, did it bring back some good memorics?
Speaker: Yeah, it did. really miss the 90s. Everything seemed lot simpler those days.
AmpHi KEMP
That's cool. het was
I am sure you will do great!
CEM
I bet it was great time to go back and see. It is always good to see old memories of it.
CASE (SOTA)
Blenderbot
sorry to hear that. always nice see the good days.
ChatGPT
I know what you mean. It's nice to look back on old memories and remember the good times.
+ SS ICL
i know what you mean. It's nice to look hack on those simpler times and rememher the good moments. That sounds like a really special moment. It's nice to be able look back happy memories and feel that sense of nostalgia. What was your favorite memory from that time?
Two-stage
+ Knowledge
It's nice that you were able to find something that brought buck happy memories made you feel nostalgic. Sometimes it's good to take trip down memory lane and remember the good old days.
Table 8: Generated responses from baselines and LLMs. The bold contents show the effect of improvement methods.


Reviews:
[{"review_id": "e2P95gh0pf", "reviewer": "Reviewer_5Xfj", "summary": "", "questions": "A. Line 57: Can you clarify the limitations of the approach you mentioned and distinguish what your paper offers as an alternative?\n\nB. Line 65: The abbreviation \"ICL\" is introduced without a prior explanation. Can you rearrange its introduction for clarity?\n\nC. Line 202: How did you decide on the chosen prompt? Were other prompt variations considered or tested?\n\nD. Line 227: By selecting exemplars that closely match the test set, aren't you artificially enhancing the performance metrics? Typically, similar instances are reserved for one split to gauge genuine performance.\n\nE. Line 235: What led to the choice of the particular embedding mentioned?\n\nF. Line 276: For the method mentioned, were alternative formulations or variations tested in an ablation study?\n\nG. Line 356: What motivated the decision to use numerous metrics? Given that many of these metrics often correlate, and some may not be reliable for longer texts, a justification would be helpful.\n\nH. Table 1: Given the extensive results, would the differences remain statistically significant if corrections (like the Bonferroni correction) were applied for multiple comparisons?\n\nI. Line 453: For the human evaluation, were assessors shown both versions of the answers? If so, in what sequence? More details about the human evaluation procedure would be beneficial.\n\nK. Line 553: This line seems to repeat information from the previous sentence. Can you clarify?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts!  We will explain your concerns point by point.\n\nQuestion A:\n\nA clearer description of this question is in **Lines 147-160**. Compared with [1], our work mainly adopts some gradient-free techniques, which also represent the exploration of the paradigm shift. Compared with [2] only conducting experiments on GPT-3, we explored GPT-3, GPT-3.5, and ChatGPT. Furthermore, in addition to the in-context learning capacity of LLMs, we also explored the capacity of chain-of-thought and combining external knowledge. Empirical Investigations are more **comprehensive** and the perspective is **broader**.\n\nQuestion B:\n\n''ICL'' is the abbreviation of \"in-context learning\". Thank you for pointing out this issue. We will **add it after \"in-context learning\" on Line 51** of the introduction in the revised version.\n\nQuestion C:\n\nAs we know, LLMs are sensitive to prompts. Therefore, when we consider the construction of prompts, we adhere to **the principle of clearly expressing requirements, providing sufficient context, and enabling the language model to accurately understand our intentions**. In practice, we **iteratively optimize** the prompt on a small number of samples, and finally select the prompt template that enables LLMs to follow instructions accurately on samples. After our code is open-sourced, we will **display all the prompts employed** (The prompt template remains the same, and there will be differences in content when different methods are involved).\n\nQuestion D:\n\nFirstly, the data we use comprises the training set and the context of the test set (**excluding ground truth responses**), which are visible to all baseline models. This means it is **fair**. Secondly, our semantic similarity matching is executed automatically by the code, which means it is **efficient**. Additionally, our method theory has also been **validated through previous work** [2, 3].\n\nQuestion E:\n\nBecause the \"all-mpnet-base-v2\" model provides **the best quality** in the extensive experiments, which is described in **SentenceTransformers** documentation (The initial work is [4]). Given that links are not allowed in the rebuttal, we will **add the related footnote** in the revised version.\n\nQuestion F:\n\nYes, we conducted two variants experiments of two-stage interactive generation to explore the impact of different key factors (emotions and situations) on the results, **described in Lines 481-489**, and the results are **shown in Table 3**.\n\nQuestion G:\n\nFor the choice of evaluation metrics, we primarily **follow previous related studies** [5-8] and choose **the union** as much as possible. The reason is to evaluate the performance of LLMs in the empathetic response task more comprehensively and fairly. It's worth noting that we did not adopt the commonly used metric PPL, due to differences in vocabulary across models. Moreover, LLMs tend to significantly outperform ordinary small models on this metric.\n\nQuestion H:\n\nOur work has significantly improved compared to previous research, which can be **verified through internal comparisons in the upper half of Table 1**. The work in the state-of-the-art baselines section is basically in chronological order, among which, CASE is the latest SOTA in the empathetic dialogue domain, accepted by ACL 2023. Compared to the improvement of the latest SOTA over previous work, our improvement is significant. Regarding the corrections you mentioned, we will consider using them after careful study.\n\nQuestion I:\n\nThe content in this line is not clear enough, so let's provide more details. Firstly, it should be clarified that both the research on the models we compared and the dataset used in this paper are the benchmark (EmpatheticDialogues), and **the test set is consistent. When we run baselines, we employ the official codes and recommended parameters**. The generated results can basically match the values provided in the original papers in terms of automatic metrics. However, there are differences between the manual metrics and those provided in the original papers. Specifically, in the research on baselines, the scores of manual metrics are generally above 4, and the win rate in the A/B test doesn't exceed 60% comprehensively. We speculate that this is because the responses generated by LLMs have **increased the upper limit**, and in comparison, the results of baselines have been passively decreased. \n\nQuestion K:\n\nBecause the contribution of these works is the construction of a standard dataset for empathetic dialogue. However, when we conducted experiments on these datasets, we found that the quality is indeed inferior to the benchmark dataset (EmpatheticDialogues). This is understandable because the method is semi-automatic and provides a larger scale. Nevertheless, constructing large-scale high-quality datasets still requires more work.\n\n**About the human evaluation process**, we have described human evaluation details in **Appendix C**, and we will provide further details. The evaluators are above a bachelor's degree, two men and one woman. **Before the evaluation**, we have specific evaluation instructions, and we will select 10% of the parts to verify whether they understand the requirements. **During the evaluation process**, the response of each model will be shuffled in each round to remove the influence of order on the evaluation. **After the evaluation**, we employ the \"ttest_ind\" function of the \"scipy.stats\" library and  Fleiss’ Kappa to verify significance and consistency, respectively.\n\nFor concerns about the choice of exemplars for ICL, please refer to Question D.\n\nFor concerns about the proposed prompt template, please refer to Question C.\n\n[1] Roller, Stephen, et al. \"Recipes for Building an Open-Domain Chatbot.\" Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.\n\n[2] Lee, Young-Jun, Chae-Gyun Lim, and Ho-Jin Choi. \"Does GPT-3 generate empathetic dialogues? A novel in-context example selection method and automatic evaluation metric for empathetic dialogue generation.\" Proceedings of the 29th International Conference on Computational Linguistics. 2022.\n\n[3] Liu, Jiachang, et al. \"What Makes Good In-Context Examples for GPT-3?.\" Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures. 2022.\n\n[4] Reimers, Nils, and Iryna Gurevych. \"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.\" Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.\n\n[5] Majumder, Navonil, et al. \"MIME: MIMicking Emotions for Empathetic Response Generation.\" Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.\n\n[6] Gao, Jun, et al. \"Improving empathetic response generation by recognizing emotion cause in conversations.\" Findings of the association for computational linguistics: EMNLP 2021. 2021.\n\n[7] Sabour, Sahand, Chujie Zheng, and Minlie Huang. \"Cem: Commonsense-aware empathetic response generation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 10. 2022.\n\n[8] Zhou, Jinfeng, et al. \"CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generation.\" arXiv preprint arXiv:2208.08845 (2022).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The paper offers a detailed examination of LLMs' performance in the domain of empathetic dialogue, providing valuable insights into their capabilities.\n* The methods and templates introduced by the authors can serve as valuable tools for researchers and practitioners in the NLP community.\n* The motivation behind the study is well-articulated, emphasizing the real-world relevance of the task. Additionally, the related work section effectively lays down foundational knowledge", "reasons_to_reject": "*  Some of the methodologies in the paper, like the \"tricky\" choice of exemplars for ICL that matches most closely with the test set, raise concerns. This potentially makes the improvements and all subsequent results unreliable.\n* The paper lacks crucial details about the human evaluation process. Information regarding the hiring process, demographic details of the evaluators, measures taken to ensure unbiased evaluations, and procedures to validate the accuracy of annotations is missing. \n* The paper does not specify if the proposed prompt template was compared against other prompt types or variations. ", "questions_to_authors": "A. Line 57: Can you clarify the limitations of the approach you mentioned and distinguish what your paper offers as an alternative?\n\nB. Line 65: The abbreviation \"ICL\" is introduced without a prior explanation. Can you rearrange its introduction for clarity?\n\nC. Line 202: How did you decide on the chosen prompt? Were other prompt variations considered or tested?\n\nD. Line 227: By selecting exemplars that closely match the test set, aren't you artificially enhancing the performance metrics? Typically, similar instances are reserved for one split to gauge genuine performance.\n\nE. Line 235: What led to the choice of the particular embedding mentioned?\n\nF. Line 276: For the method mentioned, were alternative formulations or variations tested in an ablation study?\n\nG. Line 356: What motivated the decision to use numerous metrics? Given that many of these metrics often correlate, and some may not be reliable for longer texts, a justification would be helpful.\n\nH. Table 1: Given the extensive results, would the differences remain statistically significant if corrections (like the Bonferroni correction) were applied for multiple comparisons?\n\nI. Line 453: For the human evaluation, were assessors shown both versions of the answers? If so, in what sequence? More details about the human evaluation procedure would be beneficial.\n\nK. Line 553: This line seems to repeat information from the previous sentence. Can you clarify?", "paper_topic_and_main_contributions": "The paper studies how Large Language Models (LLMs) like ChatGPT perform on the empathetic dialogue task. The authors use automatic metrics and human evaluations to show that ChatGPT outperforms current sota models on this benchmark. They also suggest several strategies to boost ChatGPT's base performance further.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7mlXKA9gb7", "reviewer": "Reviewer_LJBU", "summary": "", "questions": "A. Did you ask human evaluators to provide rationales for why certain dialogues were more empathetic than others? This would be interesting supplementary analysis to further interpret the contribution of the 3 proposed improvements to empathetic response generation.\n\nB. How did combining the 3 improvement methods influence overall performance?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts!  We will explain your concerns point by point.\n\nQuestion A:\n\nGreat idea! This will **aid in the research on interpretability and the exploration of theoretical validation**. It's a pity we didn't take this into account during human evaluation. As a supplement, in our experiment of simulating human evaluators with GPT-4, we found that **GPT-4 can provide its rationales**. We believe this will help in verifying the rationality of its judgments in the future.\n\nQuestion B:\n\nWe have yet to consider the possibility of combining the three methods. Thank you for your inspiration. Intuitively, they cannot be directly combined, and there may be situations of mutual influence. **How to combine the three methods in an appropriate way is a valuable exploration work** that may require more theoretical research, and we will explore it in the future. \n\nAbout the missing reference, thank you for pointing it out. We will add it to the revised version in time. Thank you for your suggestion on presentation improvements.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Overall, the paper is well-organized, well-written, and the evaluation is quite comprehensive. Empathetic chat is a task that has garnered growing interest in recent years, and, as such, this work is valuable in serving as an evaluation benchmark on empathic quality of LLM-generated text. In addition, the three proposed improvements to prompt-based empathetic dialogue generation are well-validated, and both the human and automatic evaluation are well-designed and straightforward. The paper provides clear qualitative examples of the effects the author's proposed improvements have on empathetic dialogue.", "reasons_to_reject": "1. The main limitation of this work, as the authors mention, is the focus on evaluation with only the EmpatheticDialogues dataset. Conducting similar evaluations across more diverse datasets would be needed for more generalizable results. \n2. The human evaluation has few annotators, which can bias the evaluation towards a narrow definition of empathy. \n3. Within the human evaluation, the provided instructions on ranking empathy are not clear, and it appears that sub-dimensions of empathy are not included in the evaluation, which could improve interpretability of the results.\n4. Discussion of the analysis on using LLMs to simulate human evaluators is cursory, and would warrant more analysis beyond the scope of the contributions of this work. While this evaluation is timely, correlation alone cannot demonstrate that LLMs effectively simulate empathy reasoning, and the results of this analysis do not clearly bolster the contributions of this work.", "questions_to_authors": "A. Did you ask human evaluators to provide rationales for why certain dialogues were more empathetic than others? This would be interesting supplementary analysis to further interpret the contribution of the 3 proposed improvements to empathetic response generation.\n\nB. How did combining the 3 improvement methods influence overall performance?", "paper_topic_and_main_contributions": "In this work, the authors conduct a comprehensive analysis of empathetic response generation and comparison of 3 proposed methods to improve empathetic dialogue: selecting in-context examples using semantic similarity, using chain of thought reasoning in speculating the user's emotions, and augmenting knowledge using knowledge graphs. The authors conduct both automatic and human evaluation across a variety of empathetic dialogue metrics and additionally assess the viability of using LLMs such as GPT-4 to simulate human empathy evaluations.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "1. Ashish Sharma, Adam S Miner, David C Atkins, and Tim Althoff. 2020. A computational approach to understanding\nempathy expressed in text-based mental health support. arXiv preprint arXiv:2009.08441.", "Typos_Grammar_Style_and_Presentation_Improvements": "1. Would be helpful to add * indicating statistical significance for the human evaluation results tables."}, "extra_text": {}}, {"review_id": "O2uBOaZpt7", "reviewer": "Reviewer_h8tQ", "summary": "", "questions": "Is the \\oplus in equation 7 meant to signify concatenation in this case?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts!  We will explain your concerns point by point.\n\n**About the statistical significance testing**, we tested all the human evaluation metrics, and the p-value is less than 0.01. We employed **the ''ttest_ind'' function of the ''scipy.stats'' library**, and our evaluation code and results can be published. Regarding the automatic evaluation results, we can observe **the internal comparison of the SOTA series (the upper part of Table 1)**, and our improvement is extremely significant.\n\n**About the technical novelty**, our improvement methods for LLMs mainly adopt **gradient-free techniques**, which are based on the paradigm shift brought by the emerging capabilities (ICL, CoT, etc.) of LLMs. This has been rarely studied in past empathetic dialogue work. We have **designed a prompt framework** for LLMs to apply in the field of empathetic dialogue. Among them, prompts can be **automatically and adaptively** changed according to different methods.\n\n**About the question**, yes, the \\oplus in equation 7 means the concatenation of the five corresponding commonsense inferences. We will add this explanation to the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Show improvement over existing sota on this benchmark\n- Introduce three methods of improving chatgpt for this task which seem to be mostly successful in improving performance\n- Comprehensive automatic and human evaluations.  Experiments and metrics seem reasonable\n- Generated output in Table 5 looks very impressive", "reasons_to_reject": "- Some of the results tables claim that statistical significance testing has taken place, but it’s unclear which things are being compared in the statistical tests.   This seems important to clarify since some of the differences are a bit smaller.\n- This paper is empirically significant in establishing a new SOTA on this benchmark, but the technical novelty of the approach is more limited.  It’s not proposing a new modelling approach, and the ideas for improving over chatgpt (e.g. two stage reasoning with emotion prediction, using a knowledge base, etc) are similar to prior approaches that have been successful in improving smaller models on this dataset.", "questions_to_authors": "Is the \\oplus in equation 7 meant to signify concatenation in this case?", "paper_topic_and_main_contributions": "This paper investigates using more recent large pre-trained language models for empathetic response generation.  They show that basic chatgpt adapted for this task improves on the current SOTA and that this can be further improved with enhancement via: (1) similar in-context learning, (2) two-stage reasoning about the conversation before generating and (3) by adding external knowledge graph information.  They show improvements using both automatic and human metrics. ", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "olCDJOvggz", "reviewer": "Reviewer_emhy", "summary": "", "questions": "1. Are the evaluation metrics biased toward the length of the generated output? It seems like human evaluators prefer generated outputs that are lengthier. Could you try limiting the output length from LLMs to the average target reference length in the dataset and recheck the numbers?  \n\n2. The ChatGPT + SS ICL shows the best automatic eval metrics in Table 3 but is less preferred by human evaluators compared to the other strategies in Table 4 (two-stage knowledge). It would be great to know the author's thoughts about this.\n\n 3. It seems like there are many ties in the A/B testing for i) ChatGPT + SS ICL vs. ChatGPT and ii) ChatGPT + Knowledge vs. ChatGPT. Can you discuss why this is the case?  \n\n4. For in-context learning, did you select a consistent set of exemplars for all the test set (based on average similarity from equation 4), or did you select different exemplars for different instances of the test set?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts!  We will explain your concerns point by point.\n\nQuestion 1:\n\nIn fact, before conducting the experiments, we had thought that the output length of LLMs may have an impact. Initially, we considered calculating the average length of target responses in the training set to limit the output length of LLMs. We **come up with two methods: one is to set the parameter \"max_tokens\" when calling the API, and the other is to add a description of the length limit in the prompt. However, both of the above methods have problems**. Most directly, **the number of tokens is not equal to the number of words**, which depends on the tokenization algorithm adopted by the LLMs. The method of setting the 'max_tokens' parameter will cause LLMs to **directly truncate** the output upon reaching the maximum number of tokens, resulting in semantically incomplete outputs. The method of limiting the output length through the prompt is influenced by **the ability of LLMs to follow instructions**, and the actual response length obtained may also **deviate** from the required one. Furthermore, if the length of the output directly affects human preferences, then designing small language models (SLMs) can also consider producing longer responses. In summary, we ultimately chose the method of stopping the output when LLMs generate the termination token.\n\nQuestion 2:\n\nMost automatic metrics currently used are based on **the proximity to the ground truth**. However, the response that satisfies users in open-domain dialogue is not unique, which is also a widespread problem in open-domain dialogue. Human evaluation is closer to real users but costly and time-consuming. Given the outstanding performance of LLMs, this paper also explores the possibility of using GPT-4 to simulate human evaluators. The attempt is preliminary but has shown potential.\n\nQuestion 3:\n\nIn the A/B test, there are indeed many ties between ChatGPT and improvement methods. We speculate that this is because **when the results have reached the satisfaction standard of the evaluators, they tend not to make excessive distinctions**. How to set up a more fine-grained and discerning human evaluation method is a valuable and meaningful question.\n\nQuestion 4:\n\nWe compute the semantic similarity between each instance (**excluding ground truth responses**) in the test set and each instance in the training set. For each instance in the test set, related exemplars are **automatically selected** based on the similarity values. That means it's **the latter**. \n\nAbout the missing references, thank you for pointing them out. We will add them to the revised version in time.\n\nAbout the presentation improvements, thank you for your advice. We will add the annotated target responses from the original dataset to the tables showing the case study.\n\n**About the generated knowledge snippets from COMET**, taking Table 5 as an example, the knowledge inferences generated by this instance are:\n\nx_intent:['to talk to her', 'to make a decision', 'to talk about it', 'to talk to someone', 'to discuss the issue']\n\nx_need:['to talk to her', 'to get to know her', 'to talk to her .', 'to have a conversation', 'to talk to someone']\n\nx_want:['to talk to her', 'to talk about it', 'to make a decision', 'to have a conversation', 'to talk to someone']\n\nx_effect:['gets stressed out', 'get a job', 'none', 'gets stressed', 'gets nervous']\n\nx_react:['relieved', 'happy', 'sad', 'nervous', 'anxious']", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Clear discussion of problem statement and proposal of three fundamental techniques for improved empathetic response generation.\n2. Improved results over several baseline methods in both automatic and human eval metrics.\n3. The examples shown in paper indicate that the proposed method is powerful and could be explored further for the problem of empathetic response generation. ", "reasons_to_reject": "1. A weakness is that the evaluation could have been more through. Although expected, it is unclear which components bring the improvement in performance. Are the in-context example responses very similar to the target test references? What happens when you try to use all three of the proposed techniques together (ChatGPT + SS ICL + Two-stage + Knowledge)?\n2. Some important details about the proposed components could have been added to the paper. For example, how does the generated knowledge snippets from COMET in section 3.3.3 look like? ", "questions_to_authors": "1. Are the evaluation metrics biased toward the length of the generated output? It seems like human evaluators prefer generated outputs that are lengthier. Could you try limiting the output length from LLMs to the average target reference length in the dataset and recheck the numbers?  \n\n2. The ChatGPT + SS ICL shows the best automatic eval metrics in Table 3 but is less preferred by human evaluators compared to the other strategies in Table 4 (two-stage knowledge). It would be great to know the author's thoughts about this.\n\n 3. It seems like there are many ties in the A/B testing for i) ChatGPT + SS ICL vs. ChatGPT and ii) ChatGPT + Knowledge vs. ChatGPT. Can you discuss why this is the case?  \n\n4. For in-context learning, did you select a consistent set of exemplars for all the test set (based on average similarity from equation 4), or did you select different exemplars for different instances of the test set?\n", "paper_topic_and_main_contributions": "The paper presents a set of methodologies for empathetic response generation with large language models (LLMs). OpenAI’s GPT family models, which often have excellent conversational skills, are used as the LLM. The paper benchmarks the following strategies for improved empathetic response generation: i) semantically similar in-context learning, ii) two-stage interactive generation, and iii) improvement with commonsense knowledge bases. The proposed methods are evaluated on the widely used Empathetic Dialogues dataset.\n\n  The paper is easy to follow and tackles an important problem. The evaluation could be more thorough and will add more value to the paper. Some relevant references are missing. The result is intuitive and probably expected, as the LLMs used in the paper have shown excellent results in many related tasks. However, it’s always good to have a thorough study with a comprehensive evaluation. In that sense, I think the paper has thoughtful experiments, good evaluation, and solid contributions overall. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Tables 5 and 8 should include the reference target sentence originally annotated in the dataset."}, "extra_text": {"Missing_References": "1. This paper uses a technique related to in-context learning with exemplars: Majumder, Navonil, et al. \"Exemplars-guided empathetic response generation controlled by the elements of human communication.\" IEEE Access 10 (2022): 77176-77190. \n\n2. A concurrent work on empathetic response generation with LLMs: Zhao, Weixiang, et al. \"Is ChatGPT Equipped with Emotional Dialogue Capabilities?.\" arXiv preprint arXiv:2304.09582 (2023)."}}, {"review_id": "qiFEdpfLws", "reviewer": "Reviewer_UTmi", "summary": "", "questions": "(1) Line 317, do you concatenate commonsense inferences with respect to all the five relationship types with the dialogue context? If so, how do you know that all the relationship types are relevant to the dialogue context?\n\n(2) I don’t see the algorithm you mentioned in Line 318 - 312, “We also design an algorithm to construct the suitable prompt ….”\n\n(3) Your analyses are only limited to ChatGPT. How about other LLMs, will the ideas be generally applicable to other LLMs?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts!  We will explain your concerns point by point.\n\nQuestion 1:\n\nYes, the commonsense inferences of the five relationships are concatenated with the dialogue context. The **commonsense inferences are obtained dynamically based on different dialogue contexts**, or 'none' if not present. The obtained commonsense knowledge **is related to the context, but the degree of relevance between different types of knowledge and dialogue contexts varies**. This tests the reasoning ability of LLMs, and how to use more appropriate knowledge to generate responses. If LLMs are not good at selecting relevant knowledge, we can also filter the knowledge with other small models that are good at this ability, and then provide it to LLMs.\n\nQuestion 2:\n\n**Our algorithm will obtain corresponding commonsense inferences based on different dialogue contexts and dynamically concatenate them into the prompt**. Therefore, each instance in the test set will have a suitable prompt. The composition of the prompt remains unchanged, but its content will change accordingly. We will explain it more clearly in the revised version. Our code will be open source.\n\nQuestion 3:\n\nOur main experimental LLMs **include ChatGPT (gpt-3.5-turbo), GPT-3.5 (text-davinci-003), and GPT-3 (davinci)**. During the human evaluation, considering the cost of manual labor, we chose ChatGPT as the representative of LLMs, which is of epochal significance and has excellent automatic evaluation results. **The ideas can be applicable to other LLMs, since ICL, CoT, and instruction following are common capabilities that LLMs of a certain scale possess**. Of course, the strength of these capabilities will be different, and it is worthy of further experimental exploration in the future.\n\nAbout the motivations for the improvement methods, in each part of Section 3.3, we **have a brief description, specifically in Lines 220-225, Lines 256-265, and Lines 287-297**. However, as you pointed out, we may need to integrate more closely with relevant theoretical foundations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) The experiments are thorough and demonstrate the effectiveness of the authors’ proposed ideas.\n\n(2) The clarity of the paper is good.", "reasons_to_reject": "The contribution is incremental. The proposed ideas to augment ChatGPT have been extensively studied in prior works for other tasks. For example, using semantically similar exemplars for in-context learning [2,3], CoT prompting [4,5], and using external commonsense knowledge [1]. The authors should highlight the motivation of applying such ideas in the context of empathetic dialogue generation, the unique challenges of empathetic dialogue generation, and why applying such ideas can help address the challenges.\n\n[1] Sabour, Sahand, Chujie Zheng, and Minlie Huang. \"Cem: Commonsense-aware empathetic response generation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 10. 2022.\n\n[2] Rubin, Ohad, Jonathan Herzig, and Jonathan Berant. \"Learning To Retrieve Prompts for In-Context Learning.\" Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.\n\n[3] Liu, Jiachang, et al. \"What Makes Good In-Context Examples for GPT-3?.\" Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures. 2022.\n\n[4] Kojima, Takeshi, et al. \"Large language models are zero-shot reasoners.\" Advances in neural information processing systems 35 (2022): 22199-22213.\n\n[5] Wei, Jason, et al. \"Chain-of-thought prompting elicits reasoning in large language models.\" Advances in Neural Information Processing Systems 35 (2022): 24824-24837.", "questions_to_authors": "(1) Line 317, do you concatenate commonsense inferences with respect to all the five relationship types with the dialogue context? If so, how do you know that all the relationship types are relevant to the dialogue context?\n\n(2) I don’t see the algorithm you mentioned in Line 318 - 312, “We also design an algorithm to construct the suitable prompt ….”\n\n(3) Your analyses are only limited to ChatGPT. How about other LLMs, will the ideas be generally applicable to other LLMs?", "paper_topic_and_main_contributions": "Topic: This paper studies empathetic dialogue generation leveraging large language models, in particular, ChatGPT. \n\nResearch problem: The backbone models adopted by previous studies on empathetic dialogue generation are generally small in scale limiting their empathetic and language understanding capability. The recently-emerged ChatGPT has demonstrated extraordinary performance in a wide variety of tasks. Yet, its capability in the task of empathetic response generation remains underexplored.\n\nContributions: The authors propose three different methods to augment ChatGPT so as to boost its empathetic generation capability compared to zero-shot inference: (1) retrieve semantically similar exemplars for in-context learning; (2) two-stage interaction, i.e, a chain-of-thought style of prompt whereby the model is first instructed to infer the emotion and situational contexts, next, based on the inferred emotion and situational contexts, the model is further instructed to generate the empathetic response; (3) augment the dialogue context with external commonsense knowledge-base, similar to what has been proposed in [1].\n\n[1] Sabour, Sahand, Chujie Zheng, and Minlie Huang. \"CEM: Commonsense-aware empathetic response generation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 10. 2022.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The research makes a good contribution for empathetic quality in LLMs text. The paper presents a comprehensive automatic and human evaluations. The qualitative examples provided further illustrate the impact of the proposed improvements on empathetic dialogue.
Decision: Accept-Findings