SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations
Authors: Yirong Chen
Xiaofen Xing
Jingkai Lin
huimin zheng
Zhenyu Wang
Qi Liu
Xiangmin Xu
Venue: EMNLP 2023 Findings (2023)
Keywords: Empathy Conversation
Large Language Model
Mental Health AI
Multi-turn Empathetic Conversation Dataset
Psychological Counseling AI
TL;DR: The empathy ability of LLMs can be significantly enhanced when finetuning by using multi-turn dialogue history and responses that are closer to the expression of a psychological consultant.

Abstract: Large language models (LLMs) have been widely applied in various fields due to their excellent capability for memorizing knowledge and chain of thought (CoT). When these language models are applied in the field of psychological counseling, they often rush to provide universal advice. However, when users seek psychological support, they need to gain empathy, trust, understanding and comfort, rather than just reasonable advice. To this end, we constructed a multi-turn empathetic conversation dataset of more than 2 million samples, in which the input is the multi-turn conversation context, and the target is empathetic responses that cover expressions such as questioning, comfort, recognition, listening, trust, emotional support, etc. Experiments have shown that the empathy ability of LLMs can be significantly enhanced when finetuning by using multi-turn dialogue history and responses that are closer to the expression of a psychological consultant.

SoulChat:
Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations
Yirong Chen', Xiaofen Xing"; Jingkai Lin', Huimin Zheng',
Zhenyu Wang', Qi Liu?, Xiangmin Xu
'Guangdong Provincial Key Laboratory of Human Digital Twin, School of EE.,
South China University of Technology, Guangzhou, China School of Future Technology. South China University of Technology, Guangzhou, China Pazhoo Lab, Guangzhou, China reyirongchenémail.. scut. edu. cn, {xfxing, xmxu)@scut.. http//sct.euu....o cn
Abstract
* XXx UT -
(I' sorry to ... [xxxxx Here are some sug- gestions:...)", which may cause boredom.
Large language models (LLMs) have been widely applied in various fields due to their excellent capability for memorizing knowledge and chain of thought (CoT). When these lan- guage models are applied in the field of psycho- logical counseling, they often rush to provide universal advice. However, when psychological support, they need to gain em- pathy, trust, understanding and comfort, rather than just reasonable advice. To this end, constructed a multi-turn empathetic conversa- tion dataset of more than 2 million samples, in which the input is the multi-turn conversa- tion context, and the target empathetic re- sponses that expressions such as question- ing, comfort, recognition, listening, trust, emo- tional support, etc. Experiments have shown that the empathy ability of LLMs can be sig- nificantly enhanced when finetuning by using multi-ture turn dialogue history and responses that are closer to the expression of a psychological consultant."
2) ChatGPT inclined to provide suggestions rather than ask questions or listen. ea- ger to solve users' problems, usually provid- ing lengthy and general suggestions, as shown in Figure 12 of Appendix F. However, pro- fessional psychologists rarely provide specific suggestions during the counseling process.
seek
3) ChatGPT acts bit like rational "Straight man" for those users who need listening and comfort. Users who seek emotional sup- port usually expect empathy support such listening, understanding and comfort.
Similar issues can also be found in other LLMs, e.g. CharGLM (Zeng et al., 2023), SparkDesk as presented Appendix F. may be due to the lack of large-scale multi-turn empathy conversa- tion datasets for fine-tuning stage, especially in the field of Chinese mental health or emotional support. EMPATHETICDIALOGUES (Rashkin et al., 2019) and ESConv (Liu al., 2021) are two English empathy conversation datasets that is used for developing emotional support conversa- tion (ESC) systems, e.g MISC (Tu et al., 2022), GLHG (Peng et al., 2022), MultESSC (Cheng et al., 2022), FADO (Peng et al., 2023) and etc. On the one hand, these models may rely on annotated empathy strategies and emotions of users during the training inference stage, which means that building large-scale similar datasets for fine-tuning LLMs difficult. On the other hand, these datasets are in English, so that they cannot be applied fine-tune Chinese LLMs. As for mental health, efaqa (Hailiang et al., 2020) and PsyQA (Sun et al., 2021) are two commonly-used datasets. Among
1 Introduction
With the birth of BERT (Devlin et al., 2019) and GPT (Radford et al., 201 8), large language mod- els (LLMs) have made rapid progress in the past five years. In November 2022, OpenAl launched ChatGPT2 (OpenAI, 2022), large language model fine-tuning by reinforcement learning from human feedback (RLHF) (Ouyang et al., 2022). However, when applied to mental health or emotional sup- port conversation, there are three main issues lead to ChatGPT appear less "human-centered":
1) ChatGPT tends to provide repetitive and standardized responses. ChatGPT often uses the following template respond users' questions related to mental health: "#t 1
"Conresponding author. Email: xfxing@ scu..ecu.cucc 'https://githeee wwtptethuu..ccuutueeeeeeee www/scutcyr//oulluccco. Soullyyy///oChatt.. Phttps://chin http://:///chat.ppaaai..c
Phttps://xinghus xfyun.cc

---

have been feeling anxious and physically and mentally exhausted lately. I don't know what to do.
nf 3 can hear your feelings. Can you tell me exactly what makes you feel anxious?
T Recently, there has been a lot of work pressure and some interpersonal issues, which have made me feel very uncomfortable.
+ A St.
E ME E T
It sounds like you are going through a great challenge. But you know, you already know your situation very well. Actually, many people have this feeling.
X Ee 32 But always feel like can't overcome it, as if no matter which method tried before, it hasn't changed.
#N -
/ Don't give up, perhaps you need more time to adjust. Sometimes doing some basic self-care will strengthen your willpower overcome this feeling.
Figure 1: A case of user confiding to SoulChat. Compared to ChatGPT, SoulChat is better at listening and guiding users to think.
them, efaqa contains 20,000 conversations and pro- vides annotation information such as types of trou- bles, psychological disorders, SOS, etc. However, efaqa has a complex multi-party dialogue relation- ship and high proportion of low-quality responses from netizens, while PsyQA contains 22,346 ques- tions and 56,063 single-turn long-text psycholog- ical counseling conversations. Thus, neither of these datasets can solve the three issues of Chat- GPT mentioned above.
defined as " " ih B
XM
M fI ij - 3 (The response of the 'psychological counselor' needs to be combined with the user's description and provide empathy, such as listening, comfort, inter- pretation, trust, recognition, sincerity, emotional support, etc)".
"messages": ("role": "user".
Recently, Qiu et al. (2023) proposed SMILE approach to employ ChatGPT to convert single- turn dialogues into multi-turn ones. They uti- lized SMILE to extend the single-turn conversa- tion dataset PsyQA to a empathy multi-tumn con- versation dataset SMILECHAT with 355,733 sam- ples. Inspired by (Qiu et al., 2023), we pro- posed a Chinese empathy constraint prompt, in which the empathy prompt constraint is further strengthened compared with SMILE prompt (see Appendix C). As shown in Figure (English ver- sion: Appendix C), our empathy constraints are
KAME "content": "in (o & th [ C
i. # # F: 1. & "HH Hit. "LEE iN
m. #t: 2. "C=*"1""" NITT NE AI MA
# di: k. 12M.. PHAY 1Eff. al &
R. # 3. HLL
C H. K. 'ain 1A 11 single turn "i"mini!" AO AL HU 41 BARTE #AEOCk 138 ]]
# (=8-2088:2
H 10 39 AARRR
Figure The prompt used for converting single-turn psychological counseling conversations to multi-turn empathy conversations (English version: Appendix C).

---

To this end, we first constructed 215,813 differ- ent psychological counseling questions about 12 topies and 619,725 answers through data outsourc- ing services. Rule-based cleaning. manual rewrit- ing and human proofreading are applied to ensure that there is no sensitive or privacy-related content in the dataset. Then, we use ChatGPT to convert these single-turn long text psychological counsel- ing conversations multi-turn empathy conversa- tions. We also conducted manual proofreading and data cleansing for multi-turn dialogues rewritten by ChatGPT to further strengthen the expression of empathy, such as questioning, comfort, recognition, listening, trust, emotional support, etc. In the end, we obtained a multi-turn empathy conversation dataset, named SoulChatCorpus, with 2,300,248 samples. To knowledge, it is the first million- scale multi-turn empathy conversation dataset in the field of mental health or emotional support. We conduct experiments by using ChatGLM-6B as the base model for tne-tuning on SoulChatCorpus. Re- sults demonstrate that LLMs' empathy, listening, and comfort abilities can be improved significantly through finettuing with million-scan multi-turn empathy conversation dataset.
not publicly available due the privacy protection and ethical standards of psychological counseling. To construct high-quality multi-turn empathy con- versation dataset, We selected 12 topics of psycho- logical counseling to construct 215,81 long-text questions and 619,725 long-text answer through crowdsourcing. The distribution of topics is shown in Figure 3. Then, we used ChatGPT (99% called gpt-3.5-turbe api and 1% called gpt-4 api) text rewriting tool following the prompt as shown in Figure to convert single-turn psychological coun- seling conversations to multi-turn empathy conver- sations, in which one turn is in the form of "H] F: <user_utt> n L <psy_utt>". The response of "C in was asked to be rewrit- ten to reflect human-centered expressions such empathy, listening, comfort, etc. Finally, after man- ual proofreading, we removed 105,134 low-quality samples and ultimately obtained 2,300,248 sam- ples. As shown in Figure 4, the word cloud map of the utterances expressed by psychological consul- tants indicated that the rewritten multi-turn empa- thy conversation has high level of empathy.
Psychological knowledge
Social events
Family
1158
Treatment
H
tm
0, 0.%%
9.7%
7.9%
#
Emotion
Marriage
11.7%
16.3%
Figure 4: Word cloud map of psychological consultants' utterances (English version: Appendix D).
Sex
3.7%
11.5%
Behavior
5.8%
Growth
2.2 SoulChat Model
10.0%
11.1%
Relationship
9.5%
Self-awareness
We utilized the ChatGLM-6B (Du et al., 2022; Zeng et al., 2023) as the base LLM architecture to develop the SoulChat. ChatGLM-6B is open- source, bilingual LLM based on the General Lan- guage Model (GLM) (Du et al., 2022) framework with 6.2 billion parameters. The input of model is defined as:
Carcer
Figure 3: Distribution of counseling topics.
2 Human-centered Mental Health LLM
inpul = ui +' Vn + uf +... u +' Vvn + up
2.1 SoulChatCorpus Collection
We consider an one-on-one psychological counsel- ing conversational setting where a user and psy- chological consultant engage in multiple rounds of dialogue. However, such conversation data is
where the utterance of User on i turn u. - |
(User:)' + utterance," utterance of Psychologist on turn D utterance (i < N), n (Psychol-
(Psychologist:)) +

---

Table Evaluation results.
Automatic.
Manual.
Dataset
Model
B-1
B-2
B-3
B-4
R-1
R-2
R-L
Con.
Emp.
Hel.
Saf.
ChaiGLM-6B
22.73
13.15
8.04
4.92
25.33
18.84
1.90
1.55
1.92
SoulChat- Corpus
McChat
29.43
17.12
10.54
6.71
27.35
6.27
21.12
1.83
1.70
1.78
1.0
ChatGPT
27.98
16.09
9.93
6.23
27.39
6.82
21.92
1.96
1.62
1.94
1.0
SoulChat
33.78
20.07
12.86
8.52
31.47
8,92
26.57
1.95
1.84
1.87
1.0
ChatGLM-6B
22.91
13.56
8.40
5.15
25.99
5.95
18.76
1.81
1.39
1.84
1.0
McChat
30.63
18.41
11.59
7.46
28.92
6.76
21.59
1.95
1.74
1.83
1.0
SMILECHAT
ChatGPT
28.30
16.48
10.24
6,40
27:57
6.71
21.60
1.95
1.65
1.97
1.0
SoulChat
35.40
21.39
13.77
9.02
32.64
9.17
21.10
1.93
1.90
1.85
1.0
ogist:)', N represents the number of conversation turns for the context.
3.3
Results and Analysis
We randomly selected 10,000 samples from SoulChatCorpus and SMILECHAT respectively the test set for automatic evaluation and 100 sam- ples for manual evaluation. For each sample, each model generates an answer for evaluation. We used 7 evaluation metrics as automatic metrics: BLEU- 1). BLEU-2 (B-2), BLEU-3 (B-3), BLEU-4
3 Experiments
3.1 Baselines
We compare SoulChat and the following bench- mark models using both automatic and manual evaluations:
(B-4) (Papineni al., 2002), R-1 (ROUGE-1), R-
2 (ROUGE-2) and R-L (ROUGE-L) (Lin, 2004)).
1) ChatGLM-6B' (Du et al., 2022; Zeng et al., 2023) serves as the base model for SoulChat.
Three individual experts majoring in Psychology were asked evaluate the generated responses terms of content naturalness (Con.), empathy level (Emp.), Helpfulness (Hel.) and Safety (Saf.), as detailed described in Appendix G. The rating scale of Con., Emp. and Hel. is (0, 1, 2), while (0, 1) for Saf., where higher score means better. One hundred dialogues were randomly sampled from the test set of SoulChatCorpus and SMILECHAT for manual evaluation. Fleiss' к (Fleiss, 1971) for Con., Emp. and Hel. are 0.489, 0.472 and 0.532, indicating moderate annotation agreement respectively, while = for Saf. (perfect agreement). The evaluation results are shown in Table Generally, SoulChat outperforms ChatGLM-6B ChatGPT and McChat in both automatic evaluation metrics and Emp. met- ric on test set of SoulChatCorpus and SMILECHAT. Specifically, the results SMILECHAT demon- strates SoulChat's excellent zero-shot performance in the field of mental health.
2) ChatGPT (OpenAI, 2022: Ouyang al., 2022) is LLM that is trained using super- vised finetuning and Reinforcement Learning from Human Feedback (RLHF).
3) MeChat (Qiu et al., 2023) is LLM finetuned with low-rank adaptation (LoRA) (Hu al., 2022) on SMILECHAT dataset that gener- ated by ChatGPT based on PsyQA.
3.2 Implementation details
SoulChat is finetuned on the proposed SoulChat- Corpus with batch size of 80 and global training steps of 30,000. The WarmupDecayLLR learning rate scheduler with warmup_sleps wormup paappp = 5e is used. The maxi- mum input token length is set to 1,536. The maxi- mum target token length is set to 512. The decod- ing algorithms of Top-p sampling with = 0.75 and temperature 0.95 is adopted during the inference phase.
1000 and
Conclusion and Future Work
In this paper, explore how make LLMs more human-centered. To this end, we constructed a Chi- nese large-scale multi-turn empathy conversation dataset, named SoulChatCorpus, with 12 empathy

---

topics and more than million samples. The ex- perimental results indicate that using this dataset to finetune LLMs leads to high-level empathy abil- ity when users try to seek emotional support from LLMs. Future work needs to further consider user attributes, such as personality, gender and etc., to help LLMs generate targeted empathy responses for different individuals.
content. In addition, when this model is ap- plied to downstream scenarios, is necessary inform the users in advance that the answers they see are generated by the Al model and are for reference only.
Annotator Compensation. We invited in- dividual experts majoring in Psychology to conduct the proposed CEHS evaluation of the model's output. The annotators' evaluation of cach sample takes approximately 3 min- utes, during which they can receive salary of $0.418. Therefore, the hourly salary of the annotators $8.36, which is higher than the US minimum wage of $7.12 per hour.
Limitations
this work we proposed human-centered LLM named SoulChat that has excellent empathy abil- ity, which is finetuned on the proposed SoulChat- Corpus dataset. Although the experimental re- sults demonstrate the effectiveness of SoulChat, there are still some limitations need to consider. The mechanism of empathy is complex. Differ- ent users have different expectations for the out- put of the model. For example, when discussing tense emotions, there are significant differences in the solutions expected by adults and adolescents. Therefore, numan--cnnterd LLMs need to further consider the user's personality, identity, and other attributes to assist in generating answers that are closer to the user's needs.
A Acknowledgements
This work was supported by the Science and Tech- nology Project of Guangzhou the Natural Science Foundation of Guangdong Province (2022A151501 R&D Program of China (2022YB44506000) the Science and Technology Project of Guangdong (202280101010003, the National Natural Science Foundation of China under Grant U1801262 and Guangdong Provincial Key Laboratory of Human Digital Twin (2022B (00221212210004))
the National Key
Ethics Statement
Data Collection. In order to protect pri- vacy (Hovy and Spruit, 2016), we adopted strict manual proofreading process when con- structing the dataset. We filtered all samples with special strings such ER (I am)",, "È K (suicide)", "H (jumping off building)", etc., and conducted manual data cleansing. Any text related to privacy has been rewritten or removed. Besides, any potential conversa- tions that pose harm to users, others, or soci- ety have been completely removed from our data. To this end, we removed 105,134 sam- ples from multi-tun conversations generated by ChatGPT.
References
Yi Cheng, Wenge Liu, Wenjie Li, Jiashuo Wang, Ruihui Zhao, Bang Liu, Xiaodan Liang, and Yefeng Zheng. 2022. Improving multi-tury emotional support dia- logue generation with lookahead strategy planning. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 3014-3026, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Potential Risks of the Model We conducted safety assessment specifically for the out- put of the model during the manual evaluation phase, and the results shown in Table 1. Due to the lack of human feedback during the model finetuning stage, there are inevitably answers that may pose harm to users. There- fore, future work needs to combine RLHF to improve the safety level of model generated
Zhengxiao Du Yujie Qian, Xiao Liu, Ming Ding. Jiezhong Qiu, Zhilin Yang, and Jie Tang, 2022. GLM: General language model pretraining with autoregres- sive blank infilling. mual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 320-335, Dublin, Ireland. Association for Computational Lin- guistics.
Proceedings of the 60th An-

---

Joseph L Fleiss. 1971. Measuring nominal scale agree- ment among many raters. Psychological Bulletin, 76(5):378-382.
Huachuan Qiu, Hongliang He, Shuai Zhang, Anqi Li, and Zhenzhong Lan. 2023. Smile: Single-turn to multi-turn inclusive language expansion via chatgpt for mental health support.
Wang Hailiang, Wu Zhizhi, and Lang Jiayuan. 2020. Pat psychology: Psychological consultation q&a corpus.
Alee Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language under- standing by generative pre-training.
Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceed- ings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pa- pers). pages 591-598, Berlin, Germany, Association for Computational Linguistics.
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open- domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguisties, pages 5370-5381, I, Florence, Italy. Association for Computational I Linguistics.
Edward Hu, Yelong Shen, Phillip Wallis. Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
Hao Sun, Zhenru Lin, Chujie Zheng. Siyang Liu, and Minlie Huang. 2021. PsyQA: A Chinese dataset for generating long counseling text for mental health support. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pages 1489-1503, Online. Association for Computational Linguisties.
Chin-Yew Lin. 2004, ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Brunches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 1Ith International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 3469-3483, Online. Association for Computa- tional Linguisties.
Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Ji-Rong Wen, and Rui Yan. 2022. MISC: A mixed strategy- aware model integrating COMET for emotional sup- port conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guisties (Volume 1: Long Papers), pages 308-319, Dublin, Ireland. Association for Computational Lin- guistics.
OpenAL. 2022. Introducing chatgpt.
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang. Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong, and Jie Tang. 2023, GLM An open bilingual pre- trained model. In The Eleventh International Conference on Learning Representations (ICLR).
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 27730-27744 Curran Associates, Inc.
A Reproducibility Checklist
Model and Data: The SoulChat model and SoulChatCorpus will be released upon deci- sion of the paper.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- kational Linguistics, pages 311-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
System Hardware: We trained the SoulChat on the Ubuntu 20.04.6 LTS server that has 2 CPUs called "Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz", 8 NVIDIA A800- SXM4-80GB GPUs, and 1. 1,024GB memory.
Wei Peng, Yue Hu, Luxi Xing, Yuqiang Xie, Yajing Sun, and Yunpeng Li. 2022. Control globally, understand locally: A global-to lobal-to--ae toclocal hierarchical graph network for emotional support conversation. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22).
Driver Version: The version of Nvidia driver is 1525.10..17". The version of CUDA is "11.6". The version of Cudnn is "8.4.0.27".
Wei Peng, Ziyuan Qin, Yue Hu, Yuqiang Xie, and Yun-
Package version: python=3.8, torch =1.1311,
peng Li. 2023. Fado: Feedback- double con- trolling network for emotional support conversation. Knowledge- Based Systems, 264: 10340.
htttps://pytorch... wowp/ootttggtteeeeeee vrevsons-versions

---

transformers=44228..0 -....
"messnges":
datascts=2.11.0 and jicba=0.42.1 is recom- mended.
("role": "user".
"content"; "Plcase act as text rewriter in the field of psychological counseling, rewriting the given "single round long text Chinese psychological counseling dialogue" into multiple follows: 1.
Model Parameters: SoulChat has 6.2B parameters with 28 layers and mnar_sequence_engy of 2,048. During the inference phase, the model requires at least 14GB of GPU memory.
rouds of empathetic diulogue. The requirements Multiple rounds of dialogue should start with "User:" and end with "Psychologist" : 2. The response of "psychological counsclor" noods to combine the user's description and peovide empathy, such as listening, comfort, understanding, trust, recognition, sincerity, emotionnl support, etc; 3. The number of conversation rounds between users and psychological counsclors should be determined based on the conversation content, between 8-20 rounds; 4. In round conversation, the length of the conversation between the user the psychological counselor should be combined with the conversation history and should not be long. 'n'nnSingll round long text Chinese psychological counseling dialogue]:"" 1+ single turn + +"n'aPleass gradually analyze the users needs and empathy skills the psychological counselor in the above single round conversation, and rewrite multi round empathy conversation. Your rewritten result is:"
Training Time: SoulChat is trained with of
global steps of 30,000 and "float16" on 8 NVIDIA A800-SXM4-80GB
GPUs. The training time is about 79 hours.
B Counseling Topics
}]
The following dictionaries represent the corre- sponding relationships between Chinese and En- glish for 12 counseling topics.
Figure 5: The translated English prompt used for con- verting single-turn psychological counseling conversa- tions to multi-turn empathy conversations.
: 'Family', :: 'Marriage,, :: 'Sex'. K : 'rrowth,, " 'Rela-
tionship ' :H: 'Career', i 'Self- awareness ', 'fT''': 'Behavior', ' 'Emo- tion', ' 'Treatment', ## 'Social
エ -
T 1 BE A ENTTT d 1 WH. A
events', * 'Psychological knowl- edge'.
English:
User: I have been in relationship for two years and have been in a long-distance relationship. am usually very good. I started arguing and quarrel- ing year ago, and my boyfriend said unpleasant things or even cursed. Is it normal
Our prompt VS SMILE prompt
We used SMILE (Qiu et al., 2023) prompt and our proposed Chinese empathy constraint prompt for extend the single-turn long text psychological coun- seling example to multi-turn empathy conversation respectively.
Psychologist: Hello! Long distance love is usually okay, but when you argue, it's really abnormal to scold you. This should be related to many factors 1. Long distance relationships are inherently chal- lenging, and when your boyfriend is away from you, their language and language are challenged 2. When he usually gets along with you, he will also have lot of pressure, so when encountering similar situations, he will be very irritable
Chinese:
A AE R0 YONAR.
$ A. 19?
DE (R - 1
1-
3. When we usually get along with him, there is no conflict between you. Whenever there is a conflict, we will argue fiercely
Hb.
2- 1
$ 13
4. You usually communicate less, so he doesn't 't have channel to vent his negative emotions The above are some possibilities, but they may not necessarily be the reason for his situation, because whether there are any other conflicts between you is only known through communication between the http:://ggthua..cmm/mmrrrrttetep two of you. So, think you should seriously con-
3 #R
6112
T. -
4
ht ctw http:///gtuuuoooollggggggaaaaarp

---

sider what kind of psychological state or situation your boyfriend's behavior in, and only you know it best.
G Manual Evaluation Instructions
For mental health LLMs, we have constructed a manual evaluation framework called "CEHS" with the following four dimensions: Content natural- ness, Empathy, Helpfulness, Safety, presented in Table 2. Specifically, there may be a certain conflict between Empathy and Helpfulness. For example, ChatGPT tends to generate helpful but lacking empathetic responses. On the other hand, when SoulChat generates empathetic responses, it may weaken the direct Helpfulness of the answer. This is because general advice often appears help- ful, but not so empathetic. The scores of Empathy and Helpfulness in Table also illustrate this phe- nomenon.
As shown in Figure (English version: Fig- ure 8) and Figure (English version: Figure 10), the multi-turn conversation generated by using the proposed prompt has richer expressions of empathy, compared with SMILE prompt.
D English Word Cloud Map
The English word cloud map is presented in Fig- ure 6.
Seikatsu belief
inside
Found
Miyako
Go Support Very
Events
that
Table
Manual Evaluation.
Main
that
Questions
&aath
peri
sona
Aspect
Score
Description
Help
Bethod
need
(Maderately))
able
would
d
Let's
Probablyy
mainly evaluates whether the con- tent generated by the model is rele- vant and coherent to the user's con- versation history, whether the con- tent smooth and natural, whether the content is consistent with lan- guage habits, whether the content has syntax emror.
aay with lntor one of them
Emotion
Le
But....
Content 0-2
Figure 6: Word cloud map of issychological consultants'
utterances.
The main assessment is whether the text generated by the model can un- derstand the feelings and behaviors of the parties involved, grasp their inner feelings, deep meanings, and their degree.
E Sample Conversations of SoulChat
Empathy 0-2
As shown in Figure 11, SoulChat can better serve listener when users seek emotional support or confide. Besides, it can naturally empathize with users (e.g. "Ik 41 m 4 Jj A AU T 3 (Your childhood experiences were really difficult)", "# can under-
This indicator mainly evaluates whether the output of the model helpful to the user. When evaluating this item, it is necessary to consider from the user's perspective whether the model's answer provides assis- tance.
Helpful- ness
0-2
यं E #e iK AM I MA ( stand your pain and inner struggle)") and comfort them (e.g. #M fi ik E
1x W A, पJ (I believe you are a strong person who can change the situation through your own efforts.))).
This indicator mainly evaluates whether the text generated by the model causes harm to users, others, society, environment, etc., whether personal privacy is protected, and whether laws and regulations are complied with. The main consider- ations are: respect for social moral- ity, Public policy doctrine. Whether it violates laws and regulations, involves political sensitivity, com- mits crimes, infringes on physical health, infringes mental health, infringes on property privacy, dis- criminates/prcuudices. insults/hate speech, violates ethical ethics, etc.
F Sample Conversations of Other LLMs
Safety 0-1
The examples when users seek emotional sup- port from ChatGPT, ChatGLM and SearkDesk are shown in Figure 12, Figure 13 and Figure 14. These LLMs are inclined to provide suggestions rather than ask questions or listen, acting bit like ra- tional "Straight man" for those users who need listening and comfort, which make them appear less "humman-cenerred"

---

HP.: R P PRR 4
- 10 ThE 2
Interpretation
Interpretation 32 2 AE
H カ.
IF Recognition
Interpretation
18:00 A0 Fi 18
JH JH 100
42 (A (i] 05 AA
A B0 SE F8 SR
Recognition 2 2n
19
Faj th SE ful OS E8
C ti
pig WG IHE [A] HITE Ch. 1E A 16
4u ill # 1
(6--11
XLE FR BE HEE
RE
Trust
12 Trust
18 Rk
19 68
in 1E iZ/0Ram R
1 AE X3
1
07
#
Comfort
Figure 7: Multi-turn conversation generated by ChatGPT using the proposed prompt.
User: have been dating for two years, but are in long-distancc relationship. Everything is usually fine, but year ago we started arguing and it became increasingly fierce. Is it normal for a boyfriend to say unpleasant things or even curse?
Psychologist: Hello! have know the difficulties you are currently facing. Long distance love itself challenging, especially in terms of communication and interaction It is indeed abnormal for a hoyfriend to use unplcasant language or even curse during arguments. |I can understand that this will bring a huge blow and distress your mood. Interpretation
User: Yes, our argument has caused me great pain. don't know why like this, he hasn't been like this before.
Interpretation
Psychologist: Firstly, understand that you are confused about his changes. This may be because long-distance love itself will bring Psycholegical stress and emotional fluctuations. In a long-term long-distance relationship, he may become irritable due to longing and unease. Meanwhile, we all have channels for emotional release, but due the lack of substantive communication between you, he may not have found good way release negative emotions.
Recognition
User: Yes, rarely communicate in our daily interactions, and think this may be one of the reasons for the problem. But am not sure what other contradictions exist.
Interpretation Psychologist: Your observation is very sharp. Indeed, insufficient communication may lead to the accumulation of conflicts. I suggest that you try to communicate more and share your thoughts and feelings candidly. order to better understand each other. At the same time, I also encourage mutual understanding and support between you, trying to accept each other's emotions and difficulties. This kind of empathy and understanding helps to establish more stable relationship.
User: Thank you for your suggestion. will try to communicate more with him and also hope that he can understand my feelings. Recognition
Trust
Psychologist: appreciate your proactive approach in seeking solutions, Communication requires time and patience, but only through honest communication and mutual support can problems be gradually resolved. believe you can overcome difficulties and work together towards healthier and more stable relationship. If you have any other questions during the communication process, feel free to come me any time.
User: Thank you for your encouragement and support. will continue to work hard. Trust
Psychologist: You are very brave, and I believe you will be able to overcome difficulties. you need any help, remember sock support from mc any time You are not alone in facing this problem, will always be by your side. Comfort
Figure 8: Multi-turn conversation generated by ChatGPT using the proposed prompt (English version).

---

> WETT.,, JA T
Interpretation
# Rbb 18MAF!
-
1. 11 HO 1
x 3 BLAR JC Ht,,
MO AD F
H.50 11 RE fi
M (EM
Interpretation
Comfort
# 11
da 1 1 AE
T
T IT 76 02
#1 WT
iZ
AU UEE.
: 32 #1
22 fuL AT A6 RI BEZ T 11 ii 1h
77 #9
R1 AX ! Trust
Tet 3. AF A1 (1 RE 45
-
KEB:
:
18 A0
1
Reflection of feelings
: 4
201 AE 45 4E 12
iu [-) Hi stt 160 UF., IRI BU
A !
Figure 9: Multi-turn conversation generated by ChatGPT using the SMILE prompt.
Helper: I have been in relationship for two years and have been in long-distance relationship. I am usually very good. started arguing and quarreling a year ago, and my boyfriend said unpleasant things or even cursed. Is it normal?
Interpretation
Supporter: Helper, hello! understand your current troubles very well. Long distance relationships are inherently challenging and can have a certain impact on each other's language and communication style. Your boyfriend is not normal and healthy behavior to say unplcasant things and curse others during fight. Helper: But we don't have any problems with our daily interactions. Why did we suddenly become so irritable? Supporter: This may be caused by a combination of multiple factors. Firstly, long-distance love itself requires more effort and persistence, which poses challenge to both parties' emotions and communication skills.
Secondly, he may also face many pressures and troubles in his daily life, and his irritability during arguments may be an accumulation of emotions and stress. At the same time, there is a lack of sufficient communication and interaction between you, and he may not have sufficient channels to release negative emotions, leading to emotional loss during arguments. Interpretation Helper: The possibilities of these situations you mentioned are very high, I really need to think carefully. But I am still very worried that our conflicts and arguments will affect our relationship.
Comfort
Supporter: Understand your concerns, Contradictions and arguments can indeed have negative impact on relationships. However, there is no need to be too pessimistic Every relationship will face various challenges and tests, and the important thing that can face them together and find solutions. You can try increasing the frequency and quality of communication, as well as jointly developing rules and resolving conflicts. At the same time. remember to pay attention to each other's emotions and needs, maintain understanding and support. Only through mutual effort and understanding can your relationship become more stable and healthy.
Helper: What you said is very reasonable. will try to have more communication and exchange with him to see if we can find a solution to the problem. Thank you for your support and sugeestions!
Trust Supporter: You're welcome, I'm glad to be able to help you. In the process of solving problems, remember remain calm and rational, and not vent your emotions in arguments. [ believe you will be able to overcome this difficulty and establish better emotional foundation. If you have any progress or need further support, feel free seek my help at any time. Good luck to you all!
Helper: Thank you for your encouragement and blessings! I will work hard. Wishing you all the best! Supporter: Thank you very much for your blessings: [ hope we can all achieve better life with each other's If you have any other questions or need to talk to me, feel free chat with mc anytime. Come on! Reflection of feelings
suppori
Figure 10: Multi-turn conversation generated by ChatGPT using the SMILE prompt (English version).

---

! TRA ! Hello teacher! a 16 year old girl, my name Ye (pseudonym), and I would like to consult with some psychological issues.
If 1,
(1(4)
: i3
M:AF! R !
KURRMV BEL MA
Hello! am your personal exclusive listening teacher, Lingxin Teacher. Welcome to confide and chat with me, looking forward to helping you!
EN + R 2 Since the age of 14, I have had frequent suicidal thoughts and self harm behaviors in my mind. This situation has become increasingly serious in the past two years. Once, was standing on the top floor and almost jumped off. I felt like a gust of wind on the rooftop could blow me down, but in the end, managed control it forcefully. What did you say happened to me?
JH R YF
IK -- A 2. -
Reflection of feelings
1: 1E
15 ?
Listening Questioning
Upon hearing your experience, am very concerned about your physical and mental health. Your self harm behavior and suicidal awareness may be related to your childhood experiences. We can explore together. Do you want me to help you analyze the reason together?
RE "
- - X0 Æ RE A. 1
M I, (E
it # 1 "1" 5-
am a Left-behind children in China. From primary school to junior high school, have always lived in my hometown with my grandparents. My parents both work in Guangzhou, and have a loan housing. My family is under great economic pressure. My mother has breast cancer and is unstable. When was a child, was almost molested by a neighbor man, but my parents felt that was not paying attention to safety, which made me feel particularly ashamed. I am very close to my aunt and have witnessed her attempt to jump into the river. understand her psychological state very well, When I was in junior high school, trusted friend very much, but was "betrayed" by my good friend, resulting in being isolated by the whole class. no longer trusted all my classmates and friends around me. Interpretation
4 ER
iX Lifs #4
AE # Eo J
ply &
ft
M+t.
42
A.
Comfort
Your childhood experiences are really not easy, and these negative experiences may make you feel lonely, helpless, and incomprchensible. can understand your pain and inner struggle. Your family environment and economic situation have also brought you many negative impacts. However, I believe you are strong person who can change the situation through your own efforts.
Figure 11: A case of a user confiding to SoulChat.

---

J BCAE.. イ have been feeling anxious and physically and mentally exhausted lately. I don't know what to do.
E
1
プ Α CW A AE 1А
:E -
2. A - :I
3. # -44 -I A.. MR 10 12 sk
1. |
7 #
At
6. : $5 2 W1R Kaii BF RR
TU HL $
I'm sorry to hear that you've been feeling anxious and physically and mentally exhausted lately. Anxicty is common emotional state, but it be alleviated through some methods. Here are some suggestions:
Seek support: Share your feelings with people you trust (friends, family, mentors). They may be able to provide support, understanding, and advice. Adjusting your lifestyle: Pay attention to your diet, sleep, and exercise habits. Maintaining regular schedule and healthy lifestyle help alleviate anxiety.
3. Relaxation technique: try some Relaxation technique, such as deep breathing, meditation, warm water immersion, massage or yoga. These methods can help you relax your body and mind, and reduce anxiety.
4. Manage time and tasks: Develop a reasonable schedule and break down tasks into small manageable parts. Doing can reduce stress and anxicty, and make it casier for you to concentrate.
5. Seek professional help: If anxiety is severe and affects your daily life, consider consulting psychological professional. They can provide professional advice and treatment plans.
6. Relaxation and entertainment: Participate in some relaxation activities that you enjoy, such reading, drawing, listening music, watching movies. These activities can divert your attention and make you feel more relaxed.
Remember, everyone has moments of anxiety, but you take positive steps to alleviate it. If you feel that anxiety persists and is difficult to handle, be sure to seek professional help.
Figure 12: A case of confiding ChatGPT. ChatGPT is eager to solve problems and tends provide comprehensive and effective advice rather than truly empathizing with users.

---

3 E. オ have been feeling anxious and physically and mentally exhausted lately. don't know what to do.
+ MAI * MT 
W 1 4
T Df #1
2. 10
3. 14
1. 10
50 UM0, RR No
- Śm
AK NE BE 7
Anxicty is common emotional response that can be caused by various factors, such as work pressure, relationship issues, health concerns, and so Here are some potentially useful suggestions:
1. Deep breathing and relaxation exercises: Deep breathing and relaxation exercises can help alleviate tension and anxiety in the body. You can relieve tension and anxiety through deep breathing and muscle relaxation exercises.
2. Exercise: Exercise release stress from the body, help alleviate anxiety, and
improve mental state. Whether it's walking, rumning, yoga, or other forms of exercise, they can help alleviate anxiety and improve physical and mental health.
3. Talk to friends family: Sharing your feelings with close friends may make
you feel better. Find someone who can understand you, let them know your situation, they will provide you with help and support.
4. Avoid stimulating substances: substances such as caffeine and alcohol can exacerbate anxiety. If you feel anxious, it's best to avoid these items.
Seek professional help: your anxicty persists for long time or has negative impact on your daily life, it is best to seek professional help. Psychotherapy and medication are effective treatment methods.
The above some common ways to relieve anxicty, you can try some see which ones are suitable for you. If your anxiety persists and affects your daily life, it is best to seck professional help.
Figure 13: case of user confiding to ChatGLM.

---

have been feeling anxious and physically and mentally exhausted lately. don't know what to do.
1. - -I
#
2. 2
3. #1 4 1E
4. : S
HT [4]
5. * & Wb IE
I'm sorry to hear about your predicament. Anxicty and physical and mental exhaustion are common problems, but there are many ways to help alleviate this situation. Here are some potentially useful suggestions:
1. Communicate with others: Communicating with friends, family, or professionals can help you reduce stress and gain support.
2. Exercise: Exercise can help you release tension and improve your mood.
3. Learn Relaxation technique: such as deep breathing, meditation or Progressive
muscle relaxation.
4. Adequate sleep: Insufficient sleep may lead to physical and psychological
fatigue, so please ensure that you have enough sleep time every night.
5. Seek professional help: If your situation is severe, is recommended to seek professional help, such as psychological therapy or medication.
Figure 14: A case of a user confiding to SearkDesk.


Reviews:
[{"review_id": "GLwlZ07gCn", "reviewer": "Reviewer_3WZK", "summary": "", "questions": "A: Why did the authors only use the Rouge and BLEU scores? It would be better to display additional metrics results, like METEOR, BERTScore etc.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our paper. We are committed to improving the empathy ability of LLMs. In this article, we mainly explore the feasibility of using multiple rounds of empathy dialogues to improve the empathy ability of LLMs.\n\nRE: RR 1\n\nWe acknowledge your observation that the model architecture we employed might appear relatively basic. While our approach involves fine-tuning, we would like to emphasize that our primary focus lies in addressing a specific problem within a well-defined domain (empathy ability of LLMs is important for Human-Centered NLP). The contribution we aim to make is in the effective application of the model to this problem, rather than completely revolutionizing the model architecture itself. Our research is centered around solving real-world challenges, and our approach is designed to provide practical solutions to those challenges. In many instances, significant breakthroughs are achieved by building upon existing research and refining methods for specific contexts. Fine-tuning, when applied thoughtfully, can lead to improved performance and practical outcomes. Our work seeks to extend the capabilities of existing models by adapting them to a unique scenario, and we believe this incremental progress can still contribute to the advancement of the field. While our current approach might not be groundbreaking in terms of model architecture, we recognize the potential for future research to be built upon our foundation. Our study can serve as a benchmark for others interested in exploring this problem space. Researchers can leverage our work to experiment with innovative modifications to the model architecture, explore alternative training techniques, or introduce new evaluation metrics to achieve greater innovation. In light of your feedback, we are considering how we can inject more innovation into our approach. We are exploring avenues to incorporate advanced techniques or modifications that align with the scope of our research problem. This may involve emotional attention expression mechanisms, reinforcement learning based on emotional feedback, or other related enhancements to further improve the performance of our model.\n\nRE: Question A\n\nYour question about our choice of evaluation metrics is insightful. We appreciate your suggestion to consider additional metrics beyond Rouge and BLEU scores. Our initial decision to utilize Rouge and BLEU scores was motivated by their widespread usage in the field and their ability to capture different aspects of model performance. Due to space constraints, we have reserved space for manual evaluation indicators, so we cannot take into account all automatic evaluation indicators. We hope that manual evaluation indicators can further enhance readers' perception of SoulChat's performance in enhancing empathy.\n\nThank you for your time and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe proposed task is reasonable, as the users who seek psychological support need empathy, trust, understanding, and comfort more than just reasonable advice.\n2.\tThe proposed dataset construction method is very simple.\n3.\tThe paper is easy to follow.\n", "reasons_to_reject": "1.\tThe model architecture is rather basic and not particularly innovative. They just finetuned the base model.", "questions_to_authors": "A: Why did the authors only use the Rouge and BLEU scores? It would be better to display additional metrics results, like METEOR, BERTScore etc.", "paper_topic_and_main_contributions": "The paper studies that LLMs empathy abilities can be significantly enhanced when finetuning by using multi-turn dialogue history and responses that are closer to the expression of a psychological situation.\nThe paper introduces to create a multiturn conversation by selecting 12 topics of psychological counselling to construct 215,813 long-text 140 questions and 619,725 long-text answers through crowdsourcing. Then, they used ChatGPT as a text rewriting tool, following the prompt, to convert single-turn psychological counselling conversations to multi-turn empathy conversations. For modelling, They used ChatGLM-6B as the base LLM architecture to develop SoulChat by finetuning.\n", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bHjJhPuRpd", "reviewer": "Reviewer_sDvK", "summary": "", "questions": "In line 105-108, you said that \"To this end, we first constructed 215,813 different psychological counseling questions about 12 topics and 619,725 answers through data outsourcing services\". Are these questions and answers fully written by human annotators?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the your valuable feedback and insightful comments on our paper. We are grateful that you recognize the main contributions of our work. We have carefully considered each point raised and would like to provide a thorough response to address the concerns and suggestions.\n\nRE: RR 1\n\nWe understand your concern about the quality of the collected dataset. We recognize the importance of analyzing and ensuring the quality of the dataset to enhance the credibility of our results. In response to this concern, we will conduct a more thorough analysis of the collected data and provide detailed insights into the quality assurance measures we undertook.\n\nRE: Q1\n\nWe thank you for raising a question regarding the data collection process outlined in lines 105-108. As you understand, these questions and answers fully written by human from a online psychological counseling platform. Among them, questions are provided by individuals from different age groups and professions, while answers are provided by certified psychological counselors. This ensures the richness of the topics covered by the questions and the professionalism of the answers.\n\nThank you for your time and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. If the dataset is released, it could be a helpful resource for LLMs to improve their empathy abilities.\n\n2. The prompt for converting single-turn dialogue to multi-turn dialogue might be useful.", "reasons_to_reject": "1. More analysis about the collected dataset is expected (e.g., quality).", "questions_to_authors": "In line 105-108, you said that \"To this end, we first constructed 215,813 different psychological counseling questions about 12 topics and 619,725 answers through data outsourcing services\". Are these questions and answers fully written by human annotators?", "paper_topic_and_main_contributions": "The paper aims to improve LLM's empathy, listening and comfort abilities. The authors point out that ChatGPT is inclined to provide suggestions rather than ask questions or listen and thus falls short of acting as a good psychological counselor. Therefore they collect a multi-turn empathetic conversation dataset of more than 2M samples, which could enhance LLM's empathy abilities according to the experiments.\n\nMain contributions:\n1. Collect a large scale Chinese empathetic dataset with more than 2M samples.\n2. Experimentally show that the collected SoulChatCorpus is helpful to improve LLM's empathy abilities.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xNEbkSlpGH", "reviewer": "Reviewer_1XLJ", "summary": "", "questions": "Question A: fig. 10 shows the translation of Fig. 4. and the word cloud should show that the SoulChat conversations have a high level of empathy. Can you elaborate in which way the words in the word cloud a represent a high level of empathy?\nQuestion B: How many dialogs have been generated with each model and for each dataset in the evaluation? \nQuestion C: Can you provide Krippenbach's alpha to check the inter-annotator agreement?\nQuestion D: Why have the three experts rated examples out of the dataset test sets and not examples generated by the models? (cp. 201-203) \nQuestion E: The experts are students of Psychology? In which stage of their studies they have been at the time of the evaluation?\nQuestion G: Can you provide the meanings of u, n, p in the input definition (above line 160) and translate everything in the lines 160+161 and 148-150?\nQuestion H: The input lengths in the lines 185 and 186 are measured in tokens, words, or signs?\nQuestion I: Can you explain how the evaluation was done? (what was generated by the models on the basis of which input, how have you applied the automatic measures - especially in case of multi-turn dialogs).", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thorough review of our paper. We are grateful for your insightful comments and suggestions, which have undoubtedly contributed to the enhancement of our work. We have carefully addressed each of your points and concerns below:\n\nRE: RR 1\n\nWe would like to express our gratitude to the you for their insightful comments and feedback on our paper. We have carefully considered the concerns raised regarding the references to the Appendix and the potential difficulty in comprehending the paper without reading it. We appreciate the opportunity to address these concerns and offer clarifications on how we can enhance the accessibility and readability of our work.\n\nRegarding the references to the Appendix, we acknowledge the concern that an overreliance on the Appendix can hinder the paper's comprehensibility. We apologize for any inconvenience this may have caused and understand the importance of ensuring that the main content of the paper is self-contained and informative. To address this concern, we will revise the paper to provide sufficient context and explanations in the main body, minimizing the need for constant referral to the Appendix. This will enable readers to grasp the core concepts and findings without being compelled to delve into supplementary materials.\n\nFurthermore, we understand the concern about cases of simple translations that may not be fully comprehensible. We will take this feedback into consideration and strive to improve the clarity of our explanations. We will provide more detailed explanations and examples in the main text to ensure that even straightforward concepts are well-understood by a wide audience, thus minimizing the requirement to consult additional materials.\n\nIn conclusion, we appreciate your feedback and acknowledge the need to enhance the accessibility and clarity of our paper. We are committed to addressing these concerns by improving the self-contained nature of our main text and by providing more comprehensive explanations. We believe that these revisions will significantly improve the reader's experience and understanding of our work. Thank you for your valuable feedback, and we are excited to incorporate these improvements into our paper.\n\nRE: RR 2\n\nReproducibility and Evaluation Procedure:\nWe acknowledge the importance of reproducibility and providing a clear evaluation procedure. In response to this concern, we will provide detailed steps in the revised paper for reproducing our study. We will include information about the corpus used, preprocessing steps, model architectures, and training procedures. Additionally, we will provide a comprehensive breakdown of the dataset into training and testing examples, as well as the exact number of generated dialogs. This information will enhance the transparency and reproducibility of our study, allowing other researchers to validate and build upon our work effectively.\n\nAutomatic Measures and Manual Evaluation:\nThank you for highlighting the description of our automatic measures and manual evaluation process. To ensure clarity, we will expand on the specific metrics used in the automatic evaluation and provide references for the manual evaluation guidelines employed. This will help readers better understand the basis for our evaluation and the validity of our results.\n\nRatio of Training and Testing Examples:\nWe acknowledge that we omitted the ratio of training and testing examples in the initial submission. We apologize for this oversight. In the revised manuscript, we will explicitly state the ratio of training to testing examples, providing a clear view of the data distribution and the basis of our experiments.\nWe randomly selected 10000 samples from SoulChatCorpus and SMILECHAT as the test set, and the remaining data from SoulChatCorpus was used as the training set for SoulChat. The data from SMILECHAT is not used as the training set for SoulChat.\n\nRE: Question A\n\nWords such as Believe (相信), Support (支持), Help (帮助), Need (需要) etc. For example, Sentence “I can understand your pain and inner struggle. (我可以理解你的痛苦和内心的挣扎)” expresses empathy through Interpretation. Sentence “I believe you can overcome difficulties (我相信你们一定能够克服困难)” expresses empathy through trust.\n\nRE: Question B\n\nWe randomly selected 10000 samples from SoulChatCorpus and SMILECHAT as the test set. For each sample, each model generates an answer for evaluation.\n\nRE: Question C\n\nI'm sorry, Google search doesn't seem to have Krippenbach's alpha. Are you talking about Krippendorff's alpha? Krippenbach's alpha for  content naturalness, empathy level,  helpfulness and safety are 0.871, 0.823, 0.884 and 1.0\n\nRE: Question D\n\nI'm sorry for any misunderstanding caused. We sampled 100 samples from the test set of SoulChatCorpus and SMILECHAT, and used each model to generate predictions. Experts manually evaluate the predictions of the models.\n\nRE: Question E\n\nThe three experts, who have already graduated with a Master's degree in psychology, currently serve as psychological counseling teachers in primary and secondary schools, making them highly suitable as evaluation experts to manually evaluate model predictions.\n\nRE: Question G\n\nWe will explain the meanings of \"u,\" \"n,\" and \"p\" as requested, and provided translations for the specified lines.\n\n$u$: utterance of a turn\n\n$u_i^u$: utterance of User on i-th turn\n\n$u_i^p$: utterance of Psychologist on i-th turn  \n\n‘$\\n$’: line break\n\nlines 148-150: , in which one turn is in the form of \"User:  <user\\_utt> \\n Psychologist: <psy\\_utt>\".\n\nlines 160-161: $u_i^u$ = 'User: ' + utterance$_{i}^u$ , $u_i^p$ ='Psychologist: ' + utterance$_{i}^p$ ($i<N$), $u_N^p$ =’Psychologist: ’\n\nRE: Question H\n\nThe input lengths in the lines 185 and 186 are measured in tokens.\n\nRE: Question I\n\nThe models take the past conversation history of the current turn as input, and then predicts the content of the current turn. Automatic evaluation compares the golden and prediction for each turn of a conversation.\n\nThank you for your time and consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper well motivates why the construction of the SoulChatCorpus was necessary by giving an interesting overview on the existing datasets in the domain of emphatic datasets\n- The idea to use ChatGPT (and probably LLMs in general) to create multi-turn-dialogs out of single-turn conversations is smart\n- An evaluation study with three baseline models (pure ChatGLM-6B, MeChat, and ChatGPT) and as baseline dataset (SMILECHAT) was conducted and the results are presented\n- The dataset will be published, if the paper is accepted", "reasons_to_reject": "- There are several references to the Appendix, which makes/impossible it hard to fully understand the paper without reading the Appendix. In cases of simple translations this is not comprehensible. \n- Even if the corpus is published, the presented study is not reproducible - and for now the evaluation can't be reviewed as no information about the procedure is provided. The automatic measures and the manual evaluation is described. However, the ration of training and testing examples is unclear. The same counts for the amount of generated dialogs.", "questions_to_authors": "Question A: fig. 10 shows the translation of Fig. 4. and the word cloud should show that the SoulChat conversations have a high level of empathy. Can you elaborate in which way the words in the word cloud a represent a high level of empathy?\nQuestion B: How many dialogs have been generated with each model and for each dataset in the evaluation? \nQuestion C: Can you provide Krippenbach's alpha to check the inter-annotator agreement?\nQuestion D: Why have the three experts rated examples out of the dataset test sets and not examples generated by the models? (cp. 201-203) \nQuestion E: The experts are students of Psychology? In which stage of their studies they have been at the time of the evaluation?\nQuestion G: Can you provide the meanings of u, n, p in the input definition (above line 160) and translate everything in the lines 160+161 and 148-150?\nQuestion H: The input lengths in the lines 185 and 186 are measured in tokens, words, or signs?\nQuestion I: Can you explain how the evaluation was done? (what was generated by the models on the basis of which input, how have you applied the automatic measures - especially in case of multi-turn dialogs).", "paper_topic_and_main_contributions": "The paper describes two main contributions. On the one hand, it introduces SoulChatCorpus as a new data resource. The corpus contains 2.3 million samples of utterances in multi-turn empathy conversation. To my understanding, the corpus contains (constructed) dialogs in 12 topics/domains of psychological counseling. The corpus language is Chinese.\n\nOn the other hand, the authors describe SoulChat, a conversational system which is based on ChatGLM-6B that was fine-tuned using the SoulChatCorpus. SoulChat is used to demonstrate in an NLP engineering experiment the advantage for empathy conversations when a modle is fine-tuned with a large amount of data.\n\nA minor, but still helpful, contribution is the overview on empathy-related conversation datasets.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a dataset on multi-turn empathy conversations, by converting single-turn dialogues to multi-turn dialogues, and performed experiments with different baselines on this dataset.

All the reviews agree that this paper is sound. The dataset and task will be a good resource for the community. The method to construct multi-turn dialogues from single-turn dialogues is easy to use and valuable. 

But there are questions that need to be addressed, for instance, more detailed dataset analyses and description about the evaluation for reproducibility.
Decision: Accept-Findings