S oul C hat: Improving LLM s’ Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations (2023)

Abstract: Large language models (LLMs) have been widely applied in various fields due to their excellent capability for memorizing knowledge and chain of thought (CoT). When these language models are applied in the field of psychological counseling, they often rush to provide universal advice. However, when users seek psychological support, they need to gain empathy, trust, understanding and comfort, rather than just reasonable advice. To this end, we constructed a multi-turn empathetic conversation dataset of more than 2 million samples, in which the input is the multi-turn conversation context, and the target is empathetic responses that cover expressions such as questioning, comfort, recognition, listening, trust, emotional support, etc. Experiments have shown that the empathy ability of LLMs can be significantly enhanced when finetuning by using multi-turn dialogue history and responses that are closer to the expression of a psychological consultant.

## introduction with the birth of bert @xcite and gpt @xcite , large language models (llms) have made rapid progress in the past five years. in november 2022, openai launched chatgpt foot_1 (openai, 2022), a large language model fine-tuning by reinforcement learning from human feedback (rlhf) @xcite . however, when applied to mental health or emotional support conversation, there are three main issues lead to chatgpt appear less "human-centered": 1) chatgpt tends to provide repetitive and standardized responses. chatgpt often uses the following template to respond to users' questions related to mental health: "我 很 抱歉...。xxx是...。以下是一些建议：...。 (i'm sorry to ...{xxx} is ...here are some suggestions:...)", which may cause boredom. similar issues can also be found in other llms, e.g. chatglm @xcite , sparkdesk foot_2 , as presented in appendix f. it may be due to the lack of large-scale multi-turn empathy conversation datasets for fine-tuning stage, especially in the field of chinese mental health or emotional support. empatheticdialogues @xcite and esconv @xcite are two english empathy conversation datasets that is used for developing emotional support conversation (esc) systems, e.g misc @xcite , glhg @xcite , multiesc @xcite , fado @xcite and etc. on the one hand, these models may rely on annotated empathy strategies and emotions of users during the training or inference stage, which means that building large-scale similar datasets for fine-tuning llms is difficult. on the other hand, these datasets are in english, so that they cannot be applied to fine-tune chinese llms. as for mental health, efaqa @xcite and @xcite are two commonly-used datasets. among them, efaqa contains 20,000 conversations and provides annotation information such as types of troubles, psychological disorders, sos, etc. however, efaqa has a complex multi-party dialogue relationship and a high proportion of low-quality responses from netizens, while psyqa contains 22,346 questions and 56,063 single-turn long-text psychological counseling conversations. thus, neither of these datasets can solve the three issues of chat-gpt mentioned above. recently, @xcite proposed a smile approach to employ chatgpt to convert singleturn dialogues into multi-turn ones. they utilized smile to extend the single-turn conversation dataset psyqa to a empathy multi-turn conversation dataset smilechat with 355,733 samples. inspired by @xcite , we proposed a chinese empathy constraint prompt, in which the empathy prompt constraint is further strengthened compared with smile prompt (see appendix c). as shown in figure 2 (english version: appendix c), our empathy constraints are defined as "'心理咨询师'的回复需要结合用户 的描述内容并提供共情，如：倾听、安慰、 理解、信任、认可、真诚、情感支持等 (the response of the 'psychological counselor' needs to be combined with the user's description and provide empathy, such as listening, comfort, interpretation, trust, recognition, sincerity, emotional support, etc)". ## conclusion and future work in this paper, we explore how to make llms more human-centered. to this end, we constructed a chinese large-scale multi-turn empathy conversation dataset, named soulchatcorpus, with 12 empathy topics and more than 2 million samples. the experimental results indicate that using this dataset to finetune llms leads to high-level empathy ability when users try to seek emotional support from llms. future work needs to further consider user attributes, such as personality, gender and etc., to help llms generate targeted empathy responses for different individuals.