ESC o T : Towards Interpretable Emotional Support Dialogue Systems (2024)

Abstract: Understanding the reason for emotional support response is crucial for establishing connections between users and emotional support dialogue systems. Previous works mostly focus on generating better responses but ignore interpretability, which is extremely important for constructing reliable dialogue systems. To empower the system with better interpretability, we propose an emotional support response generation scheme, named \textbf{E} motion-Focused and \textbf{S} trategy-Driven \textbf{C} hain- \textbf{o} f- \textbf{T} hought ( \textbf{ESCoT} ), mimicking the process of \textit{identifying} , \textit{understanding} , and \textit{regulating} emotions. Specially, we construct a new dataset with ESCoT in two steps: (1) \textit{Dialogue Generation} where we first generate diverse conversation situations, then enhance dialogue generation using richer emotional support strategies based on these situations; (2) \textit{Chain Supplement} where we focus on supplementing selected dialogues with elements such as emotion, stimuli, appraisal, and strategy reason, forming the manually verified chains. Additionally, we further develop a model to generate dialogue responses with better interpretability. We also conduct extensive experiments and human evaluations to validate the effectiveness of the proposed ESCoT and generated dialogue responses. Our dataset, code, and model will be released.

## introduction emotional support is conceptualized as expressing care, concern, affection, and interests, especially for the individuals feeling stressed or upset @xcite @xcite . incorporating emotional support can yield positive effects in many scenarios, such as therapeutic sessions @xcite , customer service counters @xcite payne, 2003). realizing reliable emotional support dialogue systems capable of automating these interactions is expected to expand the scope and efficacy of such services. moreover, a reliable emotional support dialogue system should not work like a black box, providing conversational responses but unable to explain how those responses were generated @xcite . as shown in figure 1 , let's imagine how a helpful supporter would work by considering the feelings of a seeker who asks for help: the supporter would first identify the situation and the emotion of the seeker, then understand and acknowledge the emotion, and finally choose appropriate strategies to respond in order to regulate the emotion @xcite . therefore, it is extremely desired to build a reliable and trustworthy emotional support dialogue system that can not only generate emotional support responses but also provide the reasoning or chain-of-thought (cot) behind the generated responses. some previous endeavors have attempted to improve the interpretability of emotional support dialogue systems, such as controlling the response by emotion @xcite or strategy @xcite , or using commonsense to augment the emotional support response @xcite . however, to the best of our knowledge, there is currently no such emotional support dialogue system that can provide comprehensive reasoning explanations. therefore, in this work, we aim to build an interpretable emotional support dialogue system. due to the high expertise requirements for supporter roles in emotional support conversations, building a human-annotated emotional support dialogue dataset is very costly. recently, the powerful language generation and reasoning capabilities of large language models (llms) have demonstrated a viable pathway to generate high-quality data. efforts such as augesc @xcite and @xcite have attempted to expand emotional support dialogue datasets via llms. however, the reasoning or chain-of-thought behind the dialogue responses has been overlooked. in this paper, we propose an emotional support response generation scheme, named emotionfocused and strategy-driven chain-of-thought (escot), to generate dialogue data, inspired by the human emotional support generation process of identifying, understanding, and regulating emotions. specifically, to emphasize the critical role of conversation strategies and dialogue situations, we first create diverse dialogue situations, and then enhance dialogue generation using richer emotional support strategies based on these situations. furthermore, we complement selected dialogues with the chain-of-thought (cot), which is represented as a quintuple (em, es, ia, sr, re), reflecting the process as illustrated in figure 1 . after careful manual checking, we build the first dataset for emotional support dialogue with cot (esd-cot), containing 1.7k+ dialogues. moreover, we build our emotional support dialogue system with better interpretability via supervised fine-tuning a pre-trained language model on esd-cot, providing a strong baseline for future investigation. our main contributions in this work include: (1) we develop an effective emotion-focused strategydriven chain-of-thought automatic data generation scheme called escot to increase the interpretability of emotional support response generation. @xcite we build the first chain-of-thought emotional sup-port dataset esd-cot, containing 1.7k+ dialogues through automatic generation and manual correction. @xcite we conduct human evaluations to validate from different aspects the effectiveness of our data generation scheme and the quality of our constructed dialogue dataset. @xcite we build an interpretable emotional support response generation model on esd-cot and conduct a comprehensive assessment of the performance, providing a strong baseline for future research. sharma et al. (2020) construct a dataset for supporting the epitome model based on talklife and mental health subreddits. sun et al. (2021) scrape q&a from the yixinli platform and annotate responses based on psychological counseling theories to create the psyqa chinese dataset. these datasets are non-dialogue datasets, but applications like psychological counseling need multi-turn dialogues. rashkin et al. (2019) propose a new emotional dialogue generation benchmark and create a new dataset called empatheticdi-alogues, which contains 25k dialogues in emotional contexts. liu et al. (2021) propose the esc framework and construct a dataset named esconv based on this theoretical framework. in order to obtain such datasets at a lower cost, some works leverage the power of llms to augment data for emotional support dialogues. zheng et al. (2023) introduce augesc, an augmented dataset for the esc @xcite task by leveraging fine-tuned large language models to complete dialogues. @xcite develop the smile approach by using chatgpt (openai, 2022) to transform single-turn dialogues into multi-turn conversations. however, these works merely treat data augmentation as dialogue continuation or rewriting tasks, without making specific adjustments to accommodate the features of emotional support dialogues. for instance, dialogue situations and conversation strategies have not been taken into consideration. interpretable dialogue systems deep learning models for dialogue systems are often seen as black boxes due to the complexity and opacity of their internal mechanisms. lack of interpretability can lead to safety concerns, as it's challenging to predict or understand the models' decisions in critical scenarios. to address these concerns, researchers have been exploring different methods to improve the interpretability of language models. one research direction involves integrating knowledge-based reasoning to improve the moral and ethical judgment capabilities of dialogue systems. @xcite and @xcite try to incorporate external knowledge sources and structured reasoning pathways to enhance the decision-making quality of these models, particularly in scenarios requiring moral or ethical considerations. @xcite introduce an interpretable dialogue system that employs a twostage response generation process, enhancing response diversity and system transparency. moreover, @xcite create dialogue modules based on cbt (beck, 1979) dialogue scenarios centered on socratic questioning and consider the questions about abc @xcite ). an emerging direction is to explain language models using llms. @xcite explore using @xcite to interpret and understand the behavior of neurons in language models such as gpt-2 xl @xcite . however, there is currently no emotional support dialogue system that can provide comprehensive reasoning explanations to improve the interpretability. @xcite initially introduce chain-of-thought (cot) prompting to mimic the reasoning process. following this, various works that utilize cot to prompt llm for intricate reasoning tasks spring up across different domains, such as auto-cot @xcite , sp-cot @xcite , and psycot @xcite . to address challenges when applying cot prompting in dialogues, dialogue cot @xcite propose decomposing commonsense reasoning into steps and generating rationale as a sequence of inferred commonsense knowledge required for response generation to facilitate dialogue cot reasoning. the cue-cot @xcite prompts the system to infer the user status first and then generate a response based on dialogue context and user status. however, cot for emotional ## esd-cot dataset construction due to the high expertise requirement in the supporter role, manually constructing an emotional support dialogue dataset is very costly. mimicking the human thought process of comforting others, we propose an emotion-focused and strategydriven chain-of-thought (escot) scheme to generate emotional support responses in an interpretable manner, and build the first emotional support dialogue with cot dataset (esd-cot). the construction process of esd-cot dataset can be divided into two steps: (1) esd construction where we first create diverse situations, and then generate dialogues with enriched strategies based on generated situations by leveraging llms; ## experiments we split the dataset into train, validation, and test with the ratio of 7:1:2, as shown in we evaluate the following pre-trained language models as backbone models for dialogue response ## conclusion in this paper, inspired by the human consultation process of identifying, understanding, and regulating emotions, we propose a novel emotion-focused and strategy-driven chain-of-thought (escot) emotional support response generation scheme and apply it in building the first dataset for emotional support dialogue with chain-of-thought named esd-cot. moreover, we conduct extensive experiments and human evaluations to validate that the new benchmark based on esd-cot can provide effective emotional support with better interpretability. we hope our esd-cot dataset and baseline models can facilitate further investigation into interpretable emotional support dialogue systems in the community.