Using Extracted Emotion Cause to Improve Content-Relevance for Empathetic Conversation Generation (2022)

Abstract: “Empathetic conversation generation intends to endow the open-domain conversation model with the capability for understanding, interpreting, and expressing emotion. Humans express not only their emotional state but also the stimulus that caused the emotion, i.e., emotion cause, during a conversation. Most existing approaches focus on emotion modeling, emotion recognition and prediction, and emotion fusion generation, ignoring the critical aspect of the emotion cause, which results in generating responses with irrelevant content. Emotion cause can help the model understand the user’s emotion and make the generated responses more content-relevant. However, using the emotion cause to enhance empathetic conversation generation is challenging. Firstly, the model needs to accurately identify the emotion cause without large-scale labeled data. Second, the model needs to effectively integrate the emotion cause into the generation process. To this end, we present an emotion cause extractor using a semi-supervised training method and an empathetic conversation generator using a biased self-attention mechanism to overcome these two issues. Experimental results indicate that our proposed emotion cause extractor improves recall scores markedly compared to the baselines, and the proposed empathetic conversation generator has superior performance and improves the content-relevance of generated responses.”

## introduction open-domain conversation generation has made remarkable progress over recent years, relying on deep learning and neural networks @xcite @xcite . however, previous works primarily centre around improving the linguistic quality of the generated responses, such as grammatical correctness, content variety, and topic relevance, neglecting the important factor of emotion @xcite . the information conveyed by humans during communication contains not only syntactic and semantic information but also emotional information. emotion is one of the essential cognitive behaviors in humans, and artificial intelligence has the objective of enabling machines to mimic human intelligent behaviors. as an important research branch of ai, one of the longterm goals of dialogue systems is to enable machines to perceive, comprehend and express emotions. in addition, studies @xcite @xcite have shown that introducing emotional information into conversation systems can improve user engagement and satisfaction, make human-computer conversation more natural, and reduce the number of conversation terminations. as a new research hotspot for the nlp community, most existing approaches on empathetic conversation generation focus on identifying the emotion category of the input sequence and generating a response based on the predicted emotion label. @xcite introduce an external emotion lexicon into the generation module to achieve explicit and implicit emotion expression. @xcite create a coarse-grained and fine-grained emotion representation for the input sequence by using an emotion classifier to identify the input sequence's emotion category and an external emotion lexicon to identify the emotion terms. @xcite improve the empathetic expression in the generated response by mimicking the input sequence's emotion while taking into account their emotional polarity. @xcite incorporate multiple emotions to varying degrees in the generation of responses to make the model more anthropomorphic. existing works mainly focus on emotion-related issues, focusing less on content-relevance. however, a case study of the responses generated by the existing models shows that the existing models do not guarantee the content-relevance of the generated responses very well. as shown in to this end, we propose an empathetic conversation generation model enhanced by emotion cause to improve the content-relevance of generated responses. specifically, our model involves two components, an emotion cause extractor and an empathetic conversation generator. in order to accurately identify emotion cause in the absence of large-scale labeled data, we present a semi-supervised training method to optimize the emotion cause extractor. to integrate the extracted emotion cause into the empathetic conversation generator and minimize the damage to the general language knowledge already learned by the pre-trained language model, we introduce a biased self-attention mechanism to enhance the model's attention to the emotion cause when generating responses. the contributions of our work are summarized as follows: ## approach our proposed emotion-cause-enhanced empathetic conversation generation model consists of two main modules: emotion cause extractor and empathetic conversation generator. the overview is shown in figure 1 . since there is no large-scale word-level emotion cause dataset available, we present a semisupervised training method to obtain the emotion cause extractor using small-scale labeled data jointly trained with large-scale unlabeled data. to involve the emotion cause in the generation process, we introduce multiplicative signals to implement the biased self-attention mechanism. the multiplicative signal enhances the model's attention to the emotion cause in the generation process and improves the content-relevance of the generated responses. figure 1: the overview of our proposed ece and ecg. ## conclusion in this paper, we present an empathetic conversation generation model enhanced by the emotion cause to make the generated responses more content-relevant. our proposed model comprises an emotion cause extractor and an empathetic conversation generator. to compensate for the scarcity of large-scale wordlevel emotion-cause labeled datasets, we suggest a semi-supervised training method that simultaneously uses labeled and unlabeled data for training. to integrate the extracted emotion cause into the generation process, we propose a biased self-attention mechanism that does not introduce new additional parameters. experimental results indicate that our proposed model performs superior to the baselines and the generated responses of our model are more empathetic and content-relevant.