A Taxonomy of Empathetic Response Intents in Human Social Conversations (2020)

Abstract: Open-domain conversational agents or chatbots are becoming increasingly popular in the natural language processing community. One of the challenges is enabling them to converse in an empathetic manner. Current neural response generation methods rely solely on end-to-end learning from large scale conversation data to generate dialogues. This approach can produce socially unacceptable responses due to the lack of large-scale quality data used to train the neural models. However, recent work has shown the promise of combining dialogue act/intent modelling and neural response generation. This hybrid method improves the response quality of chatbots and makes them more controllable and interpretable. A key element in dialog intent modelling is the development of a taxonomy. Inspired by this idea, we have manually labeled 500 response intents using a subset of a sizeable empathetic dialogue dataset (25K dialogues). Our goal is to produce a large-scale taxonomy for empathetic response intents. Furthermore, using lexical and machine learning methods, we automatically analysed both speaker and listener utterances of the entire dataset with identified response intents and 32 emotion categories. Finally, we use information visualization methods to summarize emotional dialogue exchange patterns and their temporal progression. These results reveal novel and important empathy patterns in human-human open-domain conversations and can serve as heuristics for hybrid approaches.

## introduction inspired by the recent success of deep neural networks for natural language processing (nlp) tasks such as language modeling @xcite and machine translation @xcite , neural response generation is currently at the forefront of research in the nlp community. recent advances in this field have proven the efficacy of deep neural networks in modelling both task-oriented and opendomain dialogue systems @xcite @xcite . most of the existing neutral conversation models are capable of generating syntactically and contextually wellformed responses. some of the work also focuses on enabling chatbots to generate emotionally colored and affect-rich responses @xcite @xcite . despite the efforts in modeling affect in natural language, work that focuses specifically on modeling empathy in chatbots is relatively limited and remains an open research question @xcite . empathy plays a vital role in human psychological processes for smooth social interaction @xcite . empathy-related responding includes caring and sympathetic concerns for other people. humans are born with core affect neural circuitry, and they gradually develop the ability to apprehend the emotional states of others and respond in an empathetic manner. empathy motivates pro-social behavior and increases the sense of social bonding @xcite . therefore, in the context of social interaction, a chatbot needs to be empathetic to maintain healthy interaction with humans and develop trust. the task of augmenting social chatbots with empathy is challenging because the generated responses have to be appropriate in terms of both content and emotion information @xcite . several neural response generation models have attempted to address this challenge in a fully datadriven manner. for example, @xcite , use the full transformer architecture @xcite pre-trained on 1.7 billion reddit conversations and fine-tuned on the empatheticdialogues dataset @xcite to generate empathetic responses. @xcite adapt the generative pretrained transformer (gpt) @xcite to empathetic response generation task by finetuning it on the personachat @xcite and empatheticdialogues datasets. even though these models are capable of mimicking human empathetic conversation patterns in some ways, it is often unpredictable what the chatbots might generate, for example, they may generate inconsiderate remarks, redundant responses, asking the same questions repeatedly, or any combinations of them. since it is really important to respond to humans' emotions appropriately, we believe controllability of response generation is essential. several other neural response generation approaches attempt to gain control over the generated response by conditioning it on a manually specified emotion label @xcite @xcite or using affective loss functions based on heuristics such as minimizing or maximizing affective dissonance between prompts and responses @xcite . these models claim to generate emotionally more appropriate responses than those generated from purely datadriven models. however, the primary concern of these handcrafted rules is its practicality. no prior work has shown normative associations between the speaker's emotions and the corresponding listener's emotions. as our work would reveal, listeners are much more likely to respond to sad or angry emotions with questioning than expressing similar or opposite emotions in the first turn. @xcite , however, has shown the benefit of incorporating dialogue acts as policies in designing a social chatbot. they were able to avoid the need to manually condition the next response with a label by jointly modeling dialogue act selection and response generation. their framework first selects a dialogue act from a policy network according to the dialogue history. the generation network then generates a response based on both dialogue history and the selected dialogue act. it is thus possible to explicitly learn human-human conversational patterns in social chitchat and generate more controlled and interpretable responses. unfortunately, they did not study empathetic response generation. to fill this gap, we have developed a taxonomy of empathetic listener intents by manually annotating around 500 utterances of the empatheticdialogues dataset @xcite , covering 32 types of emotion categories. in the following, we first describe in detail how this taxonomy was derived (figure 1 ) and how we chose the dataset to support this annotation work. to extend this subset, we employ automatic techniques to label all speaker and listener utterances, covering 25k empathetic humanhuman conversations. to be able to explain the patterns and trends of the conversation flow, we employ visualization methods to illustrate the most frequent exchanges and reveal how they temporally vary as dialogues proceed. finally, we discuss how these results can be used to derive more informed heuristics for controlling the neural response generation process. foot_0 ## related work to provide a background for this research, we begin by describing some of the existing theories related to empathy in other fields such as psychology and neuroscience and their limitations in incorporating them into the design of social chatbots. then we describe seminal work on existing neural-based open-domain response generation systems and means by which they control the generated response. these studies serve as the motivation and inspiration for our work. next, we discuss some existing dialogue-act/intent taxonomies and their limitations in modeling empathy in human social conversations. @xcite defines empathy as a social emotion in response to the emotions of others. further, he states that the evoked empathetic reaction itself constitutes an emotional experience, primarily because it is associated with increased excitement and awareness. it tends to be a feeling with or feeling for the observed party. two of the most famous theories in the psychological literature to explain the phenomenon of empathy are the "simulation theory" @xcite and the "theory-theory" (gopnik and wellman, 1995). the "simulation theory" states that a person understands another or empathizes by imagining himself in the other's situation and seeing it from his perspective. the "theory-theory" states that the ability to understand what another person is feeling is based on rules for how one should think and feel. ## taxonomy of empathetic response intents to develop the taxonomy, we investigated which intents are frequently associated with listeners when responding to different emotional situations in empatheticdialogues. we took a subset of the dataset with situations associated with the plutchik's 8 basic emotions @xcite (joyful, anticipating, trusting, surprised, angry, afraid, sad, and disgusted), and manually analysed it to derive the listener intents associated with each type of emotions. in this process, 20 dialogues belonging to each emotion were randomly selected and each sentence in all listener utterances were manually annotated by an expert evaluator with a label that best describes their intent. this resulted in 521 sentences manually annotated with intent labels. because an utterance can have multiple sentences, we decided to annotate each sentence in a listener's utterance with a unique intent label. for example, the two sentences comprising the utterance "those symptoms are scary! do you think it's corona?" would be annotated with separate intent labels "acknowledging" and "questioning", respectively. after analysing their occurrences and whether some of the intents can be grouped into a common intent, we were able to come up with a taxonomy of 15 empathetic response intents. category examples frequency 1. questioning (to know further details or clarify) -what are you looking forward to? 24.38% 2. acknowledging (admitting as being fact) -that sounds like double good news. it was probably fun having your hard work rewarded. 22.46% 3. agreeing (thinking/saying the same) -that's a great feeling, i agree! 9.60% 4. consoling -i hope he gets the help he needs. 7.87% 5. encouraging -hopefully you will catch those great deals! 5.37% 6. sympathizing (express feeling pity or sorrow for the person in trouble) -so sorry to hear that. 5.37% 7. wishing -hey... congratulations to you! 4.41% 8. suggesting -maybe you two should go to the pet store to try and find a new dog for him! 4.03% 9. sharing own thoughts/opinion i would love to have a boy too, but i'm not sure if i want another one or not. 4.03% 10. sharing or relating to own experience i had a friend who went through the same thing. 3.84% 11. advising don't take too much money with you. 2.69% 12. expressing care or concern i hope the surgery went successfully and with no hassle. 2.30% 13. expressing relief phew.. that's a relief., i am glad you were okay. 1.53% 14. disapproving but america is so great now! look at all the great things that are happening. 1.15% 15. appreciating you are very trusting. it's nice to have a friend like you. 0.95% ## conclusion in this paper, we introduced a taxonomy of empathetic response intents capable of supporting automatic empathetic communication in social chitchat. the strategies relying on this taxonomy are essential for a chatbot to engage in prosocial conversations, expressing empathetic concern for its users, and keeping the users engaged. another significant contribution from our work is to provide analysis on the em-patheticdialogues corpus after automatically annotating it based on the most frequent intents from our taxonomy and 32 types of emotion categories defined in empatheticdialogues. we illustrate the most frequent emotion-intent exchange patterns in the dataset and how they vary temporally over the course of interaction. these results further validate the taxonomy of empathetic listener intents we derived and shed light on the frequent empathetic conversation patterns seen among humans when engaged in social chitchat. we explained how our taxonomy can be utilized in the development of an empathetic chatbot to achieve more controllability and interpretability in the responses generation process. the method described here can also be used as an annotation scheme to label utterances from other datasets and analyse them in terms of their empathetic quality. as future work, we plan on using these findings to develop a social chatbot capable of effectively engaging in empathetic conversations.