Building a Dialogue Corpus Annotated with Expressed and Experienced Emotions (2022)

Abstract: In communication, a human would recognize the emotion of an interlocutor and respond with an appropriate emotion, such as empathy and comfort. Toward developing a dialogue system with such a human-like ability, we propose a method to build a dialogue corpus annotated with two kinds of emotions. We collect dialogues from Twitter and annotate each utterance with the emotion that a speaker put into the utterance (expressed emotion) and the emotion that a listener felt after listening to the utterance (experienced emotion). We built a dialogue corpus in Japanese using this method, and its statistical analysis revealed the differences between expressed and experienced emotions. We conducted experiments on recognition of the two kinds of emotions. The experimental results indicated the difficulty in recognizing experienced emotions and the effectiveness of multi-task learning of the two kinds of emotions. We hope that the constructed corpus will facilitate the study on emotion recognition in a dialogue and emotion-aware dialogue response generation.

## introduction text-based communication has become indispensable as society accelerates online. in natural language processing, communication between humans and machines has attracted attention, and the development of dialogue systems has been a hot topic. through the invention of transformer @xcite and the success of transfer learning (e.g., @xcite ; @xcite ), the performance of natural language understanding models and dialogue systems continues to improve. in recent years, there have been studies toward building open-domain neural chatbots that can generate a human-like response @xcite @xcite . one of the keys to building more human-like chatbots is to generate a response that takes into account the emotion of the interlocutor. a human would recognize the emotion of the interlocutor and respond with an appropriate emotion, such as empathy and comfort, or give a response that promotes positive emotion of the interlocutor. accordingly, developing a chatbot with such a human-like ability @xcite @xcite is essential. although several dialogue corpora with emotion annotation have been proposed, an utterance is annotated only with a speaker's emotion @xcite or a dialogue as a whole is annotated @xcite , all of which are not appropriate for enabling the above ability. in this paper, we propose a method to build an emotion-annotated multi-turn dialogue corpus, which is necessary for developing a dialogue system that can recognize the emotion of an interlocutor and generate a response with an appropriate emotion. we annotate each utterance in a dialogue with an expressed emotion, which a speaker put into the utterance, and an experienced emotion, which a listener felt when listening to the utterance. to construct a multi-turn dialogue corpus annotated with these emotions, we collect dialogues from twitter and crowdsource their emotion annotation. as a dialogue corpus, we extract tweet sequences where two people speak alternately. for the emotion annotation, we adopt plutchik's wheel of emotions @xcite as emotion labels and ask crowdworkers whether an utterance indicates each emotion label for expressed and experienced emotion categories. each utterance is allowed to have multiple emotion labels and has an intensity, strong and weak, according to the number of crowdworkers' votes. we build a japanese dialogue corpus as a testbed in this paper, but our proposed method can be applied to any language. using the above method, we constructed a japanese emotion-tagged dialogue corpus consisting of 3,828 dialogues and 13,806 utterances. 1 statistical analysis of the constructed corpus revealed the characteristics of words for each emotion and the relationship between expressed and experienced emotions. we further conducted experiments to recognize expressed and experienced emotions using bert @xcite . we defined the task of emotion recognition as regression and evaluated bert's performance using correlation coefficients. the experimental results showed that it was more difficult to infer experienced emotions than expressed emotions, and that multi-task learning of both emotion categories improved the overall performance of emotion recognition. from these results, we can see that expressed and experienced emotions are different, and that it is meaningful to annotate both. we expect that the constructed corpus will facilitate the study on emotion recognition in dialogue and emotion-aware response generation. ## conclusion we proposed a method to build an emotion-tagged multi-turn dialogue corpus to help machines recognize emotional transition in a dialogue. dialogues between two speakers are collected from twitter, and each utterance is annotated with emotions by crowdsourcing. in the annotation process, we consider the emotions expressed by a speaker who said the utterance and the emotion experienced by a listener who heard the utterance. in addition, the labels are provided with their intensity, representing the granularity of emotions. we built a japanese emotion-tagged dialogue corpus and analyzed it. the results showed the characteristics of words for each emotion, the correlation between the emotions about a certain utterance and speaker, and the tendency for speakers to become positive through a dialogue. we also developed emotion recognition models for expressed and experienced emotions based on the japanese pretrained bert models. the experimental results indicated that it is more difficult to recognize a listener's emotion than a speaker's emotion. multitask learning of expressed and experienced emotions improved the performance of the two emotion recognition tasks about an utterance and a speaker. for our future work, we will tackle response generation based on predicted emotions. with our corpus, a dialogue system is expected to predict which emotion it experiences from a given utterance and which emotion it should express for the next utterance. once such emotions are recognized, the dialogue system should be able to generate an appropriate response depending on the predicted expressed emotion. the corpus in this work is annotated only with expressed and experienced emotions about an utterance. in addition to the emotion annotation, we should also consider dialogue situations @xcite . the cause of a dialogue or an utterance helps recognize a speaker's emotion and how it changes. we can also consider non-emotional annotation, such as a dialogue's topic and an utterance's intention @xcite . the relationship between emotions and non-emotional factors is also important for machines to better recognize a speaker's emotion.