Knowledge Aware Emotion Recognition in Textual Conversations via Multi-Task Incremental Transformer (2020)

Abstract: Emotion recognition in textual conversations (ERTC) plays an important role in a wide range of applications, such as opinion mining, recommender systems, and so on. ERTC, however, is a challenging task. For one thing, speakers often rely on the context and commonsense knowledge to express emotions; for another, most utterances contain neutral emotion in conversations, as a result, the confusion between a few non-neutral utterances and much more neutral ones restrains the emotion recognition performance. In this paper, we propose a novel Knowledge Aware Incremental Transformer with Multi-task Learning (KAITML) to address these challenges. Firstly, we devise a dual-level graph attention mechanism to leverage commonsense knowledge, which augments the semantic information of the utterance. Then we apply the Incremental Transformer to encode multi-turn contextual utterances. Moreover, we are the first to introduce multi-task learning to alleviate the aforementioned confusion and thus further improve the emotion recognition performance. Extensive experimental results show that our KAITML model outperforms the state-of-the-art models across five benchmark datasets.

## introduction emotion recognition in textual conversations (ertc), which aims to identify the emotion of each utterance from the transcript of a conversation, has become a popular research topic in recent years. ertc can be widely used in various scenarios, such as opinion mining of comments in social media @xcite , emotion analysis of customers in artificial customer service, and others. in addition, it can also be applied to chat robots to analyze the user's emotional state in real time and generate emotion-aware responses @xcite @xcite . @xcite from knowledge-enriched transformer @xcite , which is the current stateof-the-art model. we notice that there are barely miss-classifications among the non-neutral categories (angry, sad, and happy). most of the errors, shown in the bold font, correspond to the confusion between a few non-neutral categories and much more neutral category (others). in this paper, we propose a novel knowledge aware incremental transformer with multi-task learning (kaitml) to address the aforementioned challenges. firstly, we enhance the background and semantic information of the given utterance to facilitate ertc with the retrieved relevant knowledge graphs from a large-scale commonsense knowledge base. specifically, we propose a dual level graph attention mechanism to encode these relevant knowledge graphs, which consists of a node-level attention to learn the importance of different neighboring nodes and a relation-level attention to learn the importance of different relations to the current node. then we apply the incremental transformer @xcite to incrementally encode multi-turn contextual utterances, which could capture the intra-utterance and interutterance correlations by the self-attention @xcite and context-attention @xcite modules, respectively. moreover, we introduce multi-task learning to alleviate the confusion between a few non-neutral utterances and much more neutral ones, as shown in in summary, this paper makes the following contributions: ## related work emotion recognition in conversations has grabbed much attention from researchers in the past few years due to the proliferation of publicly available conversational dataset @xcite @xcite and its widespread applications in opinion mining, recommender systems, emotion-aware dialogues generation, and so on @xcite . some of the deep learning-based models have been proposed for emotion recognition in conversations, in only textual and multimodal settings (containing textual, acoustic, and visual information). @xcite proposes a long short-term memory network (lstm) @xcite based model to capture contextual correlations from the utterances of a user-generated video for multimodal sentiment classification. @xcite proposes conversational memory network (cmn) that exploits distinct memory units for each speaker to model emotional dynamics and detect emotion in a dyadic conversation. later, @xcite improves upon this approach with interactive conversational memory network (icon), which utilizes the interactive memory unit to hierarchically model the self-and inter-speaker emotional influences for emotion recognition in conversational videos. @xcite proposes the dialoguernn model that exploits three gated recurrent units (gru) @xcite to capture speaker information, context and emotional information of the preceding utterances, respectively. they achieve the state-of-the-art performance on several multimodal conversation datasets. compared to these gated rnns and cnns based models, we apply the incremental transformer @xcite to incrementally encode multi-turn contextual utterances, where the shorter path of information flow in the self-attention @xcite and context-attention @xcite modules in the incremental transformer allows our model to exploit contextual information more efficiently. recently, a considerable literature has grown up around the theme of incorporating external knowledge in generative conversation systems, including question answering systems @xcite , open-domain dialogue systems @xcite @xcite , and task-oriented dialogue systems @xcite @xcite . @xcite proposes a knowledge-enriched transformer (ket) achieving the state-ofthe-art performance on multiple textual conversation datasets, where contextual utterances are encoded using hierarchical self-attention and commonsense knowledge is incorporated using a context-aware affective graph attention mechanism. however, they ignore various relations in external knowledge base, which may cause the loss of semantic information. by contrast, our dual-level graph attention mechanism, can take advantage of the various relations in external knowledge base to better augment the semantic information of the utterances. ## conclusion and future work in this paper, we propose a novel knowledge aware incremental transformer with multi-task learning (kaitml) for emotion recognition in textual conversations, which can effectively incorporate contextual information and commonsense knowledge, and alleviate the confusion between a few non-neutral utterances and much more neutral ones. moreover, extensive experimental results show that our kaitml model outperforms state-of-the-art models across five benchmark dataset. future work will focus on the following directions: 1) how to differentiate similar emotions, 2) how to recognize emotion using limited data, 3) how to incorporate multimodal information for emotion recognition in conversations.