Knowledge-Interactive Network with Sentiment Polarity Intensity-Aware Multi-Task Learning for Emotion Recognition in Conversations (2021)

Abstract: Emotion Recognition in Conversation (ERC) has gained much attention from the NLP community recently. Some models concentrate on leveraging commonsense knowledge or multi-task learning to help complicated emotional reasoning. However, these models neglect direct utterance-knowledge interaction. In addition, these models utilize emotion-indirect auxiliary tasks, which provide limited affective information for the ERC task. To address the above issues, we propose a Knowledge-Interactive Network with sentiment polarity intensity-aware multi-task learning, namely KI-Net, which leverages both commonsense knowledge and sentiment lexicon to augment semantic information. Specifically, we use a self-matching module for internal utterance-knowledge interaction. Considering correlations with the ERC task, a phrase-level Sentiment Polarity Intensity Prediction (SPIP) task is devised as an auxiliary task. Experiments show that all knowledge integration, self-matching and SPIP modules improve the model performance respectively on three datasets. Moreover, our KI-Net model shows 1.04% performance improvement over the state-of-the-art model on the IEMOCAP dataset.

## introduction emotion recognition in conversation aims to identify each utterance's emotion from a conversation, which requires machines to understand the way of emotion expression during conversations @xcite . research on erc helps in creating empathetic dialogue systems @xcite , thus improving the overall human-computer interaction experience. hence, the erc task has a wide range of applications such as social media analysis @xcite and intelligent systems like smart homes and chatbots @xcite . unlike vanilla emotion recognition of sentences, context modeling for conversations is crucial for erc models. early recurrent neural network (rnn)-based erc works adopt memory networks to store historical conversation context @xcite . recent progress in pre-trained language models (plms) like bert @xcite and xlnet @xcite has benefitted many downstream tasks, such as dialogue systems @xcite and reading comprehension @xcite . nevertheless, @xcite indicate that plms lack some dimensions of knowledge, which may limit the performance of the corresponding downstream tasks. hence most recent plm-based erc works adopt hierarchical structures that obtain word-level or utterancelevel representations via plms and then devise other elaborate modules for knowledge complement. some of them explicitly combine structured commonsense knowledge to the model and form knowledge-enriched representations @xcite . for the knowledge that is abstractive or unstructured, some other models adopt multi-task learning to compensate for missing knowledge dimensions implicitly @xcite . however, the above works do not consider internal interactions between utterance and knowledge representations when incorporating commonsense knowledge but simply concatenate them, which may negatively affect model performance as proved in the follow-up experiment. besides, the auxiliary tasks of most multi-task learning methods are emotion-indirect, such as topic inference @xcite and utterance-speaker verification @xcite , which do not involve additional affective information directly. @xcite also show that knowledge overlap between different knowledge sources is little. intuitively for the erc task, the complement of different dimensions of knowledge helps the reasoning process. in figure 1 , we illustrate a conversation where both commonsense knowledge and sentiment lexicon aid emotion detection. for example, considering the keyword "divorce" in the first utterance, with "an_affair" as a possible cause, "stop_being_married" as an action, and "di-vide_a_family" as a result, commonsense knowledge enables the model to build a semanticsenhanced chain on "divorce". the sentiment lexicon assigns extremely negative sentiment polarity intensity "-0.83" for "divorce", which directly instructs the model on determining negative emotions. obviously, in the process of judging this utterance as "frustrated", the two sources of knowledge have played their respective roles. to cope with the above challenges, we propose a knowledge-interactive network with sentiment polarity intensity-aware multi-task learning (ki-net). we first adopt a context-and dependency-aware encoder for context modeling. to further enhance the word-level representations, we leverage a largescale commonsense knowledge graph and a sentiment lexicon. inspired by @xcite , knowledge representations are incorporated into word-level representations using a self-matching mechanism, allowing a full internal interaction. we also introduce a phrase-level sentiment polarity intensity prediction (spip) as the auxiliary task, which is expected to provide more direct instructions on emotion recognition of the target utterance. in summary, this paper makes the following contributions: ## related work emotion recognition in conversation has gained attention from the nlp community only in the past few years @xcite @xcite since the growing availability of public conversational data @xcite @xcite . erc task naturally requires modeling interaction between conversation participants. considering this requirement, many works adopt rnns to model contextual utterances in a temporal sequence, such as cmn @xcite and icon @xcite . based on them, @xcite propose a attentive rnnbased model dialoguernn to model party states and emotional dynamics. transformer @xcite has also been devised to model input sequences in many recent works @xcite , which lead to better results. besides, modules such as memory networks @xcite and graph-based networks @xcite are also introduced for representation learning to better model contextual information and utterance dependencies. limited by the scale of current available highquality datasets, some works manage to incorporate task-related knowledge to boost model performance. @xcite al. (2021 ); chapuis et al. (2020) propose elaborate pre-training tasks to improve generalization of models. zhong et al. (2019); zhang et al. (2020) explicitly extract commonsense knowledge from large-scale knowledge graphs and concatenate them to word embeddings, forming knowledge-enriched representations. in addition, some works implicitly introduce knowl-sentence-level rep. sentic rep. sentence-level rep. sentic rep. interacted rep. (a) (b) happy emotion, 3.69 party, 1.58 … smile, 6.33 birthday, 4.16 bless 0.98 positive pleasant . . . … gladness peace interacted rep. utterance emotion polarity value utterance emotion polarity value context-and dependency-aware encoder edge through multi-task learning, such as label imbalance confusion @xcite , dialogue topic information @xcite and utterancespeaker relations @xcite . ## experimental setting in this section we present experimental settings such as datasets, baselines, implementation details and evaluation metrics. ## conclusion this paper proposes a ki-net for emotion recognition in conversations. our model outperforms state-of-the-art models on iemocap. extensive experiments prove the necessity of interaction between knowledge and utterance, and the new auxiliary task spip will further improve performance. utterance-level interaction and confusion of similar emotions are the focus of our following research. which dimensions of knowledge erc relies more on is also worthy of in-depth discussion.