D ialogue EIN : Emotion Interaction Network for Dialogue Affective Analysis (2022)

Abstract: Emotion Recognition in Conversation (ERC) has attracted increasing attention in the affective computing research field. Previous works have mainly focused on modeling the semantic interactions in the dialogue and implicitly inferring the evolution of the speakers’ emotional states. Few works have considered the emotional interactions, which directly reflect the emotional evolution of speakers in the dialogue. According to psychological and behavioral studies, the emotional inertia and emotional stimulus are important factors that affect the speaker’s emotional state in conversations. In this work, we propose a novel Dialogue Emotion Interaction Network, DialogueEIN, to explicitly model the intra-speaker, inter-speaker, global and local emotional interactions to respectively simulate the emotional inertia, emotional stimulus, global and local emotional evolution in dialogues. Extensive experiments on four ERC benchmark datasets, IEMOCAP, MELD, EmoryNLP and DailyDialog, show that our proposed DialogueEIN considering emotional interaction factors can achieve superior or competitive performance compared to state-of-the-art methods. Our codes and models are released.

## introduction emotion recognition in conversation (erc), aiming to recognize the emotional status of each utterance in a conversation, has attracted increasing research attention in recent years. it has rich application potentials in emotional support, mental health, and legal trials etc @xcite . unlike traditional emotion recognition based on isolated utterances, conversation context modeling is very important for the erc task @xcite . different approaches have been proposed look, you saw the way that she acted and then… well, i mean, you, you were acting kind of… what? i don't know, embarrassed of her since i came. i mean, that's the reason why i asked you here. well. i love you. i guess that's why i came. neutral happy happy happy happy sad sad sad sad emotional stimulus the emotional change of the male stimulates the female to become happy. emotional inertia no obvious stimulus and the emotion remains unchanged. figure 1 : illustration of emotional interaction in an example dialogue from the iemocap dataset. the grey arrow line indicates that the speaker's emotional state remains unchanged, and the orange arrow line indicates that the speaker is stimulated and the emotion changes, corresponding to emotional inertia and emotional stimulus respectively. to model the context of a conversation. for example, cmn @xcite and icon @xcite use speaker-specific and global recurrent networks to model the semantic context in a dialogue, and multi-hop memory networks are used to generate the summaries for prediction. dialoguegcn @xcite models the speaker-specific semantic interaction via designing different relations based on graph networks. di-alogxl @xcite utilizes the dialog-aware self-attention mechanism in a transformer structure to capture intra-and inter-speaker dependencies. these works only focus on modeling the dialogue semantic context. recently, some works have considered the dialogue emotional evolution and proposed several methods to model the emotional context. dialoguernn @xcite and cosmic @xcite use separate grus to model global and speaker-specific semantic context and utilize another gru to track the evolution of global emotional states. cesta @xcite adopts the conditional random field (crf) to learn the global emotional consistency in the conversation. iein @xcite proposes to model the global emotion interaction with emotion embeddings and an rnn-based iterative structure. however, these works only consider modeling the global emotional evolution, while ignoring the important speaker-aware emotional dependencies related to emotional inertia and emotional stimulus. according to psychological and behavioral studies, emotional inertia and emotional stimulus are important factors that affect the speaker's emotional state in dialogues. emotional inertia @xcite means that in the absence of sufficient external stimulus, the speaker tends to keep the emotional state unchanged within a dialogue. emotional stimulus refers to another phenomenon in which a subject's emotion can be aroused and affected by external events which can be words, facial expression, speech intonation or even emotions of the interlocutor @xcite . figure 1 illustrates the emotional inertia and emotional stimulus in an example. in this dialogue, the female speaker feels sad at the beginning because of some misunderstanding, but as the misunderstanding is resolved, both speakers appear to be happy. in the first two turns of the dialogue, the emotional states of the male and female speakers remain unchanged (emotional inertia). however, after listening to the explanation, the male speaker shows happiness, and his emotional change stimulates the female speaker to become happy as well (emotional stimulus). we simply call such emotional inertia and emotional stimulus information as the emotional interaction context in the dialogue, which measures how a person's emotion affects his own or his interlocutor's emotion. in this paper, we propose a novel dialogue emotion interaction network(dialogueein), to explicitly model the emotional interaction context in conversations for erc tasks. dialogueein mainly consists of a semantic interaction network and an emotional interaction network, where the former aims to capture dialog-level semantic context representations based on a transformer structure, and the latter aims to model the emotional interaction context including intra-speaker emotional inertia, inter-speaker emotional stimulus, global-and local emotional interactions through four corresponding types of dialog-aware self-attention mechanism respectively. we carry out experiments on four erc benchmark datasets, including iemocap, meld, emorynlp and dailydialog. the experiment results show that our proposed dialogueein achieves state-of-the-art performance on all datasets, which indicates that emotional interactions (such as emotional inertia and emotional stimulus) are important for tracking speakers' emotional evolution in conversations. the main contributions of this work include: ## conclusion in this work, we propose a novel emotional interaction network (dialogueein) for emotion recognition in conversation (erc). dialogueein explicitly models emotional inertia, emotional stimulus in a conversation, which most previous works have neglected. dialogueein can capture the emotion tendencies of each utterance and model the emotional dependencies based on them. an attention-based emotional interaction network is proposed to measure the emotional interactions, and four types of dialog-aware attentions are employed to simulate emotional inertia, emotional stimulus, global and local evolution of emotional states in the dialogue respectively. extensive experiments are carried out on iemocap, meld, emorynlp and dai-lydialog benchmark datasets. dialogueein significantly outperforms other state-of-the-art models on iemocap and meld datasets, and achieves competitive performance on all four datasets, which proves the effectiveness of the proposed model. moreover, several ablation studies further explore the structure of dialogueein and interpret the use of emotional interaction, which also suggests possible future research directions, such as fusing multimodality to capture emotional stimulus information more accurately, etc.