A Discourse-Aware Graph Neural Network for Emotion Recognition in Multi-Party Conversation (2021)

Abstract: Emotion recognition in multi-party conversation (ERMC) is becoming increasingly popular as an emerging research topic in natural language processing. Prior research focuses on exploring sequential information but ignores the discourse structures of conversations. In this paper, we investigate the importance of discourse structures in handling informative contextual cues and speaker-specific features for ERMC. To this end, we propose a discourse-aware graph neural network (ERMC-DisGCN) for ERMC. In particular, we design a relational convolution to lever the self-speaker dependency of interlocutors to propagate contextual information. Furthermore, we exploit a gated convolution to select more informative cues for ERMC from dependent utterances. The experimental results show our method outperforms multiple baselines, illustrating that discourse structures are of great value to ERMC.

## introduction in the past few years, emotion recognition in conversation (erc) has become increasingly popular in natural language processing (nlp) with the proliferation of open conversational data on social media platforms @xcite . similar to text sentiment analysis, erc is a task to determine the emotion of each utterance within a conversation, as shown in fig. 1 , and plays important role in many nlp applications, such as opinion mining in conversation @xcite , social media analysis @xcite and emotion-aware dialogue systems @xcite . however, erc, particularly the emotion recognition in multiparty conversation (ermc), often exhibits more difficulties than traditional text sentiment analysis due to the emotional dynamics of conversations @xcite . consequently, recognizing surprise (1) a: how does she do that? many approaches have been proposed for erc with a focus on conversational context representation and speaker-specific modeling. while earlier works on erc focus on two-party conversation and exploit recurrent neural networks (rnns) to capture sequential context features of conversations @xcite @xcite , recent studies exert more efforts on ermc and explore different techniques such as multi-task learning @xcite and pre-training language modeling @xcite to capture speaker-specific information. although these studies have greatly promoted the progress of erc, most of them ignore the important conversational discourse structures. therefore, they can only leverage cues in neighboring context of conversations, and are difficult to handle informative distant dependencies for erc. actually, conversational discourse structures contain discourse relations or discourse dependencies between utterances and thus provide a straightforward way to capture both adjacent and distant cues for ermc. fig. 1 illustrates a multi-party conversation example with its discourse structure obtained from the discourse parser proposed by @xcite . as we can see, although the first and the fourth utterances are distant in position within the conversation, they have an immediate discourse relation and are thus annotated with the same emotion type surprise. therefore, such discourse relations offer important contextual cues for ermc. on the other hand, discourse structures have proven to be useful for document-level sentiment analysis @xcite @xcite and we believe that they are also beneficial for ermc. moreover, recent progress in conversational discourse parsing @xcite makes it applicable to explore discourse structures to help model conversational contexts and speakers for ermc. however, two new problems may arise when discourse structures are applied to ermc. first, previous works have shown that speaker-specific information is very important for ermc @xcite . so it becomes a key issue how to incorporate conversational discourse structures into speaker-specific modeling for ermc. second, discourse structures involve dependent relations between utterances. however, not all information from dependent utterances is useful for conversational emotion recognition. therefore, another important problem might be how to select more informative cues for ermc. to address the aforementioned issues, we propose a discourse-aware graph neural network for emotion recognition in multi-party conversation, named ermc-disgcn. it consists of three main modules: firstly, a sequential context encoding module exploits long short-term memory (lstm) @xcite to capture the sequential features of utterances in a conversation. then, we exploit discourse dependency links and discourse relations to construct a graph, which contains two main convolution operations, namely a relational convolution and a gated convolution. the relational convolution is used to model the self-speaker dependency based on dis-course structures, where individual speakers resist the change of their own emotion against external influence @xcite , while the gated convolution adopts a gated mechanism to select informative cues for ermc from dependent utterances. similar to @xcite , we take utterances as nodes of the constructed graph. finally, a decoding module is applied to predict the emotion label for each utterance. in addition, we employ the deep sequential discourse parser developed by @xcite to obtain the explicit discourse dependency trees of input conversations. in summary, we make the following contributions: ## related work recently, erc has become a new trend due to the emergence of publicly available conversational datasets collected from social media platforms and scripted situations @xcite @xcite . earlier works focus on capturing sequential context features for emotion recognition in two-party conversation. poria et al. (2017) propose a lstmbased network to propagate contextual information within conversations. majumder et al. (2019) propose a recurrent-based model to track the speaker states and global context during conversations. jiao et al. (2019) propose a hierarchical gated recurrent unit (gru) (chung et al., 2014) structure that trains utterance-level and conversation-level encoders jointly. ghosal et al. (2019) construct a fully connected graph within a context utterance window to aggregate information. zhong et al. (2019) incorporate external commonsense knowledge and employ the transformer encoder @xcite to capture contextual information. for emotion recognition in multi-party conversation (ermc), studies exert more effort in handling speaker-specific information. @xcite represent the entire conversational corpus as a large graph to model speaker-sensitive dependency. @xcite use speaker identification as an auxiliary task to capture speaker-specific features. @xcite propose an all-in-one xl-net @xcite model with dialog-aware self-attention to deal with the multi-party structures. however, these studies neglect the informative discourse structures in multi-party conversations. to the best of our knowledge, we are the first to investigate the importance of discourse structures in handling informative contextual cues and speakerspecific features for ermc. discourse structures have been successfully applied to document-level sentiment analysis @xcite @xcite , where discourse structures are produced by rhetorical structure theory (rst) parser @xcite . recently, @xcite propose a deep sequential model for conversational discourse parsing and achieve new state-ofthe-art (sota) results. with this model, @xcite transform dialogue histories into threads for multi-turn response selection. inspired by @xcite and @xcite , we exploit discourse dependency links and discourse relations to construct a graph. especially, we stack two convolutional layers to aggregate contextual and speakerspecific information of the neighborhood for each utterance in the graph. ## conclusion in this paper, we investigate the importance of discourse structures in handling informative contextual cues and speaker-specific features for ermc. we propose a discourse-aware graph neural network and devise two graph convolutional layers to aggregate normalized contextual and speakerspecific information for each utterance in the graph. experimental results show that our proposed model outperforms all the baselines on all multi-party conversation datasets. furthermore, we apply extensive analyses for the proposed model and have the following findings. first, discourse structures are more helpful for emotion recognition in multi-party conversation than in two-party conversation. second, it is important to integrate discourse structures into speaker modeling. third, the gated mechanism helps select more informative cues from dependent utterances for ermc. in our future work, we would like to capture various speaker-specific features to deal with the emotional shifts. since our method focuses on using explicit discourse structures, we also plan to employ implicit methods to avoid error propagation and address consequent issues.