Enhancing Emotion Recognition in Conversation via Multi-view Feature Alignment and Memorization (2023)

Abstract: Emotion recognition in conversation (ERC) has attracted increasing attention in natural language processing community. Previous work commonly first extract semantic-view features via fine-tuning PLMs, then models context-view features based on the obtained semantic-view features by various graph neural networks. However, it is difficult to fully model interaction between utterances simply through a graph neural network and the features at semantic-view and context-view are not well aligned. Moreover, the previous parametric learning paradigm struggle to learn the patterns of tail class given fewer instances. To this end, we treat the pre-trained conversation model as a prior knowledge base and from which we elicit correlations between utterances by a probing procedure. And we adopt supervised contrastive learning to align semantic-view and context-view features, these two views of features work together in a complementary manner, contributing to ERC from distinct perspectives. Meanwhile, we propose a new semi-parametric paradigm of inferencing through memorization to solve the recognition problem of tail class samples. We consistently achieve state-of-the-art results on four widely used benchmarks. Extensive experiments demonstrate the effectiveness of our proposed multi-view feature alignment and memorization.

## introduction emotional intelligence is an advanced capability of conversational ai systems. a fundamental and critical task in this domain is emotion recognition in conversation (erc), which aims to identify the emotions conveyed in each utterance within the dialogue context @xcite . unlike the basic emotion classification (ec) task @xcite , erc is a more practical endeavor that involves predicting the emotion label of each utterance based on the surrounding context. previous methods @xcite @xcite ) commonly follow a two-step paradigm of first extracting semanticview features via fine-tuning plms and then modeling context-view features based on the obtained semantic-view features by various graph neural networks, such as gcn (kipf and welling, 2016), rgcn @xcite , gat @xcite . considering the complexity of emotions, some recent works @xcite use supervised contrast loss as an auxiliary loss to improve the feature discriminability between different classes of samples. although these methods have achieved excellent performance on erc task, three issues still remain: (1) as illustrated in figure 1 , the interaction between utterances in a conversation is very complex, and it's difficult to fully model this interaction simply through a graph neural network. (2) both semantic-view and context-view are important for erc. the former focuses on the emotion conveyed by the independent utterance, while the latter offers clues about the emotional context. these two views of information work together in a complementary manner, contributing to erc from distinct perspectives. however, as illustrated in figure 2 , semantic-view and context-view are not well aligned. (3) due to the smaller number of samples in the tail class, it is difficult for the parametric model to learn the pattern of the tail class during training. how to effectively recognize samples of the tail class remains an issue to be solved. to address these issues, we propose multi-view feature alignment and memorization (mfam) for erc. firstly, we treat the pre-trained conversation model (pcm) @xcite as a prior knowledge base and from which we elicit correlations between utterances by a probing procedure. the correlation between utterances will be used to form the weights of edges in the graph neural network and participate in the interactions between utterances. secondly, we adopt supervised contrastive learning (scl) to align the features at semanticview and context-view and distinguish semantically similar emotion categories. unlike the regular scl, both semantic-view and context-view features will participate in the computations of scl. finally, we propose a new semi-parametric paradigm of inferencing through memorization to solve the recognition problem of tail class samples. we construct two knowledge stores, one from semantic-view and the other from context-view. semantic-view knowledge store regarding semantic-view features and corresponding emotion labels as memorized keyvalue pairs, and the context-view knowledge store is constructed in the same way. during inference, our model not only infers emotion through the weights of trained model but also assists decisionmaking by retrieving examples that are memorized in the two knowledge stores. it's worth noting that semantic-view and context-view features have been well aligned, which will facilitate the joint retrieval of the semantic-view and context-view. the main contributions of this work are summarized as follows: (1) we propose multi-view feature alignment for erc, which aligns semanticview and context-view features, these two views of features work together in a complementary manner, contributing to erc from distinct perspectives. ## methodology we introduce the definition of erc task in section 3.1, and from section 3.2 to section 3.5, we introduce the proposed mfam in this paper. the overall framework of mfam is shown in figure 3 . ## conclusion in this paper, we propose multi-view feature alignment and memorization (mfam) for erc. firstly, in order to obtain accurate context-view features, we treat the pcm as a prior knowledge base and from which we elicit correlations between utterances by a probing procedure. then we adopt scl to align semantic-view and context-view features. moreover, we improve the recognition performance of tail-class samples by retrieving examples that are memorized in the two knowledge stores during inference. we achieve state-of-the-art results on four widely used benchmarks, ablation study and visualized results demonstrates the effectiveness of multi-view feature alignment and memorization.