Improving Contrastive Learning in Emotion Recognition in Conversation via Data Augmentation and Decoupled Neutral Emotion (2024)

Abstract: Emotion recognition in conversation (ERC) has attracted much attention due to its wide applications. While consistent improvement is being made in this area, inevitable challenge comes from the dataset. The ERC dataset exhibits significantly imbalanced emotion distribution. While the utterances with neutral emotion predominate the data, this emotion label is always treated the same as other emotion labels in current approaches. To address the problem caused by the dataset, we propose a supervised contrastive learning specifically oriented for ERC task. We employ a novel data augmentation method emulating the emotion dynamics in a conversation and formulate supervised contrastive learning method tailored for ERC addressing the predominance and the ambiguity of neutral emotion. Experimental results on four benchmark datasets demonstrate the effectiveness of our approach.

## introduction motivated by the success of chatbot services, emotion recognition in conversation (erc) has become an active research field, where the task is predicting the emotions in utterances in a conversation. the key point of erc is how to effectively model the context of each utterance and corresponding speaker. in order to capture the contextual information, existing works generally resort to recurrence-based methods @xcite , graph-based methods @xcite , knowledge-based methods @xcite @xcite , and pre-trained language model @xcite . despite the improvements, there always remain intrinsic challenges in erc dataset. one challenge comes from the erc dataset, where the emotion labels are often imbalanced. previous studies @xcite in erc have pointed out that the imbalanced datasets cause negative impact on the prediction performance. specifically, the class with the smallest number of samples suffers in the process due to the relatively insufficient amount of data for training. some of the works @xcite have been introduced to overcome the limitation of the dataset. @xcite use few-shot setting in episodic approach @xcite , which simulates a context with only few examples per class; spcl @xcite ) leverages a prototype for each category as at least one positive sample of the same category and negative samples of all other categories in contrastive learning. the second challenge of erc comes from the emotion label neutral which is the majority class dominating the dataset yet indistinct. prior studies have pointed out that the model tends to misclassify emotions to neutral @xcite @xcite . the main reason behind this is that the models tend to predict towards majority class, which is neutral in erc. besides, neutral was set as default emotion, where non-neutral emotions were annotated by human annotators only when the intensity of emotions (arousal) was sufficiently strong. while this setting can discern different emotions among non-neutral, the distinction between neutral and non-neutral becomes vague. even with these challenges, most existing studies in erc treat neutral emotions same as other non-neutral emotions for classification. only a few recent works @xcite have treated neutral in different ways from other emotions, such as alleviating the confusion between neutral and non-neutral through auxiliary tasks or detecting neutral first in coarse-grained level. however, due to the two-stage learning scheme, these models are inherently suboptimal. to tackle the limitation of erc dataset, we introduce a novel supervised contrastive learning framework which is specifically oriented for erc dataset (cled). to address the first challenge of erc dataset: data imbalance, cled employs a novel data augmentation technique that utilizes the emotion centroids obtained from the pre-trained language model (plm) embeddings. we perform interpolations on these centroids for generating augmented utterances, where the interpolation is performed reflecting the emotion shift through markovian property. from the training data, each transition probability is computed, and is fully utilized for our data augmentation. our method is unique in that the interpolation is based on the the realistic scenario with emotion shift. we further address the second challenge of the erc dataset concerning the limited use of the neutral. we design contrastive learning specifically dedicated to neutral emotion. we formulate an objective function that repels specific label, neutral, more strongly than others to clarify the boundaries of each label, considering that neutral closely intersects with other emotions. as such, cled makes non-neutral emotions more distinct from neutral emotion by applying a stronger repelling force from neutral. to verify the effectiveness, we implement our proposed scheme on six baselines including five recent erc models and a roberta-large based classifier which we additionally implement for this study. we compare the results using four benchmark erc datasets. experimental results show that two operations we propose consistently improves the performance. we further show that our method significantly outperforms other data augmentation method. our contribution can be summarized into three-fold. ## conclusion in this paper, we discuss the challenges in the erc dataset, which exhibits imbalanced label distribution and a dominance of neutral emotions that are difficult to distinguish from other emotions. we introduce novel method cled to address the challenges of erc datasets. the cled employs a novel data augmentation reflecting the context and emotion-dependency in conversation. with augmented data, we redefine a supervised contrastive learning loss specifically designed for the erc dataset to better distinguish between neutral and non-neutral emotions. we conduct extensive experiments to verify the effectiveness of our approach by constantly improving the previous baselines through plug-and-play.