From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues (2023)

Abstract: Understanding emotions during conversation is a fundamental aspect of human communication, driving NLP research for Emotion Recognition in Conversation (ERC). While considerable research has focused on discerning emotions of individual speakers in monolingual dialogues, understanding the emotional dynamics in code-mixed conversations has received relatively less attention. This motivates our undertaking of ERC for code-mixed conversations in this study. Recognizing that emotional intelligence encompasses a comprehension of worldly knowledge, we propose an innovative approach that integrates commonsense information with dialogue context to facilitate a deeper understanding of emotions. To achieve this, we devise an efficient pipeline that extracts relevant commonsense from existing knowledge graphs based on the code-mixed input. Subsequently, we develop an advanced fusion technique that seamlessly combines the acquired commonsense information with the dialogue representation obtained from a dedicated dialogue understanding module. Our comprehensive experimentation showcases the substantial performance improvement obtained through the systematic incorporation of commonsense in ERC. Both quantitative assessments and qualitative analyses further corroborate the validity of our hypothesis, reaffirming the pivotal role of commonsense integration in enhancing ERC.

## introduction dialogue serves as the predominant means of information exchange among individuals @xcite . conversations, in their various forms such as text, audio, visual, or face-to-face interactions @xcite , can encompass a wide range of languages @xcite . in reality, it is commonplace for individuals to engage in informal conversations with acquaintances that involve a mixture of languages @xcite tarihoran and kya kal tumne cricket match dekha? (did you watch the cricket match yesterday?) sahi me! it is impossible to beat him (truly! it is impossible to beat him!) yes! sachin played badhiya! (yes! sachin played great!) sumirat, 2022). for instance, two native hindi speakers fluent in english may predominantly converse in hindi while occasionally incorporating english words. figure 1 illustrates an example of such a dialogue between two speakers in which each utterance incorporates both english and hindi words with a proper noun. this linguistic phenomenon, characterized by the blending of multiple languages to convey a single nuanced expression, is commonly referred to as code-mixing. while code-mixing indeed enhances our understanding of a statement @xcite , relying solely on the uttered words may not fully capture its true intent @xcite . in order to facilitate better information assimilation, we often rely on various affective cues present in conversation, including emotions @xcite @xcite . consequently, the task of emotion recognition in conversation (erc) @xcite has emerged and gained significant attention. erc aims to establish a connection between individual utterances in a conversation and their corresponding emotions, encompassing a wide spectrum of possible emotional states. despite the extensive exploration of erc in numerous studies @xcite @xcite @xcite @xcite @xcite @xcite , the primary focus has been into monolingual dialogues, overlooking the prevalent practice of code-mixing. in this work, we aim to perform the task of erc for code-mixed multi-party dialogues, thereby enabling the model-ing of emotion analysis in real-world casual conversations. to the best of our knowledge, there is no previous work that deals with erc for code-mixed conversations, leading to a scarcity of available resources in this domain. as a result, we curate a comprehensive dataset comprising code-mixed conversations, where each utterance is meticulously annotated with its corresponding emotion label. the elicited emotion in a conversation can be influenced by numerous commonly understood factors that may not be explicitly expressed within the dialogue itself @xcite . consider an example in which the phrase "i walked for 20 kilometers" evokes the emotion of pain. this association stems from the commonsense understanding that walking such a considerable distance would likely result in fatigue, despite it not being explicitly mentioned. consequently, capturing commonsense information alongside the dialogue context becomes paramount in order to accurately identify the elicited emotion. to address this, we propose incorporating commonsense for solving the task of erc. however, the most popular commonsense graphs, such as conceptnet @xcite and comet @xcite are made for english, are known to work for the english language @xcite , and are not explored for code-mixed input. to overcome this challenge, we develop a pipeline to utilize existing english-based commonsense knowledge graphs to extract relevant knowledge for codemixed inputs. additionally, we introduce a clever fusion mechanism to combine the dialogue and commonsense features for solving the task at hand. in summary, our contributions are fourfold foot_0 : ## related work emotion recognition. earlier studies in emotion analysis @xcite @xcite @xcite dealt with only standalone inputs, which lack any contextual information. to this end, the focus of emotion detection shifted to conversations, specifically erc. while erc was solved using heuristics and standard machine learning techniques initially @xcite @xcite , the trend has recently shifted to employing a wide range of deep learning methods @xcite @xcite @xcite @xcite @xcite @xcite . ## problem statement given, as input, the contextual utterances (s 1 , u 1 ), (s 2 , u 2 ), . . . , (s m , u n-1 ) such that utterance u i is uttered by speaker s j , the task of emotion recognition in conversation aims to identify the emotion elicited in the target utterance u n , uttered by the speaker s k . all the discovered emotions come from a predefined set of emotion classes e. in this work, we consider the eckman's emotion set which includes seven emotions -anger, fear, disgust, sad, joy, surprise, and contempt, along with a label for no emotion, i.e., neutral. therefore, e @xmath0, fear, disgust, sad, joy, surprise, contempt, neutral}. ## the e-masac dataset a paucity of datasets exists for code-mixed conversations, making tasks on code-mixed dialogues scarce. nevertheless, a recent dataset, masac @xcite , compiled by extracting dialogues from an indian tv series, contains sarcastic and humorous hindi-english code-mixed multiparty instances. we extract dialogues from this dataset and perform annotations for the task of erc to create e-masac. the resultant data contains a total of 8, 607 dialogues constituting of 11, 440 utterances. data statistics are summarised in emotion annotation. given, as input, a sequence of utterances forming a dialogue, @xmath1, u 1 ), (s 2 , u 2 ), • • • , (s n , u n )}, the aim here is to assign an appropriate emotion, e i , for each utterance, u i , uttered by speaker s j . the emotion e i should come out of a set of possible emotions, e. following the standard work in erc for the english language, we use eckman's emotions as our set of possible emotions as mentioned in section 3, @xmath2, fear, disgust, sadness, joy, surprise, contempt, neutral}. e i . any discrepancies are resolved by a discussion among the annotators; however, such discrepancies are rare. we calculate the inter-annotator agreement, using kriprendorff's alpha score @xcite , between each pair of annotators, α @xmath3.84, α @xmath4.85, and α @xmath5.85. to find out the overall agreement score, we take the average score, α = 0.85. ## proposed methodology: coffee as mentioned in section 1, the manifestation of emotional concepts within an individual during a conversation is not solely influenced by the dialogue context, but also by the implicit knowledge accumulated through life experiences. this form of knowledge can be loosely referred to as commonsense. in light of this, we present an efficient yet straightforward methodology for extracting pertinent concepts from a given commonsense knowledge graph in the context of code-mixed inputs. additionally, we introduce a clever strategy to seamlessly incorporate the commonsense features with the dialogue representation obtained from a backbone architecture dedicated to dialogue understanding. figure 3 outlines our proposed approach, coffee while each of the intermediate modules is elucidated in detail below. ## conclusion in this study, we investigated the task of emotion recognition in conversation (erc) for code-mixed dialogues, the first effort in its kind. we curate an emotion lad dialogue dataset for hindi-english conversations and proposed a new methodology that leverages existing commonsense knowledge graphs to extract pertinent commonsense concepts for code-mixed inputs. the extracted common-sense is integrated into a backbone architecture through a novel fusion technique that uses context aware attention mechanism. our findings indicated that the incorporation of commonsense features significantly enhances the performance of erc, as evidenced both quantitatively and qualitatively.