From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues
Authors: Shivani Kumar
Ramaneswaran S
Md Shad Akhtar
Tanmoy Chakraborty
Venue: EMNLP 2023 Main (2023)
Keywords: Emotion Recognition in Conversation
Code-mix dialogues
Commonsense

Abstract: Understanding emotions during conversation is a fundamental aspect of human communication, driving NLP research for Emotion Recognition in Conversation (ERC). While considerable research has focused on discerning emotions of individual speakers in monolingual dialogues, understanding the emotional dynamics in code-mixed conversations has received relatively less attention. This  motivates our undertaking of ERC for code-mixed conversations in this study. Recognizing that emotional intelligence encompasses a comprehension of worldly knowledge, we propose an innovative approach that integrates commonsense information with dialogue context to facilitate a deeper understanding of emotions. To achieve this, we devise an efficient pipeline that extracts relevant commonsense from existing knowledge graphs based on the code-mixed input. Subsequently, we develop an advanced fusion technique that seamlessly combines the acquired commonsense information with the dialogue representation obtained from a dedicated dialogue understanding module. Our comprehensive experimentation showcases the substantial performance improvement obtained through the systematic incorporation of commonsense in ERC. Both quantitative assessments and qualitative analyses further corroborate the validity of our hypothesis, reaffirming the pivotal role of commonsense integration in enhancing ERC.

From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues
Shivani Kumar Ramaneswaran S Md Shad Akhtar 1 Tanmoy Chakraborty 'IIT Delhi, India, ?NVIDIA, India, 3ITT Delhi, India
shivaniku@iiitd. ac. in, s. s com, shad. akhtareiiitd..a. in, tanchakeiitd.ec. ac.
Abstract
Yeal Sachin playad badriyst (Veat Sachin played
Sahi me! it imposaible
dekha? maton yesterday?)
(Trdy impisssblee
Understanding emotions during conversation is a fundamental aspect of human communication, driving NLP research for Emotion Recognition in Conversation (ERC). While considerable re- search has focused on discerning emotions of individual speakers in monolingual dialogues, understanding the emotional dynamics in code- mixed conversations has received relatively less attention. This motivates our undertaking of ERC for code-mixed conversations in this study. Recognizing that emotional intelligence encom- passes comprchension of worldly knowledge, we propose an innovative approach that inte- grates commonsense information with dialogue context to facilitate deeper understanding of emotions. To achieve this, we devise an effi- cient pipeline that extracts relevant common- sense from existing knowledge graphs based on the code-mixed input. Subsequently, we de- velop an advanced fusion technique that seam- lessly combines the acquired commonsense in- formation with the dialogue representation ob- tained from dedicated dialogue understand- ing module. Our comprehensive experimen- tation showcases the substantial performance improvement obtained through the systematic incorporation of commonsense ERC. Both quantitative assessments and qualitative analy- ses further corroborate the validity of our hy- pothesis, reaffirming the pivotal role of com- monsense integration in enhancing ERC.
groat))
him!)
Figure I: Example of code-mixed dialogue between two speakers. Blue colour denote English words while red denotes proper noun.
Sumirat, 2022). For instance, two native Hindi speakers fluent in English may predominantly con- verse in Hindi while occasionally incorporating English words. Figure illustrates an example of such a dialogue between two speakers in which each utterance incorporates both English and Hindi words with a proper noun. This linguistic phe- nomenon, characterized by the blending of multiple languages to convey single nuanced expression, is commonly referred code-mixing.
While code-mixing indeed enhances our under- standing of statement (Kasper and Wagner, 2014), relying solely the uttered words may not fully capture its true intent (Thara and Poornachandran. 2018). In order facilitate better information as- similation, we often rely on various affective cues present in conversation, including emotions (Poria et al., 2019; Dynel, 2009; Joshi et al., 2017). Conse- quently, the task of Emotion Recognition in Conver- sation (ERC) (Hazarika al., 2018b) has emerged and gained significant attention. ERC aims to tablish a connection between individual utterances in conversation and their corresponding emotions, encompassing a wide spectrum of possible emo- tional states. Despite the extensive exploration of ERC in numerous studies (Hazarika et al., 201 Sa; Zhong et al., 2019a; Ghosal et al., 2019; Jiao et al., 2019; Shen et al., 2020; Li et al., 2020; Jiao et al.,
1 Introduction
Dialogue serves as the predominant means of in- formation exchange among individuals (Turnbull, 2003). Conversations, in their various forms such text, audio, visual, or face-to-face interactions (Hakulinen, 2009; Caires and Vieira, 2010), can encompass wide range of languages (Weigand, 2010; Kasper and Wagner, 2014). In reality, it is commonplace for individuals engage in infor- mal conversations with acquaintances that involve a mixture of languages (Tay, 1989; Tarihoran and
2020; Hazarika et al., 2021; Tu et al., 2022; Yang et al., 2022: Ma et al., 2022), the primary focus has been into monolingual dialogues, overlooking the prevalent practice of code- mide-mixing. In this work, we aim to perform the task of ERC for code-mixed multi-pane party dialogues, thereby enabling the model-

---

ing of emotion analysis in real-world casual conver- sations. To the best of our knowledge, there is no previous work that deals with ERC for code-mixed conversations, leading to a scarcity available re- sources in this domain. As a result, we curate a comprchensive dataset comprising code-mixed con- versations, where each utterance is meticulously annotated with its corresponding emotion label.
2 Related Work
Emotion recognition. Earlier studies in emotion analysis (Ekman, 1992; Picard, 1997; Cowen and Keltner, 2017; Mencattini et al., 2014; Zhang et al., 2016; Cui et al., 2020) dealt with only standalone inputs, which lack any contextual information. To this end, the focus of emotion detection shifted to conversations, specifically ERC. While ERC was solved using heuristics and standard machine learning techniques initially (Fitrianic et al.. 2003; Chuang and Wu, 2004; Li et al., 2007), the trend has recently shifted to employing wide range of deep learning methods (Hazarika et al., 2018a; Zhong al., 2019a; Li et al., 2020; Ghosal et al., 2019; Jiao et al., 2020; Hazarika et al., 2021; Shen et al., 2020; Poria et al., 2017b; Jíao et al., 2019; Tu et al., 2022; Yang et al., 2022; Ma et al., 2022).
The elicited emotion in conversation can be influenced by numerous commonly understood fac- tors that may not be explicitly expressed within the dialogue itself (Ghosal al., 2020a). Con- sider an example in which the phrase "I walked for 20 kilometers" evokes the emotion of pain. This association stems from the commonsense under- standing that walking such a considerable distance would likely result in fatigue, despite it not be- ing explicitly mentioned. Consequently, capturing commonsense information alongside the dialogue context becomes paramount in order to accurately identify the elicited emotion. To address this, propose incorporating commonsense for solving the task of ERC, However, the most popular com- monsense graphs, such as ConceptNet (Speer et al., 2017) and COMET (Bosselut et al., 2019) are made for English, are known to work for the English lan- guage (Zhong et al., 2021; Ghosal et al., 2020b), and are not explored for code-mixed input. To overcome this challenge, we develop pipeline to utilize existing English-based commonsense knowl- edge graphs to extract relevant knowledge for code- mixed inputs. Additionally, we introduce a clever fusion mechanism to combine the dialogue and commonsense features for solving the task at hand. In summary, our contributions are fourfold':
Emotion and commonsense. Given the implicit significance of commonsense knowledge the pro- cess of emotion identification, researchers have delved into the integration of commonsense for the purpose of emotion recognition. In scenarios involving standalone text, where the contextual in- formation is relatively limited, studies propose the utilization of carefully curated latent commonsense concepts that can seamlessly blend with the text (Balahur et al., 201 1; Zhong et al., 2021; Chen al., 2022). However, in situations where the context of the text spans longer sequences, like dialogues, it becomes essential to capture it intelligently. Many studies explore the task of ERC with commonsense fusion using ConceptNet (Speer et al., 2017; Zhong et al., 2021), Atomic triplets (Sap et al., 2019; Nie et al., 2023), and the COMET graph (Bosselut al., 2019; Ghosal al., 2020b; Li
1. We explore, for the first time, the task of ERC for multi-party code-mixed conversations. 2. We propose a novel code-mixed multi-party conversation dataset, E-MASAC, in which each discourse is annotated with emotions.
al., 2021).
Emotion and code-mixing. Existing research on emotion analysis for code-mixed language primar- ily focuses on standalone social media texts (Sasid- har et al., 2020; Ilyas et al., 2023; Wadhawan and Aggarwal, 2021) and reviews (Suciati and Budi, 2020; Zhu et al., 2022). While aspects such as sar- casm (Kumar et al., 2022a,b), humour (Bedi et al., 2023), and offense (Madhu et al., 2023) have been explored within code-mixed conversations, emo- tion analysis remains largely uncharted territory with no relevant literature available, to the best of our knowledge. We aim to fill this gap by investi- gating the unexplored domain ERC specifically for Hindi -nndi--Enlish rode-mixed conversations and introducing a novel dataset as well as a new model.
3. We develop COFFEE method to extract com- monsense knowledge from English-basee com- monsense graphs given code-mixed input and fuse it with dialogue context efficiently.
4. We give a detailed quantitative and qualita- tive analysis of the results obtained and exam- ine the performance of the popular large lan- guage models, including ChatGPT.
'The source code and dataset present here: https:// gi tiub. Com/LLCS-IIITD/EEE PMFTPO *Commonsense aware Fusion For Emotion rEcognition

---

3 Problem Statement
Utt len
Vocab len
Set
eDigs
aUtts
Avg sp/dig
Avg
Max
English
Hindi
Given, imput, the contextual utterances (81, un), (s2, w2),..., (Sm; um-1) such that utter- ance uy is uttered by speaker 8j. the task of Emo- tion Recognition in Conversation aims to identify the emotion elicited in the target utterance Un, ut- tered by the speaker Sy.- All the discovered emo- tions come from predefined set of emotion classes E. In this work, we consider the Eckman's is emotion set which includes seven emotions anger, fear, disgust, sad, joy, surprise, and contempt, along with label for no emotion, i.e., neutral. Therefore, e, E where {anger. fear, disgust, joy, surprise, contempl, neutral).
8506
3.60
Val
45
1354
4.13
10.12
218
14803
Test
56
1580
4.32
10.61
84
Total
8607
11440
12.05
31.55
415
Table 1: Data statistics for E-MASAC.
anger sodhess
contempt fear
surprise disqust
Joy
Train
Val
Test
Figure 2: Emotion distribution for E-MASAC.
4 The E-MASAC Dataset
ej. Any discrepancies are resolved by a discus- sion among the annotators; however, such discrep- ancies are rare. We calculate the inter-annotator agreement, using Kriprendorff's Alpha score (Krip- pendorff, 201 1), between each pair of annotators, Oabb 0.84, Otee 0.85, and Oac 0.85. To find out the overall agreement score, we take the average score, a 0.85.
A paucity of datasets exists for code-mixed con- versations, making tasks on code-mixed dialogues scarce. Nevertheless, a recent dataset, MASAC (Bedi et al., 2023), compiled by extracting dia- logues from an Indian TV scries, contains sarcastic and humorous Hindi-English code-mixed multi- party instances. We extract dialogues from this dataset and perform annotations for the task of ERC to create E-MASAC. The resultant data contains total of 8, 607 dialogues constituting of 11, 440 utterances. Data statistics are summarised in Table 1. Emotion distribution based on the annotations of the three sets² is illustrated in Figure 2.
5 Proposed Methodology: COFFEE
As mentioned in Section 1, the manifestation of emotional concepts within an individual during a conversation is not solely influenced by the dia- logue context, but also by the implicit knowledge accumulated through life experiences. This form of knowledge can be loosely referred to as com- monsense. In light of this, we present an effi- cient yet straightforward methodology for extract- ing pertinent concepts from given commonsense knowledge graph in the context of code-mixed in- puts. Additionally, we introduce clever strategy seamlessly incorporate the commonsense features with the dialogue representation obtained from a backbone architecture dedicated to dialogue under- standing. Figure 3 outlines our proposed approach, COFFEE while each of the intermediate modules is elucidated in detail below.
Emotion annotation. Given, as input, a se- quence of utterances forming a dialogue, D = ((1), 21), (82, u2),... (sn, mn)], the aim here is to assign appropriate emotion, ei, for each ut- terance, u4 uttered by speaker sj. The emotion e; should come out of set of possible emotions, E. Following the standard work in ERC for the En- glish language, we use Eckman's emotions as our set of possible emotions as mentioned in Section 3, F. (anger, disgust, sadness,joyy surprise, contempt, neutral). Each emotion, along with its definition and example, is illustrated in Appendix
A.1. We ask three annotators' (a, b, c) to anno- tate each utterance, t4, with the emotion they find most suitable for it, eς such that ef E E. ma- jority voting is done among the three annotations (ef, et, ef) to select the final gold truth annotation,
5.1 Dialogue Understanding Backbone (DUB)
For input containing long contextual history, such as dialogue, it becomes crucial capture and comprehend the entire progression leading up the present statement. Consequently, an effective dialogue understanding architecture which gives us a concrete dialogue representation is required.
"We follow the original train-val-test split as is MASAC "The annotators are linguists fluent in English and Hindi with good grasp of emotional knowledge. Their age lies hetween 25-30.

---

Input
Commonsense Extraction (CE)
Input
Language Id Scclinh bathtya
Transliterate "o' Sachis played smppa*"
Preprocess Saxiis playal Rोयीर
Saitts plged batteys
DUB
Encode
Query COMET
Translate
sachec wanty wilk..
Sachii ployed played-groo grear"
Des
Commonsense Fusion (CF)
Fusion Gate (FG)
Dce
Attention
Des
FG
Sigmoid
Fusion
6.s
Classity
Figure 3: A schematic diagram of COFFEE. The Commonsense Extraction (CE) module takes code-mixed input and provides a representation of the extracted commonsense information relevant to it. The commonsense information is fused with the representation obtained from a Dialogue Understanding Backbone (DUB) via the Commonsense Fusion (CF) and the Fusion Gate (FG) modules.
We existing Transformer based architectures (c.f. Section 6.1) our Dialogue Understanding Backbone, DUB. The given code-mixed dialogue D goes through DUB to give us the contextual dialogue representation, De- Specifically, De = DUB( D), such that Dg c Raxdd where n is the maximum dialogue length, and d is the dimensionality of the resulting vectors.
offfect oReact
impact of input on the listeners. The listeners' reaction to the input statement.
Want
The listeners' desire after hearing the input. What the input reveals about the speaker. The speaker's 's desire after uttering the input. The speaker's 's objective in uttering the input. speaker's needs according to the input. The speaker's 'ss reaction based on the input.
xAttr
EEffect
xIntent
xNeed
xReact
Want
The speaker's 's desire according to input.
5.2 Commonsense Extraction (CE)
Table 2: Commonsense effect-types returned by the COMET and their description.
While the conversational context provides insights into the participants and the topic of the dialogue, the comprehension of implicit meanings within statements can be greatly facilitated by incorporat- ing commonsense information. Therefore, in order to capture this valuable commonsense knowledge, we employ the COMET graph (Bosselut et al., 2019), which has been trained on ATOMIC triplets (Sap et al., 2019), to extract relevant commonsense information for each dialogue instance. However, is worth noting that the COMET graph is pretrained using triplets in the English language, making it particularly effective for English inputs (Ghosal et al., 2020a). Given that our input consists of mixture of English and Hindi, have devised specialized knowledge extraction pipeline tackle this challenge. The entire process of obtaining commonsense knowledge for a given mode-mixed textual input is shown in Figure 3 and is compre- hensively explained below.
Hindi words, the initial task is to determine the language of each word to appropriately handle different languages in the most suitable way.
2. Transliteration: The identified Hindi language words are transliterated to Devanagari script from roman script that language-specific pre- processing can applied to them.
3. Texi Processing: The next step is to preprocess the text. This step involves converting text to lowercase and removal of ASn-ASCII characters and stopwords. The resultant text is considered important or 'topic specifying' for the text.
4. Translation: Since COMET is trained for mono- lingual English, the query can only have English terms. Therefore, we translate the Devanagari Hindi 'topics' back to romanised English.
5. Querying COMET: Finally, all the 'topics' to- gether are sent as a query to the COMET graph. and all possible relations are obtained.
Language Identification: To handle the input di- alogue D, which includes a mix of English and
COMET provides us with a vast array of effect- types corresponding to the input text. Specifically,

---

BERT (Devlin et al., 2018) is a pre-trainee tre-trained language model that utilizes Transformer ar- chitecture and bidirectional context to understand the meaning and relationships of words in a sen- tence. RoBERTa (Liu al., 2019) is an exten- sion of BERT that improves its performance uti- lizing additional training tcchniques such as dy- namic masking, longer sequences, and more itera- tions. mBERT' (multilingual BERT) is a variant it provides us with information such as oEffect, oRe- act, oWant, xAttr, xEffect, xIntent, xNeed, xReact, xWant. Refer Table for the description of each of these values. We carefully select the relevant attributes (c.f. Section 6.2) from the extracted pairs and encode them using the BERT model (Devlin et al.. 2018). The representation obtained from BERT acts as our commonsense representation. Formally, D.s CE(D), such that Des C IRnd where m is the length of the commonsense informa- tion, and d is the vector dimension obtained from the BERT model. After we obtain the common- sense representation Des, we need to integrate with the dialogue representation De. Consequently, we devise sophisticated fusion mechanism as de- scribed in the following section.
to regulate the extent of information to integrate from the commonsense knowledge and the amount of information to retain from the dialogue context. is a learnable parameter learnt using Equation 3, where WA1: Wky Wa2 and Woz E Rdd11 are trained along with the model.
K V
AR A
K
AB Ao
Uk Up
(1
+
(2)
A A
K V
Wky W mg
U
W
o
+
Des
k2 W
) (3)
U
Finally, the commonsense knowledge infused vectors K and V are used to compute the traditional scaled dot-product attention.
5.3 Commonsense Fusion (CF)
QKT
D. = Softmar
VV
(4)
Several studies discuss knowledge fusion, partic- ularly in the context of multimodal fusion (Po- ria et al., 20)7 where the most successful ap- proaches often employ traditional dot-product- based cross cross-modal attention (Bose et al., 2021; Ma and Ma, 2022). However, the traditional attention scheme results in the direct interaction of the fused information. As each fused information can be orig- inated from a distinct embedding space, a direct fusion may be prone to noise and may not preserve maximum contextual information in the final repre- sentations. To address this, taking inspiration from Yang et al. (2019), we propose to fuse common- sense knowledge using context-awan attention mechanism. Specifically, we first generate com- monsense conditioned key and value vectors and then perform a scaled dot-product attention using them. We elaborate on the process below.
dd,
5,4 Fusion Gating (FG)
In order control the extent of information trans- mitted from the commonsense knowledge and from the dialogue context, we use a Sigmoid gate. Specifically, W E RRddd and b E Rdx1 are trainable param- eters, and @ denotes concatenation. The final information fused representation D. is given by D. De + De. D. is used to identify the emotion class for the input dialogue.
[De + D.JW + +.. Here,
Experiments and Results
6.1 Dialogue Understanding Backbone
As mentiond carlier, existing approaches for ERC predominantly concentrate the English lan- guage. Nonetheless, we incorporate two state-of- the-art techniques for ERC using English datasets and leverage four established Transformer-be based methodologies as our foundation systems to ad- dress the ERC task.
Given the dialogue representation De obtained by dialogue understanding backbone architecture, we calculate the query, key, and value vectors Q, K, and V R respectively, as outlined in Equa- tion where WQ, WK, and Wy E ddx are learn- able parameters, and n and d denote the maximum sequence length of the dialogue and dimensionality of the backbone architecture, respectively.
[QKV]= = D. [W]
(1)
On the other hand, with the commonsense vector, Des, we generate commonsense infused key and value vectors K and V. respectively as outlined in Equation 2, where Uk and U, E ddd are learn- able matrices. scalar À E Rix1 is employed
hhttps://huggingfaac Co/M-CLIP/
M- HARR--Base-iiTe

---

Model
Anger
Contempt
Disgust
Fear
Joy
neutral
Sadness
Surprise
Weighted Fl
BERT
0.23
0.18
0.11
0.20
0.45
0.54
0.16
0.32
0.40
Standard
RoBERTa
0.26
0.21
0.16
0.96
0.47
0,57
0.12
0.34
0.41
MBERT
0.10
0.00
0.23
0.30
MURIL
0.24
0.22
0.07
0.00
0.42
0.51
0.06
0.23
0.35
CoMPM
0.10
0.12
0.00
0.00
0.44
0.57
0.02
0.00
0.35
XialogXL
0.25
0.00
0.07
0.17
0.43
0.17
0.41
0.24 (10.01)
(10.02)
(10.01)
(10.01)
(10.01)
(10.02)
(10.02)
(10.03)
(10.01)
RoBERTa
0.29 (10.03)
0.24 (10.03)
0.18 (10.02)
0.10 (10.04)
0.49 (10.02)
0.61 (10.04)
0.18 (10.06)
0.34 0.000
0.44 (10.03)
MBERT
0.11 (10.01)
0,13 (10.02)
0.04 (10.04)
0,12 (10.01)
0.24 (10.01)
0.51 (10.01)
0.12 (10.01)
0.10 (10.02)
0.31 (10.01)
MURIL
0,26 (10.02)
0.21 (10.01)
0.10 (10.03)
0.01 (10.01)
0.46 (10.04)
0.52 (10.01)
0.08 (10.02)
0.22 (J0.01)
0.37 (10.02)
CoMPM
(10.01)
0.14 (10.02)
0.02 (10.02)
(10.02)
(10.01)
(10.01)
(10.01)
(70.01)
0.36 (10.01)
XL
0,26 (10.93)
0.11 (10.02)
0.10 (10.03)
0.19 (10.02)
0.44 (10.01)
0.59 0.00)
0.20 (10.03)
0.31 (10.03)
0.42 (10.01)
KET
0,14 (10.15)
0.11 (10.13)
0.09 (10.09)
0 (100110)
0.34 (10.15)
0.41 (10.20)
0.08 (10.10)
0.19 (50.15)
0.28 (10.16)
COSMIC
0.21 (10.08)
0.18 (10.06)
0.15 (10.03)
0.03 (10.07)
0.39 (10.10)
0.49 (10.12)
0.13 (10.05)
0.27 (J0.07)
0.34 (J0.10) 10)
Table Performance of comparative systems with and without incorporating commonsense via COFFEE. Numbers in parenthesis indicate the corresponding performance gain over the con-commonsense (standard) version. The last two rows compare the performance of the best performing COFFEE model (RoBERTa) with other commonsense (CS) based ERC methods.
BERT that is trained on a multilingual corpus, en- abling it to understand and process text multiple languages. MURIL (Khamuja et al., 2021) (Multi- lingual Representations for Indian Languages) is a variant of BERT specifically designed to han- dle Indian languages. CoMPM (Lee and Lee, 2022) is Transformer-basee architecture espe- cially curated for ERC. It extracts pre-trained mem- ory as an extractor of external information from the pre-trained language model and combines with the context model. DialogXL (Shen et al., 2020) addresses multi-party structures by utilizing increased memory to preserve longer historical con- text and dialog-aware self-attention. It alters XL. Net's recurrence method from segment utterance level better represent conversational data. KET (Zhong et al., 2019b) or the Khowledge-Enrichee Transformer deciphers contextual statements by employing hierarchical self-attention, while simul- taneously harnessing external common knowledge through an adaptable context-sensitive affective graph attention mechanism. COSMIC (Ghosal et al., 2020b) is framework that integrates various aspects of common knowledge, including mental states, events, and causal connections, and uses them as a foundation for understanding how partic- ipants in a conversation interact with one another. Although BERT and RoBERTa are trained us- ing monolingual English corpus, we use them for romanised code-mixed input, anticipating that fine- tuning will help the models grasp Hindi-specific nuances (c.f. Appendix A.2). To ensure fair com- parison, also include multilingual models such as mBERT and MURIL in our analysis. Addition- ally, since we are dealing with the task of ERC, we consider two state-of of-the-atteeeee
0.0
oWtnnt
xAttr
XEfruct
Ximtent
WNeedd
xWant
Figure 4: Correlation between different commonsense
attributes with the emotion attribute.
in this domain for monolingual English dialogues. namely CoMPM and DialogXL. and two state-of- of- the-art baseline that incorporates commonsense for ERC KET, and COSMIC.
6.2 Experiment Setup and Evaluation Metric
The COMET graph gives us multiple attributes for one input text (c.f. Table 2). However, not all of them contributes towards the emotion elicited in the speaker. Consequently, we examine the correla- tion between the extracted commonsense attributes with emotion labels in our train instances. We use BERT to obtain representation for each common- sense attribute and find out their correlation with the emotion labels. We show this correlation in Fig- ure 4. As can be seen, 'xWant' is most positively correlated with the emotion labels, and 'oReact' is most negatively correlated. Consequently, we select the attributes 'xWant', and 'oReact' as com- monsense. Further, for evaluating the performance, we select weighted Fl score as our metric of choice to handle the imbalanced class distribution of emo- tions present in our dataset (c.f. Figure 2).
6.3 Quantitative Analysis
Table 3 illustrates the results (Fl-scores) we obtain for the task of ERC with and without using COFFEE
the-art art baseline architectures

---

RoBERTa
Anger
Contempt
Disgust
Fear
Joy
neutral
Sadness
Surprise
Weighted Fl
Standard
0.26
0.21
0.16
0.06
0.47
0:57
0.12
0.34
0.41
Concat
0.22 (10.04)
0.19 (.0.02)
0.15 (J0.01)
0,94 (10.02)
0.44 (10.03)
0.52 (10:05)
0.09 (10:03)
0.31 (10.03)
0.37 (10.04)
DPA
0.27 (10.01)
0.21 0.00)
0. 0.00)
0.08 (10.02)
0.48 (10.01)
0.59 (10.02)
0.11 (10.01)
0.33 (10.01)
0.42 (10.01)
10FFEE
0.11 (10.15)
0.09 (20.12)
0.01 (J0.15)
((00006)
0,16 ((0.31)
0.24 (10,33)
0.02 (10.10)
0.11 (10.23)
0.16 (10.25)
CoFFEE
0.20 (10.06)
0.15 (L0.66)
0.12 (J0.05)
0.02 (10.04)
0.36 (j0.11)
0.53 (10.04)
0.12 (10.00)
0.29 (10.05)
0.35 (10.06)
0.26 0.00)
0.22 (20.0)
(10.01)
0.04 (10.02)
0.47 0.00)
0.59 0.02)
(10:04)
0.33 (10.01)
0.42 (10.0)
COFFEE,
0.27 (10.01)
0.24 (20.03)
0.17 (10.01)
0.07 (10.01)
0.43 (10.04)
0.59 (10.02)
0.18 (10.06)
0.33 (10.01)
0.41 () 0.00)
COFFEE
0.29 (10.03)
0.24 (70.03)
0.18 (10.02)
0.10 (10.04)
0.49 (10.02)
0.61 (10.04)
0.18 (10.06)
0.34 (1 0.00)
0.44 (10.03)
Table 4: Ablation results comparing different fusion techniques for the best performing system (RoBERTa). Numbers in parenthesis indicate the performance gain over the non-commonsense (standard) version. Performance when only one of the matrix or embedding language is used for experimentation is also shown. (CS: Commonsense; DPA: Dot
Product Attention; COFFEE,w: COFFEE with only xWant attribute commonsense knowledge; COFFEE,n: COFFEE with only oReact attribute as commonsense knowledge).
offfect
oftorct
<Aitr
xRffect
slairet
xNord
<Woust
representations obtained from the commonsense graph and concatenates them with the utterance representations obtained from the GRU architec- ture. Since we use the generated natural language commonsense with the smart fusion method, we hypothesize that our model is able to capture and utilize this knowledge effectively. Additionally, we performe a T-test on our results to check the sta- tistical significance of our performance gain and obtained p-value of 0.0321 for our RoBERTa model which, being less than 0.05, makes our sults statistically significant.
0.32
0.41
0.39
0.34
0.37
0.37
0.33
0.36
0.42
Table 5: Ablation results comparing different attibutes of commonsense when fused with RoBERTa using the COFFEE. The scores are weighted FI.
to incorporate commonsense knowledge. Notably, in the absence of commonsense, RoBERTa and Di- alogXL. outperform the other systems. However, it is intriguing observe that mBERT and MURIL, despite being trained on multilingual data, do not surpass the performance of BERT, RoBERTa, or XL. We provide a detailed analysis regard- ing this in Appendix A.2. Further, when com- monsense is included as part of the input using the COFFEE approach, all systems exhibit improved performance. The Fl scores corresponding to indi- vidual emotions show a proportional relationship with the quantity of data samples available for each specific emotion, as anticipated within a deep learn- ing architecture. The neutral emotion achieves the highest performance, followed by joy and surprise, as these classes possess a greater number of data samples (see Table 2). Conversely, the minority classes such as contempt and disgust consistently obtain the lowest scores across almost all systems. Furthermore, we can observe from the table that the existing strategies of commonsense fusion per- form poorly when compared with the COFFEE method. The loss in performance can be attributed to two aspects of the comparative system - KET uses NRC_ VAD (Mohammad, 2018 8), which is an English-based lexicon containing VAD scores, i.e., valence, arousal, and dominance scores, to gather words for which knowledge is be retrieved. Since our input is code-mixed with the matrix language as Hindi, using only the English terms makes the KET approach ineffective. In contrast, although COSMIC uses the COMET graph, it uses the raw
6.4 Ablation Study
Fusion methods We investigate the effectiveness of COFFEE in capturing and incorporating common- sense information. To evaluate different mecha- nisms for integrating this knowledge into the dia- logue context, we present the results in Table Ini- tially, we explore a straightforward method of con- catenating the obtained commonsense knowledge with the dialogue context and passing it through the RoBERTa model. Interestingly, this simple con- catenation leads to a decline in the performance of emotion recognition, suggesting that the intro- duced commonsense information may act noise in certain cases. This outcome can be attributed to the inherent nature of some utterances, where external knowledge may not be necessary to ac- curately determine the expressed emotion. For instance, consider the sentence "Aaj me sad hun" ("I am sad today"), which can be comprehended without relying on commonsense information to identify the emotion as sadness. In such scenar- ios, enforcing additional information may disrupt the model's behavior, resulting in suboptimal per- formance. Conversely, by allowing the model the flexibility to decide when and to what extent to incorporate commonsense knowledge, as demon-

---

strated by the attention and COFFEE approaches, we observe an improvement in system performance, with COFFEE yielding the most favorable outcomes. Effect of language In code-mixing, the input amalgamates two more languages, often with one language being the dominant one, called the matrix language, while others act as embedding languages. The foundation of grammatical struc- ture comes from the matrix language (Hindi in our case), and solely relying on the embedding lan- guage (English in our case) can lead to a decline in the model's performance. On the flip side. the embedding language plays vital role in capturing accurate contextual details within the input. There- fore, confining ourselves to only the matrix lan- guage should also result a drop in performance. To verify this hypothesis, the third and fourth row of Table 4 shows the performance of the COF methodology, using the RoBERTa model, when we use only English (the embedding language) and only Hindi (the matrix language) in our input. The results reinforce our hypothesis, where the usage of only embedding language (English only) dete- riorates the model performance extensively, while the sole use of matrix language (Hindi only) also hampers the performance when compared to the system that uses both the languages.
Predicted
Jo
26/33
5/5
8/4
2/9
L3/20
86/62
1/4
Co
664
9/16
6/6
1/2
7/8
5Q442
2/2
1/2
Gold
Fe
9/13
2/5
3/4
1/3
17/19
76555
12/17
2/6
Jo
3/8
3/4
2/2
1/9
159/171
162/139
15/8
4/8
496/450
24/32
Sa
5/11
7/6
5/3
0/1
19/26
90/71
28/35
1/2
8/6
22/18
0/1
26/31
Table 6: Confusion matrices for ERC for the best per- forming RoBERTa model (without/with commonsense). (An: Anger; Co: Contempt: Di: Disgust; Fe: Fear; Jo: Joy; Ne: Neutral: Sa: Sadness; Su: Surprise).
confusion matrices and subjective evaluations.
6.5.1 Confusion Matrix
Given the superior performance of the RoBERTa model conduct an examination of its confusion matrices with and without commonsense fusion, shown in Table 6. We observe that the RoBERTa model with COFFEE integration achieves a higher number of true positives for most emotions. How- ever, it also exhibits a relatively higher number of false negatives when compared with its standard variant, particularly for the neutral class. This ob- servation suggests that the commonsense-infuns model excels in recall but introduces some chal- lenges in terms of precision, thereby presenting an intriguing avenue for future research. Addi- tionally, we notice heightened level of confusion between neutral and joy emotions, primarily due to their prevalence in the dataset. Both models. however, demonstrate the least confusion between the disgust and surprise emotions, indicating their distinguishable characteristics.
COMET attributes We explore the utilization various COMET attributes as our commonsense information. The last three rows in Table 4 demon- strate the outcomes when we integrate the two most correlated attributes, xant and oReact with the RoBERTa backbone model using COFFEE. It is ev- ident that the individual consideration of these at- tributes does not significantly enhance the perfor- mance of ERC compared when they are com- bined. Additionally, Table 5 presents the weighted Fl scores achieved by the RoBERTa model when each commonsense attribute incorporated indi- vidually using COFFEE. These results align well with the observed correlation between the attributes and the corresponding emotion labels in Figure 4,
6.5.2 Subjective Evaluation
For the purpose of illustration, we select a single in- stance from the test set of E-MASAC and present, with it, the ground-truth and the predicted labels for the task of ERC for the best performing RoBERTa model with and without using the COFFEE approach in Table 7. It can observed that the inclusion of commonsense knowledge in the model signifi- cantly reduces errors. Comparatively, the variant of RoBERTa that does not incorporate commonsense knowledge makes errors in 5 out of 9 instances, whereas the variant utilizing commonsense knowl- edge, using COFFEE, misclassifies only 3 utterances. Within the test set, numerous similar instances ex- ist where the commonsense-infusee variant outper- forms its counterpart due to the implicit informa- tion embedded in the utterances.
6.5 Qualitative Analysis
A thorough quantitative analysis, detailed in the previous section, revealed that the integration of commonsense knowledge enhances the perfor- mance of all systems under examination. How- ever, to gain a deeper understanding of the under- lying reasons for this improvement, we conduct a comprehensive qualitative analysis, comprising of

---

Emotion
Speuker
Utterance
Gold
w/o CS
CS
ul
Maya
Khatam bo gaya Sahil it's over! (It's over Sakil!)
sadness
sadness
sadness
02
Monisha
Mummyji, tissue paper ke our boxes hai, lanti hun. (Mumoryji have two more howe of rissue paper, bving them. )
neutral
joy
neutral
u3
Sahil
Monisha, mom paper beat nhi kar thi bai... (Monisha, puper..)
tulking about
neutral
sadaess
u4
Maya
My life! Meri zindagi! Khatam be gayz hai. Can you imagine Sahil?? Uss Rita se toh Monisha ryada achi hai. Can you imagine? (My life! My life! It's over. Can you imagine, Sabil? Monisha is betler you imangine?)
sadness
sadness
sadness
than
us
Monisha Maya
Mein kya imi buri hun mummy ji? (Am that had, mammuyji?)
sadness
sadness disgust
conterppt sadness
u6
Hoan beta. Lekin wo Rita! Oh my god! Soans leti hai toh bhi checkh sunai deti hai. Jab logo ko pata chaloga ke rosesh oc loudspeaker se shsadi ki bai?! (Yes, dear. But that Rita Oh my god! Even when she breathes, she makes a sound. When people find out that Rosesh got married through a loudspaker?!)
sadness
u7
Monisha
Logo pata chal gaya mummyji... ( People found manmyji....
neutral fear neutral
surprise foar surprise
surprisee
u8
Maya
What do you mean?! (Whut do you mean?!)
contempt neutral
u9
Monisha
Wo sarita aunty kn phone aaya tha oa... (Sarita aunty called, right...)
Table 7: Actual and predicted emotions (using RoBERTa) for a dialogue having nine utterances from the test set of E-MASAC. Red-colored text represents misclassification.
6.6 ChatGPT and Code-mixing
sense is integrated into backbone architecture through novel fusion technique that uses context aware attention mechanism. Our findings indicated that the incorporation of commonsense features significantly enhances the performance of ERC, as evidenced both quantitatively and qualitatively.
Considering the emergence and popularity of Chat- GPT, it becomes imperative to conduct an analysis of it for the task of ERC in code-mixed dialogues. Although ChatGPT exhibits remarkable perfor- mance in a zero-shot setting across various tasks and scenarios, it is important to note its shortcom- ings, particularly when dealing with code-mixed input. To evaluate its performance, we extract in- stances from E-MASAC and engage ChatGPT in identifying the emotions evoked within the dia- logues. To accomplish this, we construct a prompt that includes a potential set of emotions along with the code-mixed dialogue as input. Specifically, the prompt used is as follows:
8 Acknowledgements
The authors acknowledge the support of the ihub- Anubhuti-iitd Foundation, set up under the NM- ICPS scheme of the DST. This work is also sup- ported by Infosys Foundation.
Limitations
"Out of the following emotion set (Anger, Con- tempt, Disgust, Fear, Joy, Neutral, Sadness, Sur- prise). find out the emotion for the last utterance given the following conversation. <Conv>" While ChatGPT successfully discerned emotions in short and straightforward conversations, fal- tered in identifying the appropriate emotion as the dialogue context expanded, sometimes even span- ning more than three utterances. We present more details and example instances in Appendix A.3.
This work marks the inception of emotion identi- fication in code-mixed conversations, opening up a plethora of research possibilities. One avenue for future studies is training the tokenizer specifi- cally for code-mixed input, which can enhance the model's performance. Moreover, it is worth not- ing that dataset comprises dialogues extracted from a situational comedy (sit-com). which may not encompass all real- world scenarios, potentially limiting the generalizability of our model to unseen situations. Consequently, future investigations can delve into diversifying the dataset to incorporate a broader range of contexts. Furthermore, conduct- ing in ne-depth analysis to identify the optimal com- bination of commonsense attributes from COMET for ERC would shed further light on improving the system's performance. Although these avenues were beyond the scope of our current study, they present exciting prospects for future research.
7 Conclusion
In this study, we investigated the task of Emotion Recognition in Conversation (ERC) for code-mixed dialogues, the first effort in its kind. We curate an emotion lad dialogue dataset for Hindi-English conversations and proposed a new methodology that leverages existing commonsense knowledge graphs to extract pertinent commonsense concepts for node-mixed inputs. The extracted common-

---

References
Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion, 6(3-4)1169-22
Alexandra Balahur, Jesús M Hermida, Andrés Montoyo, and Rafael Munoz. 2011. Emotinet: A knowledge base for emotion detection in text built the ap- praisal theories. In Natural Language Processing and Information Systems: 16th International Confer- ence on Applications of Natural Language to Infor- mation Systems, NLDB 2011, 1, Alicante, Spain, June 28-30, 2011. Proceedings 16, pages 27-39. Springer.
Siska Fitrianie, Pascal Wiggers, and Leon JM Rothkrantz. 2003, A multi-modal cliza using natu- ral language processing and emotion recognition. In Text, Speech and Dialogue: 6th International Confer- ence, TSD 2003. České Budéjovice, Czech Republic, September 8-12, 2003. Proceedings 6, pages 394- 399. Springer.
Manjot Bedi, Shivani Kumar, Md Shad Akhtar, and Tan- moy Chakraborty. 2023. Multi-modal sarcasm detec- tion and humor classification in code-mixed conver- sations. JEEE Transactions on Affective Computing, 14(2):1633-375..
Deepanway Ghosal, Navonil Majumder, Alexander Gel- bukh, Rada Mihalcea, and Soujanya Poria. 2020a. Cosmic: Commonsense knowledge for emotion identification in conversations. arXiv preprint arXiv:2010.02795.
Rupak Bose, Shivam Pande, and Biplab Banerjee, 2021. Two headed dragons: multimodal fusion and cross modal transactions. In 2021 IEEE International Con- ference on Image Processing (ICIP), pages 2893- 2897. IEEE
Deepanway Ghosal, Navonil Majumder, Alexander Gel- bukh, Rada Mihalcea, and Soujanya Poria. 2020b. COSMIC: COmmonSense knowledge for eMotion identification in conversations. In Findings of the As- sociation for Computational Linguistics: EMNLP 2020, pages 2470-2481, Online. Association for Computational Linguistics.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chai- tanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense transformers for auto- matic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762-4779, Flo- rence, Italy. Association for Computational Linguis- tics.
Deepanway Ghosal, Navonil Majumder. Soujanya Poria, Niyati Chhaya, and Alexander Gelbukh. 2019. Dia- loguegen: A graph convolutional neural network for emotion recognition conversation, arXiv preprint arXiv: 11540.
Luis Caires and Hugo Torres Vieira. 2010. Conversation types. Theoretical Computer Science, 4 (11(51):4399) ))4499- 4440. European Symposium on Programming 2009.
Auli Hakulinen. 2009. Conversation types. The prag- matics of interaction, pages 55-65.
Jing Chen, Tao Yang, Ziqiang Huang, Kejun Wang. Meichen Liu, and Chunyan Lyu. 2022. Incorpo- rating structured emotion commonsense knowledge and interpersonal relation into context-aware emotion recognition. Applied Intelligence, 53(4):4201-42)
Devamanyu Hazarika, Soujanya Poria, Rada Mihalcea, Erik Cambria, and Roger Zimmermann. 2018a. Icon: interactive conversational memory network for mul- timodal emotion detection. In EMNLP, pages 2594- 2604.
Ze-Jing Chuang and Chung-Hsicr Wu. 2004. Multi- modal emotion recognition from speech and text. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 9, Number 2. August 2004: Special Issue on New Trends of Speech and Language Processing, pages 45 62.
Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018b. Conversational memory net- work for emotion recognition in dyadic dialogue videos. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long Papers), pages 2122-2132, New Orleans, Louisiana.
Alan S Cowen and Dacher Keltner. 2017. Self-report captures 27 distinct categories of emotion bridged by continuous gradients. PNAS, 114(38):79000E77900
Devamanyu Hazarika, Soujanya Poria, Roger Zimmer- mann, and Rada Mihalcea. 2021. Conversational transfer learning for emotion recognition. Informa- tion Fusion, 65:1-12.
Heng Cui, Aiping Liu, Xu Zhang, Xiang Chen, Kongqiao Wang, and Xun Chen. 2020. Eeg- based emotion recognition using an end-to-end regional-asymmetric convolutional neural network. Knowledge- Based Systems, 205:10
Abdullah Ilyas, Khurram Shahzad, and Muhammad Kamran Malik. 2023. Emotion detection in code- mixed roman urdu english text. ACM Trans. Asian Low-Resown Lang, Inf. Process., 22(2).
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing, arxiv preprint arXiv:1810.04805
Wenxiang Jiao, Michael Lyu, and Irwin King. 2020. Real-time emotion recognition via attention gated hierarchical memory network. AAAI, volume 34. pages 8002-8009.
Marta Dynel. 2009. Beyond joke: Types of conversa- tional humour. Language and Linguistics Compuss, (25):12841129)

---

Wenxiang Jiao, Haiqin Yang. Irwin King, and Michael Lyu. 2019. Higru: Hierarchical gated recurrent units for utterance-level emotion recognition. arXiv preprint Ntt::/904.0444
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprini 1rXXiv1907.11000
Aditya Joshi, Pushpak Bhattacharyya, and Mark J. Car- man. 2017. Automatic sarcasm detection: A survey, ACM Comput, Surv. 50(5).
Hui Ma, Jian Wang. Hongfei Lin, Xuejun Pan, Yijia Zhang, and Zhihao Yang. 2022. A multi-view net- work for real-time emotion recognition in conversa- tions. Knowledge-Baat Baaed Systems, 107751.
Gabriele Kasper and Johannes Wagner. 2014, Conver- sation analysis in applied linguistics. Annual Review Applied Linguistics, 34:171-212.
Yukun Ma and Bin Ma. 2022. Multimodal sentiment analysis on unaligned sequences via holographic em- bedding. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 8547-8551. IEFE.
Simran Khanuja, Diksha Bansal, Sarvesh Mchtani, Savya Khosla, Atreyee Dey, Balaji Gopalan, Dilip Kumar Margam, Pooja Aggarwal, Rajiv Teja Nagipogu, Shachi Dave, Shruti Gupta, Subhash Chandra Bose Gali, Vish Subramanian, and Partha P. Talukdar. 2021. Muril: Multilingual representations for indian languages. CoRR, abs/2103.10730.
Hiren Madhu, Shrey Satapara, Sandip Modha, Thomas Mandl, and Prasenjit Majumder. 2023. Detecting offensive speech in conversational code-mixed di- alogue social media: A contextual dataset and benchmark experiments. Expert Systems with Appli- cations, 19::1.
Klaus Krippendorff. 2011. Computing krippendorff's alpha-reliability.
Arianna Mencattini, Eugenio Martinelli, Giovanni Costantini, Massimiliano Todisco, Barbara Basile, Marco Bozzali, and Corrado Di Natale, 2014. Speech emotion recognition using amplitude modulation pa- rameters and combined feature selection procedure. Knowledge- Based Systems, 63:68-81.
Shivani Kumar, Atharva Kulkarni, Md Shad Akhtar, and Tanmoy Chakraborty. 2022a. When did you be- come so smart, oh wise one?! sareasm explanation multi-modal multi-party dialogues. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers). pages 5956-5968, Dublin, Treland. Association for Computational Linguistics.
Saif Mohammad, 2018. Obtaining reliable human rat- ings of valence, arousal, and dominance for 20,000 Fnnglish words. In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). pages 174-184, Melbourne, Australia. Association for Computational Linguistics.
Shivani Kumar, Ishani Mondal, Md Shad Akhtar, and Tammoy Chakraborty. 2022b. Explaining (sarcastic) utterances to enhance affect understanding in multi- modal dialogues.
Weizhi Nie, Yuru Bao, Yue Zhao, and Anan Liu. 2023, Long dialogue emotion detection based on common- sense knowledge graph guidance. IEEE Transactions on Multimedia, pages 1-15.
Joosung Lee and Wooin Lee. 2022. CoMPM: Con- text modeling with speaker's pre-trained memory tracking for emotion recognition in conversation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 1669-5679, Seattle, United States. Association for Computational Linguistics.
Rosalind W. Picard. 1997. Affective Computing. MIT Press, Cambridge, MA, USA.
Soujanya Poria, Erik Cambria, Rajiv Bajpai, and Amir Hussain. 2017a. A review of affective computing: From unimodal analysis to multimodal fusion. Infor- mation mutionfusion, 37:98-125.
Dayu Li, Xiaodan Zhu, Yang Li, Suge Wang, Deyu Li, Jian Liao, and Jianxing Zheng. 2021. Enhane- ing emotion inference in conversations with com- monsense knowledge. Knowledge-Based Systems, 107:49.
Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Navonil Majumder. Amir Zadeh, and Louis- Louis-Philippe Philippe Morency. 2017b. Context-dependent sentiment anal- ysis in user- aser-generated videos. In ACL, pages 873- 883.
Haifang Li, Na Pang, Shangbo Guo, and Heping Wang. 2007. Research on textual emotion recognition in- corporating personality factor. In 2007 IEEE Inter- national Conference on Roboties and Biomimetics (ROBIO), pages 2222-2227. IEEE.
Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in con- versation: Research challenges, datasets, and recent advances. IEEE Access, 7: 7:100943-100953.
Wei Li, Wei Shao, Shaoxiong Ji, and Erik Cambria. 2020. Bieru: bidirectional emotional recurrent unit for conversational sentiment analysis. arXiv preprint arXiv:2000.0049..
Maarten Sap, Ronan Le Bras, Emily Allaway, Chan- dra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. 2019.

---

Atomic: An atlas of machine commonsenss for if- then reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):32773333.
Li Zhang, Kamlesh Mistry, Siew Chin Neoh, and Chee Peng Lim. 2016. Intelligent facial emo- tion recognition using moth-firefly optimization. Knowledge- Based Systems, 111:248-267.
T Tulasi Sasidhar, Premjith B, and Soman K P. 2020. Emotion detection in hinglish(hindi+eenlihhh code- mixed social media text. Procedia Computer Sci- ence, 171:1346 1352. Third International Confer- Computing and Network Communications (CoCoNet'19).
Peixiang Zhong, Di Wang, Pengfei Li, Chen Zhang, Hao Wang. and Chunyan Miao. 2021. Care: Commonsense-aware emotional response generation with latent concepts. Proceedings of the AAAI Conference on Artificial Intelligence, 35(11):14577 14585.
Weizhou Shen, Junqing Chen, Xiaojun Quan, and Zhixian Xie. 2020. Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition. arXiv preprint arXiv:2012.08695.
Peixiang Zhong, Di Wang, and Chunyan Miao. 2019a. nowleede-ennrchee transformer for emotion detec- tion in textual conversations. In EMNI.P-JJCNLP, pages 165-176, Hong Kong, China, Association for Computational Linguistics.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. open multilingual graph gen- eral knowledge. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 17, page 4444-4451. AAAI Press.
Concepinet 5.5:
Peixiang Zhong, Di Wang. and Chunyan Miao. 2019b. Knowledge-enriched transformer for emotion de- tection in textual conversations. In Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th Interna- rional Joint Conference Natural Language Pro- cessing (EMNLP-IJCNLP), pages 165-176, Hong Kong, China.
Andi Suciati and Indra Budi. 2020. Aspect-based senti- ment amalysis and emotion detection for code-mixed review. International Journal of Advanced Computer Science and Applications, 11(9).
Naf' Tarihoran and lin Ratna Sumirat. 2022. The impact of social media on the use code mixing by generation z. International Journal of Interactive Mobile Technologies (iJIM), 16(7):54-69.
Xun Zhu, Yinxia Lou. Hongtao Deng, and Donghong Ji. 2022. Leveraging bilingual-vies -iiew parallel translation code-switched emotion detection with adversarial dual-channel encoder. Knowledge-Ban -Baedd Systems, 235:107436.
Mary WJ Tay, 1989. Code switching and code mixing as a communicative strategy multilingual discourse. World Englishes, 8(3):407-417.
Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. CoRR, abs/ atss/15060065 506.06624.
S Thara and Prabaharan Poornachandran. 2018. Code- mixing: A brief survey. In 2018 International con- ference advances in computing, communications and informatics (ICACCI). pages 2382-2388 IEEE.
Geng Tu, Jintao Wen, Cheng Liu, Dazhi Jiang, Erik Cambria. 2022. Context- and awwa net- works for emotion recognition in conversation. IEEE Transactions on Artificial Intelligence, 3(5):699-708.
William Turnbull. 2003. Language action: Psycho- logical models of conversation. Routledge.
Anshul Wadhawan and Akshita Aggarwal. 2021, To- wards emotion recognition in hindi-english code- mixed data: A transformer based approach.
Edda Weigand. 2010. Language dialogue. 7(3):505- 515.
Baosong Yang, Jian Li, Derek Wong, Lidia Chao, Xing Wang. and Zhaopeng Tu. 2019. Context-aware self-attention networks. Proceedings of the AAAI Conference Artificial Intelligence, 33(01)3387- 394.
Lin Yang. Yi Shen, Yue Mao, and Longjun Cai. 2022. Hybrid curriculum learning for emotion recognition in conversation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 36, pages 11595-11603.

---

A Appendix
Engiss Hindi Code-mixed
A.1 Emotion Labels
We create E-MASAC by utilising code-mixed dia- logues from MASAC and asking annotators an- notate them with emotion labels. We consider the Eckman's emotions our choice of emotion la- bels due its prevalence in established English based ERC. This set contains seven emotions, namely anger. fear, disgust, sadness, joy, surprise, and contempl, along with a label for no emotion, i.e. neutral. Each emotion with their definition is illus- trated in Table 8.
-02-
-03-
-0,4
-0.2
(a) Pre-trained
0.3
A.2 Embedding Space for BERT
Code-mixed
We employed various Transformer based methods our dialogue understanding backbone com- plete the task of ERC. Although RoBERTa per- forming best for ERC, it is important to note that RoBERTa is pre-trained on English datasets (Book- Corpus (Zhu et al., 2015) and English Wikipedia.). In order to explore how the representation learning is being transferred to code-mixed setting, analyse the embedding space learnt by the model before and after fine-tuning for our task. We con- sidered three random utterances from E-MASAC and created three copies of them- one English, one in Hindi (romanised), and one without modifi- cation i.e. code-mixed. Figure illlustrates the PCA plot for the embeddings obtained for these nine utterance representations obtained by RoBERTa before and after fine-tuning on our task. It is inter- esting note that even before any fine-tuning the Hindi, English, and code-mixed representations lie closer each other and they shift further closer when we fine-tune our model. This phenomenon can be justified as out input is of romanised code- mixed format and thus we can assume that repre- sentations are already being captured by the pre- trained model. Fine-tuning helps us understand the Hindi part of the input.
0.1
-02-
-0.2
0.4
(b) Fine-tuned
Figure 5: Embedding space for RoBERTa before and after fine-tuning on emotion recognition conversation.
zero-shot scenarios across diverse tasks, it is essen- tial to acknowledge its limitations, especially when confronted with code-mixed input. To assess its ef- ficacy, select instances from the E-MASAC and task ChatGPT with the identification of emotions elicited within the dialogues. By subjecting Chat- GPT to this evaluation. we gain insights into its effectiveness for the ERC task. To accomplish this, we construct a prompt that includes a potential set of emotions along with the code-mixed dialogue input. Specifically, the prompt is as follows:
"Out of the following emotion set (Anger, Con- tempt, Disgust, Fear, Joy, Neutral, Sadness, Sur- prise), find out the emotion for the last utterance given the following conversation. <Conv>" Although ChatGPT demonstrated proficiency in discerning emotions within concise and uncompli- cated conversations, its performance waned when faced with the challenge of identifying the accurate emotion as the dialogue context extended, occa- sionally encompassing more than three utterances. Figure shows four such instances and Table 9 shows their translation for easy understanding.
A.3 ERC for Code-mixed Dialogues Using ChatGPT
Given the rise and widespread adoption of expan- sive language models such as ChatGPT, is cru- cial to undertake comprehensive analysis of its capabilities in the context of ERC. While Chat- GPT has demonstrated impressive performance in
"We consider its latest version picked from here: https: // //ww. paulekman. wtw.kannncoomnni..eeaae/// wwwm//nnieeraalll wereeeerllleeeeee -enrsallmmooooss//

---

Emotion
Description
Example
Anger
Arises when the target blocked from pursuing goal and/or treated unfairly Feeling of dislike for and superiority (usually morally) over another person, group of people, and/or their actions
"Get out of my way!"
Contempl
"I'm hetter than and are lesser than me.
Disgust
Feeling of aversion towards something offensive
"Who put this dead mouse here!!"
Joy
Arising from connection or sensury pleasure
"Thal was so fun!"
Fear
Arises with the threat of harm, cither physical, cmotional, or psychological, real or imagined
"Is there no way out from here?"
Sadness
Resulting from the loss of someone or something important. What causes one's sadness varies greatly based on personal and cultural notions of loss Arises when the larget encounters sudden and unexpected sounds, move- ments, situations and actions
"My dog died two days ago"
Surprise
"I did not expected it he this good!"
Neutral
When target experiencing none the emotions
"Hey. what are you doing today?"
Table Fekman's emotions, their description, and examples.
dAngan Cor Contame Elggusi, Hoourd Sushnow Suappr arettanite the
khetem gaas Samill rist
bust
given
imagines?
Rzeads szonde men gahow, hagicherid Ahanbbuics thxn pehne me samraoo He paos nuu saran sanan, meivetes. hows, mei zoneenn, covon.
charege tockspecker
pronaad Rossih yoluo tune nutin tox.
Morionia
pane vunning
Ressans Angy asplo salse para chais?
Maped What meart?
Inoiaadan Hyane sconi hal Sout harslloo ada
Monama: We aurta surty puotone
Basedion the con consessttin provited the altaranco "Wo tanta la mh warry
the muth, who vmite 1.77..
The cmst udd with tteranco appoors Surprioo
Theerodio ausociate welv rpppors Bururnce
oftheril
Contornt Dioaut, heuto,
Out the to leinng enudio oin laager Contoret Orggust, Roor, Jey hestra Sodinssss Surpetud, the the kass ute fit
Maye Whnte pleasse surpriod
nochts
Inchenarcham: Manrouch hurgiumahnn piorgg Mays Muy Subahh chaisiMaa
Rusci the conersation provided the kus uttoonne to "hanooo volo walo hal bolwas dberatts lobes your
Rasud the coneiritiinn pers tholaut unteure ree ic Mar ravt gir eatiiyyyung hare hedl srtt a times sinco menennnn
arydhing and pomster surph sor
wwwiitnnonnnnnoo aeee with wec.tteeeenneee
The smot on snsociated with th apoaan Dlggurt
Figure 6: Screenshots of ChatGPT responses when prompted to provide emotions for the last utterance in the dialogue. The translations are given in Table 9,

---

Code-mixed Dialognes
English Traansaaionn
ChatGPT Emotion
Rosesh: Pani Ki boonde bani meri gehne, bagiche ki kharchha kn
Rosesh: Water droplets have tarned into my jewelty, adorrn the ullire of the gardle's frugrance, I sway near the borders. swaying gently. am the the breeze.
Surprise
libuas pehne mei sarhndo ke paus letrun, suman suman, mei vauyu, mei
pavan.
the earth, the wind. Indravardham: Rosesh, you dida't 't write this poem.
Indravardhan: Rosesh ye kavitu tume nahin lihhi bai.
Rosesh: Arey, sapko pata
Rosesh:
did you
Indravardhum: Kyunki yo wcchi hai! Sach bata kisnee likhi!!
Indravardiaa: Because it's good! Tell me the truth, who wrote ir?!
Maya: Khatam gaya
over!
Maya: over,
Surprise
Monisha: Mummyji tissue puper ke box hai. Lasti hun...
Monisba: Munmyji. have
boxes of tissue paper. I'll
Sabit: Monisha, mom tisoue paper ki haat nhi kar thi hai.
bring them.
Maya:
bo gayi bai? you imagine Sahil? Uss Rita se toh Monisha zyada achi hai! Can imagine!
zindagi!
Sahil: Monislu, Mom not talking tissue paper...
Maya: My life! My life! It's over. Can you imagine, Sahit?
Monisho is better than Rita. Can you imagine?
Monisha: Mein imi buri hon mummyji?
Monisha: Am that had, mummyji?
Maya: Haan beta, lekin wo Rita! Oh my god! Seams leti bai toh cheek
Maya: Yes, dear. But that Rita! Ob my god! Even when she people find that Rosesh martied through loudspeaker!!
logo pata chalega
ne
breathes,
loudspeaker shuadi ki bei...
Monisha: Logo ko pata chal gaya mummyji...
Monisha: People found out, mummyji...
Maya: What you
Maya: What do you memn!!
Monisha: Wo sarita aunty ka phone anya tha na...
Monisha: Sarita aunty called, right...
Baldev: Hi Maya.
Baldev: Hi Maya.
Joy or Surprise
Maya: Hello baldev! What pleasant surprise!
Hello Baldey
a picasant surprise!
Baldev: Sorry main Bina phone a Gaya.
Baldey: Sorry, I came without calling.
Indravardban: No problem. Hamare kele wala hai na val bli baguir bole dhamakta Hai Kabhi Kabhi.
Indravardhan: No problem, banana seller also comes with-
saying anything and sometimes surprises
Indravardhan: Good morning Maya
Indravardham: Good morning, Maya.
Disgust
Maya: Good momnnn sweetic
Maya: Good moming. sweetic.
Indravardhun: Nashia nushia nushta
Indravardhan: Breakfast, breakfast, breakfast.
Maya: Vinthal Sabioo nashsa do
Maya: Vinthal, serve breakfast to everyone.
Indravardban:
hurgis piyungu. Maya Mujhe
Indravardhan:
drink milk. Maya, diarrieu times since morning.
diaria Hai Subah So 6 bar In chuka Hun Main
Table 9: Examples of dialogues we queried ChatGPT along with their English translations and emotion labels recognised by ChatGPT. The target utterances highlied with red colour.


Reviews:
[{"review_id": "GBSHI6XiAQ", "reviewer": "Reviewer_QymD", "summary": "", "questions": "A. Which resource is used to identify the word written in English as a Hindi word to convert it to Devanagari?\nB. How do you determine the relevance of Knowledge Graphs (in this case, COMET) with respect to the dataset? \nC. Is the proposed model domain agnostic? \nD. If not, what measures can be implemented to avoid Knowledge Noise?  ", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments, which we address below.\n\n**A. (Resource to identify Hindi words)** We used the Textcat module of the NLTK library to identify languages. It uses the TextCat algorithm proposed by Cavnar et al. [1].\n\n*[1] Cavnar, W. B. and J. M. Trenkle, \"N-Gram-Based Text Categorization\".*\n\n**B. (Why COMET)** Since we had open domain chit-chat-based conversations in our dataset, we wanted to apply a generic commonsense architecture that returned various characteristics of the input. Since COMET returned multiple attributes related to speakers and listeners, it fitted perfectly with our requirements. Moreover, we did a thorough literature review and found COMET to be a popular choice for the English emotion recognition task and, thus, decided to go forward with it.\n\n**C&D. (Is the model domain agnostic?)** We tested our model in the open-domain chit-chat setting, where the speakers can talk about *anything*. Consequently, we can say that the model works for the open domain category, making it domain-independent.\n\n**(Missing references and typos)** Thank you for pointing these out. We will add the suggested edits to the updated manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The work is novel and new from the perspective that in India, Hindi and English are fused in a way, that it is common practice to use English words while speaking Hindi, and it does not follow any structure. Therefore, it makes communication unusual and complicated for machines to comprehend. This work is a step toward addressing the challenges in code-mixed language in multilingual settings. The world also contributes—a dataset that is helpful for the research community, and unfolds another use case for ChatGpt. \n", "reasons_to_reject": "Since the work is novel in nature, I do not see any strong reason to reject the manuscript. ", "questions_to_authors": "A. Which resource is used to identify the word written in English as a Hindi word to convert it to Devanagari?\nB. How do you determine the relevance of Knowledge Graphs (in this case, COMET) with respect to the dataset? \nC. Is the proposed model domain agnostic? \nD. If not, what measures can be implemented to avoid Knowledge Noise?  ", "paper_topic_and_main_contributions": "The manuscript “From Multilingual Complexity to Emotional Clarity: Leveraging Commonsense to Unveil Emotions in Code-Mixed Dialogues” presents an approach that integrates commonsense information with dialogue context to interpret emotions better by means of a pipeline based on a knowledge graph to use relevant knowledge. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Minor Grammatical punction errors. "}, "extra_text": {"Missing_References": "One reference that closely aligns with the manuscript to address Knowlege Noise and Relevance of the KG (Quantitative  and Qualitative) :  \n “K-LM: Knowledge Augmenting in Language Models Within the Scholarly Domain” (DOI: 10.1109/ACCESS.2022.3201542)  "}}, {"review_id": "qJha0YOPtQ", "reviewer": "Reviewer_m45s", "summary": "", "questions": "A. What is the performance when only considering the English words in the conversation turn for commonsense?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments, which we address below.\n\n**(Comparison with commonsense ERC models)** Since the existing commonsense ERC architectures were curated only for English, we did not consider them in our comparative systems. However, considering the criticality of the reviewer's suggestion, we have run the suggested baselines on our data and provided the results in the table below. \n\n|   Model   | Anger | Contempt | Disgust | Fear | Joy | Neutral | Sadness | Surprise | Wt. F1 |\n|-----------|-------|----------|---------|------|-----|---------|---------|----------|--------|\n| KET [1]   | 0.14  |   0.11   |  0.09   | 0.0  |0.34 |  0.41   |  0.08   |  0.19    |  0.28  |\n| COSMIC [2]| 0.21  |   0.18   |  0.15   | 0.03 |0.39 |  0.49   |  0.13   |  0.27    |  0.34  |\n| **Ours**      | **0.29**  |   **0.24**   | **0.18**   | **0.10** |**0.49** | **0.61**   |  **0.18**   |  **0.34**    |  **0.44**  |\n\nWe can observe from the table that the existing strategies of commonsense fusion perform poorly when compared with the COFFEE method. The loss in performance can be attributed to two aspects of the comparative system -- \n1. KET [1] uses NRC_VAD [3], which is an English-based lexicon containing VAD scores, i.e., valence, arousal, and dominance scores, to gather words for which knowledge is to be retrieved. Since our input is code-mixed with the matrix language as Hindi, using only the English terms makes the KET approach ineffective.\n2. Although COSMIC [2] uses the COMET [4] graph, it uses the raw representations obtained from the commonsense graph and concatenates them with the utterance representations obtained from the GRU architecture. Since we use the generated natural language commonsense with the smart fusion method, we hypothesize that our model is able to capture and utilize this knowledge effectively.\n\n*[1] Zhong, Peixiang, Di Wang, and Chunyan Miao. Knowledge-enriched transformer for emotion detection in textual conversations. EMNLP-IJCNLP, 2019.*\n\n*[2] Deepanway Ghosal, Navonil Majumder, Alexander Gelbukh, Rada Mihalcea and Soujanya Poria. COSMIC: COmmonSense knowledge for eMotion Identification in Conversations. EMNLP (Findings), 2020.*\n\n*[3] Saif Mohammad. Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words. ACL, 2018.*\n\n*[4] Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chai tanya Malaviya, Asli Celikyilmaz, and Yejin Choi. COMET: Commonsense transformers for auto640 matic knowledge graph construction. ACL, 2019.*\n\n**(Performance gain by COFFEE is minimal)** We appreciate the concern regarding the performance gain. However, we would like to clarify a possible misunderstanding in this regard. In Table 3, we showed the experimental results as the raw F1 scores without converting them to percentages. Consequently, an increase of 0.03 in the F1 score for the RoBERTa model signifies a 3% increase (from 41% to 44%). We have recently performed a T-test on our results to check the statistical significance of our performance gain and obtained a p-value of 0.0321 which, being less than 0.05, makes our results statistically significant.\n\n**(Loss of knowledge due to translation)** We used the googletrans python library to translate sentences into English. This library provides the free use of Google translate API. To ensure the integrity of our dataset, we diligently performed manual checks. However, the COMET commonsense network can take input with a maximum sequence length of 17 tokens, which is less than most of the sequence lengths in our input since our average sequence length, as mentioned in Table 1 of the paper, is 31.55 tokens. To address this, we removed the stop word to adjust the sequence length while retaining information. This strategy's simplicity and effectiveness guided our choice.\n\nTo reinforce the quality of translations, we show some examples in the table below. We can observe that the translated text retains its quality, preserving the relevance of the extracted commonsense knowledge to the original input.\n\n| Input (After preprocessing) | Translation | Commonsense knowledge |\n|-----------------------------|-------------|-----------------------|\n| monisha तुम्हारे फटे हुए कुर्ते rosesh थोड़ा फरक | monisha your torn kurta rosesh slightly different | xWant: to be unique; oReact: wear a new one |\n| indravadan, कितने सालो joke मार रहे. कोई हँसा आज तक? | Indravadan, how many years have you been joking? Anyone laughed today? | xWant: to be funny; oReact: laughs |\n| एक minute एक minute kismi. इसका मतलब railway station जाकर सोने सोच | One minute one minute kismi. This means thinking of going to the railway station and sleeping. | xWant: one minute; oReact: get to sleep |\n\nAdmittedly, though the translation achieves a decent level of accuracy, we acknowledge the presence of nuances that warrant further enhancement. However, the intricate realm of language translation is a domain distinct from the focus of our current paper, and thus, we do not discuss it further in the work.\n\n**(English-only performance)** In code-mixing, input amalgamates two or more languages, often with one language being the dominant one, called the matrix language, while others act as embedding languages. The foundation of grammatical structure comes from the matrix language (Hindi in our case), and solely relying on the embedding language (English in our case) can lead to a decline in the model's performance. On the flip side, the embedding language plays a vital role in capturing accurate contextual details within the input. Therefore, confining ourselves to only the matrix language should also result in a drop in performance.\n\nThe table below shows the performance of the COFFEE methodology, using the RoBERTa model, when we use only English (the embedding language) and only Hindi (the matrix language) in our input. \n\n|  Model  | Anger | Contempt | Disgust | Fear | Joy | Neutral | Sadness | Surprise | Wt. F1 |\n|---------|-------|----------|---------|------|-----|---------|---------|----------|--------|\n| English | 0.11  |   0.09   |  0.01   | 0.0  |0.16 |   0.24  |  0.02   |   0.11   |  0.16  |\n|  Hindi  | 0.2   |   0.15   |  0.12   | 0.02 |0.36 |   0.53  |  0.12   |   0.29   |  0.35  |\n|  **Both**   | **0.29**  |   **0.24**   |  **0.18**   | **0.10** |**0.49** |   **0.61**  |  **0.18**   |   **0.34**   |  **0.44**  |\n\nWe observe the same pattern as discussed above, where the usage of only embedding language (English only) deteriorates the model performance extensively, while the sole use of matrix language (Hindi only) also hampers the performance when compared to the system that uses both the languages.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Their dataset would encourage the study of emotions in code-mixed conversations.", "reasons_to_reject": "* Translating only parts of each conversation turn may lead to a loss of meaning. As a result, the commonsense knowledge extracted might be inaccurate.", "questions_to_authors": "A. What is the performance when only considering the English words in the conversation turn for commonsense?", "paper_topic_and_main_contributions": "The author created a dataset of English-Hindi multiparty conversations with emotional annotations for each turn. They introduced a new method to extract commonsense knowledge from Hindi words from a pretrained English commonsense knowledge graph.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ks6SFBjBSP", "reviewer": "Reviewer_FCsy", "summary": "", "questions": "- l.135-7: Could you provide some more details on these DL methods for ERC? Which general trends are there?\n- l.199-200: \n(a) What is the average length of these conversations and utterances? Could you also report standard deviation for average conversation length? \n(b) In the introduction (l. 113), you state that you work on multi-party conversations, while here you state that you work on dialogues. Which of the two is true?\n- Section 4:\n(a) Could you add a table in which you calculate annotator agreement for each emotion label seperately?\n(b) How did you handle instances where multiple emotion labels were applicable for a single utterance? Which label did annotators have to choose in such cases?\n(c) Footnote 4: How were the annotators remunerated? Per hour (focus on quality) or per instance (focus on quantity)?\n(d) l.221-2: how rare? Can you give an indication of how often such discrepancies occurred relative to the total number of instances to label?\n(e) Why was Kriprendorff’s Alpha calculated on annotator pairs instead of all annotators together if all annotators labelled the same instances?\n(f) Were IAA scores calculated before or after the annotators solved their discrepancies? If after, what was the IAA before?\n- l.253-60: DUB: Did you add [SEP] tokens between utterances to signal where one utterance ends and another one starts? If not, how were utterances joint?\n- Section 6.5.2 on subjective evaluation: Could you, similarly, look at an example in which the RoBERTa model with COFFEE approach performs worse than the RoBERTa model without COFFEE approach. What might be the cause?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments, which we address below.\n\n**(Performance gain by COFFEE)** We appreciate the concern regarding the performance gain. However, we would like to clarify a possible misunderstanding in this regard. In Table 3, we reported the experimental results as raw F1 scores without converting them to percentages. Consequently, an increase of 0.03 F1 score for the RoBERTa model signifies a 3% increase (from 41% to 44%). We have recently performed a T-test on our results to check the statistical significance of our performance gain and obtained a p-value of 0.0321 which, being less than 0.05, makes our results statistically significant.\n\n**(ChatGPT section)** The section on ChatGPT was incorporated to highlight future research directions. We wanted to focus on the fact that while ChatGPT solves many problems and responds appropriately to variable queries, the research in the NLP domain is still far from complete. Consequently, we highlighted one drawback of the LLM related to how it handles a mix of languages in text since it was in line with our work. However, due to the associated cost with ChatGPT or the GPT3 API, we couldn't get the predictions for our complete test set. Thus, we couldn't provide comparable metrics for the complete set. Consequently, we randomly selected a few instances from the test set and queried them into ChatGPT. Specifically, we queried four instances and showed their results in the Appendix (Figure 6 and Table 9).\n\n**(More data statistics)** We summarize the requested dataset statistics here. We will add these statistics in the manuscript, too.\n1.  ***Average length of utterances:*** 31.55 words (Mentioned in Table 1 of the main paper)\n2.  ***Average length of conversations:*** 24.76 utterances\n3.  ***Standard deviation of conversation length:*** 14.55\n\nPlease note that we work on multi-party conversation and use the terms 'dialogue' with 'conversation' interchangeably. Specifically, the term 'dialogue' refers to a conversation between two or more people in our study. We will mention this explicitly in the manuscript and make it consistent.\n\n**(More details on annotations)** Please find the clarifications regarding emotion annotations below. We will add these to the manuscript.\n1. ***Annotator agreement for individual emotion label:***\n\n| Anger | Contempt | Disgust | Fear | Joy | Neutral | Sadness | Surprise |\n|-------|----------|---------|------|-----|---------|---------|----------|\n| 0.87  |  0.81    |  0.81   | 0.82 |0.93 |  0.89   |  0.87   |   0.83   |\n\n2. ***Multiple emotion labels for a single utterance:*** While annotating the instances, the annotators were asked to focus on the most prominent emotion in the utterance and assign that to the entire statement. We did not encounter any instance in our data where more than one emotion played a *crucial* part. Thus, our prominent emotion strategy worked for our data.\n\n3. ***Annotator remuneration:*** The annotators were linguistics experts. They were paid as per the institute rule.\n\n4. ***Annotation discrepancies:*** Out of the 11440 total utterances, there were only 43 utterances where the annotators had a disagreement over emotion labels. Most of them were one word utterances, and the discrepancies were resolved after a discussion among the annotators.\n\n5. ***IAA calculation:*** The IAA score was calculated after resolving the discrepancies. Since the number of discrepancies is low with respect to the total number of utterances, the score before discrepancy resolution is similar to the final score of 0.85.\n\n**(Utterance separator)** Yes, we used the [SEP] token to join different utterances.\n\n**(Subjective evaluation)** While there are several instances for which COFFEE outperforms the non-CS approach, there are a few samples where the opposite is observed. We show one such example in the table below.\n\n|   |   Speaker   | Utterance | Gold Emotion | Emotion w/o CS | Emotion w CS |\n|---|-------------|-----------|---------|------|--------|\n|u1 |    Maya     | Vishal ji I am so sorry, Monisha ki jidh thi ki hum log ish tara se corridor me cricket khele toh. *(Vishal ji m so sorry, it was Monisha's insistence that we should play cricket in the corridor like this.)* | neutral | neutral | sad |\n|u2 | Vishal Guha | Nehi nehi koi baat nehi. Hamari cricket career ki suruwat bhi corridor se hi huyi hai. *(No no no problem. Our cricket career also started from the corridor itself.)* | neutral | joy | joy |\n|u3 |    Indu     | Accha. Chaliye chaliye. Chaliye vishal ji game suru kijiye chaliye. *(Good. Let's go let's start the game vishal ji.)* | joy | joy | joy |\n|u4 | Vishal Guha | Bat kaun karega? *(Who will bat?)* | neutral | neutral | neutral |\n|u5 |   Monisha   | Mai. *(Me.)* | joy | joy | surprise |\n|u6 | Vishal Guha | Chalo. *(Let's go.)* | neutral | joy | neutral |\n\nThe instances where COFFE fails to capture the correct emotions might be due to the fact that the commonsense acts as noise instead of a helper and adds confusion to the system.\n\n**(Grammar, style, and presentation improvements)** Thank you for pointing these out. We will add the suggested edits to the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The authors have created the first dataset for emotion recognition in conversation (ERC) in a code-mixed setup. So far, ERC has not been investigated in a code-mixed setting and this novel resource will help future research on code-mixing.\n- The authors also present COFFEE, a method to extract commonsense knowledge from English knowledge graphs and use it in a code-mixing setup. The validity of this model is shown through various quantitative and qualitative analyses.\n- The paper is well-written and presents not only an interesting, but highly relevant contribution to the field of ERC and code-mixing.", "reasons_to_reject": "- Some details on the dataset are missing (see questions for authors).\n- Although the model COFFEE improves upon the baselines, the improvements are relatively small.\n- The section on ChatGPT (Section 6.6) is somewhat out of line with the rest of the paper and seems to have been added to an already nicely structured project. Although such an addition is valuable, the current addition does not fit the structure of the paper. If you incorporate this model, it should be included earlier throughout the experimental section, not at the end after all analyses. Moreover, performance is not clearly reported (metric, number of instances involved).", "questions_to_authors": "- l.135-7: Could you provide some more details on these DL methods for ERC? Which general trends are there?\n- l.199-200: \n(a) What is the average length of these conversations and utterances? Could you also report standard deviation for average conversation length? \n(b) In the introduction (l. 113), you state that you work on multi-party conversations, while here you state that you work on dialogues. Which of the two is true?\n- Section 4:\n(a) Could you add a table in which you calculate annotator agreement for each emotion label seperately?\n(b) How did you handle instances where multiple emotion labels were applicable for a single utterance? Which label did annotators have to choose in such cases?\n(c) Footnote 4: How were the annotators remunerated? Per hour (focus on quality) or per instance (focus on quantity)?\n(d) l.221-2: how rare? Can you give an indication of how often such discrepancies occurred relative to the total number of instances to label?\n(e) Why was Kriprendorff’s Alpha calculated on annotator pairs instead of all annotators together if all annotators labelled the same instances?\n(f) Were IAA scores calculated before or after the annotators solved their discrepancies? If after, what was the IAA before?\n- l.253-60: DUB: Did you add [SEP] tokens between utterances to signal where one utterance ends and another one starts? If not, how were utterances joint?\n- Section 6.5.2 on subjective evaluation: Could you, similarly, look at an example in which the RoBERTa model with COFFEE approach performs worse than the RoBERTa model without COFFEE approach. What might be the cause?", "paper_topic_and_main_contributions": "The authors of this paper present the first dataset for emotion recognition in code-mixed conversations, called E-MaSaC. They also propose COFFEE, a method to extract commonsense knowledge from English knowledge graphs (not trivial given the code-mixed data) and fuse this knowledge with dialogue context. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "- l.135-7: Could you provide some more details on these DL methods for ERC? Which general trends are there?"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Abstract (l.13-5): you seem to claim that your innovation is integrating commonsense knowledge with dialogue context, while this has already been studied before. Obviously, there is novelty to this approach due to the code-mixing setup, but could you rephrase this part slightly to avoid confusion?\n- l.183-4: Ekman's model consists of 6 instead of 7 basic emotions.\n- Figure 3: There are still capitals in the text of the preprocessing step\n- l. 307: Refer -> We refer to\n- l. 390-3: RoBERTa is also trained without next-sentence prediction loss, in contrast to BERT\n- l.425: contributes -> contribute\n- l.449-50: Could you summarize this detailed analysis in a sentence or two?\n- Table 5: Could the order of attributes be changed to the order in Figure 4? (easier for comparison)\n- l.508-24: You refer to the section on the ablation study the quantitative section, while the section prior to this (Section 6.3) is called \"Quantitative analysis\". Moreover, the confusion matrix in section 6.5.1 is also a quantitative analysis."}}]

Author rebuttal:

Meta-review: The paper studies the task of Emotion Recognition in Conversations (ERC) in the code-mixed setting. In particular, it studies the commonsense-aided ERC task.

Largely, the paper (a) Proposes a dataset towards this task (b) Does a study of translating code-mixed sentences to English to perform commonsense inference, and (c) Provides comprehensive experiments.

Reviewers appreciated the presentation of the paper and its technical soundness. The questions raised during the discussion were aptly addressed by the authors. Furthermore, reviewers liked the fact that ERC is being studied in code-mixed setting and appreciated the promise to release the prepared dataset.
Decision: Accept-Main