An Empirical Study on Multiple Knowledge from ChatGPT for Emotion Recognition in Conversations
Authors: Geng Tu
Bin Liang
Bing Qin
Kam-Fai Wong
Ruifeng Xu
Venue: EMNLP 2023 Findings (2023)
Keywords: Emotion Recognition in Conversations
Graph Network
Supervised Contrastive Learning

Abstract: Multiple knowledge (e.g., co-reference, topics, emotional causes, etc) has been demonstrated effective for emotion detection. However, exploring this knowledge in Emotion Recognition in Conversations (ERC) is currently a blank slate due to the lack of annotated data and the high cost involved in obtaining such knowledge. Fortunately, the emergence of Large Language Models (LLMs) holds promise in filling this void. Therefore, we propose a Multiple Knowledge Fusion Model (MKFM) to effectively integrate such knowledge generated by LLMs for ERC and empirically study its impact on the model. Experimental results on three public datasets have demonstrated the effectiveness of multiple knowledge for ERC. Furthermore, we conduct a detailed analysis of the contribution and complementarity of this knowledge.

An Empirical Study on Multiple Knowledge from ChatGPT for Emotion Recognition in Conversations
Geng Tuk Bin Liang"; Bing Qin', Kam-Fai Wong', Ruifeng Xu
'Harbin Institute of Technology, Shenzhen, China
GGuangdong Provincial Key Laboratory of Novel Security Intelligence Technologies
The Chinese University of Hong Kong, Hong Kong, China
HHarbin Institute of Technology, Harbin, China
$ PPnng Cheng Laboratory, Shenzhen, China
tugeng03138gmn com, bin. www.llennnnn.ccmmm edu. hk, xurui feng@he edu. cn
Abstract
feel con kaow, of saturally are laundry, there lan't anything can' (Scard))
seund realty
(e2) Speckerr I: That does out sound stepit to me. heu lanen, It's like dhe lint
Multiple knowledge (e-g., co-reference, topics, emotional causes, etc) has been demonstrated effective for emotion detection. However, ex- ploring this knowledge in Emotion Recognition in Conversations (ERC) is currently blank slate due to the lack of annotated data and the high cost involved in obtaining such knowledge. Fortunately, the emergence of Large Language Models (LLMs) holds promise in filling this void. Therefore, we propose Multiple Knowl- edge Fusion Model (MKFM) to effectively in- tegrate such knowledge generated by LLMs for ERC and empirically study its impact on the model. Experimental results on three public datasets have demonstrated the effectiveness of multiple knowledge for ERC. Furthermore, we conduct a detailed analysis of the contribution and complementarity of this knowledge'.
lime lad is make dianer for aqsell, after Card left aue? I'm sers, Blat all the tine we hare. Nerd on Bass... lhaak. (Phanetisl)
Speaker What sh-alif (Pousfall)
(ea) Spodker Uhoh, the lunrdy'ss It's
lunddry
Ubiak laurdry's done, slesh, whook, (Pescctuu)
(e5) Speaker Rose the matter? (Prandutt)
(40) Speodorr Nathing, oathing, Lan-ke, the boundry's dkine. (Mlad)
(ITT) Spoaker En Coree on. show me. (Pranctutt)
(RO Specker All right, all right. Ih's jus that you left not week in with wlll your whils, and new, everythings kinda pisk. (Sul)
Spenker Oh, enerything's pilk, (Soll)
(RE8) Spenker Veal, ub, exeept for mach which
happos
(MED) Speaker Excegt It hegpeced
pad, gress
hig peep. delleg Idsieg? My Father's can't
Kanstedge BALK TE' W.w.LLLL... MP A. L.R.EAA
Kasediedge 21 ALK
Knsedtedgg 3. M.KKK
RC-asseme 1718 and se1)
rooued pi L.L.R, Cheeaat alle Mi,
ACS Iedamals 181h, Pi)
Specker that
EC 2f Expresses scif doans croiity feeling toopuile of living ber loundy.
dring
CS of #10c Speaker I tims io veeaol Speaker anknow claiig dec stocke reavaning them couit happes to sepono The evcatio behind eo resparty and moserraain
US of Spokeer beceess meretttrua ixtependentlyy ende single talo 3eke leandry The emorice behind statesest Staatation depecaaiio
actt-bottt -panttoiing thit dniiiy to
Spaaker moiig
1 Introduction
happon aepoooo implyinn li milusoo
ACS of dil: Sell-rnican and dount Speaker troeppoue ufff-cocte and critcioox their ability love independently based the kondy cistap..
Emotion recognition in conversations (ERC) has gamered considerable research interest in recent years, as evidenced by several studies such (Tu et al., 2022b; Xie et al., 2021; Mao et al., 2021; Tu et al., 2023b). The goal of ERC is to identify the emotion of each utterance in conversations.
Figure I: Examples of utterances, reflecting the process of introducing multiple knowledge for ERC.
generative knowledge generated by the pre-trained commonsense transformer (COMET) (Bosselut et al., 2019). The other is concepts retrieved from the external knowledge base ConceptNet (Speer et al., 2017) or SenticNet (Cambria al., 2020).
Existing efforts in ERC have traditionally fo- cused on modeling context- and ppeaker-sensitvv dependencies (Lian et al., 2021). including recurrent- based network (Hazarika et al., 2018; Majumder al., 2019; Ghosal et al., 2020: Jiao al., 2020; Li et al., 2022b), transformer-based network (Lian et al., 2021; Shen et al., 2021a; Ong et al., 2022), and graph-based network (Ghosal et al., 2019; Shen et al., 2021b; Saxena et al., 2022). Additionally, recent ERC models (Tu et al., 2022a; Ghosal et al., 2020; Zhong et al., 2019; Jiang et al., 2022) have begun to leverage commonsense knowl- edge, which mainly includes two categories. One is
However, relying solely on commonsense knowl- edge as background information (Bauer et al.. 2018) is insufficient for a deep understanding and recognition of emotions. Multiple knowledge (e.g.. co-reference, topics, emotional causes, etc) are also crucial for emotion detection, but integrating this knowledge into the ERC model poses challenges due the limited availability of annotated data and the high cost of manual tagging. Fortunately, Large Language Models (LLMs) such as ChatGPT pro- vides an opportumity to overcome these limitations, as illustrated in Fig. 1. We divide this knowledge
"Corresponding authors.
'The code is available at htttps://githh: comm TuGengs/NKFM.

---

from ChatGPT into three categories based on data formats, namely Auxiliary Contextual Knowledge (ACK with index list format); Co-reference (CR) is crucial for modeling multi-party dialogue in im- proving the dialogue understanding (Li and Zhao, 2021); Emotional Cause (EC2) reasoning involves identifying the reasons behind emotions and how contextual utterances contribute to them (Poria al., 2021); Context (CT) is indicated im- portant in emotion generation theory (Gross and Barrett, 201 I), Auxiliary Label Knowledge (ALK with label format): Topics (TP) play a significant role as emotions are closely tied to specific sub- jects, and the same words can have different emo- tional meanings depending on the topic (Zhu et al., 2021); Sarcasm (SC) is an important linguistic tool that uses irony express contempt and af- feets the accurate prediction of emotional mean- ing (Poria al., 2019b); Metaphor (MP) is com- mon language expression that conveys nonliteral meanings and emotions by comparing or connect- ing concepts (Mohammad et al., 2016), and Aux- iliary Utterance Knowledge (AUK with sentence format): Commonsense knowledge (CS) related to the utterances; Affective commonsense knowledge (ACS) that bridges the cognitive and affective gap between word-level natural language data and the concept-level sentiments conveyed by them (Cam- bria et al., 2012); Another format of Emotional Cause (EC). Then we introduce a Multiple Knowl- edge Fusion Model (MKFM) that combines three types of knowledge via utterance-level encoder for AUK, graph context encoder for ACK, and Super- vised Contrastive Learning (SCL) module for ALK.
et al., 2020). Furthermore, the participants in ERC are crucial (Wen et al., 2023), leading the de- velopment of speaker-specific models (Kim and Vossen, 2021; Majumder et al., 2019; Ghosal et al., 2020), graph-based models (Nie et al., 2021; Shen et al., 2021b; Ghosal et al., 2019), and so on. How- ever, these approaches lack commonsense knowl- edge, which is important for human-like perfor- mance (Tu al., 2023a). Therefore, researchers have integrated external knowledge sources like COMET (Bosselut et al., 2019), SenticNet (Cam- bria et al., 2022), and ConceptNet (Speer et al., 2017) into their models (Zhao et al., 2022; Ghosal et al.. 2020; Fu et al., 2021: Jiang et al., 2022). Contrastive Learning: Chen al. (2020) intro- duced SimCLR, a comparative learning network for visual representation using image augmenta- tion. In NLP, Yan et al. (2021) proposed a self- supervised CL method for fine-tuning BERT, ad- dressing BERT's poor performance in semantic text similarity tasks. To incorporate label informa- tion, Gunel et al. (2020) extended self-supervised CL to a fully-supervised CL framework, which im- proved performance in few-shot learning scenarios. In ERC, Li et al. (2022a) and Song et al. (2022) applied SCL to separate utterances with different emotions, enhancing emotion identification. How- ever, there is no effort unsupervised CL in ERC.
3 Methodology
3.1 Task Definition
Let C be a conversation consisting of n utter- ances. Each utterance u; is spoken by one of the speakers in the set S, which contains m speakers: {s1 $2, ..., }m The objective of ERC is to predict the emotion label y, for each utterance u.
Our main contributions are (1) the first explo- ration of multiple knowledge for ERC, (2) the proposed MKFM for integrating this knowledge, which outperforms state-of of-the-art baselines, and (3) conducting further analysis on the contribution and complementarity of multiple knowledge.
3.2 Overview
In the MKFM shown in Fig. 2, we incorporate multiple knowledge. Based on how the model inte- grates these knowledge types, we categorize them into three classes. In the following sections, we provide detailed descriptions of each module.
2 Related Work
Emotion Recognition in Conversations: The emotion generation theory (Gross and Barrett, 2011) 1) emphasizes the role of contextual informa- tion in identifying emotions. RNN-based mod- els (Poria et al., 2017) are commonly used to cap- ture context dependencies but struggle with distin- guishing between historical utterances (Lian et al., 2021). Memory networks have been proposed to address this limitation (Hazarika et al., 2018; Jiao
3.3 Utterance-level Encoder
Following Ghosal al. (2020), we utilize the pre- trained model RoBERTa (Liu et al., 2019) to en- code the input vector u, To enhance the utterance representations using AUK, we include this knowl-

---

Utterance-leve Encoder (AUK).
Graph Context Encoder (ACK)
Contrastive Learning (ALK)
Lavers
i EC
ACS
H
CT
CR
cc2
Graph Attention
H
Ug
RoBerta Encoder
12g BC
ACS
H
CT
CR
:C2
Classifier
Lec
H
11g
üg EC
i
CT
CR
BC2
SCL
L L/tt/m
H
11 EC
ACS
Eittereat Speakers
T
M
Intra- and Inter-peaker
Intra- and Inter-CT
Intra- and Inter-(RR
U5
üs EC
Inter-EC2
H
Figure 2: Illustration of MKFM, where mathematical symbols are consistent with the formulas in this paper.
edge by utilizing the concatenation operation.
context dependencies. u, E V and rg R. repre- sent the utterance node and edge type, respectively. (1) Øi (u,, Pk; u,) E & denotes the edge between node and j. where > i. The weight of eij is denoted as os.j E A. The utterance-level encoder initializes each node u, The edge types in R. are determined by the speaker and various ACK.
H Linear(x;)
x; - û. @ EC, @ CS; @ ACS;
(2)
û RoBERTa(u;)
(3)
where 0; is the d, dimension hidden states of u, H()) E Rd is the vector representation af- ter linear transformation Linear. @ represents
3.4.2 Information Aggregation
For each node at layer &, the graph context encoder aggregates its neighboring nodes as follows.
the concatenation operation. EC, dh rep- resents the emotion cause of the i-th utterance, CS; E Rd represents the commonsense knowl- edge, and ACS; c IRd, represents the affective commonsense knowledge.
H ΣS JEN, M + (Q
(4)
0 19) (9
aij
(5)
16) 0
Softmax ( H P
(6)
3.4 Graph Context Encoder
(5) = CRU, ") H
ERC, extracting contextual information from surrounding utterances is crucial. However, rely- emotion
where N, denotes the set of neighboring nodes. (E) is the context representation of u, w!
ing on future utterances to determine the current utterance is not practical in real-life situations (Poria et al., 2017; Ghosal et al., 2021a). Therefore, we define the context of the current ut- terance the preceding utterances Uj Vj < i. Previous research has focused modeling intra- and inter-speaker context dependencies capture emotional dynamics in conversations (Poria al., 2019b). To enhance the contextual information, we employ a graph network that utilizes different edge types guided by ACK to integrate contexts.
Reda x2d and (( C R. xda represent projection parameters of the model. M,(-) = (m= and (-)-) 1 in denote the indicator function. used to model the different context dependencies. In previous methods, 17 was generally equal s. used model inter- and inter-speaker context rep- resentation (E) However, by utilizing ACK, 1 can represent the CR relationship. CT semantic cor- relation, and EC2 to obtain the knowledge-enriched contextual representation KH.
3.4.1 Graph Construction
Unlike the previous graph construction (Ghosal al., 2019; Li et al., 2021; Tu al.. 2022b), they incorporate future utterances to update the current utterance node. Following Shen et al. (2021b), we build directed graph G = (V,& E R.A) to model
H (5) GRÜ
h CH
H.
(5-1))
(8)
(5) w +
CH
(9)
KH.
CT
CR 5 EC2
(10)

---

where CT.}) CR and EC2 ( denote the context representation by using ACK-enriched
Dialogues
Utterances
Dataset
train
val
test
train
val
test
(5)
IEMOCAP
120
5,810
1,623
CH; denotes the knowledge-enriched contextual representation. w Id4 xdda represents pro- jection parameters of the model.
MELD
1039
114
280
9,989
1,109
2610
EmoryNLP
659
7,551
954
984
Datuset
Classes
Metric
3.5
Contrastive Learning Module
IEMOCAP
6
Weighted Avg. FI
MELD
Weighted Avg. F]
For the classifier, we employ a feed-forward neural network to predict the distributions of emotions.
BmoryNLP
7
Weighted Avg. F1
Table 1: Statistics of experimental datasets.
Y Argmmax(Sottmax W,O, bx))
(11)
Ô, = ReLU b,)
(12)
IEMOCAP comprises dyadic sessions where ac- tors engage in improvisations scripted scenarios. Each utterance in this dataset is labeled with one of the following emotions: happy, angry, neutral, sad, excited, or frustrated. As there is no validation set in this dataset, we adopt the approach from Shen et al. (2021b) by utilizing the last 20 dialogues from the training set for validation.
C ... @ H 99 + X
O, =
(13)
where W, E ((C 1)d, 4da) and W, E Rd xd, are projection parameters. b, and bn are biases. Y E N is the predicting emotional label set, where N is the total number of utterances.
Lee CrossEntropy Y, Y) 3 ||0122 (14)
MELD is multi party conversation dataset col- lected from the TV show Friends, which is tension of the Emotion dataset (Hsu al., 2018). Each utterance is annotated with one emo- tion from the set: surprise, fear, disgust, anger, sadness, neutral, or joy, along with one sentiment from the set: neutral, negative, or positive.
where Lee is the classification loss. + is set of projection parameters. 3 represents the coefficient of I-gregulariization To distinguish between ordi- nary utterances and SC utterances, as well as MP utterances, and to obtain vector representations for utterances related to the TP. we introduce the SCL loss items L., f. and Lt, as follows.
EmP consists of multi-party sessions from the TV show Friends and each utterance is la- beled with one emotion from the set: joyful, scared, peaceful, sad, powerful, mad, neutral, along with one sentiment, as suggested in Ghosal et (2020), from the set: neutral, negative, or positive.
L= + 1/ + Vr vm
(15)
1
fa/t//m
log (P (S/TT/M))
(16)
71b
[1 [,ر= E(0;,0 Oj) 10,00 02)
P(A)
(17)
En 11 [( L(O,,
4.2 Comparison Models
We compare our proposed framework with vari- ous ERC baselines, including RNN-based models: CauAIN (Zhao et al., 2022), COSMIC (Ghosal et al., 2020), DialogueRNN (Majumder al., 2019); Memory networks: ICON (Hazarika et al., 2018), AGHMN (Jiao et al., 2020); Graph-based models: DialogueGCN (Ghosal et al., 2019), DAG- ERC (Shen et al., 2021b), SKAIG (Li et al., 2021); Transformer-
where ny is the size of mini-batch sample. tha th and tm are tuned hyper-parameters. ht/mm//t// ny E S/M/T indicates whether the i-th utterance is a satirical expression, metaphori- cal expression, and its topic, respectively. S/T/M is the subset of ALK, generated by ChatGPT to promote the utterance representations. e(*, /// where the temperature parameter and simi(.) denotes the cosine similarity function.
models: KET (Zhong et al., 2019), BERT BASE (Kenton and Toutanova, 2019). RoBERTa (Liu al., 2019); Genera- tive models: CoG-BART (Li et al., 2022a). Curie (Olmo et al., 2021), ChatGPT (Ouyang et al., 2022)
4 Experiments
4.1
Datasets
We conduct experiments on three datasets: IEMO- CAP (Busso et al., 2008), EmoryNLP (Zahiri and Choi, 2018), and MELD (Poria et al., 2019a). The dataset statistics are presented in Table 1.
'Appendix A. details the prompt of ChatGPT and Curie for FRC, and Appendix A.2 explains the prompi used to ohtain multiple knowledge mentioned in this paper.

---

Methods
IEMOCAP
MELD
EmoryNLP
Porfix mance of the mode wi MP difleent positions in eneverutioos
Fl Libta Loves Lifligg aooee
Baseline
67.45
64.76
38.46
TP
67.74
64.89
38.72
w/ SC
67.64
65.07
38.56
20
:20
W/ MP
68.26
65.12
38.63
Perfommnce the model ACS diffeeen posticess convenstioon
ALK
w/ TP + SC
67.90
A 65.02
38,89
w/ TP
67.73
65.16
38.88
w/ SC MP
68.31
65.19
38,90
20
40
93
80
100
170
Perfommance model
w/TP + SC MP
& 67.66
65.17
38,95
EC
65.13
38.70
w/ CS
67.86
65.16
39.07
20
100
130
w/ ACS
68.88
65.28
38.55
AUK
w/ EC + CS
67.85
65.25
4 38.87
Figure 3: Lifting performance of the model on different positions conversations.
EC ACS
67.53
65.16
38.72
w/ CS ACS
67.97
A 65.27
A 39.06
w/ EC + CS + ACS
67.88
65.34
4 38.89
Performance of the model en diftrrett categones and phanommnne
CR
68.02
65.15
38.78
w/ CT
67.66
65.19
38.66
w/ FC2
67.68
65.07
38.79
ACK
w/ CR + CT
67.87
65.25
39,20
w/ EC2
68.49
65.28
38.83
199
Noosa (24425))
Freeused (244011))
Hoppy 33,05
w/ CT EC2
68.46
65.24
38.89
w/ CR CT FC2
67.59
65.34
38.86
Figure 4: Lifting performance of the model ES prob- lems and different emotional categories.
Table 2: Average weighted Fl score (%) of Bascline (MKFM w/o any knowledge) based homologous knowledge. The best test scores are highlighted with an underline. A, A, and represent performance greater than any individual knowledge, not less than certain knowledge, lower than any knowledge, respectively.
CR demonstrates the most significant performance improvement on the IEMOCAP dataset, with a rise of 0.57%. Most importantly, to further investigate the contributions of MP, ACS, and ACS in enhanc- ing performance, we present their effects across different emotional categories, various positions in conversations, and their impact utterances w/ emotion shift (ES) and w/o ES.
4.3 Experimental Settings
We perform hyper-parameter search for MKFM on each dataset with a validation set, including learning rate, batch size, dropout rate, tuned hyper- parameters V/ *//t/m> Twe and ny. And we let d, = 1024, r 0.07, dg 768, and d, 300 on each d The reported results all based on the average weighted F1 score of 5 runs the test set.
Contribution of knowledge to utferances in dif- ferent positions: Fig. 3 show that improvements become noticeable after the 100th utterance and become significant after the 120th utterance. How- ever, there a slight dip performance between the 80th and 100th utterances, followed by a grad- ual recovery and stabilization. The utterances performance enhancement are mainly observed to- ward the end of conversations, possibly because ERC models prioritize context in immediate prox- imity (Ghosal et al., 2021b) due to their similar semantics, making it challenging detect utter- ances requiring longer contexts. By incorporating knowledge, the model can consider semantic ele- ments of utterances from greater distances, which is crucial for handling long conversation data. Contribution knowledge to utterances in dif- ferent emotional categories: Regarding different categories, as depicted in Fig. 4, all this knowledge
5 Experimental Results
5.1 Contributions of Multiple Knowledge
Table 2 presents the performance of the model us- ing each piece of knowledge. We can observe that almost all of the knowledge sources have positive impact on the ERC task. Additionally, based on the results from the three datasets, we can that for ALK, MP shows the most significant improve- ment on the IEMOCAP dataset, with an increase 0.81%. For AUK, ACS exhibits the most pow- erful performance improvement on the IEMOCAP dataset, with an increase of 1.43%. As for ACK,
AAppendix A.3 details more hyper-parameter settings.

---

consistently boosts the performance of minority classes, such as 'happy'. On the other hand, for the majority classes, such as 'neutral', except for the model w/ ACS, leveraging other knowledge ex- hibits a decline performance. Notably, the model w/ ACS achieves promising performance across all classes, except for slight decrease in the emotion 'frustrated', which further explains its significant performance improvement. Overall, the incorpora- tion of external knowledge appears alleviate the issue of class imbalance in ERC.
Methods
IEMOCAP
MELD
EmoryNLP
Baseline
67.45
64.76
38.46
w/TPP BC
67.82
à 65.09
39.30
w/ TP + CS
2 67.20
& 65.10
A 39.50
w/ TP ACS
& 67.84
à 65.16
39.22
ALK AUK
w/ SC EC
4. 67,70
65,24
39.33
w/ SC + CS
68.04
65.25
39.17
w/ SC ACS
68.47
65.22
39.29
w/ MP EC
A. 68.19
A 65.34
38.90
MP
65.38
39.21
W/PP ACS
Ø 68.16
& 65,19
39.27
w/ BC CR
68.29
65,31
39.23
w/ EC CT
68.38
65.34
39.11
Contribution of knowledge to utterances in ES problems: Concerning ES problems, although ing this knowledge contributes to performance im- provements, we find that the enhancements are more prominent for utterances w/o ES, This sug- gests that the ES problem in ERC remains a chal- lenge, and solely relying on the introduced knowl- edge is not sufficient for effective resolution.
w/ BC EC2
68.05
66.32
39.06
ALK ACK
w/ CS + CR
68.18
65.33
39.53
w/ CS CT
68.22
65.39
39.19
W/ BC2
68 68.45
65.39
39.28
w/ ACS CR
A. 68.8.00
65.38
39.08
w/ ACS + CT
4 68.51
65.34
39.26
ACS EC2
A 68.08
65.50
39.76
w/ TP CR
67.39
65.43
38.97
w/ TP + CT
Ø 67.36
65.35
39.22
AUK ACK
EC2
67.84
65,41
39.02
5.2 Complementarity of Knowledge
w/ SC CR
68.55
65.44
39.40
w/ SC CT
68.79
65.36
39.30
Complementarity of homologous knowledge: In Table 2, we can observe that there is strong syn- ergy between the homologous knowledge sources ALK and ACK. However, the complementarity performance of the model with AUK is notably poor, which could be attributed the introduction of excessive noise associated with this knowledge source (Jiang et al., 2022; Tu et al., 2022a).
SC
68.34
65.34
39.25
w/ MP + CR
68.57
66.48
39.07
MP CT
68.51
65.46
39.17
w/ MP BC2
68.65
63.40
39.24
w/TP + EC CR
Ø 67.66
65.19
39.20
+
67.05
65.26
39.06
T/ EC EC2
Ø 66.99
65.29
39.17
w/TP CR
Ø 67.44
65,29
A 39.04
w/TP CS CT
Ø 66.74
65.34
A 39.00
Complementarity of non-homologous knowl- edge: Moving on to non-homologous knowledge sources, in Table 3, the combination of TP per- forms poorly in the JEMOCAP dataset. In con- trast to other knowledge types, some utterances may lack specific topic assignments during anno- tation. This inconsistency in annotation hinders the complementarity of the model, and this phe- nomenon is further amplified in long conversations. This observation holds true even when combined with three knowledge sources ALK + AUK + ACK. Furthermore, we also explore simpler approach to integrating knowledge without considering the complementarity of heterogeneous knowledge, as shown in Table 5. This approach selects the best- performing knowledge or combination of knowl- edge from each homogeneous source. Experimen- tal results show that this method further improved the MELD and EmoryNLP datasets. However, per- formance is slightly decreased on the long dialogue dataset IEMOCAP. This decrease can mainly attributed to the dominance of ACS knowledge on
w/TP + BC2
2 66.64
65.27
39.11
TP ACS
65,43
39.07
w/ TP ACS UCS++ CT
Ø 67.38
& 65,26
38.94
TP ACS FC2
A 67.75
65.37
38.99
w/SC EC CR
68.56
65.38
38.97
ALK AUK ACK
w/ SCS EC CT
68.26
65.32
39.12
SC EC2
65.37
39.17
w/ CS CR
68.42
65,41
& 39.00
w/ SC + CS + CT
68.38
65.45
39.25
SC EC2
68.32
65.47
39.05
w/ SC ACS CR
A 68.26
65.51
39.56
w/ SC ACS CT
A 68.65
65.46
39.44
w/ WCSC ACS FC2
& 68.22
65.47
39.34
w/ MP MM++EC+++ CR
68.27
65.41
38 97
w/ EC CT
68.33
65.37
39.07
w/ MP + +C BC2
4 68.17
65.40
38 18.96
w/ MP + CR
68.53
65.36
39.25
w/ MP +P+CS++ CT
68.69
65.38
39.45
w/ MP + CS + EC2
A 68.00
65.44
39.45
ACS CR
à 68.41
65.46
39.52
w/ MP ACS CT
A 68.43
65.44
39.28
w/ MP + ACS EC2
à 68.67
65.47
39.61
Table Average weighted Fl score (%) of Baseline based on non- non-hooologous knowledge.

---

SC + MP SC
SC+ MP + SC
#Ratio
The mumber of utterances correctly peodicted in the model we MP but not the model w/ CS cifforent positirs conversations
ALK
94.69%
95.20%
0.9946
SC MP MP
SC MP MP
#Ratio
94.69%
95,20%
0.9946
20 number uttermene comectly pretictaal the MP:CS on different pusitions in conversation
+ ACS -
ACS + CS
#Ratio
AUK
91.54%
91.54%
1.0000
Nartee
CS + ACS + ACS
CS ACS ACS
#Ratio
20
120
91.54%
91.8%%
0.9962
BC2 -
EC2 <-
#Ratio
Figure 3: Lifting performance the model on different positions conversations.
ALK AUK ACK AUK + ACK ALK ACK ALK + AUK ACK
94.13%
94.82%
0. 0.9928
CR BC2 EC2
CR + BC2 BC2
#Ratiio
94.40%
95.70%
0.9865
Performance ofe model validation oct EEMDAP dataset
MP + CS MP
MP CS MP
#Ratio
1.0-
91.87% MP + CS CS
93.03%
0.9875
Littig oare
MP+ CS
#Raaioo
0.0
91.96%
94.66%
0.9714
SC CT - SC
SC CT SC
#Ratio
95.13% SC + CT CT
95.48%
0.9964
SC CT CT
#Ratio
94.23%
95.26%
0.9892
Figure 6: Lifting performance of the model based on different coefficients loss items.
ASC + FC2 - ASC
ASC EC2 <- ASC
#Ratio
94.13%
92.72%
1.0152
ASC EC2 EC2 92.61%
ASC BC2 EC2
#Ratio
92.44%
1.0019
using single or multiple knowledge (TT) by those correctly predicted using multiple knowledge alone (TF), Element 2 represents the consistency of the model by dividing the number of TT by those cor- rectly predicted using single knowledge alone (FT), Ideally, higher consistency and lower diversity are desired, indicating smaller ratios of Element to 2 representing better complementarity. The reported results in Table 4 highlight the most significant performance improvement from different types of knowledge and their combinations. The results indicate that using MP + SC yields stronger com- plementary effect, while ASC + EC2 shows the least favorable complementarity.
MP + CS + CT - MP
MP + CS + CT < MP
#Ratio
94.67%
94.84%
0.9982
MP CS CT CS
MP CS CT CS
#Ratio
94.49%
96.23%
0.9819
MP CS + CT + CT
MP + CT CT
#Ratio
93.59%
94.44%
0.9910
Table 4: Complementarity of the model based on ho- mologous and non-homologous knowledge, + f- correspond elements and 2, respectively. A smaller ratio of elements and 2 indicates stronger complemen- tarity the model using this knowledge.
Methods
IEMOCAP
MELD
EmoryNLP
Baseline
67.45
64.76
38.46
w/ ALK AUK
68.27
65.41
39.43
Further analysis to the complementarity of knowledge: To provide a specific analysis, set of utterances is obtained that could be correctly recognized by the model with SC but not by the model with MP + SC. Among these utterances, 27 have no ES, while 31 have ES. The similar num- bers indicate that ES has a negligible impact on model consistency. However, a significant portion of these utterances (70%) belong to the 'frustrated' and 'hapy'' classes, indicating challenges in en- suring effective consistency for extreme classes during the fusion process. Additionally, in Fig. 5, visualize the utterances correctly identified by MP + SC but not by SC alone, well as the sam- ples correctly identified by MP SC but not by SC or MP alone. Interestingly, the distribution these samples did not show significant changes,
ACK
65.66
38.84
w/ AUK ACK
68.67
65.34
39.61
w/ AUK AUK ACK
68.01
65.54
38.84
Table 5: Performance of the model based on the combi-
nation of best homologous knowledge.
that dataset, which makes simple concatenation in- effective in complementing other knowledge. This aspect further demonstrated in the complemen- tary heterogeneous knowledge.
Metrics for complementarity of knowledge: To quantify the complementarity of knowledge sources, a ratio of two elements was introduced.
Element denotes the diversity of the model by dividing the number of correctly predicted samples

---

suggesting that the diversity of the model with MP SC primarily arises from coordinating the two types of knowledge rather than supplementing a distinct knowledge type. This further confirms the strong complementarity between the two knowl- edge sources.
Methods
IEMOCAP
MELD
EmoryNLP
CharGPT
40.07
54.37
37.55
Curie
57.33
65.01
37.40
BERT_BASEE
BERT _BASE
61.19
56.21
33.15
RoBERTa
54.55
62.02
37.29
EmoBERIa
68.57
65.61
DialogueRNN
61.21
56.27
31.70
AGHMN
62.70
58.10
5.3 Analysis of Loss Coefficients
KETF
59.56
58.18
34,39
DAG-ERC
68.03
63.65
39.02
In Fig. 6, we show the impact of different loss coef- ficients on model performance. Increasing V/s and i' results in fluctuating pattern but overall con- sistent improvement in the model's performance. However, the behavior of 4/: is different. Initially, increasing value improves the model's perfor- mance, but then it rapidly declines. Even with some improvement afterward, the performance re- mains lower than that of the original model. This is because not every utterance is assigned specific topic. Increasing 2/2 amplifies the semantic gap be- tween utterances with and without topic, leading degradation in performance.
COSMIC
65.28
65.21
38.11
SKAIG?
66.96
65.18
38.88
CoG-BART
66.18
64.81
39.04
CauAIN
67.61
65.46
Baseline
67.45
64.76
Baselinci w/ ACS
68.88
65.28
38.55
Baseline w/ ALK ACK
68.
65.66
38,
Baseline w/ ACS
68.08
65.50
39.76
Table 6: Average weighted F1 score (%) of different methods. $ represents our implemented model. Results with F and è are respectively retrieved from Zhong et al. (2019) and Ghosal al. (2020). Results with are retrieved from the original papers.
Performance
Different Knowledge
5.4 Comparison with Various Baselines
Latteg worm 2003
2.3
KET MELD
KET EmaryNLP
KET LEMOCAP
Table 6, fine-tuning at the utterance level alone is inadequate for ERC due to reliance on context and the speaker's state information. EmoBERTa, a modified version of RoBERTa, addresses this limitation by adjusting input structures and improv- ing performance. Graph-based models outperform RNN-based models in IEMOCAP and EmoryNLP datasets by effectively capturing local context in lengthy conversations. However, in the MELD dataset (TV show data), the coherence between con- secutive utterances may be lacking, reducing the advantage of graph-based models. Large models struggle in ERC, possibly due to difficulties in cap- turing intricate interactions, especially in lengthy conversations, as seen in their performance on the IEMOCAP dataset. And prompt-tuning the Curie model alone is insufficient for comprehensive emo- tional comprehension. Generative methods are not ineffective for ERC, task-specific factors like imbalanced samples and long-term context model- ing need to be considered (Li et al., 2022a).
0.0
Performance of the COSMIC model on Different Knowledge
Littog soore 660
3.5
C0S30C EnseyNL
COSMIC TEMOCAP
11
MP
BC
AC3
B62
Figure Lifting performance of different ERC models based on various knowledge.
son with existing methods. The results, reported in Table 6, clearly indicate that our method surpasses state-of-the-an benchmarks, which demonstrates both the feasibility of the proposed MKFM and the efficacy of such knowledge for the ERC task. Ad- ditionally, we conduct a paired t-test on the results, revealing a significant p-value (< 0.05) between the original baseline results and the baseline results enhanced with different knowledge.
Although LLMs often struggle to outperform smaller models in complex tasks like FRC, they still contribute advancements in ERC by leveraging rich knowledge. Therefore, we utilize ChatGPT acquire diverse knowledge and pro- posed MKFM, graph-based model for integrating this knowledge. We select several representative results from Table 3, and 5 to conduct a compari-
5.5 Generalization Analysis
To validate the generalization of multiple knowl- edge other ERC methods, we conduct exper- iments using the RNN-based approach COSMIC and the Transformer baassorrmme----eddddd approach KET. In Fig. 7, our findings indicate that while the KET model and

---

ACK do not fit well in the MELD dataset, lever- aging this knowledge demonstrates significant per- formance improvements other cases. This ob- servation proves the generalization ability of this knowledge for the ERC methods.
may be limited, leading to potential performance limitations. While the emergence of ChatGPT has made accessing such knowledge easier, adjusting prompts remains an unavoidable challenge, intro- ducing a different form of "annotation" cost. Fur- thermore, except for AUK and ALK, the introduc- tion of ACK necessitates certain modifications to the structure of non-graph-baser models, which adds to the workload of incorporating knowledge.
6 Conclusion
In this paper, we propose MKFM for integrating multiple knowledge sources in ERC, which sur- passes state-of-the-art baselines. We also conduct further analysis of the contribution and comple- mentarity of this knowledge. The contribution of knowledge focuses on end-of-converssation ut- terances, particularly those from minority classes. However, there still unresolved ES issues. When it comes to the complementarity of homol- ogous knowledge, apart from AUK's performance decline caused by noise, other cases generally show satisfactory results, As for the complementarity of non- honohoollgouus knowledge, the combination including TP struggles with inconsistent annota- tions leads to persistently poor performance in long conversations. other cases show satisfactory results. While the complementary improvement does not address ES problems effectively, it has a notable im- pact on utterances from minority classes, aligning with the contribution of this knowledge. Addition- ally, we showcase the impressive generalization ability of this knowledge in other ERC models.
References
Lisa Bauer, Yicheng Wang. and Mohit Bansal, 2018, Commonsense for generative multi-hop question an- swering tasks. In Proceedings of the 2018 Confer- ence on Empirical Methods Natural Language Processing, pages 4220-4230.
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chai- tanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. Comet: Commonsense transformers for auto- matic knowledge graph construction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4762-4779.
Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jean- nette N Chang. Sungbok Lee, and Shrikanth S Narayanan. 2008. Temocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42(4):335-33)
Erik Cambria, Yang Li, Frank Xing, Soujanya Poria, and Kenneth Kwok. 2020. Senticnet 6: Ensemble application of symbolic and subsymbolic ai for sen- timent analysis. In Proceedings of the 29th ACM international conference on information & knowl- edge management, pages 105-114.
A Acknowledgements
We thank the anonymous reviewers for their valu- able suggestions to improve the overall quality of this manuscript. This work was partially supported by the National Natural Science Foundation of China (62006062, 62176076), Natural Science Foundation of GuangDong 2023A1515012922. Key Technologies Research and Development Program of Shenzhen
Erik Cambria, Qian Liu, Sergio Decherchi, Frank Xing. and Kenneth Kwok. 2022. Senticnet 7: A commonsense-basse neurosymbolic ai framework for explainable sentiment analysis. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3829-3839.
Erik Cambria, Yunqing Xia, and Amir Hussain. 2012. Affective common sense knowledge acquisition for sentiment analysis. In LREC, pages 3580-3585.
Shenzhen Foundational Research Fund- and JCYJ202208 11022008811121111 1032, Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies 2022B 121200005..
ing JCYJ2020010911344111
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Pro- ceedings of the 37th International Conference on Machine Learning, pages 1597-1607.
Limitations
Yahui Fu, Shogo Okada, Longbiao Wang, Lili Guo, Yaodong Song. Jiaxing Liu, and Jianwu Dang. 2021. Consk gons conversational semantic-am and knowledge- oriented graph convolutional network for multimodal emotion recognition. In 2021 IEEE International Conference on Multimedia and Expo, pages 1-6. IBEE.
Our approach requires diverse knowledge encom- passing co-reference. topics, and emotional causes, which can be time-consuming, labor-intensive, and resource-demanding to acquire. As a result, the availability of knowledge for the MKFM model

---

Deepanway Ghosal, Navonil Majumder, Alexander Gel- bukh, Rada Mihalcea, and Soujanya Poria. 2020. Cosmic: Commonsense knowledge for emotion iden- tification in conversations. In Findings of the Associ- ation,fr Computational Linguistics: EMNLP 2020, pages 2470-2481.
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: tre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-4186,
Taewoon Kim and Piek Vossen. 2021. Emoberta: Speaker-aware emotion recognition in conversation with roberta. arXiv preprint arXiv:2108. 12009.
Deepanway Ghosal, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2021a. Exploring the role of context in utterance-level emotion, act and intent classification in conversations: An empirical study. In In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1435-1449.
Jiangnan Li, Zheng Lin, Peng Fu, and Weiping Wang- 2021. Past, present, and future: Conversational emo- tion recognition through structural modeling of psy- chological knowledge. In In Findings of the Associ- ation furonfor Computational Linguistics: EMNLP 2021, pages 1204-1214.
Deepanway Ghosal, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2021b. Exploring the role of context in utterance-level emotion, act and intent classification in conversations: An empirical study. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1435-1449.
Shimin Li, Hang Yan, and Xipeng Qiu. 2022a. Contrast and generation make bart good dialogue emotion recognizer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10002- 11010.
Deepanway Ghosal, Navonil Majumder. Soujanya Poria, Niyati Chhaya, and Alexander Gelbukh. 2019. Dia- loguegen: A graph convolutional neural network for emotion recognition in conversation. In In Proceed- ings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Inter- national Joint Conference Natural Language Pro- cessing, pages 154-164.
Wei Li, Wei Shao, Shaoxiong Ji, and Erik Cambria. 2022b. Bieru: Bidirectional emotional recurrent unit for conversational sentiment analysis. Neurocomput- ing, 467:73-82.
Yiyang Li and Hai Zhao. 2021. Self-and pseudo-self- supervised prediction of speaker and key-utterance for multi-party dialogue reading comprehension In Findings of the A Association for Computational Lin- guistics: EMNLP 2021, pages 2053-2063.
James J Gross and Lisa Feldman Barrett. 2011. Emo- tion generation and emotion regulation: One or two depends on your point of view. Emotion Review, 3(1).
Zheng Lian, Bin Liu, and Jianhua Tao. 2021. Ct- net: Conversational transformer network for emo- tion recognition. IEEF/ACM Transactions on Audio, Speech, and Language Processing, 29:985-1000.
Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoy- anov. 2020. Supervised contrastive learning for pre- trained language model fine-tuning, arXiv preprint arXiv:2011.01403.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907.11977.19
Devamanyu Hazarika, Soujanya Poria, Rada Mihalcea, Erik Cambria, and Roger Zimmermann. 2018. Icon: Interactive conversational memory network for multi- modal emotion detection. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages
Navonil Majumder, Soujanya Poria, Devamanyu Haz- arika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: An attentive rnn for emotion detection in conversations. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 33, pages 68 18-8225.
Chao-Chun Hsu, Sheng- Ych Chen, Chuan-Chun Kuo, Ting-Has Hao Huang, and Lun-Wei Ku. 2018. Emotion- lines: An emotion corpus of multi-party conversa- tions. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation.
Yuzhao Mao, Guang Liu, Xiaojie Wang, Weiguo Gao, and Xuan Li. 2021. Dialoguetrm: Exploring multi- modal emotional dynamics Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 2694-2704.
conversation. In
Dazhi Jiang, Runguo Wei, Jintao Wen, Geng Tu, and Erik Cambria. 2022. Automl-emo: Automatic knowl- edge selection using congruent effect for emotion identification in conversations. IEEE Transactions on Affective Computing.
Saif Mohammad, Ekaterina Shutova, and Peter Turney. 2016. Metaphor as a medium for emotion: An empir- ical study. In Proceedings of the Fifth Joint Confer- ence on Lexical and Computational Semantics, pages 23-33.
Wenxiang Jiao, Michael Lyu, and Irwin King. 2020. Real-time emotion recognition via attention gated hierarchical memory network. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 8002-8009.
Weizhi Nie, Rihao Chang, Minjie Ren, Yuting Su, and Anan Liu. 2021. I-gen: Incremental graph convo- lution network for conversation emotion detection. IEEE Transactions on Multimedia, pages 4471-448)

---

Alberto Olmo, Sarath Sreedharan, and Subbarao Kamb- hampati. 2021, Gpt3-to- to-3-to--p plan: Extracting plans from text using gpt-3. arXiv preprint arXiv:2106. 2tt::2/06.001311
Xiaohui Song, Longtao Huang, Hui Xue. and Songlin Hu. 2022. Supervised prototypical contrastive learn- ing for emotion recognition in conversation. arXiv preprint arx.v:2210.08713.
Donovan Ong, Jian Su, Bin Chen, Anh Tuan Luu, Ashok Narendranath, Yue Li, Shugi Sun, Yingzhan Lin, and Haifeng Wang. 2022. Is discourse role important for emotion recognition in conversation? In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 11121-11129.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of gen- cral knowledge. In Proceedings of the 31 AAAI con- ference on artificial intelligence, pages 4444 4451.
Geng Tu, Bin Liang, Dazhi Jiang, and Ruifeng Xu. 2022a. Sentiment-emotiionann and context-guided knowledge selection framework for emotion recogni- tion in conversations. IEEE Transactions on Affective Computing, (01):1-14.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong 7hang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models follow instruc- tions with human feedback. Advances in Nearal Information Processing Systems, 35:27730-27744,
Geng Tu, Bin Liang, Xiucheng Lyu, Lin Gui, and Ruifeng Xu. 2023a. Do topic and causal consistency affect emotion cognition? graph interactive net- work for conversational emotion detection. In The 26th European Conference on Artificial Intelligence (ECAI'23), pages
Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Philippe Morency. 2017. Context-dependent sentiment anal- ysis in user- gerrggeeerted videos. In Proceedings of the 55th annual meeting of the association for computa- tional linguistics, pages 873-883.
Navonil Majumder, Amir Zadeh, and Louis-Philippe
IOS Press.
Geng Tu, Bin Liang, Ruibin Mao, Min Yang, and Ruifeng Xu. 2023b. Context or knowledge is not always necessary: contrastive learning framework for emotion recognition in conversations. In Find- ings of the Association for Computational Linguistics: ACL 2023. pages 44454-10065
Soujanya Poria, Devamanyu Hazarika, Navonil Ma- jumder, Gautam Naik, Erik Cambria, and Rada Mi- halcea. 2019a. Meld: A multimodal multi-pan party dataset for emotion recognition conversations. In Proceedings of the 57th Annual Meeting of the Associ- ation for Computational Linguistics, pages 527-536.
Geng Tu, Jintao Wen, Hao Liu, Sentao Chen, Lin Zheng, and Dazhi Jiang, 2022b. Exploration meets exploitation: Multitask learning for emotion recog- nition based on discrete and dimensional models. Knowledge- Based Systems, 235:107598.
Soujanya Poria, Navonil Majumder, Devamanyu Haz- arika, Deepanway Ghosal, Rishabh Bhardwaj. Sam- son Yu Bai Jian, Pengfei Hong. Romila Ghosh, Ab- hinaba Roy, Niyati Chhaya, et al. 2021. Recognizing emotion cause conversations. Cognitive Computa- tion, 13:1317-3332.
Jintao Wen, Dazhi Jiang, Geng Tu, Cheng Liu, and Erik Cambria. 2023. Dynamic interactive multiview mem- ory network for emotion recognition in conversation. Information Fusion, 91:123-133.
Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019b. Emotion recognition in con- versation: Research challenges, datasets, and recent advances. IEEE Access, 111100943-10055..
Yunhe Xie, Kailai Yang. Cheng Jie Sun, Bingquan Liu, and Zhenzhou Ji. 2021. Knowledge-interacti net- work with sentiment polarity intensity-awan multi- task learning for emotion recognition in conversa- tions. In Findings of the Association for Computa- rional Linguistics: EMNLP 2021, pages 2879-2889.
Prakhar Saxena, Yin Jou Huang, and Sadao Kurohashi. 2022. Static and dynamic speaker modeling based on graph neural network for emotion recognition in conversation. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, pages 253.
Yuanmeng Yan, Rumei Li. Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021. Consert: A con- trastive framework for self-supervised sentence repre- sentation transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing, pages 3065-5075.
Weizhou Shen, Junqing Chen, Xiaojun Quan, and Zhix- ian Xie. 2021a. Dialogxl: All-in-one xlnet for multi- party conversation emotion recognition. In Proceed- ings of the AAAT Conference on Artificial Intelligence, volume 35, pages 13789-1377 13797.
Sayyed M Zahiri and Jinho Choi. 2018 Emotion detection on tv show transcripts with sequence-based convolutional neural networks. In Workshops at the thirty-second AAAI conference on artificial intelli- gence, pages 44-52.
Weizhou Shen, Siyue WU, Yunyi Yang, and Xiaojun Quan. 2021b. Directed acyclic graph network for conversational emotion recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing, pages 1551 1-560.
Weixiang Zhao, Yanyan Zhao, and Xin Lu. 2022. Cauain: Causal aware interaction network for emo- tion recognition in conversations. In Proceedings of the 31st International Joint Conference on Artificial Intelligence, pages 4530-

---

Peixiang Zhong, Di Wang, and Chunyan Miao. 2019. Knowledge-enriched transformer for emotion detec- tion in textual conversations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 165-176.
The Somsus Ballnss
You tevalabls babii mvtoan undd dig
cencxulee.
zdox vasance
upda
Drone gemenace inturemen'' si. japkk 2ijaarmncd' aumrmnnei 18..-- You sheall as saarnnecs af aesud of crnare anassac..
You dhelkd soking porr
Figure 10: Prompt template for TP.
Lixing Zhu, Gabriele Pergola, Lin Gui, Deyu Zhou,
You an Rnviluable ossiina in and anilyoing which isprssi ixpoocica samanaa in convanation
and Yulan He. 2021. Topic-driven and knowledge- aware transformer for dialogue emotion detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ith International Joint Conference on Natural Lan- guage Processing (Volume I: Long Papers), pages
The Serwas on Salkow loger farwa
Spuater:
Yoar sple feveat Tor
Not..)
the and uucnass nau cousenad rote usever You' of Ny You dhaaldrup redingg fermal
1571-1582.
Figure I I: Prompt template for SC.
Boc sncce cilipii INCOROO
You tnvakatte antiraat suilyiing wiuch arsernce econcsion of maraptor coesinatios Gise
Spalerot did tidestittoo Spakerte Soul
Spakes Wart
The Senuu ase Salvowe
lapei Karmal
Spalero iile-
who
ou outti KilLi modld
Spotker
hoout reginggin
Your apte fummat
varttill
not
For untrraan you choild rodl wouees "Nad in New sleakingg siingg fermel
12 Spciantt You boaw ve dil was lave are clarking co vutetil your up du lidk cor noot
Speiter wh anvennae la ye sen feriiar th tes par bike wok
Figure 12: Prompt template for MP.
Speciee Free tey
the spuctars doci aratan aiea
Braccercnaaes maree, La coaatiee relicied is de carce stomaanc may 301 soleflecnne by Englean pacioos aiconac selicncod be empicce cancel skonnce you mnverri if tfflil encdiend
dlsaea au beay anp
Figure 13: Prompt template for EC.
Figure 8: Prompt template for ChatGPT.
Fig. 9 to perform prompt-tuning which enhances the model's performance in generating emotionally appropriate responses.
you mnddd woeh obourhon didoseei corkgreet bos tweal
B Prompt Templates for Knowledge
fos
In this section, we explain the methodology used to obtain multiple knowledge. We mainly use various prompt templates (Figs. 10-18) to extract the rele- vant knowledge from each utterance in conversa- tion. Due to the diversity of outputs from ChatGPT, also introduce some cases standardize the for- mat of generated knowledge such as CR, as shown in Fig. 16. Especially, we conduct manual eval- uation of extracted knowledge. Specifically, we randomly select 10% conversations to evaluate the quality of this knowledge. Three human evaluators rate each extracted knowledge on scale of 0 to 5: O for extremely poor, for poor, 2 for fair, 3 for good, 4 for very good, and 5 for excellent quality. The weighted average of these scores determines the knowledge quality. Only knowledge with an av- erage score above 3 is retained. Otherwise, adjust the prompt template necessary, re-extract, and reevaluate.
of
plene dey bookki-io man plg in ler..
Tymegh
Figure 9: Prompt template for the Curie model.
A Prompt Templates for ERC
this section, we primarily focus on explaining the methodology used to obtain the results for the ChatGPT and prompt-tune Curie model in this pa- per. For ChatGPT, we employ a prompt template, as depicted in Fig. 8, to extract the emotion of each utterance in conversation. This approach en- sures the utilization of contextual information and facilitates the generation of formmaedttt output results. On the other hand, for the Curie model, we directly access OpenAl's API for implementa- tion. We utilize the prompt example presented in

---

innskaticc quonaat audying laanlegge stonac cencnutee.
Methods
IEMOCAP
MELD
EmoryNLP
Cuerr mone Kanalely reliers peoples yreual Eoodliiie mordly wodk cemenn tod couple
Learning rate
0.0005
0.00001
0.0005
lis sxdemtatingg
comeae da karen ferisla,, lot lall pance ney
Butch size
8
Semuat Salowe
Dropout rale
0.2
0.1
0.3
bipel Sereal
The number layers
2
Your sgée lonnal
1/
0.5
0.3
0.2
anchrooo unply comaaco
ficied sools
0.8
0.5
0.5
You mas sanver aidu bersu
0.8
Figure 14: Prompt template for CS.
Table 7: Search results of hyper-parameterss.
mn innkarii conrcrations
endlying ufferiine conmancene harvicdge dle acch milee
Altectiv ormacesctos Anzn
oenzen ougazine undemandin of crurimn a landova poopli ye geaues leape anioozadig expaossag caua. oonmu atag crntioes Attratie
single NVIDIA A 1100-PCTE-40GB GPU and the reported results are all based on the average score of 5 random runs on the test set.
rmpp rieegis
gronilly
aedcasa aadhers
Trper Simua:
ansranee Spuzker aucunce
D Glossary of acronyms
Your oplyy Borear
pertsifecte coneneneau Luosdddyy offeatee com troodadgg attactce conecesare
In this section, compile thorough glossary of acronyms, as shown in Table to assist readers in comprehending the concepts the paper.
uesozcon somteond comeanon Levaldyy waach
Figure 15: Prompt template for ACS.
conelenting withe maits.
which chanchh andithu untondimas cana anditir Pleas carwer
Paow amaue wablis che emmuaate aue, was adeasuaa afd catoonors sarundlss
Nowr pive
manggles
Spmskrtt Jilli,, nice
Spalic aa (nec.kand.yee
Spukar like
comett muverris
Opeaker 11
Speckor litt (oodd1911]. (911.
11
Teer Jeer.. bow
Figure 16: Prompt template CR.
sdova aattaa ateranc))
Figure 17: Prompt template for CT.
Irscmnnmatios, the deceeaiinn of eareet sitenan mo on inlland to yedkie ahmomao bokker Your focmut
alle contionee stannnc).
Figure 18: Prompt template for EC2.
C Hyper-parameter settings
We perform hyper-parameter search for Baseline (MKFM w/o any knowledge) each dataset with validation set, including learning rate, batch size, dropout rate, and so on. The seeds are 200, 201, 202, 203, and 204. The detailed search results of hyper- parameaassesers are shown in Table Addition- ally, each training and testing process is run on a

---

Acronym
Explanation
ERC
Emotion Recognition in Conversations, the research task of this paper.
ES
Emotion Shift, where two consecutive utterances conversation exhibit different emotions.
SCL
Supervised Contrastive Learning, a training methodology for classification tasks.
MKFM
Multiple Knowledge Fusion Model, integrates three different knowledge: Utterance-level
Encoder for AUK, Graph Context Encoder for ACK, and Contrastive Learning module for ALK.
AUK
Auxiliary Utterance Knowledge, category of knowledge represented by one several sentences for each utterance in conversation.
CS
Sens knowledge of utterances, type of AUK.
ACS
Affective CommonSense knowledge of utterances, a type of AUK with focus on emotions and sentiments.
EC
Emotional Cause of utterances, type of AUK focusing on context clues triggers giving rise to emotions and sentiments.
ACK
Auxiliary Context Knowledge, category of knowledge represented index list for each utterance in a conversation.
CR
Co- Co-Reference relationships between utterances, type of ACK represented an index list of historically related utterances.
EC2
Emotional Cause relationships between utterances, type of ACK represented as index list indicating emotional causes behind the current utterance.
CT
Context for better understanding of utterances, type of ACK represented as an index list of historically relevant utterances.
ALK
Auxiliary Label Knowledge, a category of knowledge represented by a label for each utterance a conversation.
TP
Topics of utterances, a type of ALK represented as the topic label of each utterance in conversation.
SC
Sarcasm indication for utterances, a type of ALK denoted by labels or 0. Metaphor indication for utterances, a type of ALK denoted by labels I or O.
MP
Table Glossary acronyms.


Reviews:
[{"review_id": "e0ffT9IfHh", "reviewer": "Reviewer_poob", "summary": "", "questions": "A. How exactly is the contrastive learning part of the model used? Is it used to extract information such as sarcasm and metaphor to incorporate as part of the extra knowledge used to predict the emotion of the utterance?\n\nB. Why this particular typology of knowledge information? Can the authors introduce their own knowledge categorization and make the prompt engineering easier?\n\nC. How do you validate the output of the ChatGPT?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our great appreciation for your valuable comments to improve the quality of this manuscript.  \n\n**Answer of Reject#1:**  \n&ensp;&ensp;Thanks for your valuable suggestion. We will include a comprehensive glossary of acronyms in the appendix, as shown below, to assist readers in comprehending the concepts in the paper. Thank you again for your insightful input; we believe this addition will significantly enhance the clarity and comprehensibility of our paper.  \n| Acronym | Explanation                                                                                                  |\n|:---------:|:--------------------------------------------------------------------------------------------------------------:|\n| ERC     | ERC represents Emotion Recognition in Conversations, which is also the research task of this paper.                              |\n| ES      | ES represents Emotion Shift, where two consecutive utterances in a conversation exhibit different emotions.                   |\n| SCL     | SCL represents Supervised Contrastive Learning, which is a training methodology for classification tasks.                          |\n| MKFM    | MKFM represents Multiple Knowledge Fusion Model, which is used to integrate three different knowledge. Specifically, MKFM employs the Utterance-level Encoder to integrate AUK, the Graph Context Encoder to incorporate ACK, and utilizes the Contrastive Learning module to fuse ALK. |\n| AUK     | AUK represents Auxiliary Utterance Knowledge, which refers to a category of knowledge represented by one or several sentences, corresponding to each utterance in a conversation.                    |\n| CS      | CS belongs to AUK and represents CommonSense knowledge of utterances. Unlike other forms of knowledge like triplets, obtaining utterance knowledge is simpler.                                |\n| ACS     | ACS belongs to AUK and represents Affective CommonSense knowledge of utterances. ACS is similar in form to CS but with a greater emphasis on knowledge related to emotions and sentiments.                     |\n| EC      | EC belongs to AUK and represents Emotional Cause of utterances. EC is also similar in form to CS but focuses more on context clues or triggers that give rise to emotions and sentiments.                                  |\n| ACK     | ACK represents Auxiliary Context Knowledge, which refers to a category of knowledge represented by an index list, corresponding to each utterance in a conversation.                    |\n| CR      | CR belongs to ACK and represents Co-Reference relationships between utterances. CR is an index list of historical utterances with a co-reference relationship to the current utterance.              |\n| EC2     | EC2 belongs to ACK and represents Emotional Cause relationships between utterances. EC2 is an index list of historical utterances indicating the emotional causes behind the current utterance.               |\n| CT      | CT belongs to ACK and represents Context for a better understanding of utterances. CT is an index list of historical utterances that help in understanding the current utterance.          |\n| ALK     | ALK represents Auxiliary Label Knowledge, which refers to a category of knowledge represented by a label, corresponding to each utterance in a conversation.                            |\n| TP      | TP belongs to ALK and represents Topics of utterances. TP is the topic label of each utterance in a conversation.                                                      |\n| SC      | SC belongs to ALK and represents whether the utterance is expressing SarCasm or not, typically denoted by the labels 1 or 0.                                    |\n| MP      | MP belongs to ALK and represents whether the utterance is expressing MetaPhor or not, typically denoted by the labels 1 or 0.                                 |\n  \n**Answer of Reject#2:**  \n&ensp;&ensp;Thanks for your insightful comment. We divide this knowledge, demonstrated effective for emotion detection, into three categories (AUK, ACK, and ALK) based on data formats. Specifically, AUK refers to a category of knowledge represented by one or several sentences corresponding to each utterance. ACK refers to a category of knowledge represented by an index list corresponding to each utterance. ALK refers to a category of knowledge represented by a label corresponding to each utterance. For instance, for SC in ALK, we can utilize ChatGPT to determine whether an utterance carries sarcasm undertones, which serves as a form of label information. CR in ACK focuses on enhancing contextual understanding. Determining if an utterance refers back to something mentioned earlier might not be applicable to SC. Furthermore, the approach of obtaining labels or list-based information for CS in AUK is unconventional, making it more suitable for aiding the semantic understanding of utterances.  \n&ensp;&ensp;Intuitively, AUK, ACK, and ALK have distinct roles in ERC. AUK enhances the semantics of utterances, ACK enriches contextual information, and ALK introduces labels for various tasks to learn task-specific latent features, thus improving the model's emotional understanding capability. Additionally, more details on the labeling and their meanings can be found in the **Answer of Reject#1**. The provided example in Figure 1 enhances the clarity of our illustration for AUK, ACK, and ALK.\n\n**Answer of Reject#3:**  \n&ensp;&ensp;Thanks for your insightful comment. For integrating ALK, the obvious approach is to employ multi-task learning. However, this could lead to increased workload and additional training complexity, which may overly complicate the baseline. Hence, we leverage Supervised Contrastive Learning (SCL), which pulls together utterance representations with the same label and pushes apart those with different labels. This is achieved without escalating model complexity and raising training expenses. It only adds an additional loss term while still delivering effective results.\n\n**Answer of Reject#4:**  \n&ensp;&ensp;Thanks for your valuable comment. In the process of extracting knowledge from ChatGPT, noise does indeed emerge. The noise introduced by irrelevant knowledge has consistently remained one of the challenges in ERC. Even without using ChatGPT to acquire knowledge, similar noise is inevitable (Tu et al., 2022a).  \n&ensp;&ensp;To mitigate the impact of noise, we conducted a manual evaluation of extracted knowledge. Specifically, we randomly selected 10% conversations to evaluate the quality of this knowledge. Three human evaluators rated each extracted knowledge on a scale of 0 to 5: 0 for extremely poor, 1 for poor, 2 for fair, 3 for good, 4 for very good, and 5 for excellent quality. The weighted average of these scores determined the knowledge quality. Only knowledge with an average score above 3 was retained. Otherwise, adjust the prompt template if necessary, re-extract, and reevaluate. We will detail it in the revision.\n\n*Tu, G., Liang, B., Jiang, D., & Xu, R. (2022). Sentiment-Emotion-and Context-guided Knowledge Selection Framework for Emotion Recognition in Conversations. IEEE Transactions on Affective Computing.*\n\n**Answer of Reject#5:**  \n&ensp;&ensp;Thanks for your insightful comment. As for why it is worth expending the extra effort to acquire this knowledge, we can elaborate based on the following key points.\n1. Some settings as shown in Table 6 show significant improvements, highlighting the potential for outperforming strong baselines. Further optimizing the method of knowledge integration could lead to additional performance improvement.\n2. Although the improved performance is marginal with different settings on the whole, it becomes significant after the 100th utterance as shown in Figure 3, effectively compensating for the limitations of ERC models in modeling long-term contextual dependencies.\n3. The incorporation of external knowledge appears to alleviate the issue of class imbalance in ERC as depicted in Figure 4.\n4. The strong generalizability of this knowledge is evident from Figure 7. Our results demonstrate significant performance improvements in different ERC models that leverage this knowledge.\n\n**Answer of Question A:**  \n&ensp;&ensp;Thanks for your insightful question. Contrastive learning module, as mentioned in this paper, is primarily applied to integrate ALK rather than knowledge extraction. Knowledge extraction for SC (or MP) is achieved using ChatGPT. This knowledge belongs to ALK and represents whether the utterance is expressing sarcasm (or metaphor) or not, typically denoted by the labels 1 or 0. In the contrastive learning module, we leverage Supervised Contrastive Learning (SCL) to pull together utterance representations with the same label and to push apart those with different labels.\n\n**Answer of Question B:**  \n&ensp;&ensp;Thanks for your insightful question. We categorize this knowledge based on its data format due to our consideration of knowledge attributes. Specifically, AUK refers to a category of knowledge represented by one or several sentences corresponding to each utterance. ACK refers to a category of knowledge represented by an index list corresponding to each utterance. ALK refers to a category of knowledge represented by a label corresponding to each utterance. For instance, for SC in ALK, utilizing ChatGPT to determine whether an utterance carries sarcasm undertones is intuitively reasonable, and serves as a form of label information. CR in ACK focuses on enhancing contextual understanding. Determining if an utterance refers back to something mentioned earlier might not be applicable to SC. Obtaining labels or index lists for CS in AUK is unconventional, making it more suitable for enhancing the semantics of utterances.  \n&ensp;&ensp;Additionally, introducing our own knowledge categorization for facilitating prompt engineering is a promising and theoretically feasible concept. Thank you again for your insightful input; We will attempt this approach to simplify prompt engineering in the source code upon paper acceptance.\n\n**Answer of Question C:**  \n&ensp;&ensp;Thanks for your valuable question. We conducted a manual evaluation of extracted knowledge. Specifically, we randomly selected 10% of conversations to evaluate the quality of this knowledge. Three human evaluators rated each extracted knowledge on a scale of 0 to 5: 0 for extremely poor, 1 for poor, 2 for fair, 3 for good, 4 for very good, and 5 for excellent quality. The weighted average of these scores determined the knowledge quality. Only knowledge with an average score above 3 was retained. We will detail it in the revision.\n\n**Answer of Typos Grammar Style And Presentation Improvements:**  \n&ensp;&ensp;Thanks a lot for your valuable suggestions, we have corrected them in the revision and will avoid these mistakes in the future.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper has the following strengths:\n\n1. The authors overcome the problem of annotating the conversation datasets by employing ChatGPT prompts which they engineer to extract the needed knowledge information. This approach is very promising for other NLP tasks to compensate for the lack of gold-standard annotations specially for challenging tasks.\n\n2. The paper includes examples of the prompts used for producing the multiple knowledge categories used in training the emotion detection model. The prompt templates can be useful for future research.\n\n3. The in-depth analysis of how each category of the extra knowledge has helped in improving the model's performance as well as which emotional categories has been positively affected.\n\n4. The authors show that their model's performance surpasses a number of benchmarks.", "reasons_to_reject": "There a few issues which the authors may want to consider:\n\n1. The long list of acronyms used in the paper makes it very difficult to read. It would be very helpful if there is a glossary of acronyms in the appendix if the authors insist on using all these acronyms to explain their experiments.\n\n2. The difference between the three knowledge categories is not very clear. Some of the categories seem to relate to each other and are not really distinguishable, others seem to be difficult to understand. The authors relied on the categorization of other research from different domains but some of the labeling and their meanings are not clear.\n\n3. The contrastive learning module section is difficult to understand.\n\n4. Although the approach of obtaining knowledge from ChatGPT prompting is a good idea, there is no mention of any type of validation for the produced information. The extra knowledge can constitute noise rather than incorporate extra features.\n\n5. The improved performance is marginal with different settings. The paper needs to justify the extra effort that is exerted to obtain this knowledge as compared to the marginal improvement in detecting emotions in the conversation datasets used.", "questions_to_authors": "A. How exactly is the contrastive learning part of the model used? Is it used to extract information such as sarcasm and metaphor to incorporate as part of the extra knowledge used to predict the emotion of the utterance?\n\nB. Why this particular typology of knowledge information? Can the authors introduce their own knowledge categorization and make the prompt engineering easier?\n\nC. How do you validate the output of the ChatGPT?", "paper_topic_and_main_contributions": "The paper tackles the problem of emotion recognition in conversations. The authors introduce \"multiple knowledge\" such as co-reference, topics of the utterance, the emotional cause, sarcasm, metaphor, etc. as features to improve the detection of emotion in conversation. They acquire the extra knowledge information through ChatGPT prompts which they engineer in a way to extract the information in their particular format. They categorize the knowledge information based on different literature research.\n\nThe introduced model is a pretrained Roberta which encodes the utterances and enhances the utterance representations with the auxiliary information they obtained from the ChatGPT output. The model also employs graph networks to capture contextual information in the conversation. Their model graph utilizes information such as coreference and emotional cause to represent relationships between nodes. The classifier used for predicting the emotion in conversation employs contrastive learning.\n\nThe authors run several experiments with different settings for the multiple knowledge they annotate to conversation emotion detection datasets. They conclude that generally speaking the model benefits from the incorporated extra knowledge. They show, however, that the improvements fluctuates as the number of utterances increases but the performance is typically better towards the end of the conversation.\n\nThey also illustrate the effect of the extra knowledge on the prediction of emotional categories and show that the contribution of knowledge boosts the performance of the minority classes in the datasets such as 'happy'.\nThey compare their model's performance with other baselines and they show that their model surpasses the state-of-the-art benchmarks on the datasets they use for experiments.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Please check language on lines:\n\n079, 106-107"}, "extra_text": {}}, {"review_id": "4s5qXvTmFm", "reviewer": "Reviewer_7phw", "summary": "", "questions": "Are you sure that the context data obtained from ChatGPT is correct and not subject to hallucination?\nHave you thought of manually validating at least some of them? To assess its quality and reliability?\nDo you think the approach is generalizable with a good level of reliability?\n\n(issues are effectively resolved by authors)", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our great appreciation for your valuable comments to improve the quality of this manuscript.\n\n**Answer of Reject#1:**  \n&ensp;&ensp;Thanks for your valuable comment. We will include a comprehensive glossary of acronyms in the appendix, as shown below, to assist readers in comprehending the concepts in the paper. Thank you again for your insightful input; we believe this addition will significantly enhance the clarity and comprehensibility of our paper.\n| Acronym | Explanation                                                                                                  |\n|:---------:|:--------------------------------------------------------------------------------------------------------------:|\n| ERC     | ERC represents Emotion Recognition in Conversations, which is also the research task of this paper.                              |\n| ES      | ES represents Emotion Shift, where two consecutive utterances in a conversation exhibit different emotions.                   |\n| SCL     | SCL represents Supervised Contrastive Learning, which is a training methodology for classification tasks.                          |\n| MKFM    | MKFM represents Multiple Knowledge Fusion Model, which is used to integrate three different knowledge. Specifically, MKFM employs the Utterance-level Encoder to integrate AUK, the Graph Context Encoder to incorporate ACK, and utilizes the Contrastive Learning module to fuse ALK. |\n| AUK     | AUK represents Auxiliary Utterance Knowledge, which refers to a category of knowledge represented by one or several sentences, corresponding to each utterance in a conversation.                    |\n| CS      | CS belongs to AUK and represents CommonSense knowledge of utterances. Unlike other forms of knowledge like triplets, obtaining utterance knowledge is simpler.                                |\n| ACS     | ACS belongs to AUK and represents Affective CommonSense knowledge of utterances. ACS is similar in form to CS but with a greater emphasis on knowledge related to emotions and sentiments.                     |\n| EC      | EC belongs to AUK and represents Emotional Cause of utterances. EC is also similar in form to CS but focuses more on context clues or triggers that give rise to emotions and sentiments.                                  |\n| ACK     | ACK represents Auxiliary Context Knowledge, which refers to a category of knowledge represented by an index list, corresponding to each utterance in a conversation.                    |\n| CR      | CR belongs to ACK and represents Co-Reference relationships between utterances. CR is an index list of historical utterances with a co-reference relationship to the current utterance.              |\n| EC2     | EC2 belongs to ACK and represents Emotional Cause relationships between utterances. EC2 is an index list of historical utterances indicating the emotional causes behind the current utterance.               |\n| CT      | CT belongs to ACK and represents Context for a better understanding of utterances. CT is an index list of historical utterances that help in understanding the current utterance.          |\n| ALK     | ALK represents Auxiliary Label Knowledge, which refers to a category of knowledge represented by a label, corresponding to each utterance in a conversation.                            |\n| TP      | TP belongs to ALK and represents Topics of utterances. TP is the topic label of each utterance in a conversation.                                                      |\n| SC      | SC belongs to ALK and represents whether the utterance is expressing SarCasm or not, typically denoted by the labels 1 or 0.                                    |\n| MP      | MP belongs to ALK and represents whether the utterance is expressing MetaPhor or not, typically denoted by the labels 1 or 0.                                 |\n  \n**Answer of Reject#2:**  \n&ensp;&ensp;Thanks for your valuable comment. We performed a paired t-test on the experimental outcomes presented in Table 6. The results indicate a p-value of < 0.05 between the original baseline results and the baseline results enhanced with different knowledge. Even in the scenario with the smallest observed enhancement—specifically, between the baseline with ACS and the baseline on the EmoryNLP dataset—the p-value stood at 0.03, highlighting statistical significance. We will detail it in the revision.\n\n**Answer of Reject#3:**  \n&ensp;&ensp;Thanks for your insightful comment. In our study, we have carried out experiments utilizing three distinct datasets: IEMOCAP, EmoryNLP, and MELD. The detailed statistics pertaining to these datasets are presented in Table 1.\n\n**Answer of Reject#4:**  \n&ensp;&ensp;Thanks for your insightful comment. We will release the source code after the acceptance of this paper to facilitate reproducibility by readers.\n\n**Answer of Questions:**  \n&ensp;&ensp;Thanks for your valuable questions. The concerns regarding the accuracy of the context data and the hallucination issue are indeed crucial considerations. To mitigate the risks of errors and the hallucination issue, we conducted a manual evaluation of extracted knowledge.  \n&ensp;&ensp;Specifically, we randomly selected 10% of conversations to evaluate the quality of this knowledge. Three human evaluators rated each extracted knowledge on a scale of 0 to 5: 0 for extremely poor, 1 for poor, 2 for fair, 3 for good, 4 for very good, and 5 for excellent quality. The weighted average of these scores determined the knowledge quality. Only knowledge with an average score above 3 was retained. Otherwise, adjust the prompt template if necessary, re-extract, and reevaluate. We will detail it in the revision.  \n&ensp;&ensp;In Section 5.5, we validated the generalizability of our approach, as depicted in Figure 7. Our results show performance improvements in different ERC models, which utilize the MKFM to integrate this knowledge derived from ChatGPT. This observation confirms the capacity of our approach to generalize across various ERC methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Technically Robust\n- Exhaustive Experimental Session\n- Innovative model is proposed\n- Trending and relevant topic\n- New research frontiers", "reasons_to_reject": "- Complex to read and follow\n- Results are not statistically validated (issues are effectively resolved by authors)\n- Only one dataset (issues are effectively resolved by authors)\n- Low replicability (issues are effectively resolved by authors)", "questions_to_authors": "Are you sure that the context data obtained from ChatGPT is correct and not subject to hallucination?\nHave you thought of manually validating at least some of them? To assess its quality and reliability?\nDo you think the approach is generalizable with a good level of reliability?\n\n(issues are effectively resolved by authors)", "paper_topic_and_main_contributions": "The authors propose a Multiple Knowledge Fusion Model (MKFM) to effectively integrate Multiple knowledge (e.g., co-reference, topics, emotional causes, etc) generated by prompting LLMs for Emotion Recognition in Conversations (ERC).", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "--", "Typos_Grammar_Style_and_Presentation_Improvements": "--"}, "extra_text": {}}, {"review_id": "AwJo4FSjQR", "reviewer": "Reviewer_9VVj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our great appreciation for your valuable comments to improve the quality of this manuscript.\n\n**Answer of Reject#1:**  \n&ensp;&ensp;Thanks for your valuable comment. We will include a comprehensive glossary of acronyms in the appendix, as shown below, to assist readers in comprehending the concepts in the paper. Thank you again for your insightful input; we believe this addition will significantly enhance the clarity and comprehensibility of our paper.\n| Acronym | Explanation                                                                                                  |\n|:---------:|:--------------------------------------------------------------------------------------------------------------:|\n| ERC     | ERC represents Emotion Recognition in Conversations, which is also the research task of this paper.                              |\n| ES      | ES represents Emotion Shift, where two consecutive utterances in a conversation exhibit different emotions.                   |\n| SCL     | SCL represents Supervised Contrastive Learning, which is a training methodology for classification tasks.                          |\n| MKFM    | MKFM represents Multiple Knowledge Fusion Model, which is used to integrate three different knowledge. Specifically, MKFM employs the Utterance-level Encoder to integrate AUK, the Graph Context Encoder to incorporate ACK, and utilizes the Contrastive Learning module to fuse ALK. |\n| AUK     | AUK represents Auxiliary Utterance Knowledge, which refers to a category of knowledge represented by one or several sentences, corresponding to each utterance in a conversation.                    |\n| CS      | CS belongs to AUK and represents CommonSense knowledge of utterances. Unlike other forms of knowledge like triplets, obtaining utterance knowledge is simpler.                                |\n| ACS     | ACS belongs to AUK and represents Affective CommonSense knowledge of utterances. ACS is similar in form to CS but with a greater emphasis on knowledge related to emotions and sentiments.                     |\n| EC      | EC belongs to AUK and represents Emotional Cause of utterances. EC is also similar in form to CS but focuses more on context clues or triggers that give rise to emotions and sentiments.                                  |\n| ACK     | ACK represents Auxiliary Context Knowledge, which refers to a category of knowledge represented by an index list, corresponding to each utterance in a conversation.                    |\n| CR      | CR belongs to ACK and represents Co-Reference relationships between utterances. CR is an index list of historical utterances with a co-reference relationship to the current utterance.              |\n| EC2     | EC2 belongs to ACK and represents Emotional Cause relationships between utterances. EC2 is an index list of historical utterances indicating the emotional causes behind the current utterance.               |\n| CT      | CT belongs to ACK and represents Context for a better understanding of utterances. CT is an index list of historical utterances that help in understanding the current utterance.          |\n| ALK     | ALK represents Auxiliary Label Knowledge, which refers to a category of knowledge represented by a label, corresponding to each utterance in a conversation.                            |\n| TP      | TP belongs to ALK and represents Topics of utterances. TP is the topic label of each utterance in a conversation.                                                      |\n| SC      | SC belongs to ALK and represents whether the utterance is expressing SarCasm or not, typically denoted by the labels 1 or 0.                                    |\n| MP      | MP belongs to ALK and represents whether the utterance is expressing MetaPhor or not, typically denoted by the labels 1 or 0.                                 |\n  \n**Answer of Reject#2:**  \n&ensp;&ensp;Thanks for your valuable comment. We have revised the manuscript to include a detailed section devoted to the methodology of utilizing ChatGPT for acquiring multiple knowledge. In this process, the most crucial aspect is to obtain a suitable prompt template for each type of knowledge. Therefore, we randomly selected 10% of conversations to evaluate the quality of this knowledge. Three human evaluators rated each extracted knowledge on a scale of 0 to 5: 0 for extremely poor, 1 for poor, 2 for fair, 3 for good, 4 for very good, and 5 for excellent quality. The weighted average of these scores determined the knowledge quality. Only knowledge with an average score above 3 was retained. Otherwise, adjust the prompt template if necessary, re-extract, and reevaluate. Next, we can utilize the prompt templates to acquire this knowledge, as demonstrated in the example given in Figure 1. We will detail it in the revision.\n\n**Answer of Reject#3:**  \n&ensp;&ensp;Thanks for your insightful comment. In this paper, MKFM employs the Utterance-level Encoder to integrate AUK, the Graph Context Encoder to incorporate ACK and utilizes the Contrastive Learning module to fuse ALK. To emphasize the details of knowledge integration, we will restructure the methodology section according to the framework outlined in the introduction, as per your advice. We appreciate your attention to the clarity and structure of our methodology section. Your comment has been invaluable in guiding us toward a more comprehensive and reader-friendly presentation of our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The idea presented in this paper is intriguing and warrants the investigation.\n- The extensive number of experiments and analyses illustrates the advantages of leveraging diverse knowledge sources for the task of emotion recognition.", "reasons_to_reject": "- One of the primary drawbacks of this paper is the excessive use of abbreviations, which creates confusion for readers and requires constant backtracking.\n\n- Another issue is the mention of utilizing ChatGPT for obtaining multiple knowledge sources, without a dedicated section describing its methodology. Given that ChatGPT is referenced in multiple sections of the paper (e.g., title and introduction), it would have been beneficial to provide a thorough explanation of its role in the methodology.\n\n- Although the paper discusses the various categories of knowledge sources in the third paragraph of the introduction, there is insufficient description of these sources in the methodology section and how the proposed model (MKFM) effectively incorporates them. To address this, I recommend following the structure outlined in the introduction, dividing the methodology section into three categories: 1. Auxiliary Contextual Knowledge, 2. Auxiliary Label Knowledge, and 3. Utterance Knowledge. This approach would help readers understand how the proposed model integrates different types of knowledge and facilitate the connection between the three auxiliary knowledge types.\n\nIn short, this paper is limited by inadequate clarification of the use of ChatGPT, excessive use of abbreviations, and lack of organizational structure.\n\n-------------------------------------------------------------\nI thank the authors for providing a detailed response. While I appreciate the inclusion of a glossary of abbreviations, I would recommend reducing the number of abbreviations to only a few. This will enhance the paper's readability and ease of navigation. I am pleased to note that the authors have addressed the remaining comments I had.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The purpose of this paper is to acquire valuable insights (such as co-reference, topics, emotional causes, etc.) to improve the task of emotion recognition in conversation. In order to accomplish this, the authors introduce a Multiple Knowledge Fusion Model (MKFM) that integrates diverse knowledge sources to enhance the performance of emotion recognition. Subsequently, they perform a series of experiments and analyses using established emotion datasets.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: - The paper uses many abbreviations and it is hard to read the manuscript.
- An excessive number of experiment is done but the results are marginally better
- The model is complex and that makes reproducibility a challenging task
- The method is tested only on one dataset
Decision: Accept-Findings