An Empirical Study on Multiple Knowledge from C hat GPT for Emotion Recognition in Conversations (2023)

Abstract: Multiple knowledge (e.g., co-reference, topics, emotional causes, etc) has been demonstrated effective for emotion detection. However, exploring this knowledge in Emotion Recognition in Conversations (ERC) is currently a blank slate due to the lack of annotated data and the high cost involved in obtaining such knowledge. Fortunately, the emergence of Large Language Models (LLMs) holds promise in filling this void. Therefore, we propose a Multiple Knowledge Fusion Model (MKFM) to effectively integrate such knowledge generated by LLMs for ERC and empirically study its impact on the model. Experimental results on three public datasets have demonstrated the effectiveness of multiple knowledge for ERC. Furthermore, we conduct a detailed analysis of the contribution and complementarity of this knowledge.

## introduction emotion recognition in conversations (erc) has garnered considerable research interest in recent years, as evidenced by several studies such as @xcite @xcite . the goal of erc is to identify the emotion of each utterance in conversations. existing efforts in erc have traditionally focused on modeling context-and speaker-sensitive dependencies @xcite , including recurrent-based network @xcite @xcite @xcite , transformer-based network @xcite @xcite , and graph-based network @xcite @xcite . additionally, recent erc models @xcite @xcite have begun to leverage commonsense knowledge, which mainly includes two categories. one is generative knowledge generated by the pre-trained commonsense transformer (comet) @xcite . the other is concepts retrieved from the external knowledge base conceptnet @xcite or senticnet @xcite . however, relying solely on commonsense knowledge as background information @xcite is insufficient for a deep understanding and recognition of emotions. multiple knowledge (e.g., co-reference, topics, emotional causes, etc) are also crucial for emotion detection, but integrating this knowledge into the erc model poses challenges due to the limited availability of annotated data and the high cost of manual tagging. fortunately, large language models (llms) such as chatgpt provides an opportunity to overcome these limitations, as illustrated in fig. 1 . we divide this knowledge from chatgpt into three categories based on data formats, namely auxiliary contextual knowledge (ack with index list format): co-reference (cr) is crucial for modeling multi-party dialogue in improving the dialogue understanding @xcite ; emotional cause (ec2) reasoning involves identifying the reasons behind emotions and how contextual utterances contribute to them @xcite ; context (ct) is indicated as important in emotion generation theory @xcite , auxiliary label knowledge (alk with label format): topics (tp) play a significant role as emotions are closely tied to specific subjects, and the same words can have different emotional meanings depending on the topic @xcite ; sarcasm (sc) is an important linguistic tool that uses irony to express contempt and affects the accurate prediction of emotional meaning @xcite ; metaphor (mp) is common language expression that conveys nonliteral meanings and emotions by comparing or connecting concepts @xcite , and auxiliary utterance knowledge (auk with sentence format): commonsense knowledge (cs) related to the utterances; affective commonsense knowledge (acs) that bridges the cognitive and affective gap between word-level natural language data and the concept-level sentiments conveyed by them @xcite ; another format of emotional cause (ec). then we introduce a multiple knowledge fusion model (mkfm) that combines three types of knowledge via utterance-level encoder for auk, graph context encoder for ack, and supervised contrastive learning (scl) module for alk. our main contributions are (1) the first exploration of multiple knowledge for erc, (2) the proposed mkfm for integrating this knowledge, which outperforms state-of-the-art baselines, and (3) conducting further analysis on the contribution and complementarity of multiple knowledge. ## related work emotion recognition in conversations: the emotion generation theory @xcite emphasizes the role of contextual information in identifying emotions. rnn-based models @xcite are commonly used to capture context dependencies but struggle with distinguishing between historical utterances @xcite . memory networks have been proposed to address this limitation @xcite . furthermore, the participants in erc are crucial @xcite , leading to the development of speaker-specific models @xcite @xcite , graph-based models @xcite @xcite , and so on. however, these approaches lack commonsense knowledge, which is important for human-like performance @xcite . therefore, researchers have integrated external knowledge sources like comet @xcite , senticnet @xcite , and conceptnet @xcite into their models @xcite @xcite . contrastive learning: chen et al. (2020) introduced simclr, a comparative learning network for visual representation using image augmentation. in nlp, @xcite proposed a selfsupervised cl method for fine-tuning bert, addressing bert's poor performance in semantic text similarity tasks. to incorporate label information, @xcite extended self-supervised cl to a fully-supervised cl framework, which improved performance in few-shot learning scenarios. in erc, @xcite and @xcite applied scl to separate utterances with different emotions, enhancing emotion identification. however, there is no effort on unsupervised cl in erc. ## experiments 4.1 datasets we conduct experiments on three datasets: iemo-cap @xcite , emorynlp (zahiri and choi, 2018), and meld @xcite . the dataset statistics are presented in meld is a multi-party conversation dataset collected from the tv show friends, which is an extension of the emotionlines dataset @xcite . each utterance is annotated with one emotion from the set: surprise, fear, disgust, anger, sadness, neutral, or joy, along with one sentiment from the set: neutral, negative, or positive. emorynlp consists of multi-party sessions from the tv show friends and each utterance is labeled with one emotion from the set: joyful, scared, peaceful, sad, powerful, mad, or neutral, along with one sentiment, as suggested in @xcite , from the set: neutral, negative, or positive. ## conclusion in this paper, we propose mkfm for integrating multiple knowledge sources in erc, which surpasses state-of-the-art baselines. we also conduct further analysis of the contribution and complementarity of this knowledge. the contribution of knowledge focuses on end-of-conversation utterances, particularly those from minority classes. however, there are still unresolved es issues. when it comes to the complementarity of homologous knowledge, apart from auk's performance decline caused by noise, other cases generally show satisfactory results. as for the complementarity of non-homologous knowledge, the combination including tp struggles with inconsistent annotations leads to persistently poor performance in long conversations. other cases show satisfactory results. while the complementary improvement does not address es problems effectively, it has a notable impact on utterances from minority classes, aligning with the contribution of this knowledge. additionally, we showcase the impressive generalization ability of this knowledge in other erc models. {"prompt":"1. speaker0: guess what? ->","completion":" excited"} {"prompt":"2. speaker1: what? ->","completion":" neutral"} {"prompt":"3. speaker0: i did it, i asked her to marry me. ->","completion":" excited"} {"prompt":"4. speaker0: yes, i did it. ->","completion":" excited"} {"prompt":"5. speaker1: when? ->","completion":" excited"} {"prompt":"6. speaker0: oh my god, it was just last weekend. ->","completion":" excited"} {"prompt":"7. speaker1: oh, what, how-where how did you do it? ->","completion":" excited"} {"prompt":"8. speaker0: she--,well, she said yes, first of all, let me say that right off the bat. well, i would like to assume, too. but you never know these things, right? ->","completion":" happy"} {"prompt":"9. speaker1: okay good. i i assumed. ->","completion":" excited"} {"prompt":"10. speaker1: oh-->","completion":" happy"} {"prompt":"11. speaker0: i did it up at yosemite. we went camping right like we usually do. you know we go camping up there all the time. but, um-the waterfalls had died down a little bit, you know. and so uh-we-i had her climb up. ->","completion":" happy"} {"prompt":"12. speaker0: you know-we did some-you know some rock climbing up the waterfalls and went up to this little pool that was up there. and then, i-it's great. um-you have to climb this little rock-->","completion":" happy"} {"prompt":"13. speaker1: uh-huh ->","completion":" neutral"} {"prompt":"14. speaker0: and you can climb it and it's covered in algae you know. so, i wanted her to climb it to go take a picture, like, back in this little waterfall area, and she got back there and she took her picture. and then, i had her come back. and i was going to do the same thing.