Global-Local Modeling with Prompt-Based Knowledge Enhancement for Emotion Inference in Conversation (2023)

Abstract: The ability to recognize emotions in conversations is necessary and important for the online chatbot to do tasks such as empathetic response generation and emotional support. Present researches mainly focus on recognizing emotions through a speaker’s utterance, while research on emotion inference predicts emotions of addressees through previous utterances. Because of the lack of the addressee’s utterance, emotion inference is more challenging than emotion recognition. In this paper, we propose a global-local modeling method based on recurrent neural networks (RNN) and pre-trained language models (PLM) to do emotion inference, which utilizes the sequence modeling ability of RNNs and abundant knowledge from PLMs. Moreover, we take the whole dialogue history as input of PLM to generate knowledge by in-context learning. Experimental results show that our model with knoledge enhancement achieves state-of-the-art performance on all three datasets.

## introduction the task of emotion recognition in conversation (erc) @xcite aims to identify emotion labels of an utterance, where the whole dialogue history along with the current utterance is given. however, in emotion inference in conversation (eic) the current utterance is lacking but the dialogue history and the current addressee are known @xcite . for example, figure 1 shows a conversation between a and b. in the third turn, erc detects a's emotion using all available information while eic predicts a's emotion using all the information except the last utterance. erc is a popular task that has been explored widely and deeply, while eic is a new task that measures the emotion understanding ability of models from a different perspective. person a person b [neutral]: hi, what are you doing here? [happy]: i am celebrating my child's birthday. would you like to join us? [happy]: thank you! it's so nice of you to invite me. in erc, some previous works utilize sequencebased neural networks to model the context and speaking parties @xcite @xcite . these approaches first finetune a model on utterances to classify emotions. then this model is used to extract features of utterances. as shown in figure 1 , b makes a happy because he/she is inviting a to join in a party. however, b feels happy because he/she is celebrating his/her child's birthday. the finetuning tends to keep the semantics that is helpful to classify the current emotion and the feature extraction compresses the utterance's information, which may cause information loss that is valuable to infer a's emotion. some works model dialogues at utterance-level with graph-based models @xcite . they have the same problem as sequence-based models. also, it becomes difficult for them to distinguish similar emotions as their layers deepen @xcite . pre-trained language models do not need to do the feature extraction process and they contain knowledge suitable for eic. however, these models can not naturally process sequential utterances from different parties. motivated by this, we propose global-local modeling method to combine different abilities from these models. specifically, we use a sequence-based model to get the representation of the dialogue history and a pre-trained model to process utterances that are close to the addressee's turn in which his/her emotion is to be inferred. in our framework, the global representation and local utterances can attend to each other, which we believe is helpful for eic. some researchers introduce external knowledge to improve the performance of emotion detection @xcite . they generate commonsense knowledge using comet @xcite which is trained on atomic @xcite . however, this knowledge is limited to certain event types. also, it is generated based on a single utterance instead of the whole dialogue, which further limits the quality of the knowledge. recent advancements of in-context learning @xcite show that it is possible to generate high-quality knowledge when language models are provided with appropriate examples. based on the above analysis, we propose a knowledge generation method specially designed for eic task based on prompt learning. specifically, we use templates to obtain two kinds of knowledge: i. we let gpt fill the dialogue, thus we get pseudo utterances that may be spoken by the addressee and take them as knowledge. ii. we ask gpt how the addressee feels and take generated texts as knowledge. our knowledge is more precise since we take the whole dialogue history as input. also, it is more diverse because gpt is trained on a large number of texts in different fields. ## related work emotion recognition in conversation has been a popular area where different models have been proposed. we divide them into three categories: sequence-based, graph-based and transformerbased. dialoguernn @xcite models different parties and global state by different recurrent neural networks. dialogueinfer @xcite adopts two lstms to process utterances by whether they are from addressees. dialoguecrn @xcite iteratively does retrieving and reasoning process to extract and integrate emotional clues. dialoguegcn @xcite utilizes graph neural networks to connect utterances with surrounding utterances. dag @xcite uses a directed acyclic graph network to gather information over long distances. cog-bart @xcite adopts supervised contrastive learning and response generation as auxiliary tasks. compm (lee and lee, 2022) combines speaker's memory using a pre-trained model as an extractor. some works focus on introducing knowledge to help detect emotions. ket @xcite retrieve commonsense knowledge from conceptnet @xcite and nrc_vad @xcite . cosmic @xcite , dialogueinfer @xcite and todkat @xcite incorporates commonsense knowledge generated by comet @xcite . gkp @xcite generates knowledge from language models with prompt learning to do commonsense reasoning. ## conclusions in this paper we combine the ability of sequence models and pre-trained models and propose globallocal modeling method to do emotion inference in conversation. moreover, we take the whole dialogue as input and generate knowledge with prompt learning. experiments show that our model has achieved state-of-the-art performance on three datasets. ablation studies show the effectiveness of different modules in our model.