P et K az at S em E val-2024 Task 3: Advancing Emotion Classification with an LLM for Emotion-Cause Pair Extraction in Conversations (2024)

Abstract: In this paper, we present our submission to the SemEval-2023 Task 3 “The Competition of Multimodal Emotion Cause Analysis in Conversations”, focusing on extracting emotion-cause pairs from dialogs. Specifically, our approach relies on combining fine-tuned GPT-3.5 for emotion classification and using a BiLSTM-based neural network to detect causes. We score 2nd in the ranking for Subtask 1, demonstrating the effectiveness of our approach through one of the highest weighted-average proportional F1 scores recorded at 0.264.

## introduction developing dialog systems is a complex task that has attracted considerable attention from many technology companies and universities over the last 70 years since the introduction of eliza in 1966 @xcite . modern large language models (llms) like @xcite are trained to avoid causing harm and often assert their lack of personal opinions on intricate matters, which is not at all natural for conversations. they do not respond in a way that is truly human, and they do not understand the range of feelings that words can cause. recognizing the emotional implications of an utterance provides a deeper understanding of dialog, enabling the development of more human-like dialog systems. these systems could navigate conversations using a comprehensive understanding of emotional dynamics and planning responses based on this understanding rather than just predicting likely outcomes. to bridge the gap between machine-generated dialogs and rich, complex human communication, we develop models for semeval-2024 task 3 "the competition of multimodal emotion cause analysis in conversations" 1 (ecac) @xcite . we focus only on subtask 1, "textual emotion-cause pair extraction in conversations" (ecpe), foot_0 where the goal is to classify emotions and extract the corresponding textual causal spans. to accomplish this, we propose a two-stage pipeline: (1) first, emotions are classified using a fine-tuned llm, and then (2) causes are extracted with a simple neural network consisting of bilstm and linear layers (see figure 1 ). our system achieved a weightedaverage proportional f 1 score of 0.264, the primary metric in this competition's evaluation phase on the test set. consequently, our team ranked 2nd out of 15 participating teams based on this metric. we provide an extensive analysis of the model's performance in section 5.2. ## related work recent research in the field of dialog systems and emotion-cause extraction has seen significant advancements through various innovative approaches, some of which we overview in this section. for instance, @xcite introduce a novel technique that uses graphs to model "causal skeletons" alongside a causal autoencoder (cae) for refining these models by integrating both implicit and explicit causes. following closely, @xcite present dual graph attention networks (dualgats) that leverage discourse structure and speaker context through a combination of discourse-aware gat (disgat) and speaker-aware gat (spkgat), enriched with an interaction module for effective information exchange and context capturing. rachel oh that is what you want. monica yes. rachel fine! monica fine! phoebe there we go. you know what, if we were in prison, you guys would be, like, my *******. llm : anger : anger one-hot : 001 : 001 : 010 : pre-trained sentence embeddings speaker vectors emotion labels ... ... : bilstm : feed-forward network utterances embeddings figure 1: the pipeline for ecpe. utterances are classified with emotion labels e i , and speakers are represented with one-hot vectors s i . utterances are then encoded with pre-trained sentence embeddings u i and enriched with context by bilstm ūi . for each target utterance u t , we detect whether any other utterance from the conversation history h(u t ) is causal using a feed-forward network. ūi , s i (of a potential causal utterance), ūt , s t , and e t are concatenated, and then, binary classification is performed. the pipeline outputs labelled emotion-cause pairs (u i , u t , e t ). finally, @xcite focus on improving dialog systems' empathetic response generation by identifying emotion causes. their framework combines an emotion reasoner for predicting emotion and its cause with a response generator that employs a gated attention mechanism to emphasize important words, exploring both hard and soft gating strategies. ## system overview our pipeline consists of two stages. specifically, to identify emotion-cause pairs and emotion types, dialogs are passed through the following modules: the full pipeline is shown in figure 1 . due to the limitations of the data, we perform the tasks separately, and we elaborate on each of the stages in the following sections. ## results our final submission was evaluated on the test set and achieved the following results: as a result, we score second out of fifteen teams participating in subtask 1 according to the main shared task metric -w-avg. proportional f 1 . ## conclusions our work presents a novel approach to emotioncause pair extraction in conversations, using the capabilities of an llm (specifically, gpt-3.5) for emotion classification. this methodology is further enhanced by the use of a bilstm-based neural network for extracting causes. our system outperforms most of the submissions to the shared task, scoring 2nd in the overall ranking according to the main metric of weighted-average proportional f 1 . for future enhancements to our pipeline, we consider the following improvements: