PWEITINLP at S em E val-2024 Task 3: Two Step Emotion Cause Analysis (2024)

Abstract: ECPE (emotion cause pair extraction) task was introduced to solve the shortcomings of ECE (emotion cause extraction). Models with sequential data processing abilities or complex architecture can be utilized to solve this task. Our contribution to solving Subtask 1: Textual Emotion-Cause Pair Extraction in Conversations defined in the SemEval-2024 Task 3: The Competition of Multimodal Emotion Cause Analysis in Conversations is to create a two-step solution to the ECPE task utilizing GPT-3 for emotion classification and SpanBERT for extracting the cause utterances.

## introduction this paper introduces an approach for the emotioncause extraction problem in dialogues. an emotion cause is defined and annotated in the given subtask as a textual span. input to the model is a conversation containing the speaker and the text of each utterance. the model output should include all emotion-cause pairs, where each pair contains an emotion utterance along with its emotion category and the textual cause span in a specific cause utterance, e.g.(u3_joy, u2_"you made up!"). our contribution to solving subtask 1: textual emotion-cause pair extraction in conversations defined in the semeval-2024 task 3: the competition of multimodal emotion cause analysis in conversations @xcite is as follows: i) utilize gpt-3 for emotion classification, ii) utilize spanbert architecture to extract the cause utterances in dialogues as q&a task, iii) contribute to solving the ecpe by finding its possible solutions in other nlp fields. the task was separated into two parts -emotion classification, called subtask 1.1, and emotioncause pair extraction, termed subtask 1.2. we have used gpt-3 and the spanbert model for these subtasks. in this paper, we also reflect on the results we got in the case of both subtasks, namely emotion classification and emotion-cause pair extraction. our model, with one test entry, scored 9th in the competition. 1 ## related work in recent years, many authors have suggested their approach to solving the ecpe task. @xcite defined the ecpe task and proposed a two-step framework. first, independent multitask learning (named indep) consisting of bilstm modules and interactive multi-task learning (called inter-ec for a model that uses emotion extraction to improve cause extraction and inter-ce for a model that uses cause extraction to enhance emotion extraction) was used to extract a set of emotion cases and a set of cause clauses. secondly, the sets were paired to yield a set of candidate emotion-cause pairs. finally, a logistic model regression was used to filter the pairs. this two-step framework suffers from error propagation from the first step to the second step. @xcite has also proposed a one-step framework that takes emotion-cause pairs as a 2d representation scheme with bilstm modules. these representations are forwarded into the 2d transformer framework to capture pair interaction. finally, binary classification is conducted to extract valid emotion-cause pairs. the new proposed framework outperforms the two-step framework by 7.6 percentage points of the f1 score. regarding the joint framework, @xcite have proposed a sliding window multi-label learning scheme named ecpe-mll. it works on the assumption that all clauses in a document are emotion clauses, and an emotion-oriented sliding window is built centered on each emotion clause. in each window, the emotion clause extracts one or more of the corresponding cause clauses (the iterative synchronized multi-task learning (isml) model is introduced to solve these subtasks). the results of this learning can be transformed into emotion-cause pairs. this approach serves an excellent advantage over the two-step framework proposed before. @xcite have proposed two alignment mechanisms with a model named a foot_1 n et. text documents are encoded with bert and a partition filter network (pfn) to implement the first alignment mechanism: feature-task alignment to produce emotion-specific, cause-specific, and interaction features. the features are applied for ee (emotion and interaction features), ce (cause and interaction features), and ecpe tasks (all features). the inter-task alignment reduces then the inconsistency between label spaces among all tasks. the proposed framework achieves a higher f1 score and recall in the ecpe task, a higher f1 score in the ee task, and a higher recall and f1 score in terms of the ce task when compared to ecpe-2d. ## methodology the emotion extraction cause task consists of two components -emotion extraction from the conversation and emotion cause span extraction. the first one could have been considered as a baseline for the second one, as we needed to identify which emotion and utterance should be used in the process of the cause search. there are two ways of approaching this problem. we could have created one model for both subtasks or separated it into two subsequent tasks, where each could be implemented using different models. we have decided to go with the second approach, as we concluded that those less complex parts could have better quality in the end, even though we are aware of the error propagation, which definitely will be present in such a case. ## experimental results this section presents the experiment results (before evaluation phase) in terms of both subtasks. we have checked the accuracy of the gpt-3 model by utilizing 400 randomly selected utterances. assistant, which was enhanced by adding a knowledge base, was using another randomly selected (but not similar to the ones in the test dataset) 500 records from the training dataset. during the testing phase, we calculated the predicted labels' f1-score, accuracy, and recall and created a confusion matrix. figures 1 , 2 and tables 2, 3 refer to the confusion matrix for the assistant with and without knowledge base accordingly. while analyzing the confusion matrix and values of the metrics for the test data, one can see that when considering the best gpt-3 model with the knowledge base and all bilstm models, the gpt-3 overperforms the bilstm model in all presented metrics. it became clear that if we want to use bilstm models for the classification tasks where context plays an important role, there should be more complex preprocessing techniques and feature extraction for both the model input and the attention layer (both utterances and context values) that would be solved by utilizing gpt-3 model. ## evaluation and conclusions for the evaluation phase, we have used the evaluation data provided by the organizers. the data was emotion-classified using gpt-3, and the data was suited for span extraction as in section 3.2. we have also tried to use bilstm in this case, however, its capabilities were very limited when processing data with unknown words and short sentences (the probability of each occurrence of emotion was nearly identical). the results from span-bert were answers to questions built upon classified emotions. obtained answers were added to the original evaluation dataset's utterances (called main utterances in the following text) based on the created by author unique id. answers also contained spans of text that could occur in different utterances, so the utterances that did not belong to the main utterance were placed in different lists (meaning multiple cause spans) in each matched main utterance. based on the main utterance answer and additional answers, emotion-cause pairs were created in a manner that the "emotion-cause_pairs" list contained emotion utterance along with its emotion category and a cause utterance id followed by position indexes of predicted cause span within the utterance. the position index starts from 0, and the ending index is the index of the last token plus 1 excluding the punctuation token at the beginning and end. the evaluation phase ended for 1 entry uploaded on the codalab @xcite submission as follows: w-avg. strict f1: 0.0449, w-avg. proportional f1: 0.0723, strict f1: 0.0462, proportional f1: 0.0717, resulting in 9th place out of 29 teams. the results showed that each of the presented subtasks, namely emotion classification and emotion-cause pair extraction, could perform better in terms of classifying emotions and extracting spans. the changes could improve the overall score by employing gpt-4 architecture and experimenting with span extraction model architecture as well.