A Unified Sequence Labeling Model for Emotion Cause Pair Extraction (2020)

Abstract: Emotion-cause pair extraction (ECPE) aims at extracting emotions and causes as pairs from documents, where each pair contains an emotion clause and a set of cause clauses. Existing approaches address the task by first extracting emotion and cause clauses via two binary classifiers separately, and then training another binary classifier to pair them up. However, the extracted emotion-cause pairs of different emotion types cannot be distinguished from each other through simple binary classifiers, which limits the applicability of the existing approaches. Moreover, such two-step approaches may suffer from possible cascading errors. In this paper, to address the first problem, we assign emotion type labels to emotion and cause clauses so that emotion-cause pairs of different emotion types can be easily distinguished. As for the second problem, we reformulate the ECPE task as a unified sequence labeling task, which can extract multiple emotion-cause pairs in an end-to-end fashion. We propose an approach composed of a convolution neural network for encoding neighboring information and two Bidirectional Long-Short Term Memory networks for two auxiliary tasks. Experiment results demonstrate the feasibility and effectiveness of our approaches.

## introduction recently, causal relationships between human emotions and their corresponding causes have received a lot of attention. recognizing the causes of a specific emotion in a document is considered as more useful than only identifying the emotions, due to the great potential of helping people make reasonable decisions and avoid unnecessary loss @xcite @xcite . the first task concerning causal relationships between emotions and causes is the emotion cause extraction (ece) task proposed by @xcite , which aims to extract the causes for the annotated emotion in a document. to avoid the limitation that emotions must be annotated before cause extraction in the ece task, @xcite proposed the emotion-cause pair extraction (ecpe) task to extract emotions and causes as pairs from documents without the emotion annotations, where the extracted pairs are called "emotion-cause pairs (ecps)". to address the ecpe task, they proposed a two-step framework, where emotions and causes are first extracted separately by two binary classifiers, and then another binary classifier is trained to pair them up. however, such two-step approaches may suffer from two major shortcomings as follows. firstly, binary classifiers cannot distinguish ecps of differernt emotion types, which limits the applicability of the proposed approaches since the extracted ecps may not be directly usable in real-world applications, such as discovery and analysis of the possible causes for a specific emotion type. to further see this, let us consider a document with two ecps shown in taking these two problems into consideration, the main challenge is to extract multiple ecps of different emotion types in an end-to-end fashion. in this paper, we reformulate the ecpe task as a sequence labeling task, which can assign different labels to different clauses and extract multiple ecps of different emotion types simultaneously by simply pairing up clauses with matching labels. to pair up emotion clauses and cause clauses, we design a unified label for each clause, which is composed of two types of labels: a causal identity label denoting the clause being a cause or emotion clause (e.g., cause(c), emotion(e)), and an emotion type label denoting the related emotion type of the clause (e.g., happiness(h), sadness(sa)). with such a formulation, we devise our end-to-end sequence labeling approach based on a stacked neural network framework, namely ie-cnn, where the upper network predicts the unified labels and the lower networks perform auxiliary prediction tasks. specifically, since our unified labels consist of causal identity labels and emotion type labels, two bidirectional long-short term memory (bilstm) networks are employed in the lower level to separately predict these two types of labels for each clause, and obtain feature vectors with causal identity information and emotion type information for the subsequent unified label prediction. then, based on the observations made by @xcite that most emotion clauses and their corresponding cause clauses appear near each other, the resultant feature vectors of the two bilstms are concatenated and fed to a convolutional neural network (cnn) in the upper level to encode the neighboring information. experiment results show that our approach significantly outperforms the existing baseline models. the main contributions of this work can be summarized as follows: ## problem formulation in this section, we formally redefine the ecpe task. in the benchmark ecpe dataset, the following observations form the basis of our task reformulation. firstly, there are possibly multiple ecps in a document, and each emotion clause can be paired up with multiple cause clauses. secondly, a clause can be both a cause clause and an emotion clause, and each clause will only be associated with at most one emotion type, which means that there will not be multiple emotion clauses of the same emotion type, or a single clause with multiple emotion types in one document. therefore, formulating the ecpe task as a sequence labeling task is a suitable choice since ecps of different emotion types can be extracted simultaneously from the final label sequence. we design a set of unified labels to pair up emotion and cause clauses. specifically, except "o" which denotes that a clause is neither a cause clause nor an emotion clause, each unified label contains a causal identity label and an emotion type label. the set of causal identity labels is denoted by: y @xmath0, cause(c), emotion(e), both-cause&emotion(b)}, and the set of emotion type labels is denoted by: y @xmath1, happiness(h), sadness(sa), anger(a), disgust(d), surprise(su), fear(f)}. with these two sets of labels, the unified label set is constructed as: y @xmath2, c-h, c-sa, c-a, c-d, c-su, c-f, e-h, e-sa, e-a, e-d, e-su, e-f, b-h, b-sa, b-a, b-d, b-su, b-f}. using these unified labels, emotion and cause clauses of the same emotion type can be easily paired up. for example, c-h denotes the clause being a cause clause of emotion happiness, e-sa denotes the clause being an emotion clause of sadness, while b-a denotes the clause being both cause and emotion clauses of anger. these specially-designed unified labels enable the proposed model to handle documents with multiple ecps of different emotion types as desired. thus, ecpe can be redefined as: given a clause sequence @xmath3, ..., x t } with t clauses, predict a label sequence y @xmath4, ..., y u t }, where y u @xmath5. ## approach as shown in figure 1 , our proposed ie-cnn is based on a stacked neural network framework, where the upper network can utilize the useful information delivered from the lower networks. since our unified labels contain two parts, namely, causal identity labels and emotion type labels, we first employ two bilstm networks, bilstm i and bilstm e , in the lower level to predict these two types of labels for each clause, respectively. then, the hidden representations of two bilstm networks are concatenated to construct feature vectors with both causal identity and emotion type information, which are further fed to the cnn layer in the upper level to encode the neighboring information. in the following, we introduce our proposed framework in detail. ## experiments in this section, we present our experimental studies to evaluate the feasibility and effectiveness of our proposed approach. ## related work recently, there have been several works focusing on the ecpe task, aiming to propose end-to-end approaches to avoid the possible cascading errors brought by the two-step manner in the three approaches proposed by @xcite . @xcite proposed a rankcp model which makes use of the graph attention network to propagates information among clauses and ranks the candidate ecps based on the learned pair representations to get the prediction results. tang et al. (2020) 2020 ) treated the ecpe task as a link prediction problem and proposed e2eecpe to address the ecpe task, which utilizes a biaffine attention module to model the relationships between any two clauses. the ecpe-2d model proposed by @xcite deployed a hierarchical self-attention module to calculate the biaffine attention matrix among all clauses and a 2d transformer to model the interaction among ecps. all of the above works are constructed based on calculating the attention value between any two clauses of the given document. @xcite proposed to address the ecpe task from a different angle by modeling the extraction of the ecp as performing a sequence of transitions and actions. specifically, they constructed a transitional directed graph for each given document and transformed the original dataset into sequences of transitions and actions, based on which they trained a model to predict the next state of the sequence given the current state and the predicted transition. different from the above works, we reformulate the ecpe task as a unified sequence labeling problem and design our unified labels based on two sets of labels: causal identity labels and emotion type labels. such a formulation can help us address the two problems analyzed in section 1 (i.e., uninformative extraction results and possible cascading error of the two-step approaches), and enable our model to extract multiple ecps of different emotion types simultaneously. a conventional solution for sequence labeling with two sets of labels is to train, jointly, two models to predict these two sets of labels separately @xcite . however, using such approaches, the two sets of labels may not match correctly, resulting in meaningless labels. for example, a clause can be assigned with "cause" as its causal identity label, and "outside" as its emotion type label which means the clause is not related to any emotion. in this case, the predicted causal identity label and emotion type label cannot be combined to construct a meaningful unified label. therefore, a better solution is directly predicting unified labels so as to avoid the matching error @xcite . in this regard, many existing frameworks (e.g., a stacked recurrent neural network framework proposed by @xcite ) have been proposed and are applicable to address the ecpe task. in our approach introduced in section 3, we adopt a stacked neural network framework containing two bilstms and a cnn, given that our unified labels contain two parts and encoding neighboring information may improve the task performance. ## conclusion in this paper, we have investigated the ecpe task which aims to extract emotion-cause pairs from documents. to tackle the limitation that the existing two-step approaches cannot distinguish emotion-cause pairs of different emotion types, our proposed approach assigns emotion type labels to both emotion and cause clauses. besides, we have reformulated the ecpe task as a sequence labeling task, and proposed a unified sequence labeling approach to extract multiple emotion-cause pairs in an end-to-end fashion. this allows us to handle documents with multiple types of emotions and avoid cascading errors compared to the existing baselines. based on stacked neural networks, our proposed approach consists of two bilstm networks in the lower level to separately predict causal identity label and emotion type label, and a cnn layer in the upper level to encode the neighboring information. experiment results show that our approach achieves better performance in pair extraction compared with the existing baseline without bert, and we believe that adding bert in our proposed model can further improve the prediction performance. furthermore, ablation study well reveals the effectiveness of our devised framework and individual components.