Bidirectional Hierarchical Attention Networks based on Document-level Context for Emotion Cause Extraction (2021)

Abstract: Emotion cause extraction (ECE) aims to extract the causes behind the certain emotion in text. Some works related to the ECE task have been published and attracted lots of attention in recent years. However, these methods neglect two major issues: 1) pay few attentions to the effect of document-level context information on ECE, and 2) lack of sufficient exploration for how to effectively use the annotated emotion clause. For the first issue, we propose a bidirectional hierarchical attention network (BHA) corresponding to the specified candidate cause clause to capture the document-level context in a structured and dynamic manner. For the second issue, we design an emotional filtering module (EF) for each layer of the graph attention network, which calculates a gate score based on the emotion clause to filter the irrelevant information. Combining the BHA and EF, the EF-BHA can dynamically aggregate the contextual information from two directions and filters irrelevant information. The experimental results demonstrate that EF-BHA achieves the competitive performances on two public datasets in different languages (Chinese and English). Moreover, we quantify the effect of context on emotion cause extraction and provide the visualization of the interactions between candidate cause clauses and contexts.

## introduction in recent years, emotion cause extraction (ece), which aims to identify the causes with respect to certain emotion in the text @xcite @xcite , has obtained increasing attention in academics and industry. mining the causes of certain emotion has a wide range of applications, such as public opinion monitoring and product review mining @xcite @xcite @xcite . the goal of ece is to find out the cause clause (e.g., c 3 ) that contains the emotion cause for the given emotion clause (e.g., c 4 ), as presented in example 1. example 1 (c 1 ) wu was diagnosed with advanced liver cancer at the beginning of 2014 (c 2 ) since he began to update his health condition in microblog (c 3 ) if wu didn't update his microblog for a long time (c 4 ) people worried that he may have passed away we divide the existing works related to ece into rule-based methods, traditional machine learning algorithms and deep learning methods. the rule-based methods are dependent on linguistic rules and common-sense knowledge, which requires plenty of manual operations @xcite . the traditional machine learning algorithms generally rely on feature engineering to manually select the features as the inputs of model @xcite . recently, a number of works adopted deep neural networks like self-attention @xcite , coattention @xcite , hierarchical attention @xcite to capture the relations among clauses. some works utilized the multi-task learning @xcite @xcite to extract emotion cause clauses. for example 1, if the contextual clauses (e.g., c 1 and c 2 ) are ignored, there may be no direct causal relationship between the emotion clause (e.g., c 4 ) and the cause clause (e.g., c 3 ), since not updating one's social media account will not cause worried. in fact, context has been utilized in many causal relation tasks to provide semantic information and improve the model performance @xcite @xcite . @xcite has mentioned that the causal relationship between the emotion and cause clauses may only be valid in a specific context. however, few works related to emo-tion cause extraction consider the document-level context information. to determine whether a clause is the cause of a certain emotion clause, it actually requires to understand the entire document. in this paper, we take each clause in the document as the candidate cause clause of the given emotion, and propose a bidirectional hierarchical attention networks (bha) to capture the document-level context for a specific candidate cause clause in a structured and dynamic manner. the "bidirectional" denotes the document-level context is divided into forward and backward context based on the position of current candidate cause clause in document, and "hierarchical" denotes that we use hierarchical attention networks to selectively focus on the context information related to the current candidate cause clause at word and clause levels. in contrast to the hierarchical attention networks @xcite and self attention model @xcite , the bha allows dynamically access to the context from two directions and distinguishes the effects of forward and backward context on candidate cause clause. moreover, the previous works are generally limited to calculate the relative position of candidate cause clause to the emotion clause for leveraging emotion clause @xcite , ignoring the role of emotion clause in filtering the irrelevant information. so we design an emotional filtering module (ef), which computes a gate score for each layer of graph attention networks (gat) based on the emotion clause to filter the irrelevant information. combining the bha and ef, we propose a bidirectional hierarchical attention networks (ef-bha) with emotional filtering to appropriately encode contextual features into the clause representation for emotion cause extraction. the main contributions of this paper are summarized as follows: recently, a number of works adopted deep learning networks to solve ece task. based on the original memory network @xcite , @xcite proposed to store local context of each word in different memory slots to extract emotion cause clause. @xcite captured the interactions between emotion classification and cause detection in an end-to-end fashion. @xcite built the co-attention relationship between emotion clause and each candidate clause with emotional context. @xcite viewed ece as a reordered prediction problem and incorporated the dynamic global labels into the model. @xcite firstly employed transformer @xcite to encode the global level information on all the clauses rather than relying solely on the hidden state of one clause. @xcite proposed a graph convolutional structure with fusion of semantics and structural constricts (fss-gcns) to automatically learned how to selectively attend the relevant clauses useful for emotion cause extraction. however, we notice that these methods related to ece task ignore two major issues: 1) pay few attentions to the effects of document-level context on ece task. 2) lack of sufficient exploration about how to effectively use the annotated emotion clause. in this paper, we try to incorporate the documentlevel context into the ece task and use the emotion clause to filter the irrelevant contextual information simultaneously. ## conclusions and future work in this work, we propose the ef-bha to model the relations between candidate cause clauses and the contexts. especially, ef-bha extracts the relevant contextual information according to the candidate cause clause, and then summarize contextual information with different granularity into clause representation. moreover, we propose to add an emotional filtering module for each layer of gats to filter the irrelevant information. the experimental results on two public datasets demonstrate that the proposed ef-bha achieves competitive performance in comparison with the existing methods, thereby validating the effectiveness. we further visualize the attention weight extracted by the bidirectional hierarchical context attention module, aiming to provide the visualization of the interactions between candidate cause clauses and contexts. ef-bha may introduce some irrelevant information when integrating context information. in future work, we will only use the previous |w| clauses and the following |w| clauses of the candidate cause clause as the context to alleviate this problem.