Position Bias Mitigation: A Knowledge-Aware Graph Model for Emotion Cause Extraction (2021)

Abstract: The Emotion Cause Extraction (ECE) task aims to identify clauses which contain emotion-evoking information for a particular emotion expressed in text. We observe that a widely-used ECE dataset exhibits a bias that the majority of annotated cause clauses are either directly before their associated emotion clauses or are the emotion clauses themselves. Existing models for ECE tend to explore such relative position information and suffer from the dataset bias. To investigate the degree of reliance of existing ECE models on clause relative positions, we propose a novel strategy to generate adversarial examples in which the relative position information is no longer the indicative feature of cause clauses. We test the performance of existing models on such adversarial examples and observe a significant performance drop. To address the dataset bias, we propose a novel graph-based method to explicitly model the emotion triggering paths by leveraging the commonsense knowledge to enhance the semantic dependencies between a candidate clause and an emotion clause. Experimental results show that our proposed approach performs on par with the existing state-of-the-art methods on the original ECE dataset, and is more robust against adversarial attacks compared to existing models.

## introduction instead of detecting sentiment polarity from text, recent years have seen a surge of research activities that identify the cause of emotions expressed in text @xcite @xcite @xcite . in a typical dataset for emotion cause extract (ece) @xcite , a document consists of multiple clauses, one of which is the emotion clause annotated with a pre-defined emotion class label. in addition, one or more clauses are annotated as the cause clause(s) which expresses triggering factors leading to the emotion expressed in the emotion clause. an emotion extraction model trained on the dataset is expected to classify a given clause as a cause clause or not, given the emotion clause. figure 1 : the distribution of positions of cause clauses relative to their corresponding emotion clauses in the ece dataset @xcite . nearly 87% of cause clauses are located near the emotion clause (about 55% are immediately preceding the emotion clause, 24% are the emotion clauses themselves and over 7% are immediately after the emotion clause). however, due to the difficulty in data collection, the ece datasets were typically constructed by using emotion words as queries to retrieve relevant contexts as candidates for emotion cause annotation, which might lead to a strong positional bias @xcite . figure 1 depicts the distribution of positions of cause clauses relative to the emotion clause in the ece dataset @xcite . most cause clauses are either immediately preceding their corresponding emotion clauses or are the emotion clauses themselves. existing ece models tend to exploit such relative position information and have achieved good results on emotion cause detection. for example, the rel-ative position augmented with dynamic global labels (pae-dgl) @xcite , rnn-transformer hierarchical network (rthn) @xcite and multi-attention-based neural network (mann) @xcite all concatenate the relative position embeddings with clause semantic embeddings as the clause representations. we argue that models utilising clause relative positions would inherently suffer from the dataset bias, and therefore may not generalise well to unseen data when the cause clause is not in proximity to the emotion clause. for example, in a recently released emotion cause dataset, only 25-27% cause clauses are located immediately before the emotion clause @xcite . to investigate the degree of reliance of existing ece models on clause relative positions, we propose a novel strategy to generate adversarial examples in which the relative position information is no longer the indicative feature of cause clauses. we test the performance of existing models on such adversarial examples and observe a significant performance drop. to alleviate the position bias problem, we propose to leverage the commonsense knowledge to enhance the semantic dependencies between a candidate clause and the emotion clause. more concretely, we build a clause graph, whose node features are initialised by the clause representations, and has two types of edges i.e., sequence-edge (s-edge) and knowledge-edge (k-edge). a s-edge links two consecutive clauses to capture the clause neighbourhood information, while a k-edge links a candidate clause with the emotion clause if there exists a knowledge path extracted from the con-ceptnet @xcite between them. we extend relation-gcns @xcite to update the graph nodes by gathering information encoded in the two types of edges. finally, the cause clause is detected by performing node (i.e., clause) classification on the clause graph. in summary, our contributions are three-fold: the presented work is closely related to two lines of research in emotion cause extraction: positioninsensitive and position-aware models. position-insensitive models. a more traditional line of research exploited structural representations of textual units relying on rule-based systems (lee @xcite or incorporated commonsense knowledge bases @xcite for emotion cause extraction. machine learning methods leveraged text features @xcite and combined them with multi-kernel support vector machine (svm) @xcite . more recent works developed neural architectures to generate effective semantic features. cheng et al. (2017b) employed lstm models, gui et al. (2017) made use of memory networks, while li et al. (2018) devised a convolutional neural network (cnn) with a co-attention mechanism. (chen et al., 2018) used the emotion classification task to enhance cause extraction results. position-aware models. more recent methodologies have started to explicitly leverage the positions of cause clauses with respect to the emotion clause. a common strategy is to concatenate the clause relative position embedding with the candidate clause representation (ding et al., 2019; xia et al., 2019; li et al., 2019). the relative position augmented with dynamic global labels (pae-dgl) (ding et al., 2019) reordered clauses based on their distances from the target emotion clause, and propagated the information of surrounding clauses to the others. xu et al. (2019) used emotion dependent and independent features to rank clauses and identify the cause. the rnn-transformer hierarchical network (rthn) (xia et al., 2019) argued there exist relations between clauses in a document and proposed to classify multiple clauses simultaneously. li et al. (2019) proposed a multi-attention-based neural network (mann) to model the interactions between a candidate clause and the emotion clause. bi-lstm bi-lstm . . . transformer 𝑝 1 . . . 𝑝 2 bi-lstm 𝑠 1 𝛼 1 𝛼 𝐾 s-edge k-edge 𝐶 ) 1 𝐶 ) 𝐸 𝐶 ) 8 . . . 𝐶 ) 2 𝐶 ) 6 c1 c8 c5 p 1 p 2 bi-lstm 𝛼 2 𝐷 p 1 p 2 e 15 𝐶 ) 𝐸 conceptnet ℎ / ℎ 0 softmax 𝑦 2 𝟏 document (b) path representations. (a) document encoding. (c) clause graph update. (d) classification. . . . . . . . . . 𝐶 4 bi-lstm figure 2: the framework of our proposed kag. given an input document consisting of eight clauses (c1 • • • c8), we first extract knowledge paths from conceptnet between each candidate clause and the emotion clause ( §3.1), e.g., two knowledge paths, p 1 and p 2 , are extracted between c 1 and the emotion clause c 5 . (a) document encoding. clauses are fed into a word-level bi-lstm and a clause-level transformer to obtain the clause representations ĉi . the document embedding d is generated by dot-attention between the emotion embedding ĉe and clause embeddings. (b) path representations. the extracted knowledge paths are fed into bi-lstm to derive path representations. multiple paths between a clause pair are aggregated into s i based on their attention to the document representation d. (c) clause graph update. a clause graph is built with the clause representations ĉi used to initialise the graph nodes. the k-edge weight e ie between a candidate clause ĉi and the emotion clause ĉe are measured by their distance along their path s i . (d) classification. node representation h i of a candidate clause c i is concatenated with the emotion node representation h e , and then fed to a softmax layer to yield the clause classification result ŷi . the generated representations are fed to a cnn layer for emotion cause extraction. the hierarchical neural network @xcite aimed at narrowing the gap between the prediction distribution p and the true distribution of the cause clause relative positions. ## knowledge-aware graph (kag) model for emotion cause extraction we first define the emotion cause extraction (ece) task here. a document d contains n clauses @xmath0, one of which is annotated as an emotion clause c e with a pre-defined emotion class label, e w . the ece task is to identify one or more cause clauses, c t , 1 ≤ @xmath1, that trigger the emotion expressed in c e . note that the emotion clause itself can be a cause clause. we propose a knowledge-aware graph (kag) model as shown in figure 2 , which incorporates knowledge paths extracted from conceptnet for emotion cause extraction. more concretely, for each document, a graph is first constructed by representing each clause in the document as a node.