Towards Causal Explanation Detection with Pyramid Salient-Aware Network (2020)

Abstract: Causal explanation analysis (CEA) can assist us to understand the reasons behind daily events, which has been found very helpful for understanding the coherence of messages. In this paper, we focus on Causal Explanation Detection, an important subtask of causal explanation analysis, which determines whether a causal explanation exists in one message. We design a Pyramid Salient-Aware Network (PSAN) to detect causal explanations on messages. PSAN can assist in causal explanation detection via capturing the salient semantics of discourses contained in their keywords with a bottom graph-based word-level salient network. Furthermore, PSAN can modify the dominance of discourses via a top attention-based discourse-level salient network to enhance explanatory semantics of messages. The experiments on the commonly used dataset of CEA shows that the PSAN outperforms the state-of-the-art method by 1.8% F1 value on the Causal Explanation Detection task.

## introduction causal explanation detection (ced) aims to detect whether there is a causal explanation in a given message (e.g. a group of sentences). linguistically, there are coherence relations in messages which explain how the meaning of different textual units can combine to jointly build a discourse meaning for the larger unit. the explanation is an important relation of coherence which refers to the textual unit (e.g. discourse) in a message that expresses explanatory coherent semantics @xcite . as shown in figure 1 , m1 can be divided into three discourses, and d2 is the explanation that expresses the reason why it is advantageous for the equipment to operate at these temperatures. ced is important for tasks that require an understanding of textual expression @xcite . for example, for question answering, the answers of questions are most likely to be in a group of sentences that contains causal explanations @xcite . furthermore, the summarization of event descriptions can be improved by selecting causally motivated sentences @xcite . therefore, ced is a problem worthy of further study. the existing methods mostly regard this task as a classification problem @xcite . at present, there are mainly two kinds of methods, feature-based methods and neural-based methods, for similar semantic understanding tasks in discourse granularity, such as opinion sentiment classification and discourse parsing @xcite @xcite . the feature-based methods can extract the feature of the relation between discourses. however, these methods do not deal well with the implicit instances which lack explicit features. for ced, as shown in figure 1 , d2 lacks explicit features such as because of, due to, or the features of tenses, which are not friendly for feature-based methods. the methods based on neural network are mainly tree-lstm model @xcite and hierarchical bi-lstm model @xcite . the tree-lstm models learn the relations between words to capture the semantics of discourses more accurately but lack further understanding of the semantics between discourses. the hierarchical bi-lstm models can employ sequence structure to implicitly learn the relations between words and discourses. however, previous work shows that compared with tree-lstm, bi-lstm lacks a direct understanding of the dependency relations between words. therefore, the method of implicit learning of inter-word relations is not prominent in the tasks related c c l 2 0 2 0 the ability to operate at these temperature is advantageous, the devices need less thermal insulation. but the devices still have to be able to operate at different temperatures. m1 d1: d2: d3: causal explanation s advantageous (root) ability is operate the at to nsubj cop infmod det prep aux temperature pobj these det devices insulation thermal the less nsubj dobj nmod det advmod need (root) have (root) the at to det prep aux devices able nsubj xcomp to be operate aux cop xcomp temperatures pobj different amod syntactic structure of d1, d2 and d3 figure 1 : instance of causal explanation analysis (cea). the top part is a message which contains its segmented discourses and a causal explanation. the bottom part is the syntactic dependency structures of three discourses divided from m1. to understanding the semantic relations of messages @xcite . therefore, how to directly learn the relations between words effectively and consider discourse-level correlation to further filter the key information is a valuable point worth studying. further analysis, why do the relations between words imply the semantics of the message and its discourses? from the view of computational semantics, the meaning of a text is not only the meaning of words but also the relation, order, and aggregation of the words. in other simple words is that the meaning of a text is partially based on its syntactic structure @xcite . in detail, in ced, the core and subsidiary words of discourses contain their basic semantics. for example, as d1 shown in figure 1 , according to the word order in syntactic structure, we can capture the ability of temperature is advantageous. we can understand the basic semantic of d1 which expresses some kind of ability is advantageous via root words advantageous and its affiliated words. additionally, why the correlation and key information at the discourse level are so important to capture the causal explanatory semantics of the message? through observation, the different discourse has a different status for the explanatory semantics of a message. for example, in m1, combined with d1, d2 expresses the explanatory semantics of why the ability to work at these temperatures is advantageous, while d3 expresses the semantic of transition. in detail, d1 and d2 are the keys to the explanatory semantics of m1, and if not treated d1, d2, and d3 differently, the transitional semantic of d3 can affect the understanding of the explanatory semantic of m1. therefore, how to make better use of the information of keywords in the syntactic structure and pay more attention to the discourses that are key to explanatory semantics is a problem to be solved. to this end, we propose a pyramid salient-aware networks (psan) which utilizes keywords on the syntactic structure of each discourse and focuses on the key discourses that are critical to explanatory semantics to detect causal explanation of messages. first, what are the keywords in a syntactic structure? from the perspective of syntactic dependency, the root word is the central element that dominates other words, while it is not be dominated by any of the other words, all of which are subordinate to the root word @xcite . from that, the root and subsidiary words in the dependency structure are the keywords at the syntax level of each discourse. specifically, we sample 100 positive sentences from training data to illuminate whether the keywords obtained through the syntactic dependency contain the causal explanatory semantics. and we find that the causal explanatory semantics of more than 80% sentences be captured by keywords in dependency structure foot_0 . therefore, we extract the root word and its surrounding words on the syntactic dependency of each discourse as its keywords. next, we need to consider how to make better use of the information of keywords contained in the syntactic structure. to pay more attention to keywords, the common way is using attention mechanisms to increase the attention weight of them. however, this implicitly learned attention is not very interpretable. inspired by previous researches @xcite , we propose a bottom graph-based word-level salient network which merges the syntactic dependency to capture the salient semantics of discourses contained in their keywords. finally, how to consider the correlation at the discourse level and pay more attention to the discourses that are key to the explanatory semantics? inspired by previous work @xcite , we propose a top attention-based discourse-level salient network to focus on the key discourses in terms of explanatory semantics. in summary, the contributions of this paper are as follows: self-attention mechanism: self-attention has been introduced to machine translation by vaswani @xcite for capturing global dependencies between input and output and achieved stateof-the-art performance. for language understanding tasks, shen @xcite exploits self-attention to learn long-range dependencies. tan @xcite applies self-attention for semantic role labeling task and achieves state-of-the-art results. in this paper, we utilize a multi-head self-attention encoder to capture the representation of words. ## methodology the architecture of our proposed model is illustrated in figure 2 . in this paper, the pyramid salient-aware network (psan) primarily involves the following three components: (i) input processing module (ipm), which processes and encodes the input message and its discourses via self-attention module; (ii) bottom word-level salient-aware module (b-wsm), which captures the salient semantics of discourses contained in their keywords based on the syntactic-centric graph; (iii) top discourse-level salient-aware module (t-dsm), which modifies the dominance of different discourse based on the message-level constraint in terms of explanatory semantic via an attention mechanism, and obtain the final causal explanatory representation of input message m. ## conclusion in this paper, we devise a pyramid salient-aware network (psan) to detect causal explanations in messages. psan can effectively learn the key relation between words at the word level and further filter out the key information at the discourse level in terms of explanatory semantics. specifically, we propose a bottom word-level salient-aware module to capture the salient semantics of discourses contained in their keywords based on a the syntactic-centric graph. we also propose a top discourse-level salient-aware module to modify the dominance of different discourses in terms of global explanatory semantic constraint via an attention mechanism. experimental results on the open-accessed commonly used datasets show that our model achieves the best performance.