ERGO : Event Relational Graph Transformer for Document-level Event Causality Identification (2022)

Abstract: Document-level Event Causality Identification (DECI) aims to identify event-event causal relations in a document. Existing works usually build an event graph for global reasoning across multiple sentences. However, the edges between events have to be carefully designed through heuristic rules or external tools. In this paper, we propose a novel Event Relational Graph TransfOrmer (ERGO) framework for DECI, to ease the graph construction and improve it over the noisy edge issue. Different from conventional event graphs, we define a pair of events as a node and build a complete event relational graph without any prior knowledge or tools. This naturally formulates DECI as a node classification problem, and thus we capture the causation transitivity among event pairs via a graph transformer. Furthermore, we design a criss-cross constraint and an adaptive focal loss for the imbalanced classification, to alleviate the issues of false positives and false negatives. Extensive experiments on two benchmark datasets show that ERGO greatly outperforms previous state-of-the-art (SOTA) methods (12.8% F1 gains on average).

## introduction event causality identification (eci) is the task of identifying if the occurrence of one event causes another in text. as shown in figure 1 , given the text "... the outage 2 was caused by a terrestrial break in the fiber in egypt ...", where "outage 2 " and "break" are event triggers, an eci model should predict if they have a causal relation or not. discovering causal relationships not only helps to deeply understand how the world progresses, but also is an important goal of empirical research in various areas, such as machine reading comprehension @xcite , question answering @xcite , future event forecasting @xcite , and event knowledge graph construction @xcite . causality is usually implicit in natural language @xcite , especially when events scatter in a document, a.k.a. document-level eci (deci). recent methods typically construct an event graph to assist the global inference across multiple sentences, where nodes are events and edges are their relations, such as linguistic dependency or adjacent contexts @xcite . however, there are two major issues. first, the edges heavily rely on external tools and heuristic rules, which are not always reliable and may introduce noise @xcite . second, it is like a chicken-egg problem -to identify (causal) relations between events, you need to extract their relations to build the graph first. in this paper, we propose a novel event relational graph transformer (ergo) framework for deci, which doesn't require any external tools and can effectively alleviate the noise issue. different from conventional event graphs, the basic idea is to build an event relational graph that naturally converts eci into a node classification problem, where each node denotes a pair of events, and all edges among nodes are initialized to capture potential causal chains, following the assumption "preserving transitivity of causation" @xcite . if event a causes event b and event b causes event c, then we have event a causes event c. that is, if the node of (a, c) receives positive predictions (i.e., causal relation) from nodes (a, b) and (b, c), it is positive, too. by contrast, if either/both of nodes (a, b) and (b, c) are negative (i.e., no relation), it is not necessary that (a, c) is negative, either. to this end, we leverage a graph transformer to model the graph and assign a lower weight to such uninformative edges, paying more attention to other paths or its own textual contexts. although the proposed graph directly models all event pairs for causality identification, it poses a great challenge of false positive and false negative issues. first, most of the event pairs have no causal relations. that is, negative nodes are dominant, and the imbalanced classification will easily confuse the model into false-negative predictions. second, to ease the graph construction, we assume that all nodes can pass information with each other via the complete graph structure. while there are many spurious correlations between events, which can be incorrectly propagated to neighbor nodes, leading to severe false positives. for example, "treatment" and "death" frequently co-occur in the same document, but there is no causality between them since it is not "treatment" that causes "death". to address the above issues, we further design a criss-cross constraint and an adaptive focal loss. the criss-cross constraint simplifies the paths of each pair of events for global inference. instead of a complete graph, we assume that there is an edge between two nodes, only if the two pairs of events share at least one event. clearly, if they have no common event, there must be no direct causal effect between them. the adaptive focal loss re-weights positive and negative samples to tackle the imbalance issue. on the one hand, we leverage a weighting factor to balance two classes' training. on the other hand, we also introduce a scaling factor to focus more on difficult samples. our contributions can be summarized as follows: ## related work eci has attracted much attention in recent years. in terms of text corpus, there are mainly two types of methods: sentence-level eci (seci) and deci. in the first research line, early methods usually design various features tailored for causal expressions, such as lexical and syntactic patterns @xcite , causality cues or markers @xcite @xcite , statistical information @xcite , and temporal patterns @xcite . then, researchers resort to a large amount of labeled data to mitigate the efforts of feature engineering and to learn diverse causal expressions @xcite . to alleviate the annotation cost, recent methods leverage pre-trained language models (plms, e.g., bert @xcite ) for the eci task and have achieved sota performance @xcite @xcite . to deal with implicit causal relations, @xcite incorporate the external knowledge from conceptnet @xcite for reasoning, which achieves promising results. @xcite learn contextspecific causal patterns from external causal statements and incorporate them into a target eci model. @xcite propose a data augmentation method to further solve the data lacking problem. along with the success of sentence-level natural language understanding, many tasks are extended to the entire document, such as relation extraction @xcite , natural language inference @xcite , and event argument extraction @xcite . a concurrent and relevant work is @xcite , which also leverages focal loss for entity relation extraction. the difference is that the focal loss in @xcite is used to make long-tail (positive) classes contribute more to the overall loss, while the focal loss in our ergo tackles the imbalance issue of deci task by focusing more on difficult samples. we further leverage a weighting factor in the focal loss to balance two classes' training, which is not considered in @xcite . moreover, in section 4.6, we have given a more detailed analysis of the impact of adaptive focal loss on the deci task. compared with seci, deci not only aggravates the lack of clear causal indicators but also poses a new challenge of cross-sentence inference. @xcite use integer linear programming (ilp) to model the global causal structures; @xcite proposes a document-level contextbased graph inference mechanism to capture interaction among events; richgcn @xcite constructs document-level interaction graphs and uses graph convolutional network (gcn, @xcite ) to capture relevant connections. however, the construction of the aforementioned global structure or graph requires sophisticated feature extraction or tools, which may introduce noise and mislead the model @xcite . compared with them, we formulate deci as an efficient node classification framework, which could capture the global interactions among event pairs automatically, as well as alleviate the imbalanced and noisy issues. ## methodology the goal of our proposed framework ergo is to capture potential causal chains for documentlevel reasoning. there are three main components: ## conclusion in this paper, we regard deci as a node classification task by constructing an event relational graph. we propose a novel event relational graph transformer (ergo) framework that could capture potential causal chains and mitigate the false positive and false negative issues for deci. extensive experiments show great improvements of ergo under both intra-and inter-sentence settings on two widely used benchmarks. further analysis provides insights into our approach and the deci task. in the future, we will consider introducing commonsense reasoning and auxiliary tasks to discover more reliable causality.