Graph Convolutional Networks for Event Causality Identification with Rich Document-level Structures (2021)

Abstract: We study the problem of Event Causality Identification (ECI) to detect causal relation between event mention pairs in text. Although deep learning models have recently shown state-of-the-art performance for ECI, they are limited to the intra-sentence setting where event mention pairs are presented in the same sentences. This work addresses this issue by developing a novel deep learning model for document-level ECI (DECI) to accept inter-sentence event mention pairs. As such, we propose a graph-based model that constructs interaction graphs to capture relevant connections between important objects for DECI in input documents. Such interaction graphs are then consumed by graph convolutional networks to learn document context-augmented representations for causality prediction between events. Various information sources are introduced to enrich the interaction graphs for DECI, featuring discourse, syntax, and semantic information. Our extensive experiments show that the proposed model achieves state-of-the-art performance on two benchmark datasets.

## introduction event causality identification (eci) is an important problem in information extraction that seeks to predict causal relation between a pair of events mentioned in text. for instance, in the sentence "the building was nearly destroyed by a fire early tuesday morning.", an eci system should be able to recognize the causal relation between the two events triggered by "destroyed" and "fire" (called event mentions), i.e., "fire" cause ---→ "destroyed". eci finds its applications for a wide range of problems in natural language processing (nlp), including machine reading comprehension @xcite , future event forecasting @xcite , and why-question answering @xcite . the early approach for eci has involved featurebased methods @xcite @xcite … nugroho told agence-france presse : "we have recorded 24 people dead and 249 people injured", adding that more than 300 buildings have been damaged 1 due to the quake 1. rescuers and other assistance teams have arrived in bener meriah, while the air force have dispatched a helicopter and a cn-235 aircraft to the region, nugroho said. "we are now concentrating on searching for people who may be trapped under the rubble," said rusli m . saleh, the deputy district chief of bener meriah. he said at least 25 of the injured in his district were hospitalized in intensive care. as the quake hit, villagers in the area ran out of their homes in panic and screamed for help. "i see many houses were damaged 2 and their roofs fell onto some people," bensu elianita , a 22-year-old resident of bukit sama village in central aceh district, said shortly after the quake 2 hit. … causal relation coreference figure 1 : an example for document-level eci. @xcite while the recent approach has examined deep learning methods to deliver state-of-the-art performance for this task @xcite . despite the good performance, the existing deep learning methods for eci are limited in that they only model the context at the sentence level, assuming the event mention pairs of interest to be in the same sentences (i.e., intra-sentence setting). on the one hand, this assumption fails to cover the inter-sentence scenario where the input pairs of event mentions can appear in different sentences in the documents, e.g., in the recent eventstoryline dataset for eci @xcite . on the other hand, the sole modeling of sentence context cannot benefit from the document-level information that can provide useful evidence to facilitate the causality prediction for events. an example can be seen in figure 1 where the interested pair of event mentions involves damaged 2 and quake 2 in the last (green) sentence. a system that only considers sentence context might find it challenging to predict causal relation in this case due to the long distance and the appearance of many irrelevant words between damaged 2 and quake 2 in the sentence. however, if a system relies on documentlevel information and recognizes the coreference of the event mention pairs (damaged 2 , damaged 1 ) and (quake 2 , quake 1 ), it can exploit the clear ev-idence of "damaged 1 due to the quake 1 " to infer the causal relation for damaged 2 and quake 2 . to fill this gap, this work aims to develop a deep learning model for document-level eci (deci) where input event mentions can reside in different sentences of an input document. as such, a major challenge in modeling document-level context with deep learning involves capturing necessary interactions/connections between relevant objects for eci. for instance, in our example in figure 1 , relevant objects include the event mentions and the important context words (i.e., "due to) while necessary connections involve event coreference and interactions of event mentions with context words (i.e., between damaged 1 , quake 1 , and "due to"). motivated by this intuition, we design the graph-based model for deci where interaction graphs over relevant objects for documents are explicitly generated and consumed by graph convolutional networks (gcn) @xcite to induce representation vectors for prediction. to our knowledge, this is the first work that employs interaction graphs for documents and gcns for eci. how can interaction graphs for documents (i.e., nodes and edges) be formed to learn effective representation vectors for eci? first, the intuitive approach to design nodes for interaction graphs is to leverage relevant objects for eci in documents. accordingly, we employ all the words, event mentions and entity mentions in a document to establish nodes for its interaction graph. here, we note that entity mentions (e.g., names, pronouns, nominals) might also be helpful for eci as entity mentions can serve as arguments (participants) of events and events with the same arguments might have better chance to involve in the causal relation. second, for edges of interaction graphs, we propose to exploit different knowledge sources or information types to create different types of connections for the graph nodes. such connection types are then combined to produce a single rich interaction graph for an input document for representation learning in eci. in particular, we focus on three major types of information for node connections for eci in this work, i.e., discourse-based, syntax-based, and semantic-based information. as such, the discourse-based information explores the sentence boundary and coreference of entity/event mentions in documents to link the nodes in interaction graphs (motivated by our example in figure 1 ). the syntax-based information connects words based on their syntactic relations in dependency trees of sentences, suggested by the use of shortest dependency paths between event mentions as features for eci in prior work @xcite . in contrast, the intuition for semantic-based information is that semantically related words/entity/event mentions in documents can also provide useful evidences to infer the causal relation for events. for instance, consider the following sentence: here, the causal relation between "attack" and "condemnation" can be easily predicted due to the direct evidence in the context (i.e., via "provoked"). however, the more complicated and implicit context between "bombings" and "condemnation" would make it more difficult for eci systems to realize the causality in this case. fortunately, the systems can combine the causal relation between "attack" and "condemnation" and the close semantic similarity between the two events "bombings" and "attack" to facilitate the causality prediction between "bombings" and "condemnation". finally, we propose a novel mechanism to regularize interaction graphs and representation vectors to further improve the representation learning for deci. as such, we aim to constrain the model so edges with small weights in the generated graphs have minimized contribution to representation vectors. in this way, we expect the model to be more robust against irrelevant/noisy edges in the graphs and still promote useful edges for representation learning. we conduct extensive experiments on two datasets for deci. the results demonstrate the effectiveness of the proposed model and lead to state-of-the-art performance for deci. ## model we formulate deci as a binary classification problem. the input to the models include a document @xmath0, w 2 , . . . , w n (of n words/tokens) that can have multiple sentences, and two event mentions of interest e s and e t in d. the goal of deci is to predict whether there exists a causal relation between e s and e t in d. our model for deci involves three major components: (i) document encoder to transform the words into representation vectors, (ii) structure generation to generate an interaction graph for d, and (iii) representation regularization to regularize the representation vectors. we provide details for these components below. ## related work the early feature-based methods for eci has explored different features and resources to improve the performance, including lexical and syntactic patterns @xcite , causality cues/markers (e.g., "because") @xcite , statistical co-occurrence of events @xcite @xcite , temporal patterns @xcite , lexical semantics of events @xcite , and weakly supervised data @xcite . although we also apply related features and resource for eci (e.g., syntax, wordnet), our model employs such resources to build interaction graphs for documents to induce more abstract representations with gcns. recently, deep learning has been applied to solve eci, leveraging advanced language models (e.g., bert) @xcite and common-sense knowledge resources (i.e., conceptnet) @xcite to produce state-of-the-art performance. however, none of these deep learning models has explored document-context modeling with rich information for graph construction and gcns as we do. recently, there have been much interest in designing task-specific graphs to learn representation vectors for different nlp tasks, including sentence-level graphs for event factuality identification @xcite and event ar-gument extraction @xcite , and document-level graphs for relation extraction @xcite @xcite and event argument extraction @xcite . our model is different from such related work in that we design document-level interaction graphs that are tailored to our eci task. in addition, our model is also the first model that employs the inherent structure of external knowledge graphs (i.e., word-net) to augment interaction graphs for documents in representation learning. ## conclusion we present a novel deep learning model for document-level eci to address the limitation of prior deep learning models that only focus on causal prediction for inter-sentence event mention pairs. our model designs interaction graphs to capture important objects and connections for input documents, leveraging gcns to induce representation vectors for causal prediction. we introduce several information sources to enrich the interaction graphs based on discourse, syntax, and semantic information. the experiments confirm the effectiveness of the proposed information sources and models for deci. in the future, we plan to extend our model to other related tasks, e.g., event coreference resolution @xcite .