SEAG : Structure-Aware Event Causality Generation (2023)

Abstract: Extracting event causality underlies a broad spectrum of natural language processing applications. Cutting-edge methods break this task into Event Detection and Event Causality Identification. Although the pipelined solutions succeed in achieving acceptable results, the inherent nature of separating the task incurs limitations. On the one hand, it suffers from the lack of cross-task dependencies and may cause error propagation. On the other hand, it predicts events and relations separately, undermining the integrity of the event causality graph (ECG). To address such issues, in this paper, we propose an approach for Structure-Aware Event Causality Generation (SEAG). With a graph linearization module, we generate the ECG structure in a way of text2text generation based on a pre-trained language model. To foster the structural representation of the ECG, we introduce the novel Causality Structural Discrimination training paradigm in which we perform structural discriminative training alongside auto-regressive generation enabling the model to distinguish from constructed incorrect ECGs. We conduct experiments on three datasets. The experimental results demonstrate the effectiveness of structural event causality generation and the causality structural discrimination training.

## introduction event causality plays an essential role in natural language processing @xcite . event causality extraction aims to recognize events and their inter-causal relations from text. as shown in figure 1 , given the input text, the model should be able to identify three events, i.e., "suffered", "invasion" and "destroyed", and the causal relations in between. extracting event causality has impactful applications such as question answering @xcite , event fore- * *corresponding authors. casting @xcite and reading comprehension @xcite . for the purpose of event causality extraction, current methods break down the task into two subtasks, i.e. event detection (ed) @xcite @xcite and event causality identification (eci) @xcite @xcite . then the solution of event causality extraction is integrating these two sub-tasks. although such a pipelined method is feasible to a certain extent, two limitations deteriorate the efficacy. first, in the view of task formulation, it over-simplifies the problem as two local extraction processes with ignorance of cross-task dependencies. such separation hinders feature and knowledge sharing when extracting events and their causal interdependence. it also can result in error propagation. as shown in figure 1 , although the event "abandon" has no causal relations with other events in the context, the ed model still detects it neglecting the cross-task dependencies with eci. second, from the event causality perspective, the events and their causal relations form a global event causality graph (ecg). the pipelined models break the innate structural causality of all events in the context when extracting the events and their inter-relations without capturing the structural interactions within the ecg. for example in figure 1 , events "suffered", "invasion" and "destroyed" form an ecg. the pipelined models fail to leverage this structural information and then miss the causality between "destroyed" and "suffered" by only counting on these two events themselves. without such structural understanding, models are likely prone to extract events and causal relations by the perception of superficial linguistic features @xcite and make biased predictions. to bypass the absence of the task dependency brought by pipelining separation, we propose structure-aware event causality generation (seag), a novel paradigm for event causality extraction. specifically, seag linearizes the ecg with a label semantic enhanced template and then generates the extraction results in a text2text generative format based on a generative pre-trained language model @xcite . seag generates all events and relations in one-pass, avoiding local prediction in either ed or eci. this end-toend extraction can further mitigate the error propagation. moreover, regarding the ecg as the output, seag enables the reasoning and interactions in the whole structure. such a structural extraction process maintains the ecg and its semantics intact. to further improve the understanding of the ecg, we adopt causality structural discrimination. we first sample negative events and relations to compose negative ecgs. then we conduct discriminative learning to foster the model's awareness of the positive ecg. during this discriminative process, the model can learn to comprehend the in-depth event causality semantic of ecg structures, not learn only on superficial features. we conduct experiments on three event causality extraction datasets to testify to the effectiveness of seag. evaluation results show that the end-toend extraction of events and their causal relations in a text2text format is effective. moreover, causality structural discrimination further improves the model's extraction ability with a better understanding of event structure. we summarize our contribu-tions as follows: ## preliminaries task formulation. event causality extraction aims to extract events and their inter-causal relations from a text sequence. formally, given an input text sequence x consisting of n words pipelined extraction. pipelined methods break event causality extraction into event detection (ed) and event causality identification (eci). they solve the task as learning the probability p (r|e, x ) • p (e|x ) which breaks the joint probability of p (e, r|x ). event causality graph. an event causality graph (ecg) is a graph @xmath0, v). e @xmath1. in the directed setting, g is a directed acyclic graph while in the undirected case, g is an undirected graph. structural generation. we model event causality extraction as structural generation. we first compose the causal triplet set s as an ecg. we then learn a model p (g|x ) to identify the ecg in a text2text paradigm given the context x . s e a g r 14 e 4 e 1 e 1 e 5 e 4 e 5 e 1 e 6 e 2 e 4 e 6 e 6 e 1 eos negative ecg construction e 3 e 5 e 4 e 2 . . . e 1 negative graph e 1 e 5 e 4 e 6 e 1 e 4 e 2 positive event positive graph negative event negative graph . . . graph linearization input text e 3 e 5 e 4 e 2 e 1 structural discriminative training inference train eos e 2 r 12 e 3 e 4 e 5 r 23 r 45 r 16 r 62 r 64 r 15 r 54 e 2 . . . sep sep sep sep sep sep sep sep sep eos figure 2: overview of our seag. seag completes event causality extraction by structure-aware event causality generation. the graph linearization module linearizes all positive g and negative gs into y and ỹs. then seag conducts structural discriminative training to foster ecg understanding. 3 method model overview. our seag first linearizes the ecg g with a label semantic enhanced template. then it leverages a generative pre-trained language model to predict the results. at last, it improves the ecg structural comprehension via causality structural discrimination. the overview of seag is in figure 2. in the rest of this section, we first detail the graph linearization in section 3.1. then we elaborate the generation process in section 3.2. finally, we present the causality structural discrimination in section 3.3. ## experiments this section first gives the datasets we want to use in section 4.1. we elaborate on the evaluation metrics for event causality extraction in section 4.2. the baselines and the implementation details are in section 4.3 and 4.4 respectively. we finally report the experimental results in section 4.5. ## conclusion we propose a novel structure-aware event causality generation (seag) for event causality extraction. we model this task as structural generation and design the novel ecg linearization. we also adopt the causality structural discrimination training to foster the model's understanding of the ecg. we conduct experiments on two settings of three datasets. results demonstrate that seag outperforms the pipelined models for event causality extraction on all datasets.