Event Causality Extraction via Implicit Cause-Effect Interactions (2023)

Abstract: Event Causality Extraction (ECE) aims to extract the cause-effect event pairs from the given text, which requires the model to possess a strong reasoning ability to capture event causalities. However, existing works have not adequately exploited the interactions between the cause and effect event that could provide crucial clues for causality reasoning. To this end, we propose an Implicit Cause-Effect interaction (ICE) framework, which formulates ECE as a template-based conditional generation problem. The proposed method captures the implicit intra- and inter-event interactions by incorporating the privileged information (ground truth event types and arguments) for reasoning, and a knowledge distillation mechanism is introduced to alleviate the unavailability of privileged information in the test stage. Furthermore, to facilitate knowledge transfer from teacher to student, we design an event-level alignment strategy named Cause-Effect Optimal Transport (CEOT) to strengthen the semantic interactions of cause-effect event types and arguments. Experimental results indicate that ICE achieves state-of-the-art performance on the ECE-CCKS dataset.

## introduction event causality extraction (ece) is an emerging yet challenging natural language processing (nlp) task, which requires extracting the whole event structure of cause and effect events, including event type and event arguments. as shown in figure 1 , given the input text, an ece system is expected to identify the cause event (i.e., the event type is price increase, the argument oil plays the role of product, the argument worldwide as region), and similarly identify the corresponding effect event. recognizing event causality can provide support for many downstream nlp tasks, including machine reading comprehension @xcite , intelligent search @xcite , future event forecasting @xcite , and why-question answering @xcite . existing studies mainly regard this task as a classification-based problem. @xcite and @xcite extracted the events and classified the causal relation in a pipelined manner. @xcite leveraged grid tagging to simultaneously extract the events and causality pairs. @xcite also designed a dual grid tagging scheme, which aims at modeling the correlations between different event arguments. compared to the classification-based methods, generationbased models can take full advantage of pre-trained language models (plms) by designing flexible prompt templates @xcite . as a result, there is a trend to cast the extraction task as a conditional generation problem. although generation-based approaches have achieved remarkable success, the limited interactions between the cause-effect events impede the model's ability to reason effectively between events. intuitively, the privileged information @xcite , which stands for the ground truth information of event types or event arguments, could provide valuable knowledge for inferring causal clues. take figure 1 as an example, if the model has already known the cause event is price increase, and the product and region are oil and worldwide in this event, the inter-event interactions could benefit the extraction of effect event. similarly, if the model has known price increase causes demand increase during training, the intra-event interactions may help the model to capture event arguments more accurately. by incorporating different kinds of privileged information, the model could make full use of the implicit interactions and be guided to extract causal clues. however, due to the unavailability of such privileged information in practice, incorporating it naively will result in inconsistencies between training and test phases and may affect the model performance. several methods @xcite in other nlp fields have provided solutions to overcome this problem, but they are not applicable when ece is modeled as a generative problem. moreover, generation-based methods are typically trained via maximum likelihood estimation (mle) @xcite , which maximizes the likelihood of the next word conditioned on its previous ground truth words. then, it leverages cross-entropy loss to measure the difference at each position of the target sequence. nevertheless, since mle only emphasizes strict word-level alignment, it struggles to consider the semantic information from the perspective of event type or event arguments. for instance, as shown in figure 1 , when training the word new from the effect event argument new energy, mle ignores new energy is a whole unit as the industry, which results in a partial loss of semantics. the event type and event arguments from the cause-effect event pairs could also be regarded as different wholes, and by interacting with each other, the model could implicitly incorporate such event-level semantic information. in this paper, we propose an implicit cause-effect interaction (ice) framework for ece to address the above issues. specifically, we formulate the ece task as a template-based conditional generation problem, which takes the context and prompt template as the input, and decodes event causality and event structure from the generated sequence. to capture the implicit intra-and inter-event interactions, we feed different privileged information to the input template and train two well-informed teacher models. then a student model is driven by imitating the behaviors of teachers to narrow the input difference of training and test phases through knowledge distillation @xcite . furthermore, to facilitate knowledge transfer and strengthen the interactions between cause and effect events, we design a cause-effect optimal transport (ceot) mechanism by treating the event type and event arguments as model units, which could implicitly incorporate the event-level semantic information. in summary, the contributions of this paper are as follows: however, these works fail to model the implicit cause-effect interactions, making it difficult to extract causal clues. knowledge distillation. this technique is first proposed by @xcite , which aims to transfer knowledge from a well-trained teacher to a student model. @xcite al. (2020) and sanh et al. (2019) used knowledge distillation for compressing large-scale pre-trained language models. @xcite and @xcite adopted multipleteacher knowledge distillation to improve the effectiveness of distillation. @xcite proposed to incorporate related arguments knowledge through knowledge distillation for event argument extraction. nevertheless, they are designed for classification-based methods and struggle to migrate to the ece task under the generative pattern. optimal transport. ot has a wide range of applications in nlp domains @xcite @xcite . @xcite proposed using optimal transport to tackle the exposure bias issue in training generative models by maximum likelihood estimation. @xcite modeled events in the sequence as units and adopted optimal transport to explicitly extract the event semantics for generating temporally-ordered event sequences. in this paper, we employ optimal transport to improve the semantic interactions of event type and event argument for cause and effect events. ## methodology our ice framework formulates ece as a templatebased generation problem and implicitly incorporates privileged information for reasoning. under this paradigm, we train two well-informed teacher models by incorporating different privileged information into model inputs. then we adopt a knowledge distillation mechanism to drive a student model to capture implicit cause-effect interactions, which could alleviate the difference of unavailable privileged information in the test stage. during the training phase, a ceot strategy is adopted to improve the semantic interactions of cause-effect events and promote the training of the student. the overview of ice is shown in figure 2 . ## conclusions in this paper, we propose an implicit cause-effect interaction (ice) framework to improve the reasoning ability of the model, which tackles ece in a generative manner. the proposed method incorporates privileged information for reasoning to capture implicit intra-and inter-event interactions, and utilizes a teacher-student learning framework to bridge the gap between training and test stages. besides, we introduce a cause-effect optimal transport (ceot) strategy to improve the event-level semantic interactions of cause and effect events. experimental results indicate that ice outperforms all the baselines on the ece-ccks dataset, demonstrating the effectiveness of this work.