Event Causality Identification via Derivative Prompt Joint Learning (2022)

Abstract: This paper studies event causality identification, which aims at predicting the causality relation for a pair of events in a sentence. Regarding event causality identification as a supervised classification task, most existing methods suffer from the problem of insufficient annotated data. In this paper, we propose a new derivative prompt joint learning model for event causality identification, which leverages potential causal knowledge in the pre-trained language model to tackle the data scarcity problem. Specifically, rather than external data or knowledge augmentation, we derive two relevant prompt tasks from event causality identification to enhance the model’s ability to identify explicit and implicit causality. We evaluate our model on two benchmark datasets and the results show that our model has great advantages over previous methods.

## introduction event causality identification (eci) is an important natural language processing (nlp) task, which aims at identifying causality between events in sentences. event causality identification supports a variety of nlp applications, e.g., machine reading comprehension @xcite and event prediction @xcite . @xcite . as shown in figure 1 , an eci model identifies the causalities in sentences s 1 and s 2 : (i) practice cause -→ won in s 1 ; (ii) attack cause -→ killed in s 2 . the causality between events in a sentence mainly contains two types: explicit causality and implicit causality. for instance, the causality practice cause -→ won in s 1 is an explicit causality, which is triggered by the explicit cue words in the sentence. eci models can take causal cue words as the shortcut for explicit causality identification. as a contrast, the causality * corresponding author. figure 1 : examples of different causalities. s 1 contains explicit causality between practice and won. s 2 contains implicit causality between killed and attack. s 3 has the same patterns with s 2 , but it does not contain any causality. attack cause -→ killed in s 2 is an implicit causality, because none of explicit cue words is mentioned in s 2 . and as shown in figure 1 , comparing s 2 and s 3 , we may not always derive the existence of causality from two highly similar expressions without explicit cue words. in other words, implicit causality must be inferred from the semantics and contexts of events. most existing methods regard eci as a classification task, and train customized eci models on annotated data by supervised learning @xcite . however, the large-scale annotated datasets of eci are relatively hard to collect, referring to that the so far largest dataset eventstoryline @xcite only contains 258 documents and 4316 sentences. therefore, eci models are challenged by the data scarcity problem in supervised learning. to address this problem, various methods have been proposed to leverage either the augmented dataset @xcite or external knowledge @xcite . @xcite exploit weakly supervised method to construct eci datasets. @xcite and @xcite attempt to introduce external structure knowledge to identify causality. however, the model may fail to capture the differences between the explicit causality and implicit causality without modelling these different types respectively, especially when the eci model is trained with only insufficient annotated dataset. in this paper, we propose a new drivative prompt joint learning (dpjl) method for eci, which identifies different causalities effectively without incorporating either more annotated instances or external knowledge. firstly, as shown in figure 2 , we introduce a new prompt-based learning paradigm to eci, i.e. converting eci into a language modelling format and using pre-trained language model (plm) to identity causalities. since plms accumulated abundant knowledge @xcite @xcite through the self-supervised training on large-scale corpora, such a prompt-based paradigm may elicit the potential eci ability of plm to remedy the scarcity of annotated data. then, we design two derivative prompt tasks for eci to identify the explicit causality and implicit causality: (i) causal cue word detection (cwd), which aims to detect the causal cue word of event pairs in a sentence; and (ii) causal event detection (ced), which aims to detect the cause or effect of an event in a sentence. intuitively, cwd is a straightforward way to identify explicit causality, and ced is helpful to identify implicit causality by capturing the semantic relevancy between the contextual events. finally, given the above intuitions, we further propose a joint learning method for event causality identification enhanced by the tasks of cwd and ced. note that the training data of derivative tasks are generated from eci, without the cost of additional human annotation. our contributions are summarized as following: event causality identification event causality identification (eci) is a crucial information extraction task. early causal identification methods include rule-based methods @xcite @xcite and statisticsbased methods @xcite . recently, some benchmarks on the event causality have been released, e.g. causal-timebank @xcite , eventstoryline corpus @xcite and because @xcite . based on these annotated datasets, a number of supervised learning-based methods of eci have emerged @xcite . however, the scale of labeled data in most datasets is relatively small. to solve this problem, @xcite exploited weakly supervised method to construct eci datasets. some methods introduce additional knowledge to strengthen the eci model @xcite . @xcite improved the performance of eci with distantly supervised labeled training data. these methods introduce the pre-trained language model to generate the high-quality text coding required by the eci model. but these methods ignore the potential ability of pre-training language model to identify the causality between events, and may fail to capture the differences between the explicit and implicit causalities in a low-resource scenario without modelling these two types of causalities respectively. prompt-based learning recently, pre-trained language models like gpt @xcite , bert @xcite , roberta @xcite and t5 @xcite can capture rich knowledge @xcite ) from massive unlabeled corpora. but there is a big gap between pre-training objectives and fine-tuning objectives, that is, downstream tasks still need to build task-specific models after plms, and use task-specific annotated data to fine-tune them. to solve this problem, promptbased method (brown et al., 2020) converts the downstream task into the same form as pre-training task. to better build task prompts, automatic search of discrete prompts @xcite , gradientguided search @xcite and continuous prompts (li and liang, 2021) are successively proposed. to the best of our knowledge, there is no work that uses prompt-based method for eci task. however, only using prompt-based methods lacks the modeling of different causalities mentioned above. ## methodology in this section, we first introduce problem definition of event causality identification. then we will show the overview of our proposed model. after that, we introduce a prompt-based eci method to elicit knowledge from plms. then, we describe the details of two derivative prompt tasks and the joint learning method of derivative prompts. finally, we introduce the training and prediction process of our model. ## experiments our experiments aim to verify (1) whether the prompt-based method can improve the generalization of the eci model, and (2) whether the joint learning of derivative prompts can enhance the model's ability to identify different causalities. our source code is available on github 1 ## conclusion and future work in this paper, we first introduced a new promptbased paradigm to event causality identification and proposed a new derivative prompt joint learning method, i.e, dpjl. the proposed method adopts two new derivative tasks, i.e., the causal cue word detection and causal event detection, to strengthen the ability of an eci model in identifying the explicit causality and implicit causality respectively. the experimental results demonstrate that the proposed method achieves the state-of-the-art performance with at least 11 percent f1 improvement on both of two well-known benchmarks, i.e., eventsto-ryline and causal-timebank. additionally, the detailed analysis suggests the effectiveness of jointlearned prompt-based derivative tasks on performance improvement in downstream tasks. in the future, we will try knowledge-enhanced methods to construct both of derivative tasks and data for an eci model, which may fill the gap between knowledge and samples.