Identifying while Learning for Document Event Causality Identification (2024)

Abstract: Event Causality Identification (ECI) aims to detect whether there exists a causal relation between two events in a document. Existing studies adopt a kind of identifying after learning paradigm, where events’ representations are first learned and then used for the identification. Furthermore, they mainly focus on the causality existence, but ignore causal direction. In this paper, we take care of the causal direction and propose a new identifying while learning mode for the ECI task. We argue that a few causal relations can be easily identified with high confidence, and the directionality and structure of these identified causalities can be utilized to update events’ representations for boosting next round of causality identification. To this end, this paper designs an iterative learning and identifying framework: In each iteration, we construct an event causality graph, on which events’ causal structure representations are updated for boosting causal identification. Experiments on two public datasets show that our approach outperforms the state-of-theart algorithms in both evaluations for causality existence identification and direction identification.1

heading: 1 Introduction
text: Event Causality Identification (ECI) is the task of identifying whether there exists a causal relation between two events. ECI can facilitate a wide range of practical applications, including knowledge graph construction (Chen et al., 2019; Al-Khatib et al., 2020), question answering (Oh et al., 2017), and information extraction (Xiang and Wang, 2023). The ECI task can be divided into the sentence-level ECI (two events are in the same sentence) and document-level ECI (two events may be in different sentences).
∗Corresponding author 1 The source code is available at https://github.com/
LchengC/iLIF
In this paper, we focus on the document-level ECI task, which faces greater challenges due to the requirement of comprehending long texts for crosssentence reasoning. The traditional feature-based methods (Gao et al., 2019) utilize Integer Linear Programming (ILP) to model the document causal structure. In order to better capture the interactions among events, recent methods (Phu and Nguyen, 2021; Chen et al., 2022, 2023; Fan et al., 2022) usually construct document-level undirected graphs to facilitate cross-sentence causal reasoning. Other methods (Yuan et al., 2023) use sparse attention to address the issue of long-distance dependencies and distinguish between intra- and inter-sentential reasoning.
Modeling the interactions among events has been proven effective for the document-level ECI task, however, almost all existing methods focus on only identifying the existence of causal relation between the event ei and ej , yet without considering the causality direction being from ei to ej (or from ej to ei). In this paper, ei → ej indicates that “event ei causes ej”. This may lead to the learning of events’ representations towards capturing events’ correlations, but correlations may not be directly mapped into causalities (Pearl and Mackenzie, 2018). Furthermore, undirected connections may also lead to incorrect causality identifications, as some properties of causal structures cannot be
3815
respected without directionality. There are three basic causal structures, namely, the chain, fork, and collider (He et al., 2021). Causality identification without directionality cannot well exploit causal structures. As shown in Figure 1, “Shootinge1”−→“killinge4”−→“arrestede5” is a chain causal structure. For a model considering causality direction, if the two directional causal relations, i.e., e1 → e4 and e4 → e5, can be first identified with high confidence, then this can help to identify the causal relation between e1 and e5 due to the causal transmission in the chain structure. We argue that events’ causal relation should be with directionality, and considering causal directions could further boost event causality identification.
Besides ignoring directionality, existing solutions for the ECI task adopt a kind of identifying after learning paradigm. That is, learning events’ representations first via some advanced neural networks, and then identifying causal relations for all event pairs at only one pass. However, it could happen that some causal relations can be easily identified with high confidence. As reported by Yuan et al. (2023), identifying intra-sentence events’ causality (two events in a same sentence) is often easier and with better accuracy than identifying inter-sentence events’ causality (two events in different sentences). This motivates us to propose a new identifying while learning mode for the ECI task. That is, identifying some events’ causal relations with high confidence, and then utilizing the directionality and structure of such identified causalities to update events’ representations for boosting next round of causality identification.
Motivated from the aforementioned considerations, this paper proposes an iterative Learning and Identifying Framework (iLIF) for the documentlevel event causality identification. For an event ei, we not only encode its contextual text representation hi, but also update its causal structure representation zi in each iteration. Causality identification is modeled as a classification issue based on the representation hi and zi of an event pair. Initially, we employ a pretrained language model to encode hi. In each iteration, we first construct a directed event causality graph (ECG) based on the identified causalities, and propose a causal graph encoder to next update zi on the ECG. After the termination, we output the directed ECG as the final causality identification results. In order to differentiate the importance of iterations, we design a novel iteration discounted loss function to mitigate
the error propagation issue. We conduct experiments on two public datasets: The EventStoryLine(v0.9) dataset (Caselli and Vossen, 2017) and MAVEN-ERE dataset (Wang et al., 2022) and consider both direction and existence settings for causal relations. We preprocess the EventStoryLine dataset 2 to ensure that each ground truth ECG is a directed acyclic graph (Gopnik et al., 2007). Experiment results validate that our iLIF outperforms the state-of-the-art competitors for the document-level ECI task in evaluations for both causality existence identification and direction identification.

heading: 2 Related work
text: Sentence-level ECI Early methods in feature engineering construct classifiers by searching for effective features, such as connective word categories (Zhao et al., 2016), syntactic features (Pitler et al., 2009), and contextual semantic features (Do et al., 2011). Some studies have employed external knowledge bases or linguistic tools. Cao et al. (2021) induce descriptive knowledge and relation path knowledge from the ConceptNet for reasoning. Liu et al. (2021) enhance model recognition capability by mining context-specific patterns from the ConceptNet. Zuo et al. (2020) design data augmentation methods based on lexical knowledge bases like the WordNet (Fellbaum, 1998) and VerbNet (Schuler, 2005) to generate more training data. Zuo et al. (2021) introduce specific causal patterns and transfer them to the target model using a contrastive transfer learning framework. Hu et al. (2023) utilize an AMR parser (Banarescu et al., 2013) to transform text into semantic graphs, enabling explicit semantic structure modeling and implicit association mining. Shen et al. (2022) leverage the prompt learning paradigm to jointly construct derived templates in order to utilize latent causal knowledge.
Document-level ECI Compared to the sentencelevel ECI, document-level ECI holds greater potential for applications but also faces greater challenges, such as weak long-term dependencies and a lack of clear causal indicators. To address these challenges, researchers attempt to construct the topological structure of events for global reasoning. Gao et al. (2019) propose an approach based on Integer Linear Programming (ILP) to model the
2See Appendix A for more details.
global causal structure. Phu and Nguyen (2021) construct a document-level event graph and utilize GNN to learn structural features. Chen et al. (2022) build a relational graph and modeled the interactions between event pairs. Furthermore, Chen et al. (2023) introduce prior knowledge such as central events and coreference to construct an event interaction graph for achieving global reasoning. Yuan et al. (2023) propose a method that utilizes sparse attention to learn high-quality representations and distinguish between intra- and inter-sentential reasoning.
To the best of our knowledge, existing methods ignore casuality direction and adopt the identifying after learning paradigm for the ECI task. In this paper, we take care of causality direction and propose an identifying while learning mode.

heading: 3 Methodology
text: We propose an iterative Learning and Identifying Framework (iLIF) for document-level event causality identification. The basic idea is to iteratively update events’ representations for causality identification by exploiting causal structures on the most recent event causality graph. As shown in Figure 2, the iLIF includes four main module: (1) Contextual Text Encoder (CTE); (2) Causal Graph Encoder (CGE); (3) Event Causality Identification (ECI) (4) Causality Graph Construction (CGC).

heading: 3.1 Contextual Text Encoder
text: The CTE module is to encode contextual text information for each event mention. Given a document D with n sentences and the j-th sentence Sj contains m words. We use a Pretrained Language Model (PLM), say the BERT-base (Devlin et al., 2018), to sequentially encode the n sentences and output the encodings for all the words in the document D. For the i-th event ei, its contextual text representation hi is an average of individual token representations of the event trigger words.

heading: 3.2 Causal Graph Encoder
text: Based on the contextual text representation hi, we can apply a simple neural classifier, such as a multi-layer perceptron, to identify the existence of a causal relation between two events. Although this naive approach is often not with excellent performance, some identified causal relations may be with high confidence. So we can choose some identified causal relations with high confidence to construct a event causality graph (ECG), denoted by G. We will discuss how to construct and update G in the next subsection.
Besides describing the causality of two events, the ECG G contains more information about causal relations for all events in a document. Furthermore, some causal structures in G can be used to boost inferring new causal relations (such as the chain/fork causal structure) or to help correcting
unreasonble causal relations (such as the collider causal structure). To encode causal direction and structure information from G, we propose to learn a causal graph representation for each event node, denoted by zi (with potentially different cardinality F).
We construct G as a heterogeneous directed graph containing one type of event node and two types of directed edges: Intra-sentence and Inter-sentence. We first utilize the selfattention (Vaswani et al., 2017) on the event nodes using a shared attentional mechanism attention : RF × RF → R to compute the type-specific importance coefficient (Velickovic et al., 2017). For two events ei and ej with an edge rtij of type t, the type-specific importance coefficient ctij of r t ij is computed as follows:
ctij = attention(Wzi,Wzj ; t) (1)
where W is a learnable parameter matrix. We note that the two edge types have different attention network parameters.
For an event node ei, let N ti denote the set of its neighbors each with an edge etji. That is, ej ∈ N ti indicates a directed edge from ej to ei. We next compute the weight wtji for the edge eji as follows:
wtji = exp(ctji)∑
k∈N ti exp(c t ki)
. (2)
Now, we update the causal graph representation for the event node ei by the following multi-head attention mechanism as follows:
zti ←∥Kk=1 σ( ∑
j∈N ti
wtjiW kzi) (3)
where σ(·) denotes an element-wise activation function and ∥ the concatenation operation.
For an event ei with only one edge type, then its causal graph representation is updated by Equation (3). For an event ei connected by two types of edges (t1 for intra-sentence and t2 for intersentence edges), we combine these two types of features with different weights to distinguish their confidence in ECI. Since two events within the same sentence are usually easier to identify, we tend to give more weight to the features related to intra-sentence edges. Finally, we combine the features zt1i and z t2 i as follows:
zi ← βzt1i + (1− β)zt2i . (4) event pairs within the same sentence is relatively easier to identify

heading: 3.3 Event Causality Identification
text: After learning the contextual text representation h and causal graph representation z, we use a MultiLayer Perceptron (MLP) to output a causal relation vector pij ∈ R3 for two events ei and ej as follows:
pij = softmax([hi ∥ hj ∥ (zi − zj)]W), (5)
where ∥ stands for concatenation operation and W is a learnable parameter matrix. We note that the subtraction of zi and zj is to emphasize the causal directionality, as they are learned from a directed causal graph. We write pij = (pnij , p c ij , p e ij) with the element pnij/p c ij/p e ij denoting the probability of a NONE/CAUSE/EFFECT relation between the two events.

heading: 3.4 Causality Graph Construction
text: The CGC module is to construct a document-level event causality graph (ECG) in each iteration. The ECG G = (N ,R) contains events as nodes, and event causal relations as edges. Considering the different information density (Yuan et al., 2023) and recognition difficulty for inter-sentence event pairs and intra-sentence event pairs, we define two types of edges in R: (1) Intra-sentence edges for two events in the same sentence, e.g., the green edge of rammed→ smashing in Figure 2. (2) Intersentence edges for two events in two different sentences, e.g., the blue edge of drove→ stealing in Figure 2.
To minimize error propagation, we resort to the causal relation vector pij by Equation (5) to employ only those identified causal relations with high confidence as edges during iteration. We define ω as the relation confidence threshold. For two events ei and ej , if pcij is the largest and p c ij ≥ ω, then a directed edge eij is constructed from ei to ej ; If peij is the largest and p e ij ≥ ω, then a directed edge eji is constructed from ej to ei. After processing all event pairs’ pij , we construct the ECG G with an adjacency matrix A: Aij = 1, if there exists a directed edge from ei to ej ; Otherwise, Aij = 0
In our iterative Learning and Identifying Framework, the ECG G is first initialized and then iteratively updated till the termination.
Initialization: We initialize the causal graph representation for each event node as the event contextual text representation, i.e., z(0) = h, and construct an ECG G(0) based on the z(0) and h.
Iteration: In the l-th iteration, we first use the previous ECG G(l−1) to learn the new causal graph
representations z(l) for all nodes, as described in Section 3.2. We next construct a new ECG G(l) based on the z(l) and h.
Termination: To address the scale differences among documents and prevent over iteration, we design an iteration condition that terminates upon meeting either of the following two criteria. (1) The iteration count l reaches a predefined maximum iteration number L. For a document with n < L sentences, we set its maximum number of iterations as n. (2) If the structural difference between G(l) and G(l−1) is less than a predefined threshold:
∑
i
∑
j
|A(l)ij −A (l−1) ij | ≤ δ. (6)
After terminating the iteration, all the directed edges in the final ECG G are the identified document-level causal relations.

heading: 3.5 Training strategy
text: Intuitively, the ECG G(l) in each iteration carries different significance for the final ECG. We differentiate the iterations by first calculating an iterationlevel loss Ll for the l-th iteration. We adopt the α-balanced variant of focal loss to address the issue of class imbalance and compute Ll by
Ll = − ∑
ei,ej∈D αc(1− p̂cij)γ log(p̂cij), (7)
where p̂cij is the predicted probability of the true class c, αc is a weighting factor for the true class c. Additionally, γ represents a predefined focusing hyper-parameter.
We note that during the identifying while learning process, if some misidentification happens in an iteration, it may propagate to the later iterations. To penalize such propagations, we emphasize the importance of earlier iterations and introduce a balancing factor inversely proportional to the iteration count in the final loss function. We define the final loss function as follows:
L = LD∑
l=1
1 l Ll, (8)
where LD denotes the actual iteration counts of the document D.

heading: 4 Experiments

heading: 4.1 Experimental Settings
text: Datasets Details We evaluate our iLIF on the widely used EventStoryLine (v0.9) dataset and
MAVEN-ERE dataset. The EventStoryLine dataset comprises 22 topics, 258 documents, 5,334 event mentions, 1,770 intra-sentence causal event pairs and 3,855 inter-sentence causal event pairs (Caselli and Vossen, 2017). Following (Caselli and Vossen, 2017; Gao et al., 2019), we designate event pairs annotated with ‘PRECONDITION’ as CAUSE relation, and ‘FALLING_ACTION’ as EFFECT relation. In both settings, we utilize the last two topics as the development dataset, leaving the remaining 20 topics for 5-fold cross-validation.
The MAVEN-ERE dataset is a large-scale dataset, which comprises 4,480 documents, 103,193 events, and 57,992 causal event pairs. These causal event pairs are annotated as ‘CAUSE’ or ‘PRECONDITION,’ both representing CAUSE relations. We randomly reverse half of the event pairs with a CAUSE relation to represent the EFFECT relation. As MAVEN-ERE did not release the test set, following Tao et al. (2023), we use the original development set as the test set. Additionally, we sample 10% of the data from the original training set to form the development set.
implementation details Our method is implemented based on the PyTorch version of Huggingface Transformer (Wolf et al., 2020). We use the uncased BERT-base (Devlin et al., 2018) as the base PLM and fine-tune it during the training process. We optimize our model using AdamW (Loshchilov and Hutter, 2018), with a linear warm-up for the first 10% of steps. The learning rate for the PLM is set to 2e-5, while for other modules, it is set to 1e-4. The batch size is set to 1. More details can be found in Appendix B.
Evaluation Metrics We adopt the commonly used Precision (P), Recall (R), and F1-score (F1) as the evaluation metrics. In the direction evaluation, we calculate the micro-averaged results for Precision, Recall, and F1-score specifically for the CAUSE and EFFECT classes. In the existence evaluation, for the EventStoryLine dataset, we follow the same approach as previous methods to ensure fair comparison (Phu and Nguyen, 2021).
Direction and Existence Settings In the direction setting, we utilize three labels: NONE, CAUSE, EFFECT, which respectively represent the noncausal relation, the cause relation, and the effect relation. The direction identification results of competitors are derived by expanding the classification module of the competitors’ algorithms from binary
to ternary classification. In the existence setting, we adopt two labels: NONE, CAUSAL, which respectively represent noncausal relation and causal relation. For MAVEN-ERE dataset, we combine the event pairs identified as the CAUSE class and the EFFECT class in the direction experiment into the CAUSAL class event pairs to obtain the existence identification result.

heading: 4.2 Competitors
text: We compare iLIF with the following competitors: PLM-base concatenates two events’ contextual text representations and then identifies causality relation using a MLP. We use BERT-base (Devlin et al., 2018), RoBERTa-base (Liu et al., 2019), and Longformer-base (Beltagy et al., 2020) as the PLM.
ERGO (Chen et al., 2022) builds a relational graph to model interactions between event pairs.
SENDIR (Yuan et al., 2023) leverages intrasentence event pairs to construct a reasoning chain, facilitating inter-sentence causality reasoning.
Large Language Models (LLMs). Gao et al. (2023) conduct zero-shot ECI experiments using OpenAI’s official API 3, covering three versions of ChatGPT: text-davinci-003, gpt-3.5-turbo and
3https://platform.openai.com/
gpt-4. We also test another popular LLM, the LLaMA2 (Touvron et al., 2023) of Llama-2-7bchat version. Appendix D reports the designed prompts for the LLMs.
We consider both existence identification and direction identification for performance evaluations. The existence identification means to only identify the existence of causal relation between two events with causality direction. The direction identification means to correctly identify the causal direction, if existing, between two events.

heading: 4.3 Overall Results
text: Table 1 and Table 2 compare the overall results on the EventStoryLine and MAVEN-ERE dataset, respectively. We note that all the competitors adopt the identifying after learning paradigm; While our iLIF adopts the identifying while learning. Our iLIF achieves the best overall F1 results (intra+inter) on the two datasets in both existence and direction evaluation. This validates the superiority of our iLIF model with the new yet effective identifying while learning mode for the ECI task. For those LLMs, although they are capable of zero-shot causal reasoning, their performance is much worse than those methods based on the fine-tuned small PLMs. This suggests that LLMs, like ChatGPT and
LLAMA, may not be effective causal reasoners for complex causal reasoning tasks.
Taking a close observation of the two tables, all models perform better on identifying causality in intra-sentence than that in inter-sentence. This is in accordance with the report in Yuan et al. (2023), as comprehending events in one sentence is generally easier with the same sentential context. We note that our iLIF achieves large improvements on the Precision in the intra-sentence causality identification. This can be attributed to our using a high confidence threshold, by which the intra-sentence causal relations are often with high identification confidence; While this also leads to a relatively lower Recall compared with other models. However, using such identified intra-sentence relations with high confidence can help improving intersentence identification, as the constructed event causality graph is becoming more confident with the iterations, on which events’ causal structure representations can be well learned to further boost inter-sentence causality identification. This is evidenced from the high Precision and F1-score in the inter-sentence identification, and as a result, the overall intra+inter identification of our iLIF per-
forms better than the competitors.

heading: 4.4 Ablation Study
text: Table 3 presents the ablation studies for examining the module functionalities.
(1) iLIF w/o Direction, which removes the directionality of the ECG. The removal of directionality causes a decrease in F1-scores by 4.8% and 4.6% in the two evaluations, respectively. Additionally, the decrease is primarily concentrated in the Precision. These results imply the importance of encoding causal direction and structure information on a directed ECG for learning potential events’ interactions. We also observe that removing the direction increases the Recall in the direction evaluation. One possible reason is that the absence of direction increases the number of edges for encoding more interaction information. However, this also introduces more noisy edges, leading to inac-
curate causal structure representations and worse F1 scores.
(2) iLIF w/o Heterogeneity, which only set one type of edges in the ECG, without distinguishing the intra-sentence and inter-sentence edges. We observe that removing edge heterogeneity results in a reduction in the Precision and F1-scores. One possible reason is that recognition difficulty does differ in the intra- and inter-sentence cases; While neglecting such differences cannot well utilize more confident intra-sentence relations when encoding on the ECG.
(3) iLIF w/o Iteration, which constructs the ECG only once for learning causal graph representations to output final identifications. As it does not evolve with the re-identified causal relations, it cannot enjoy the potentials of learning more accurate events’ causal graph representations in the later iterations, so resulting in decreases of identification performance. The results again validate the effectiveness of our identifying while learning mode.

heading: 4.5 Parameter Study
text: Relation Confidence Threshold Figure 3 presents the results on examining the impact of using different relation confidence thresholds. It can be observed that using too low or two high thresholds introduces performance decrease. This is not unexpected. Using too low threshold would introduce more edges in the ECG, yet some may be incorrect ones, leading to inaccurate causal graph representation learning on the ECG. Setting the threshold too high results in a sparse ECG, which restricts the quality of the learned causal graph representation. Experiment results suggest to set 0.6 as the relation confidence threshold.
Number of Iterations Figure 4 presents the results on examining using different numbers of iterations. We can fix the number of iterations or use our termination criterion to adaptively adjust maximum iterations for different documents. As different documents may contain different numbers of sentences, it can be observed that using such fixed setting cannot well adapt to the length of documents. Furthermore, our criterion also enables to terminate iterations for two consecutive ECGs with small topological difference, hence helping to balance the under-fitting or over-fitting issue in the causal representation learning process.
Increasing the number of iterations not only affects the results but also increases the time complexity of the algorithm. We compare the complexity of different algorithms from both temporal and spatial perspectives. Experimental details can be found in Appendix C. The results demonstrate that our algorithm achieves a trade-off between algorithmic complexity and identification performance.

heading: 4.6 Exploration Study
text: Figure 5 compares the identification results of the initial and final ECG on the EventStoryLine testing dataset, which contains 32% intra-sentence and 68% inter-sentence causal relations. Compared with the initial ECG, the final ECG can increase the ratio of correct identification and decrease the ratio of un-identification for both intra-sentence and inter-sentence causal relations. This indicates that causality identification can benefit from the updating of events’ representations during the identifying while learning process. Figure 6 presents a case of the identifying while learning process: Compared with the annotated intra-sentence and inter-sentence causality (i.e., the ground truth), the
initially constructed ECG is with 3 mis-identified and 4 un-identified causal relations; While after learning and updating events’ causal graph representations from a few ECGs, 2 mis-identified relations can be corrected and 1 un-identified can be correctly identified in an intermediate ECG. At the termination, the final ECG reduces mis-identified relations from 3 to 1 and un-identified from 4 to 1.

heading: 5 Conclusion
text: In this paper, we propose a novel identifying while learning mode where the central idea is to iteratively update events’ representations for boosting next round of causality identification. Within the proposed iterative Learning and Identifying Framework, an event causality graph is constructed in each iteration based on previously identified causal relations with high confidence, which helps to mine events’ directed interactions for updating their representations. Experiments on two widely used datasets have validated the superiority of this new working mode.

heading: Acknowledgements
text: This work is supported in part by National Natural Science Foundation of China (Grant No:62172167). The computation is supported by the HPC Platform of Huazhong University of Science and Technology.
Limitation
We preprocess the EventStoryLine dataset (Caselli and Vossen, 2017) to ensure that each ground truth ECG is a directed acyclic graph (Gopnik et al., 2007). However, We do not guarantee that the final event causality graph is a directed acyclic graph.
In future work, we plan to introduce the directed acyclic constraint into the causality graph construction process to enhance the practical application effectiveness of the model.
Ethics Statement
Our research work meets the ethics of ACL. The proposed identifying while learning model can identify causal relations among events in a document. However, the algorithm is not perfect and may result in erroneous predictions. Therefore, researchers should not rely solely on the model to make real-world decisions.

heading: A EventStoryLine Dataset Preprocessing
text: Following the assumption that “provided that pairs of events have a purely causal relationship, that is edges represent causal relations between the events, we will have a directed acyclic graph” (Gopnik et al., 2007). However, in the EventStoryLine dataset, not all document-level event causality graphs (ECG) constructed with event nodes and ground truth causal relations as edges are directed acyclic graphs.
A.1 Directed Acyclic Detection For a document, we can construct an ECG G with an adjacency matrix A: Aij = 1 if there exists a causal relation from ei to ej ; otherwise, Aij = 0, based on ground truth causal relations. We employ the directed acyclic constraint proposed by NOTEARS (Zheng et al., 2018) on A:
H(A) = tr(eA⊙A)− d = 0 (9)
where tr(·) denotes the matrix trace operation, ⊙ is the Hadamard product, and d is the number of nodes. If the adjacency matrix A satisfies Equation (9), then G is a directed acyclic graph.
Finally, we find that 16 documents in the EventStoryLine dataset are not satisfied with this constraint.
A.2 Event Conflict Relation Detection We note that Caselli and Vossen (2017) only label 2,265 causal relations, and the rest are extended using within-document event co-reference chains. One possible reason is that the expansion process introduces false causal relations. Considering the mutually exclusive relation between co-reference and causality in event relations, we conduct a further analysis of the dataset. We observe that some co-referenced event pairs are incorrectly labeled as causality. Since the EventStoryLine dataset is composed by the Event Coreference Bank+ corpus, we remove causal relations that conflict with co-reference relations in all documents.
A.3 Manual Check After Event Conflict Relation Detection, we note that there are still 6 documents that do not satisfy the directed acyclic constraint. We invite 3 ECI task researchers to check the causal relations in these 6 documents according to the annotated requirements Caselli and Vossen (2017), aiming to minimize causal loops in the ground truth causal
relations. Table 4 presents the statistics of the EventStoryLine v0.9 dataset before and after preprocessing.
B Implementation Details
Our method is implemented based on the PyTorch version of Huggingface Transformer (Wolf et al., 2020). We use the uncased BERT-base (Devlin et al., 2018) as the base PLM and fine-tune it during the training process. We optimize our model using AdamW (Loshchilov and Hutter, 2018), with a linear warm-up for the first 10% of steps. The learning rate for the PLM is set to 2e-5, while for other modules, it is set to 1e-4. The batch size is set to 1. The number of attention head K is set to 4. The weights β is set to 0.7. The relation confidence threshold ω is set to 0.6. We set the maximum iteration number L to 9 for the EventStoryLine dataset and 19 for the MAVEN-ERE dataset. The structural difference threshold δ is set to 2. For loss function, we set the focusing parameter γ to 2, the weighting factor of CAUSE/EFFECT class α to 0.75, and the weighting factor of NONE class is 1−α. The MLP is a two-layer fully connected network, in which the activation function is LeakyReLU and the rate of dropout is 0.4.
The MAVEN-ERE primarily comprises lengthy documents with a large number of sentences. Capturing sufficient semantics using a single-sentence encoding approach is challenging. Therefore, for experiments on the MAVEN-ERE dataset, we employed the same encoding method as Chen et al. (2022), leveraging dynamic window and event marker techniques. In addition, in the existence setting, to fulfill the requirement of constructing a directed event causality graph, we set all the positive samples to the CAUSE direction for the EventStoryLine dataset.

heading: C Algorithm Complexity Comparison
text: Table 5 compares the average resource consumption of various algorithms on the EventStoryLine (v0.9) dataset, considering both time and space perspectives on an NVIDIA RTX 3090 GPU with 24GB memory.
From the Table, we observe that the directional causal graph generation and iterations for causality updates increase the time complexity of algorithm. However, due to the excellent identification performance of our model, we believe that the time cost is within an acceptable range, especially compared to the Yuan et al. (2023). Additionally, due to the adopted ECG updating approach, our method performs exceptionally well in terms of spatial complexity.
We argue that the key to balancing algorithmic complexity and recognition performance lies in the actual number of iterations. Therefore, we propose termination criterion to achieve the trade-off between algorithm complexity and identification performance.

heading: D LLaMA2 Prompt Details
text: We evaluate LLaMA2’s performance under zeroshot settings. Following previous works (Gao et al., 2023), only the top 20 topics of EventStoryLine dataset are used for evaluation. For the event ei and event ej , we design two prompt templates as follows:
• causality existence prompt
Input: {document content}
Question: is there a causal relationship between " {ei} " and " {ej} " ? Let’s think step by step.
• causality direction prompt
Question: is the causal direction between " {ei} " and " {ej} " from " {ei} " to " {ej} "? Let’s think step by step.
where we fill the templates with event mentions in the document to reference specific events.
We first input the causality existence prompt to obtain the existence identification results. If there is a causal relation, we input the causality direction prompt for continuous questioning to obtain the direction identification results. Figure 7 shows an example of LLaMA2 reasoning ECI task.

heading: Future Work
text: In future work, we plan to introduce the directed acyclic constraint into the causality graph construction process to enhance the practical application effectiveness of the model