Event Causality Extraction via Implicit Cause-Effect Interactions
Authors: Jintao Liu
Zequn Zhang
kaiwen wei
Zhi Guo
Xian Sun
Li Jin
Xiaoyu Li
Venue: EMNLP 2023 Main (2023)
Keywords: Event Causality Extraction
Generative Language Models
Knowledge Distillation
Optimal Transport

Abstract: Event Causality Extraction (ECE) aims to extract the cause-effect event pairs from the given text, which requires the model to possess a strong reasoning ability to capture event causalities.
However, existing works have not adequately exploited the interactions between the cause and effect event that could provide crucial clues for causality reasoning.
To this end, we propose an Implicit Cause-Effect interaction (ICE) framework, which formulates ECE as a template-based conditional generation problem. The proposed method captures the implicit intra- and inter-event interactions by incorporating the privileged information (ground truth event types and arguments) for reasoning, and a knowledge distillation mechanism is introduced to alleviate the unavailability of privileged information in the test stage. 
Furthermore, to facilitate knowledge transfer from teacher to student, we design an event-level alignment strategy named Cause-Effect Optimal Transport (CEOT) to strengthen the semantic interactions of cause-effect event types and arguments. 
Experimental results indicate that ICE achieves state-of-the-art performance on the ECE-CCKS dataset.

Event Causality Extraction via Implicit Cause-Effect Interactions
1:2 Jintao Liu Zequn Zhang' 12 Kaiwen Wei Zhi Guo 1.2 Xian Sun
Li Jin Xiaoyu Li
Key Laboratory of Network Information System Technology, Aerospace
Information Research Institute, Chinese Academy of Sciences School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing, China liujintao20 @mails. ucasu ac. cn, zqzhang Laahanng@mmill @mail. 1, ie ac.
Abstract
Cause: Price Increase
Event Causality Extraction (ECE) aims to tract the cause-effect event pairs from the given text, which requires the model to possess strong reasoning ability to capture event causal- ities. However, existing works have not ade- quately exploited the interactions between the cause and effect event that could provide cru- cial clues for causality reasoning. To this end, we propose Implicit Cause-Efffcct interaction (ICE) framework, which formulates ECE as a template-basee based conditional generation prob- lem. The proposed method captures the im- plicit intra- and inter-event interactions hy in- corporating the privileged information (ground truth event types and arguments) for reason- ing. and a knowledge distillation mechanism is introduced alleviate the unavailability of privileged information in the test stage. Fur- thermore, to facilitate knowledge transfer from teacher student, design event-level alignment strategy named Gtuse-Effee Opti- Transport (CEOT) to strengthen the seman- tic interactions of cause-effect event types and arguments. Experimental results indicate that ICE achieves sfate-of-the-art performance on the ECE-CCKS dataset.
Region
Product
The worldwide rise of gil prices stimulates the demand for new energy such as Ammonia fuel.
Industry
Product
Effect: Demand Increase
Causc: Price Increuse
Effict: Demand Increase
Region:
worldwide
Region:
None
Industry:
None
Industry:
energy
Product:
oil
Product:
Ammonia fuel
An instance event causality extraction. The cause event marked in yellow and the effect event is marked in green. The cause-effect interactions include intra- and inter-cvent interactions. Solid lines indicate intra-event interactions between the event type and the corresponding arguments, while the dashed line indi- cates inter-event interaction between the cause-effect event.
Figure 1:
intelligent search (Rudnik et al., 2019), future event forecasting (Hashimoto, 2019), and why question answering (Oh et al., 2017).
Existing studies mainly regard this task a classification-based problem. Du and Cardie (2020) and Wang et al. (2019) extracted the events and classified the causal relation in pipelined manner. Wang al. (2020) leveraged grid tagging si- multaneously extract the events and causality pairs. Cui et al. (2022) also designed a dual grid tagging scheme, which aims at modeling the correlations between different event arguments. Compared to the classification-basee methods, generation- based models can take full advantage of Pre-trained Language Models (PLMs) by designing flexible prompt templates (Ma al., 2022: Liu al., 2022). a result, there is a trend to cast the extraction task as a conditional generation problem.
1 Introduction
Event Causality Extraction (ECE) is an emerging yet challenging natural language processing (NLP) task, which requires extracting the whole event structure of cause and effect events, including event type and event arguments. As shown in Figure 1, given the input text, an ECE system is expected identify the cause event (i.e., the event type Price Increase, the argument oil plays the role of Product, the argument worldwide as Region), and similarly identify the corresponding effect event. Recognizing event causality can provide support for many downstream NLP tasks, including ma- chine reading comprehension (Berant al., 2014),
"Coreesponding author.
Although generation-barn based approaches have

---

achieved remarkable success, the limited interac- tions between the cause-effect events impede the model's ability to reason effectively between events. Intuitively, the privileged information (Xu et al., 2020), which stands for the ground truth informa- tion of event types or event arguments, could pro- vide valuable knowledge for inferring causal clues. Take Figure 1 as an example, if the model has al- ready known the cause event is Price Increase, and the Product and Region are oil and worldwide in this event, the inter-event interactions could bene- fit the extraction of effect event. Similarly, if the model has known Price Increase causes Demand Increase during training, the intra-event interac- tions may help the model to capture event argu- ments more accurately. By incorporating different kinds of privileged information, the model could make full use of the implicit interactions and be guided to extract causal clues. However, due to the unavailability of such privileged information in practice, incorporating it naively will result in inconsistencies between training and test phases and may affect the model performance. Several methods (Liu et al., 2017; Wei et al., 2021) in other NLP fields have provided solutions to overcome this problem, but they are not applicable when ECE is modeled as generative problem.
template as the input, and decodes event causality and event structure from the generated sequence. To capture the implicit intra- and inter-event inter- actions, we feed different privileged information to the input template and train two well-informed teacher models. Then a student model is driven by imitating the behaviors of teachers to narrow the input difference of training and test phases through knowledge distillation (Hinton et al., 2015). Furthermore, facilitate knowledge transfer and strengthen the interactions between cause and ef- feet events, we design a Cause-Effect Optimal Transport (CEOT) mechanism by treating the event type and event arguments as model units, which could implicitly incorporate the event-level seman- tic information. In summary, the contributions of this paper follows:
1) This work proposes an ICE framework, which models event causality extraction in a generative paradigm and incorporates privileged knowledge for reasoning.
2) The proposed method implicitly captures the intra- and inter-event interactions through knowl- edge distillation, and employs a CEOT strategy to strengthen the semantic interactions of cause and effect events.
3) Experimental results show that our model achieves state-of-the-art performance, improving the Fl-score by 8.39% on the ECE evaluation benchmark.
Moreover, generation- baased methods typi- cally trained via maximum likelihood estimation (MLE) (Salakhutdinov, 2015), which maximizes the likelihood of the next word conditioned on its previous ground truth words. Then, it leverages cross-entropy loss to measure the difference at each position of the target sequence. Nevertheless, since MLE only emphasizes strict word-level alignment, it struggles consider the semantic information from the perspective of event type or event argu- ments. For instance, as shown in Figure 1, when training the word new from the effect event argu- ment new energy, MLE ignores new energy is a whole unit as the Industry, which results in a par- tial loss of semantics. The event type and event arguments from the cause-effect event pairs could also be regarded as different wholes, and by inter- acting with each other, the model could implicitly incorporate such event-level semantic information.
Related Work
Event Causality Extraction. ECE is derived from the previous event causality identification (ECT), which aims to recognize the causal relations be- tween the given events in text (Zuo et al., 2021; Phu and Nguyen, 2021). Early methods mainly focus syntactic and lexical features (Gao et al., 2019),
causality patterns (Hidey and McKeown, 2016).
and statistical causal clues (Hu and Walker, 2017).
Recent works seek to employ external knowledge (Liu et al., 2020; Cao et al., 2021) or prompt-based models (Shen et al., 2022; Liu et al., 2023) for this task. But these methods only identify the causality of events expressed by a word or phrase, without considering the event type and event arguments. Cui et al. (2022) first proposed the ECE task and exploited the argument correlations extract event causality and event structure. Some variant meth- ods from relation extraction have also been applicd to this task (Wang et al., 2020; Wei et al., 2020).
In this paper, we propose Implicit Cause- Effect interaction (ICE) framework for ECE ad- dress the above issues. Specifically, we formulate the ECE task as a template- based conditional gener- ation problem, which takes the context and prompt

---

However, these works fail model the implicit cause-effect interactions, making it difficult ex- tract causal clues.
event pairs { (Eengs Befiliz where Bea; and Befa indicate the cause and effect event of the i-th pair, respectively. The event structure E (1, A) con- tains the event type t and the argument set A. Each argument in corresponds to role.
Knowledge Distillation. This technique is first proposed by Hinton et al. (2015), which aims to transfer knowledge from well-trained teacher to student model. Jiao et al. (2020) and Sanh et al. (2019) used knowledge distillation for compress- ing large-scale pre-trained language models. Wu et al. (2021) and Li al. (2022) adopted multiple- teacher knowledge distillation to improve the ef- fectiveness of distillation. Wei et al. (2023a) pro- posed to incorporate related arguments knowledge through knowledge distillation for event argument extraction. Nevertheless, they are designed for classification-basse methods and struggle to mi- grate to the ECE task under the generative pattern. Optimal Transport. OT has a wide range of ap- plications in NLP domains (Chen et al., 2019; Xu et al., 2021; Wei et al., 2023b). Li et al. (2020) proposed using optimal transport to tackle the ex- posure bias issue in training generative models by maximum likelihood estimation. Zhou al. (2022) modeled events in the sequence as units and adopted optimal transport to explicitly extract the event semantics for generating temporally-orderr event sequences. this paper, we employ optimal transport to improve the semantic interactions of event type and event argument for cause and effect events.
3.2 Generative Template-based ECE Model
Template Creation. At the input stage, we first construct a specific task-related template for ECE. Following Ma et al. (2022) and Liu et al. 1, we design a soft template that contains learnable pseudo tokens and slots for all compo- nents we require extracting. Figure 2(c) shows the ECE template in our model, where <Cause>, <Effect>, <type?, </type>, etc. are specific learnable pseudo tokens. Then we concatenate the context and template, and feed them into a Transformer-based model to generate the output sequence, where the slots the template will be filled with concrete event type or event arguments of the cause event and effect event.
Target Output Sequence. For the cause-effect event pair in context, construct the target output sequence Y for conditional generation by filling the ground truth cause event and effect event into the template. Note that when there is more than one argument corresponding to a role, they will be con- catenated by a special token <and>. If some roles of the event have no arguments, the corresponding positions in the target output sequence will be filled with <None>.
3 Methodology
Training. In the training process, we adopt the Transformer-basee pre-trained language model BART (Lewis et al., 2020) as our basic model archi- tecture, which consists an encoder and a decoder.
Our ICE framework formulates ECE as a template- based generation problem and implicitly incorpo- rates privileged information for reasoning. Under this paradigm, train two well-informed teacher models by incorporating different privileged in- formation into model inputs. Then we adopt knowledge distillation mechanism to drive a stu- dent model to capture implicit cause-effect inter- actions, which could alleviate the difference of un- available privileged information in the test stage. During the training phase, CEOT strategy is adopted to improve the semantic interactions of cause-effect events and promote the training of the student. The overview of ICE shown in Figure 2,
He Encoder(X (X)
(1)
H3 Decoder(Y H.)
where X denotes the concatenation of context and template. The training target to maximize the likelihood of the next token conditioned on the previous ground truth tokens in the sequence:
Cgen Σ
(2)
where L is the length of the target output sequence. Inference. After obtaining the generated sequence, we decode the event type and event arguments of cause-effect events from corresponding slots with a rule-based matching algorithm. Then we check
3.1
Task Formulation
The goal of ECE is to extract event causality and event structure from the text. Formally, given a context, ECE aims to extract a set of cause- cause-effect

---

(a) Overview of learning frumework
(b) Cause-Effect Optimal Transport (CEOT)
Knowledge-
M
Enriched
Encoder
Decoder
Template
Teacher(Event Argument Extractor)
IPOT
Fill type
CEOT
Knowledge
Context
Raw Template
Encoder
Decoder
Gruup
Group
Fill in event type and segumants canse event
Knowledge Distillaioon
CEOT
Knowledge- Enriched
Encoder
Decoder
Knowledge-Enride Enriched Template
Template
Template
Teacher(Effect Event Extractor)
(c) Raw Template Creation
-Cunses stype Tuur Slype Stegion> Roginn Stregiont sindusiry-
<lindsstry Sproduct ProductOOCaas siproductz
<Effects stype= Tiee citype <region> OrrtloOOiiffle </region> <industry> Indlrtrr <industry> "product> ProductOiifeet <prroduct>.
Figure 2: (a) The overview of the ICE learning framework, (b) The process of Cause-Effect Optimal Transport (CEOT). (c) The creation of the template. The cause event is marked in orange and the effect event is marked in green. Underlined tokens indicate slots for event types and event arguments of cause-effect events. And tokens with <angle brackets> denote specific pseudo tokens.
whether the event type is in the predefined event type set and whether each event argument is a span of the context.
construction of knowledge-enriched templates in Appendix B. Next, we concatenate the context and the knowledge-enriched templates as inputs to train teachers.
3.3 Teacher-Student Distillation Learning
In the knowledge distillation stage, use H to denote the hidden states of the i-th decoder layer of the student model, and H to denote the teacher's. s. We adopt the mean squared error (MSE) to encourage the student model to match the hidden states of corresponding layers of the decoder in the teacher:
Based on the generative paradigm, we adopt a teacher-student distillation learning framework to capture the implicit intra- and inter-event interac- tions, which could implicitly incorporate the privi- leged information for event causality reasoning and narrow the inconsistencies between training and test phases.
N Lmse ΣΣ
Specifically, we train two well-informed teach- ers, including an event argument extractor and an effect event extractor. The event argument extrac- tor aims to extract event arguments with the event type of cause event and effect event given, and the effect event extractor seeks recognize event type and event arguments of effect event with knowing the information of cause event. With template- based generation formulation, we could introduce the privileged information into the model ible way. As shown in Figure 2, for the event argument extractor, we construct a knowledge- enriched template by filling the ground truth event types of the cause and effect event in the tem- plate. Likewise, for the effect event extractor, we fill the ground truth event type and argu- ments of the cause event to form a new knowledge- enriched template. We give an example for the
MSE H H.))
(3)
1=1
where N is the number of decoder layers. We also employ KL-Divergence to encourage the student to match the probability distribution of the teacher over the next possible word at each position:
Lu KL (L(WE.Pf)
(4)
flex-
where P and pi are probability distributions of student and teacher over the next possible token at position 7. Please note that the teachers and stu- dent exploit the same model architecture and train- ing objectives, but do not share model parameters. And the parameters of teachers are fixed during the training of the student. The overall loss for training

---

student model with a single teacher is:
Split
#Sents
#Pairs
#Events
kkd = gon + alfsse + BL
(5)
Train
5600
6318
12636
Dev
700
791
1582
where a and [ are weight coefficients.
Test
700
799
1598
3.4 Cause-Fffect Optimal Transport
Table I: Statistics of the ECTE-CCKS dataset.
To improve knowledge transfer, seek to model semantic information from the perspective of event type or event arguments and introduce a CEOT strategy to promote interactions cause and effect events, which is achieved by event-level alignment of teacher and student representations.
We use cosine distance as the cost function and adopt the IPOT algorithm to compute the OT loss:
Leot = IPOT K
(7)
Meanwhile, the last hidden state of the BART decoder for teacher and student is denoted H and H We first an argmax function decode the output sequence.
Optimal transport defines distance metric be- tween two probability measures Given two discrete probability measures μ E ugdz, and V وド=1 Uy0y,, where òz, is the Dirac function centered on 2, the weights Am
domain.
linear layer followed by
Then the representations are divided into several groups based on the event type or event argument together with specific tokens before and behind it. Note that when some specific tokens are missing in the output, we remove the corresponding groups. Likewise,
w = {= E An and e = { satisfy the constraints Σ u, = Σ By 1. Under this setting, the OT distance is formalized as the following problem (Luise et al., 2018):
obtain two sequences KÅ and K! via average operation and compute the OT loss:
Lor(urv)
min Mell(u.))
Mú c(x;,))) v))
Laot = IPOT(K KT)
(8)
min MEll(u.v)
(M,C)
The overall loss of the model defined as:
(6)
C. Lkd XLoot nLoot where À and 7) weight coefficients.
where IT(,,)) = (M E TIXMIMM M 1n = v). In denotes an n-dimensional all-one vector, C' is cost matrix defined Cij c(w; 4;). M is the transportation plan, and (M, C) Tr(M C*) denotes the Frobenius in- ner product. Xie et (2019) proposed an approx- imate algorithm IPOT to solve Eq. 6, which is illustrated in Appendix A. After solving M, we use OT distance as loss to update model parame- ters.
3.5 Training Objectives
Since the two teachers exhibit different difficulties in extracting the same sample, motivated by Zhang et al. (2022), we use adaptive weights to control the importance of different teachers for a specific sample:
Specifically, the representation of the template for student is denoted as H which obtained from the last hidden state of the BART encoder student corresponding to the template. We first partition H into n groups, where each group cor- responds to the representation of the slot together with specific tokens before and behind it (e.g., <type> TypeOfCause </type> belong to group). Then average the representations of tokens in each group obtain the sequence K.O {h&) ) where he; denotes the representation of the i-th group, n is the number of groups. Similarly, we can get the sequence KT = {h? y by group and average the template representations of teacher.
wk = exp( (
geney
(10)
Σ, exp(Cyenn)
where Lk denotes the prediction loss of the k- th teacher. The total loss under the multi-teacher knowledge distillation framework is calculated as:
L = Σ
wk Ch
(11)
&
4 Experiments
4.1 Experiment Setup
Dataset. Our experiments are conducted on the ECE-CCKS (Cui et al., 2022) dataset, which is

---

EAE
CET
ECE
Method
P
R
F1
P
R
F1
P
R
F1
Novel-tagging
59.40
28.47
38.49
49.79
61.70
55.11
51.52
26.75
35.22
CasECE
36.88
36.72
36.80
58.26
59.70
58.97
31.30
41.81
35,80
Pair-tagging
47.08
46.49
46.79
55.78
62.95
59.14
39.24
47.69
43.05
DualCor
58.05
47.60
52.31
61.75
58.19
59.92
48.56
44.85
46.63
BART-ECE
64.62
50.09
56.43
69.43
60.83
64.84
54.74
47.21
50.70
Student
67.81
54.37
60.35
67.57
59.20
63.11
55.47
49.76
52.46
MKD
68.83
53.59
60.26
68.29
59.82
63.78
57.02
49.04
52.73
CE-Pipeline
66.50
52.69
58,80
69.00
60.45
64.44
55.01
48.36
51.47
TA-Pipeline
67.01
53.55
59.53
68.86
60.33
64.31
54.27
47.98
50.93
ICE (Ours)
69.69
54.57
61.21
70.00
61.33
65.38
59.13
51.44
55.02
Table Overall performance compared the state-of-the-art methods the test set. P, and Fl denote precision (%). recall (%), and Fl-score (%). The best results are denoted in bold.
derived from the corpus released by Tianchi (2021). The dataset is annotated with 39 event types and event roles, and the statistic information is listed in Table
which first extracts the cause event and then recog- nizes the effect event conditioned on the former pre- diction. (3) Pair-linking a grid tagging method based on Wang et al. (2020), which uses event- type-level pair linking as conditional information for token-pair linking to extract event arguments. (4) DualCor (Cui et al., 2022) designed dual grid tagging scheme to capture the argument correla- tions for ECE
Evaluation Metric. We use precision (P), recall (R), and Fl-score (FFI evaluation metrics. A predicted cause-effect event pair is considered cor- rect when the event type and event arguments of the cause event and effect event are correctly ex- tracted. To prove fair comparison with previous methods (Cui et al., 2022), we also report results on the following two tasks: Event Argument Extrac- tion (EAE), which measures the model's ability to extract event arguments; Cause-Effect Type ex- traction (CET), which aims to recognize whether the predieted event types of cause and effect event are correct.
Generation-base based Method. Since there no gen- crative method for ECE in cxisting studies, we im- plement the following baselines: (1) BART-ECE achieves event causality extraction with template- based conditional generation method in natural lan- guage form. (2) Student is the generative model in- troduced in Section 3.2. (3) MKD employs multi- teacher knowledge distillation framework where each teacher is trained without privileged informa- tion. (4) CE-Pipeline is a pipelined generation model, which first extracts the cause event and then predicts the effect event based on the cause. (5) TA Pipielee is also pipelined generation model that first identifies the event type of the cause and effeet event, conditioned which to detect the event arguments.
Implementation Details. All experiments are con- ducted on NVIDIA Tesla V100 GPU with Py- torch framework. We use the pre-trained BART- base from Hugging-Face's Transformers library (Wolf et al., 2020) as the encoder-decoder language model. Our model is optimized by the AdamW weight decay strategy with learning rate of 3e-5. The coefficient À is set a and le-3 and 0.5. The model is trained for epochs with batch size 16.
0.1 and is 0.1. We
4.3 Overall Performance
The experimental results are reported in Table 2. We can draw the following conclusions: (1) The proposed method achieves the best performance, outperforming the previous state-of-the-e model. DualCor, by 8.90% 5.46%, and 8.39% on EAF,, CET, and ECE in terms Fl-score. The significant improvements demonstrate the effectiveness our
4.2 Baseline Methods
Classification-bsse Method. (1) Novel-tagging is introduced to ECE by combining causality, event types, event roles, and argument span into a unified label space (Zheng et al., 2017). (2) CasECE is a pipelined method inspired by Wei et al. (2020),

---

Method
EAE
CET
ECE
Pairs
Method
P
R
F1
ICE
61.21
65.38
55.02
Pair-linking
40.31
54.32
46.28
-w/o EAE
60.43
65.11
53.20
DualCor
49.39
51.56
50.46
Single
-w/o EEE
60.97
64.71
53.51
Student
55.83
55.58
55.71
-w/o EAE&EEE
60.35
63.11
52.46
ICE
57.74
57.78
57.76
-w/o CEOT
60.97
65.18
54.06
Pair-linking
32.15
24.82
28.03
-w/o AW
60.45
63.11
54.30
DualCor
43.65
22.72
29.89
Multiple
Student
59.91
27.48
37.68
Table 3: Experimental results (Fl-score) of ablation
ICE
63.06
28.93
39.66
study on the test set.
Table 4: Experimental results under different number of cause-effect event pairs in sample on ECE task.
model. (2) The generation-based methods gener- ally produce better performance than classification- based methods. This suggests that the generation- based model could make full use of the knowledge in PLMs and exhibit strong advantages on ECE. (3) Compared with Student, our method performs better on the three tasks. We credit the reason that the student model has difficulty in extract- ing implicit causal clues, while ICE can equip the model with more implicit event causality reasoning knowledge via a multi-teacher knowledge distilla- tion framework, thus boosting the model perfor- mance. (4) With the same model architecture, the performance of ICE exceeds MKD. The improve- ments indicate that the privileged information could help to train rell-informed teachers, which guide the student to capture the intra- and inter-event interactions..5) CE-Pipeline and TA-Pipeline per- form poorly among the generation baase methods. The results illustrate that the pipelined methods suf- fer from error accumulation problems, while ICE is an end-to-end method with privileged information the supervision, which could avoid introducing noise irrelevant information.
Temp.
EAE
CET
ECE
CA
61.88
64.84
53.56
MA
60.95
64.44
54.27
SF
61.21
65.38
55.02
Table 5: Experimental results (F1-score) of using dif- ferent types of templates. CA: Concatenation Template, MA: Manual Template, SF: Soft Template.
distillation framework. (2) The model performance decreases after removing CEOT, which illustrates this strategy could promote the training process by enhancing the event-level semantic information in- teractions of cause event and effect event. (3) The adaptive weights mechanism can assign different reliability for each teacher according to the char- acteristics of the samples, further improving the performance of the student.
Effect of the Number of Cause-Effect Pairs. To evaluate the effect of different numbers of event causality pairs in sample, we conduct experiments by dividing the test set into two subsets: Single, which indicates the subset with one pair in sam- ple; Multiple, which denotes the subset with more than one pairs. The results are reported in Table It can be observed that: (1) ICE achieves the best performance among all the baselines on Sin- gle and Multiple subsets, which shows the gener- alization ability and robustness of ICE. (2) The methods on Multiple generally suffer from weak performance. The reason may be that multiple event causality pairs impose difficulties in captur- ing implicit causal clues, leading to extremely low recall. (3) The performance gaps between Student and ICE on the two subsets further demonstrate that the knowledge distillation framework could utilize the privileged information to improve the
4.4 Further Discussion
Ablation Study. To evaluate the contribution of each component, we conduct ablation studies by re- moving event argument extractor (w/o EAE), effect event extractor (w/o EEE), Cause-Effect Optimal Transport (w/o CEOT), and adaptive weights (w/o WA). The experimental results are shown in Ta- ble 3. We observe that: (I) Removing the event argument extractor or effect event extractor will result in performance decay, demonstrating that both extractors are beneficial for event causality extraction. This is because they could capture the implicit intra- and inter-event interactions to per- form event causality reasoning, and the knowledge is transferred to the student via a teacher-student

---

(a) Results on EAE
(b) Results CET
(c) Results ECK
50
30
BART-ECE
20
20
DualCor
ICE
10
10
DualCor(Pult)
01
0
ICB(Full)
5
10
20
30
5
10
20
30
5
10
20
30
Ratio of Training Datai%)
Ratin Training Data(%)
Ratio of Training Data(5))
Figure 3: Experimental results (FI-score) under different ratios of training data on three tasks.
The growth of (global ReeioonOJCaase {gasoline ProducrOjCaase demand is sluggish, and the combination of excess supply and unsatisfactory demand has led to a continuous decline in [gasoline] | ProductO/Effeec cracking spreads this year.
Examplel
BART-ECE: [TypeOfCause: Supply Increase, TypeOfffffect: Demand Drop] ICE: (TypeOfCCause: Demand Drop, TypeOffkffect: Price Drop)
The amount [silicon) ProductOfCCuee used in June increased compared to May, and the inerease came from polysilicon, while the supply of (monosilicon) ) ProductOfCamee decreased.
Example2
Therefore. it is expected that the price of [monosilicon] ProdectOjEffee will rise after July. BART-ECE: ('SppeOfCause: Supply Decrease, TypeOfffffect: Price Increase] ICE: (fypeOfCause:: Supply Decrease, TypeOffiffect: Price Increase)
Table 6: Case study on the test set. The correctly predicted event type or event arguments are marked in teal, while wrong predictions by BART-ECE are marked in red.
event causality reasoning ability of the model, thus driving a more effective model.
find that the soft template is superior the manual template and concatenation template, which illus- trates the effectiveness of our template. Although the manual template can elicit pre-training knowl- edge in cloze formulation, it is labor-intensive and hard to achieve optimal. However, the soft template can avoid this laborious process and take the best advantage of the PLMs.
Performance in Low-resource Scenarios. To in- vestigate the model performance in low-resource scenarios, we adopt different low proportions of training data to conduct experiments on the three tasks. As shown in Figure 3, it can be observed that: (1) ICE achieves the best performance under different ratios of training data on three tasks. This suggests that our model can extract event causal- ities and event structure effectively with a small scale of annotated data, which is more practical to use. (2) ICE using small amount of data can even exceed DualCor using full data, and the per- formance gap becomes larger with the decrease of training data. These observations indicate that ICE can elicit knowledge in PLMs and has a strong abil- ity to capture causal clues, which is beneficial for the model to perform the ECE task.
Case Study. We present case studies to further illustrate the performance of the proposed method. As shown in Table 6, for Examplel, BART-GEN gives wrong predictions about the event type of the cause event and effect event. The reason may be that BART-ECE has difficulty capturing implicit cause-effect interactions, so it fails to recognize the causality between Demand Drop and Price Drop. For Example2, both methods produce the correct cause-effect event type, while BART-GEN fails to predict the event argument monosilicon of the cause event. We credit the reason to that our model could leverage the event argument extractor and effect event extractor trained with privileged information to guide the training process of the student, thus obtaining better performance in extracting event causality and event structure.
Effect of the Prompt Template. We study how different types of prompt templates affect the model performance by conducting experiments with the following templates: Concatenation Tem- plate, Manual Template, and Soft Template. The creation of the three templates is shown in Ap- pendix C and the results are listed in Table 5. We

---

5 Conclusions
Liqun Chen, Guoyin Wang. Chenyang Tao, Dinghan Shen, Pengyu Cheng, Xinyuan Zhang, Wenlin Wang. Yizhe Zhang, and Lawrence Carin. 2019. Improv- ing textual network embedding with global attention via optimal transport. In Proceedings of the 57th Conference of the A Association for Computational Lin- guisties, ACI. 2019, Florence, Italy, July 28- August 2, 2019, Volume I: Long Papers, pages 5193-5200 Association for Computational Linguisties.
In this paper, we propose an Implicit Cause-Effeet interaction (ICE) framework to improve the reason- ing ability of the model, which tackles ECE in a generative manner. The proposed method incorpo- rates privileged information for reasoning to cap- ture implicit intra- and inter-event interactions, and utilizes reacher-student learning framework to bridge the gap between training and test stages. Be- sides, we introduce a Cause-Effect Optimal Trans- port (CEOT) strategy to improve the event-level semantic interactions of cause and effect events. Experimental results indicate that ICE outperforms all the baselines on the ECE-CCKS dataset, demon- strating the effectiveness of this work.
Shiyao Cui, Jiawei Sheng. Xin Cong, Quangang Li, Tingwen Liu, and Jinqiao Shi. 2022. Event causality extraction with event argument correlations. In Proceedings of the 29th International Confer- ence on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 2300-2312 International Committee on Com- putational Linguistics.
Xinya Du and Claire Cardie. 2020. Document-level event role filler extraction using multi- coalti-graullarttyy contextualized encoding. In Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2020, Online, July 5-10, 2020, pages 8010-8020 Association for Computational Linguistics.
Limitations
The multi-teacher knowledge distillation mecha- nism utilized in the ICE framework may increase the computational time during the training process. However, only the student model is leveraged dur- ing the test process, and the test time is identical models. The problem
Lei Gao, Prafulla Kumar Choubey, and Ruihong Huang. 2019. Modeling document- causal structures for event causal relation identification. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers). pages 1808- 1817. Associa- tion for Computational Linguistics.
to regular generation- of relatively long training time can be mitigated by strategies such as GPU parallelization. Consid- ering the significant improvement brought by the ICE framework, we believe the cost is acceptable.
A Acknowledgements
The research is supported by the National Nat- ural Science Foundation of China under Grant 62206267.
Chikara Hashimoto. 2019. Weakly supervised multilin- gual causality extraction from wikipedia. In Proceed- ings of the 2019 Conference on Empiricul Methods in Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Pro- cessing, EMNLP-IJCNI 2019, Hong Kong. China, November 3-7, 2019, pages 2986-2997. Association for Computational Linguistics.
References
Jonathan Berant, Vivek Srikumar, Pei-Chun Chen, Abby Vander Linden, Brittany Harding, Brad Huang. Peter Clark, and Christopher D. Manning. 2014. Modeling biological processes for reading compre- hension. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. EMNLP 2014, October 25-29, 2014, Doha, Qatar, meeting of SIGDAT, Special Interest Group of the ACL. ACL.
Christopher Hidey and Kathy McKeown. 2016. Iden- tifying causal relations using parallel wikipedia arti- cles. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL. 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Lin- guistics.
Pengfei Cao, Xinyu Zuo, Yubo Chen, Kang Liu, Jun Zhao, Yuguang Chen, and Weihua Peng. 2021. event causality identification via latent structure induction networks. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/J/NNPP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 4862 4872. Associa- tion for Computational Linguistics.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in neural network. CoRR, abs/ 503.022 .05531.
knowledge-enriched
Zhichao Hu and Marilyn A. Walker. 2017. Inferring narrative causality between event pairs in films. Proceedings of the 18th Anmual SIGdial Meeting on Discourse and Dialogue, Saarbrücceen, Germany, Au- gust 15-17, 2017, pages 342-351. Association for Computational Linguistics.

---

Xiaoqi Jiao, Yichum Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. Tinybert: Distilling BERT for natural language un- derstanding. In Findings of the Association for Com- putational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Find- ings of ACI., pages 4163-4174, Association for Com- putational Linguistics.
Xiao Liu, Yanan Zheng. Zhengxiao Du Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021. GPT understands, too. CoRR, abs/2 103. 10385.
Giulia Luise, Alessandro Rudi, Massimiliano Pontil, and Carlo Ciliberto. 2018. Differential properties of sinkhorn approximation for learning with wasser- stein distance. In Advances in Neural Information Processing Systems 31: Annual Conference on Neu- ral Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 5864-5874.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: denoising sequence-to-sequenee pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, ACI. 2020, Online, 5-10, 2020, pages 7871-7880. Association for Computational Linguistics.
Yubo Ma. Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, and Jing Shao. 2022. Prompt for extraction? PAIE: prompting argument interaction for event argument extraction. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers). ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 6759- 6774. Association for Computational Linguistics.
Jianqiao Li, Chunyuan Li, Guoyin Wang, Hao Fu, Yuh-
Chen Lin, Liqun Chen. Yizhe Zhang, Chenyang Tao. Ruiyi Zhang. Wenlin Wang. Dinghan Shen, Qian Yang, and Lawrence Carin. 2020. Improving text generation with student-forcing optimal transport. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 9144 9156. Association for Computational Linguisties.
Jong-Hoon Oh, Kentaro Torisawa, Canasai Kruengkrai, Ryu lida, and Julien Kloetzer. 2017. Multi- column convolutional neural networks with causality- attention for why-question answering. In Proceed- ings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, Cam- bridge, United Kingdom, February 6-10, 2017, pages 41 415-424. ACM.
Zhuoran Li, Chunming Hu, Xiaohui Guo, Junfan Chen, Wenyi Qin, and Richong Zhang. 2022. An unsuper- vised multiple-task and multiple-teadee teacher model for cross-lingual named entity recognition. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), ACI 2022, Dublin, Ireland, May 22-27, 2022, pages 170-179. Association for Computational Linguistics.
Minh Tran Phu and Thien Huu Nguyen. 2021. Graph convolutional networks for event causality identifica- tion with rich document- structures. In Proceed- ings of the 2021 Conference of the North American Chapter of the Association for Computational Lin- guisties: Human Language Technologies. NAACL- HLT 2021, Online, June 6-11, 2021, pages 3480- 3490. Association for Computational I. Linguistics.
Jian Liu, Yubo Chen, and Jun Zhao 2020. Knowl- edge enhanced event causality identification with mention masking generalizations. In Proceedings of the Twenty-Ninte International Joint Conference on Artificial Intelligence, IJCAI 2020, pages 3608- 3614. ijcai.org.
Charlotte Rudnik, Thibault Ehrhart, Olivier Ferret, De- nis Teyssou, Raphaël Troncy, and Xavier Tannier. 2019. Searching news articles using an event knowl- edge graph leveraged by wikidata. In Companion proceedings of the 2019 world wide web conference, pages 1232-1239.
Jintao Liu, Zequn Zhang, Zhi Guo, Li Jin, Xiaoyu Li, Kaiwen Wei, and Xian Sun. 2023. KEPT: knowledge enhanced prompt tuning for event causality identifi- cation. Knowl. Based Syst., 259: 10064.
Ruslan Salakhutdinov. 2015. Learning deep generative models. Annual Review of Statistics and Its Applica- rion, 2:361-385.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019, Distilbert, distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01
Shulin Liu, Yubo Chen, Kang Liu, and Jun Zhao. 2017. Exploiting argument information to improve event
detection via supervised attention mechanisms. In Proceedings of the 55th Annual Meeting of the As- sociation for Computational Linguistics ( (Volume 1: Long Papers), pages 1789- 1798.
Shirong Shen, Heng Zhou, Tongtong Wu, and Guilin Qi. 2022. Event causality identification via derivative prompt joint learning. In Proceedings of the 29th International Conference on Computational Linguis tics, COLING 2022, Cyeongju, Republic of Korea, October 12-17, p02e. 2288-2299, 2288-2299.nntemmaionnl International Committee on Computational Linguistics.
Xiao Liu, Heyan Huang, Ge Shi, and Bo Wang. 2022. Dynamic prefix-tuning for generative template-based event extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27. 2022, pages 5216-5228 Associ- ation for Computational Linguistics.
Tianchi. 2021. 2cks2021 the dataset for financial event and causal relation extraction.

---

Xiaozhi Wang, Xu Han, Zhiyuan Liu, Maosong Sun, and Peng Li. 2019. Adversarial training for weakly supervised event detection. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-IILT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 998-10088 Associa- tion for Computational Linguistics.
16-20, 2020. pages 38-45, Association for Computa- tional Linguistics.
Chuhan Wu, Fangzhao Wu, and Yongfeng Huang. 2021. One teacher is enough? pre-trained language model distillation from multiple teachers. In Find- ings of the Association for Computational Linguis- rics: ACI/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/UCNLP 2021 of Findings of ACL, pages 4408 4413. Association for Computational Linguistics.
Yucheng Wang, Bowen Yu, Yueyang Zhang, Tingwen Liu, Hongsong Zhu, and Limin Sun. 2020. Tplinker: Single-stage joint extraction of entities and relations through token pair linking. In Proceedings of the 28th International Conference on Computational Lin- guisties, COLING 2020, Barcelona, Spain (Online). December 8-13, 13, 2020, pages 1572-1582. Interna- tional Committee on Computational Linguistics.
Yujia Xie, Xiangfeng Wang, Ruijia Wang, and Hongyuan Zha. 2019. A fast proximal point method for computing exact wasserstein distance. In Pro- ceedings of the Thirty-Fiiik Conference on Uncer- tainty in Artificial Intelligence, UAI 2019, Tel Aviy, Israel, July 22-25, 2019, volume 115 of Proceed- ings of Machine Learning Research, pages 433-453. AUAI Press.
Kaiwen Wei. Xian Sun, Zequn Zhang. Li Jin, Jingyuan Zhang. Jianwei Lv, and Zhi Guo. 2023a. Implicit event argument extraction with argument-argument relational knowledge. IEEE Trans. Knowl. Data Eng. 35(9):8855-887)
Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun. Jian Wu, Hanxiao Sun, and Wenwu Ou. 2020. Privileged features distilla- tion at taobao recommendations. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2590- 2598.
Kaiwen Wei, Xian Sun, Zequn Zhang. Jingyuan Zhang, Zhi Guo, and Li Jin. 2021. Trigger is not sufficient: Exploiting frame-awar aware knowledge for implicit event argument extraction. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing, ACIJJICNLP 2021, (Volume 1: Long Papers), Virtual Event, Au- gust 1-6, 2021, pages 4672-4682, Association for Computational Linguistics.
Jingjing Xu, Hao Zhou, Chun Gan, Zaixiang Zheng, and Lei Li. 2021. Vocabulary learning via optimal transport for neural machine translation. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing, ACLAJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 7361-7373. Associa- tion for Computational Linguistics.
Kaiwen Wei, Yiran Yang, Li Jin, Xian Sun, Zequn Zhang, Jingyuan Zhang, Xiao Li, Linhao Zhang, Jin- tao Liu, and Guo Zhi. 2023b. Guide the many-to- one assignment: Open information extraction via iou-aware optimal transport. In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), ACI. 2023, Toronto, Canada, July 9-14, 2023, pages 4971- 4984. Association for Computational Linguistics.
Hailin Zhang, Defang Chen, and Can Wang. 2022. Confidence-aware multi-teacher knowledge distilla- tion. In IEFF International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23-27 May 2022, pages 4498 4502. IEEF
Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 August 4, Volume 1: Long Papers, pages 1227-1236. Associa- tion for Computational Linguistics.
Zhepei Wei, Jianlin Su, Yue Wang. Yuan Tian, and Yi Chang. 2020. A novel cascade binary tagging framework for relational triple extraction. In Pro- ceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, ACL 2020, On- line, July 5-10, 2020, pages 1476-1488, Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer. Patrick von Platen, Clara Ma. Yacine Jemiie,, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger. Mariama Drame Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Pro- ceedings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing: System Demon- strations, EMNLP 2020 Demos, Online, November
Bo Zhou, Yubo Chen, Kang Liu, Jun Zhao. Jiexin Xu, Xiaojian Jiang, and Qiuxia Li. 2022. Generating temporally-orden -ordered event sequences via event op- timal transport. In Proceedings of the 29th Inter- national Conference on Computational Linguistics, COLING 2022, Gyeongju. Republic of Korea, Oc- tober 12-17, 2022, pages 1875-1884. International Committee on Computational Linguistics.
Xinyu Zuo. Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Weihua Peng, and Yuguang Chen. 2021.

---

concate- Learnda: Learnable wwwwwledg-guiddeed data augmen- tation for event causality identification. In Proceed- ings of the 59th Annual Meering of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL/JJNNPP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 3558-3571. Associa- tion for Computational Linguistics.
D Compare with ChatGPT
In this section, we conduct experiments to evaluate the performance of ChatGPT on the ECE task. A well-designed prompt template is as follows: Suppose you are now an event causality extrac- tion model. Given sentence, please give the cause event and result event respectively, where the event contains the event type and the argu- ments corresponding to each role. The list of event types is: ('Typhoon', 'Demand Increase", 'Price Decrease', 'Cold Wave', 'Price Increase', 'Other Natural Disasters', 'Supply Decrease', 'Supply In- crease', 'Sales Decrease', 'Demand Drop', 'Im- port Decrease', 'Flood', 'Other Trade Frictions', 'Negative Impact', 'Swine Fever', 'Sales Increase', "Limited Production', 'Operating Costs Increased 'Other Livestock Epidemics', 'Positive Impact', 'Drought', 'Operating Cost Decrease", 'Export De- crease', 'Frost', 'Other or Unclear', 'Import In- crease', 'Bird Flu', Earthquake ', Anti-dumping Against China', Exporrts Increase', 'Add Tariffss to China', 'Decrease in Product Profits', 'Increase in Product Profits', Foot-and- Disease of Pigs', 'Anti-dumping Against Other Countries', 'Unsal- able', 'Cattle Foot and Mouth Disease', 'Flash Flood', 'Hail'1. The list of event argument roles is: ['Region', 'Industry', 'Product']. Given a sen- tence:"Thh worldwide rise of oil prices stimulates the demand for new energy such as Ammonia fuel.", please extract the event type and arguments corresponding to cause and effect event. If no argu- ment corresponds to role, the argument content returns "None". : If multiple arguments corresponds to a role. the arguments are connected with "and". types of the cause and effect event in the raw tem- EEE
A
The details of the IPOT algorithm
The details of the IPOT algorithm shown in Al- gorithm
Algorithm IPOT algorithm
Input: Feature vectors S (hi) (.
Generalized stepsize
Output: (M,C) C)
I: a 1 1m. M(1) 1 m
2: Cj - chi, h2)), Aij = = C₁
3: for t = 1, 2.3..... do
4: Q A @ ( // @ is Hadamart product
5: for k = 1,2, K do =1 in practice
6: 5 nQu T
7: end for
8: M(++1) diag(ō) Q diag(o)
9: end for
10: return (M,C)
B An example of the construction of knowledge-enriched templates
We show an example of the construction of knowledge-enriched templates for EAE and EEE in Table 7. The knowledge- wew.eedgeeeercched template for EAE is constructed by filling the ground truth event
As shown Table 9, we can observe that ICE outperforms ChatGPT by a large margin on EAE, CET, and ECE tasks. The results indicate that ChatGPT has difficulty solving such complex event causality extraction tasks without any fine-tuning or training update parameters.
plate: The knowledge-enriched template is constructed by filling the ground truth event type and arguments of the cause event the raw tem- plate.
C The creation of different types of templates
We construct three types of templates for compar- ison: (1) Concatenation Template, where all slots for event type and event arguments
nated; (2) Manual Template, where event types and event arguments are integrated into templates in natural language form; (3) Soft Template, which is used our method. And detailed information of the templates shown in Table 8.

---

Temp.
Example
<Cause> <type> TypeOfCause </typ>> <region> RegionOfCause </region>
<industry> IndusstryOfCause </inusstry> <product> ProductOOCauss </product>
Raw Temp.
<Effect> <typ>> TypeOfEffecct <region> </reeioo
<industry> Indussyyffft </industry> <
</product>.
<Cause> <lype> Price Increase </type> <region> RegionOffCuss </region>
<industry> IndustryOfCause </industry> <product> ProductOfjausse </product>
Temp. for EAE
<tffeect><typ>> <type> Demand Increase </type> <region> RegionOfttifect </regei
</region>
<produutt Andustry>/Iddusrryy h p </product>
<Cause> <Cause><<yper <type> Price Increase </type> <region> None </region>
<industry> new energy </industry> <product> Ammonia fuel </product>
Temp. for EEE
<Effect> <lype> TypeOfEffect </type> <region> RegionOfEffect </region>
<industry> IndustryyfEffectt </industry> <product> ProductOffffeet </product>.
Table 7: An example of the construction of knowledge-enriched templates for EAE and EEE.
Temp.
Example
TypeOfCause RegionOfCause IndustryOfCause ProductOfCause
CA
TypeOfEffect Regiom@Offffccctndsssttyfffffff Produffffee....fffffff..fee.
The cause TypeOfCause, the region RegionOfCause, the industry IndustryOfCause, the product ProductOfCause, leads to the effect TypeOfEffect, the region RegionOfFffect, the industry IndustryOfEfffect, the product ProductOfEffecc.
MA
<Cause> <type> TypeOfCause </type> <region> RegionOfCause </region> </reein
<industry> IndustryOfCause </industry> <product> ProductOfCause </product>
SF
<Effect> <type> TypeOfEffect </type> <region> RegionOfEffect </region>
<industry> IndustryOfEfftect </industry> < <product>> #rrduuttffffeett</pnouuue
Table 8: The creation of different types of templates. CA: Concatenation Template, MA: Manual Template, SE:
Soft Template.
EAE
CET
ECE
Method
P
R
F1
P
R
F1
P
R
F1
ChatGPT
13.16
16.26
14.54
17.85
15.64
16.68
6.00
8.04
6.87
ICE
69.69
54.57
61.21
70.00
61.33
65.38
59.13
51.44
55.02
Table 9: Overall performance compared to ChatGPT on the test set. P. R, and Fl denote precision (%), recall (%), and FI-score (%). The best results denoted in bold.


Reviews:
[{"review_id": "5wswuJH47d", "reviewer": "Reviewer_BTRu", "summary": "", "questions": "Q1: How do you obtain the templates used by your proposed method ? Am I right in thinking that they are used by the student model too ? If those aren't obtained automatically, how do you justify your extractive method having access to high-quality information (these templates) while the other baseline methods do not ?\n\nQ2: is the \"context\" the same as the \"text\" from which causal relations are extracted ? (l207-208)", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much. We feel sorry for some misunderstandings due to the lack of details, please give us a chance to explain them one by one. \n\n**Q**: How do you obtain the templates?   \\\n**A**: The raw template is manually designed by inserting special pseudo tokens, where the slots remain empty. The knowledge-enriched template is automatically constructed by filling the raw template’s slots with ground-truth event type or event arguments.\n\n**Q**: Is the template used by the student model too?\\\n**A**: The student and teacher use raw template and knowledge-enriched template, respectively. The raw template does not contain any ground-truth information, while the knowledge-enriched template contains some ground-truth information. (We adopt knowledge distillation to transfer knowledge from teacher to student, which can drive a well-formed student model to perform causality reasoning and alleviate the unavailability of ground-truth information in the test stage.)\n\n**Q**: How to justify your method with template while other baselines do not?\\\n**A**: The generation-based baselines also utilize template as part of the input, which is consistent with our method. Therefore, it is fair to compare baselines with our method.\n\n**Q**: What are details of the rule-matching algorithm?\\\n**A**: The rule-matching algorithm uses regular expressions to match the text span between special pseudo token pair from the output sequence as corresponding event type or argument, e.g., we can get “Price Increase” as cause event type from “<Cause><type>Price Increase</type>...”.\n\n**Q**: Is the \"context\" the same as the \"text\"?\\\n**A**: Yes, “context” and “text” have same meanings here.\n\n**Q**: Does l229-233 talk about the training of the teacher models?\\\n**A**: This is the ground-truth output for student and teacher during the training stage, which is used to compute the generation loss. The ground-truth output is constructed by filling **all** the slots in the raw template with ground-truth labels. \nWhile the construction of the template is in the input stage. The knowledge-enriched template is constructed by filling **certain** slots according to the teacher's functions (The event argument extractor teacher needs to fill ground-truth event type of cause-effect event; The effect event extractor teacher needs to fill ground-truth event type and arguments of cause event.)\n\n**Q**: Are un-instantiated arguments given as-is?\\\n**A**: The un-instantiated arguments in raw template remain empty in the input stage and the model is trained to fill the slots with ground-truth labels.\n\n**Q**: Which one does \"the teacher\" in l292-293 refer to?\\\n**A**: Either. For convenience, we use the same equation to describe the distillation process of the two teachers, because they only have different input templates, and the distillation method is the same.\n\n**Q**: What are the training steps of the method?\\\n**A**: We first train teacher with knowledge-enriched template. Then we train student model with raw template and the supervision of the teacher. The parameters of the teacher model are fixed when training student.\n\n\n**Q**: Are the generation-based methods variations of the proposed method?\\\n**A**: BART-ECE, CE-Pipeline, and TA-Pipeline are additionally proposed generation-based methods for comparison either in a joint or pipelined way. Student and MKD are variant methods, where Student is the generative model without the supervision of teachers, and MKD adopts raw template to train teachers instead of knowledge-enriched template. We will give a more detailed descriptions in Appendix.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The task dealt with is a relatively important task in the NLP field. Alongside that, the proposed model shows strong performance over existing methods.\n\nThe proposed method is original and, even if not directly usable in other fields, could be used as a model for other tasks.\n\nBesides one glaring problem (see reasons to reject), the paper is overall well written, and the experimental setup is strong.", "reasons_to_reject": "The proposed method relies on templates which are used to guide the extraction process. As far as I understood, the templates are provided as input to the student model, i.e. the one used to evaluate the method. The authors do not explain at all how these templates, which are a crucial component of the proposed method, are obtained.\n\nThe paper severely lacks a detailed overall description of the proposed method (the explanation at lines 191-207 is quite short), and most of all actual examples of what is given as input to which model, what is obtained as output, how the output is parsed... As it stands, the proposed method is hard to understand and some details (like details of the rule-matching algorithm) are missing.", "questions_to_authors": "Q1: How do you obtain the templates used by your proposed method ? Am I right in thinking that they are used by the student model too ? If those aren't obtained automatically, how do you justify your extractive method having access to high-quality information (these templates) while the other baseline methods do not ?\n\nQ2: is the \"context\" the same as the \"text\" from which causal relations are extracted ? (l207-208)", "paper_topic_and_main_contributions": "This paper proposes a generative method to extract causal relations (cause-effect pairs) from text. Unlike previous methods that rely on extracting causal relation candidates and classifying them, the proposed method directly extract causal relations from text.\n\nIt also uses (1) a system of templates which are used to guide the extraction process, and (2) a distillation process in which a student model is trained to mimic the performance of two teacher models which are given additional information during training. Experimental results a provided which show a strong performance gain obtained by the proposed method over state of the art methods.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Note: questions given here aren't to be answered, but should provide insight for the authors as to what information/details are missing from their paper.\n\nThe proposed method is complex, many its broad explanations are short. These could be improved.\n\nAs stated in \"reasons to reject\", the paper severely lacks detailed examples of data and explanations of data flow, and an overall detailed explanation of the proposed method that relies on these examples.\n\nl3 \"the model\": no model was introduced yet\n\nl211-213: unclear (surely related to lack of examples)\n\nl229-233: so, since knowledge is given, this section talks about the training of the teacher models, right ? This is unclear.\n\nAre un-instanciated arguments given as-is ? If so, are these words trainable ?\n\nl292-293 \"the teacher\": which one ?\n\nl305-306: this entails that there are several steps to training, which wasn't said until then.\n\nl319-321: sentence without a verb\n\nl429-441: are these variations of the proposed methods ? If so, this is unclear, and I'm also unsure which one has which component of the proposed method."}}, {"review_id": "67PRTbaxE7", "reviewer": "Reviewer_dU5i", "summary": "", "questions": "Could an LLM performing SFT, instructing tunning on the ECE dataset, be a strong baseline? ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much. We feel sorry for the lack of some analysis.\n\n**Q**: The effect of different hyper-parameters is under-explored.\\\n**A**: The main hyper-parameters include $\\lambda$, $\\eta$, $\\alpha$, and $\\beta$. We conduct experiments to illustrate the effect of each hyper-parameter. \\\nWe set $\\eta$ equal to $\\lambda$ and their value is selected from [0.05, 0.1, 0.2].\n|$\\eta$&$\\lambda$| EAE|CET|ECE|\n|-|-|-|-|\n|0.05| 61.67| 64.58| 54.09|\n|0.1  |61.21| 65.38| 55.02|\n|0.2 |60.82 | 65.51| 54.14|   \n\nThe value of $\\alpha$ is selected from [5e-4, 1e-3, 2e-3].\n|$\\alpha$| EAE|CET|ECE|\n|-|-|-|-|\n|5e-4|   61.40| 65.64| 54.78|\n|1e-3 |61.21| 65.38 |55.02|\n|2e-3 |61.48 |65.17 |54.51|\n\nThe value of $\\beta$ is selected from [0.3, 0.5, 0.7].\n|$\\beta$| EAE|CET|ECE|\n|-|-|-|-|\n|0.3| 61.36| 65.38| 54.42|\n|0.5 |61.21 |65.38 |55.02|\n|0.7 |61.46 | 66.44 |54.69|\n\nWe will add more experiments to prove the effectiveness of the hyper-parameter selection.\n\n**Q**: Ablation results show that the incomplete versions are comparable to the full version. Does the performance gain come from a careful tuning BART?\\\n**A**: All ablation experiments were performed with the same parameters. The variant models in ablation study only remove the corresponding component. \n\n**Q**: The results should reflect the variance of the scores from multiple experiments with different seeds.\\\n**A**: We report the average results over 3 runs with different seeds on the main experiments:\n| | P(EAE) | R(EAE) | F1(EAE)| P(CET) |R(CET)| F1(CET)|P(ECE)|R(ECE)|F1(ECE)|\n|-|-|-|-|-|-|-|-|-|-|\n|ICE|69.97$\\pm$0.49|52.83$\\pm$0.19|61.48$\\pm$0.28|70.76$\\pm$0.55|61.99$\\pm$0.48|66.09$\\pm$0.51|59.07$\\pm$0.30|51.52$\\pm$0.06|55.04$\\pm$0.12|\n\n\n**Q**: Could an LLM be a strong baseline?\\\n**A**: We have shown the results of ECE with ChatGPT in the following table. The experiments illustrate that ChatGPT fails to effectively perform causality reasoning and accurate extraction of spans. Due to computational resource constraints, it is difficult for us to perform instruction tuning on the ECE task with LLMs. We argue that the proposed method is superior to LLM in terms of time and space complexity. Therefore, LLM fails to serve as a strong baseline for the ECE task.\n| | P(EAE) | R(EAE) | F1(EAE)| P(CET) |R(CET)| F1(CET)|P(ECE)|R(ECE)|F1(ECE)|\n|-|-|-|-|-|-|-|-|-|-|\n|ChatGPT | 13.16 |16.26| 14.54|17.85| 15.64 |16.68| 6.00| 8.04| 6.87| \n|ICE |   69.69| 54.57| 61.21| 70.00| 61.33| 65.38| 59.13| 51.44| 55.02|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Pos:\n\n* The problem is well motivated - event causality extraction is an important but challenging NLP task, and modeling implicit interactions between events can provide useful reasoning knowledge.\n\n* The proposed ICE framework is novel, incorporating privileged information and teacher-student distillation to capture intra- and inter-event interactions. The Cause-Effect Optimal Transport strategy is also an interesting technique for aligning event representations.\n\n* The experiments are comprehensive, comparing multiple strong baselines on the ECE-CCKS dataset. The results demonstrate clear improvements, especially in low-resource scenarios. The ablation studies also verify the contribution of each model component.\n\n* The writing is clear and easy to follow. The methodology is well-explained.", "reasons_to_reject": "Neg:\n\n* The effect of different hyper-parameters is under-explored.\n\n* Ablation results show that the incomplete versions are comparable to the full version. Does the performance gain come from a careful tuning BART? \n\n* The results should reflect the variance of the scores from multiple experiments with different seeds.", "questions_to_authors": "Could an LLM performing SFT, instructing tunning on the ECE dataset, be a strong baseline? ", "paper_topic_and_main_contributions": "This paper proposes an ICE framework that models implicit cause-effect interactions using privileged information and teacher-student learning. A Cause-Effect Optimal Transport technique is introduced to align event representations and strengthen semantic interactions. Extensive experiments demonstrate the performance advantage.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hVtUnuouxB", "reviewer": "Reviewer_ttgs", "summary": "", "questions": "Please refer to the Reasons To Reject.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much. We feel sorry for not explaining some details clearly.\n\n**Q**: Is “Implicit” equivalent to “limited” in the paper？\\\n**A**: \"Implicit\" is the opposite of “explicit”, meaning that there is no obvious marker word to suggest the occurrence of event causality or event arguments, while “limited” refers to the lack of interactions. And event causality and arguments are implicit in most cases.\n\n**Q**: Why privileged information is unavailable?\\\n**A**: Since the privileged information represents ground-truth label, it is not available in the testing phase or in real applications. In our method, only the teacher model can use privileged information and guide the training of the student model through knowledge distillation, which enables the student model to effectively perform causality reasoning without privileged information in the testing phase.\n\n**Q**: Why is the distillation conducted with MSE and KL simultaneously?\\\n**A**: MSE is used to distill the feature representation of each layer of the decoder, while KL is utilized to distill the output probability. Using these two methods at the same time can not only promote the student model to imitate the representation of each layer of the teacher, but also make the prediction probability of the student model approach the teacher model, thereby promoting the training process of the student model.\n\n**Q**: What is the difference between BART-ECE and the student model?\\\n**A**: We adopt the generative model with raw template in Section 3.2 as Student. And BART-ECE uses manual template in natural language form without special pseudo tokens. We will revise the descriptions and add more details.\n\n**Q**: Show the model performances of the two teacher models and the student model.\\\n**A**: We report the experimental results in the following table:\n| | P(EAE) | R(EAE) | F1(EAE)| P(CET) |R(CET)| F1(CET)|P(ECE)|R(ECE)|F1(ECE)|\n|-|-|-|-|-|-|-|-|-|-|\n|EAE(teacher) | 79.71|\t 61.27|  69.28|  100.00|  87.61|   93.40|  79.21|   68.77 |  73.62|\n|EEE(teacher)  |84.97|  65.54|   73.97|   86.14|  74.47|   80.45|  77.53|   67.56|   72.20|\t\n|Student| 67.81| 54.37| 60.35| 67.57| 59.20| 63.11| 55.47| 49.76| 52.46| \n|ICE |   69.69| 54.57| 61.21| 70.00| 61.33| 65.38| 59.13| 51.44| 55.02|\n\nEAE(teacher) and EEE(teacher) indicates event argument extractor teacher and effect event extractor teacher, respectively.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.The proposed method is reasonable to tackle the claimed challenge.\n\n2.Figure 2 goes clearly to illustrate the whole model architecture.\n\n3.Experiments on the public benchmark demonstrate the effectiveness of the proposed method.", "reasons_to_reject": "1.The word “Implicit” in the title is not significantly explained in the paper. I try to explore the clear definition to “Implicit”, but still confused. Is the word ‘”limitted” in Line60 equivalent to “Implicit”？ Besides, Line63-78 just describes how the privileged information/cause-effect interaction helps to guide task predictions, but does not state clearly why such interaction is unavailable or implicit. \n\n2.In Line294 and 299, why is the distillation  conducted with MSE and KL simultaneously? Please add some explaination.\n\n3.Line429-435 explains the BART-ECE and Student implicitly. Is (2) Student the student model with a raw template in Section 3? What is the difference between BART-ECE and the teacher model?\n\n4.Going on with 3, please show the original model performances of the two teacher models and the student model.", "questions_to_authors": "Please refer to the Reasons To Reject.", "paper_topic_and_main_contributions": "This paper studies the task of Event Causality Extraction(ECE). Following the recent trend, ECE is formulated in a generative paradigm, and the authors aims to improve the model performances from two aspects: 1)  enhancing the limited interaction between cause-effect events; 2) exploring the semantic information of event types or arguments. To this end, two strategies are conducted in this paper: 1) knowledge distillation works to distill the privileged information from the teacher models with well-informed templates to the student model with a raw template. 2) Optimal Transport further facilitates the knowledge transfer between teacher models and student models.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "None"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper formulates event causality extraction as a template-based conditional generation problem to capture interactions between the cause and effect event. All the reviewers agree that this paper provides the sound method and makes significant contributions. The authors also give the detailed responses to the questions. The reviewers suggest some valuable methods for improving the paper. It would be nice if the authors could take them into consideration.
Decision: Accept-Main