Complex Event Schema Induction with Knowledge-Enriched Diffusion Model
Authors: Yupu Hao
Pengfei Cao
Yubo Chen
Kang Liu
Jiexin Xu
Huaijun Li
Xiaojian Jiang
Jun Zhao
Venue: EMNLP 2023 Findings (2023)
Keywords: Complex Event Schema Induction
Diffusion Model
Large Language Model

Abstract: The concept of a complex event schema pertains to the graph structure that represents real-world knowledge of events and their multi-dimensional relationships. However, previous studies on event schema induction have been hindered by challenges such as error propagation and data quality issues. To tackle these challenges, we propose a knowledge-enriched discrete diffusion model. Specifically, we distill the abundant event scenario knowledge of Large Language Models (LLMs) through an object-oriented Python style prompt. We incorporate this knowledge into the training data, enhancing its quality. Subsequently, we employ a discrete diffusion process to generate all nodes and links simultaneously in a non-auto-regressive manner to tackle the problem of error propagation. Additionally, we devise an entity relationship prediction module to complete entity relationships between event arguments. Experimental results demonstrate that our approach achieves outstanding performance across a range of evaluation metrics.

Complex Event Schema Induction with Knowledge-Enrichel
Diffusion Model
Yupu Hao!? Pengfei Cao Yubo Chen -Kan , Xu², Huaijun Li", Xiaojian Jiang", J Jun Zhao
The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China 2 School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
3 China Merchants Bank haoyupu2023@iaaccoom ppengfei.cao, yubo.chen, kliu, jzhao) @ @nlpr.ia.ac.cn
Abstract
The concept of complex event schema per- tains to the graph structure that represents real- world knowledge of events and their multi- dimensional relationships. However, previous studies on event schema induction have been hindered by challenges such as error propaga- tion and data quality issues. To tackle these challenges, we propose a knowledge-enriched discrete diffusion model. Specifically, dis- till the abundant event scenario knowledge of Large Language Models (LLMs) through an object-oriented Python style prompt. We in- corporate this knowledge into the training data, enhancing quality. Suhsequently, we em- ploy discrete diffusion process generate all nodes and links simultaneously in a non- auto-regressive manner to tackle the problem of error propagation. Additionally, vise an entity relationship prediction module complete entity relationships between event arguments. Experimental results demonstrate that our approach achieves outstanding perfor- mance across range of evaluation metrics.'
two police
thnee oeher resporaiing and called
officees subuegurity evacuaed
calnforemmeus, while a stoch officer stayed os the street to redirect gedestrans, The vehicle exploded b:30lam Eight people were treated at huquitels for injurins and laker dischirgel least thrre veliches burnd afte banbing, lnse busineses dumaged, street, away from she bombing. collapsed.
building
Raw texts
System
LOC
place
PER
Assembler
place
Dammage
Callayee
Assemble
Desonate
Injure
Interviiio
TransporoO
target
Lvaccstion
Ternsprt:
Instánce graphs
de-
instrument
VEII Aushicle
LOC
Damage
Transport
Alboo
Ariest PER
Assemble
Desonate
Assemble
Injune
Ineevention
Bay
Foccution
Broadcast
Victim
Tesmsgurt
PER
Event Schema
1 Introduction
Figure example schema induction process for complex event "Bombing".
Event schema induction aims to summarize com- mon patterns and structures from historical events. Current studies mainly induce the atomic schema for each independent event type and their argu- ments separately (c.g. "Attack" event with the ar- guments: "Attacker", "Target", "Instrument" and "Place"), without considering the correlation be- tween events (Chambers and Jurafsky, 2008; Cham- bers, 2013; Nguyen et al., 2015). However, some real-world events are usually very complex, con- sisting of multiple events and their relations. For example Figure Bombing a complex event, which involves some fine-grained events, such as Assemble, Detonate and Injure. Therefore, some
researchers attempt to study the complex event schema induction task, which abstracts typical structures for complex events from event data. Fig- ure illustrates an example of the complex event schema induction process for the scenario of Bomb- ing. Initially, an information extraction (IE) tool (Du et al., 2022) is utilized to extract instance graphs from raw texts. Subsequently, we inducc the event schema based these extracted instance graphs. The resulting event schema represented as a graph, where events are interconnected through temporal links (c.g., Damage occurs after Deto- nate) and their argument relations (e.g., the target
'Code is available at https://github... com/ hypasd-art/KON

---

of the Detonate event assumes the victim role in the subsequent Injure event).
ing to our statistics on the Suicide-IED dataset (Li et al., 2021). The need to generate so many nodes and edges will inevitably exacerbate the problem of error accumulation. Thus, it is essential to ad- dress the error propagation problem during schema graph generation.
However, indueing complex event schema is non- trivial. As shown in Figure 1. it necessitates the model's ability to summarize the events within in- stance graphs and possess profound understand- ing of the multi-dimensional relationships between these events. Recently, graph-based methods are proposed for this task by utilizing graph generation techniques (Li et al., 2021; Jin al., 2022). For ex- ample, Li et al. (2021) proposes an auto- regreseere generation method that generates the schemn fol- lowing event temporal order. Similarly, Jin et al. (2022) leverages an auto-encoder to encode the global skeleton information and decode the schema graph event by event. Despite successful efforts, these methods still face two critical challenges:
In this paper, we propose a novel method termed as Knowledge-Enriched Diffusion Model (KDM) to address aforementioned problems. Firstly, to im- prove the knowledge coverage of instance graphs, we devise a Instance Graph Expansion module. As Large Language Models (LLMs) are trained on vast corpora of texts (Touvron et al., 2023; Zhao et al., 2023; Wang et al., 2023) and therefore pos- sess extensive event and entity knowledge of the real world, we leverage the LLMs (Chowdhery et al., 2022; Ouyang et al., 2022) as the knowledge databases to inject knowledge into instance graphs. The module utilizes Python style object-oriented prompt to extract event knowledge from LLMs, and adds the knowledge into the instance graphs. Sec- ondly, to tackle error propagation of ruge-seeressive decoding. we propose an Event Skeleton Gener- ation module, which utilizes discrete diffusion model to predict all nodes and links simultaneously in non-auto-regreesivv manner but not generates individually based on time series, which alleviate the error propagation problem (Austin et al., 2021; Yang et al., 2023; Vignac et al., 2022). Finally, we devise an Entity Relation Prediction module, which expands the event skeleton with correspond- ing arguments and predicts their relations to get a complete schema.
Knowledge Coverage of Instance Graphs: The event schema induction task summarizes the in- stance graphs to obtain the event schema. Thus, the quality of the instance graphs is crucial for the event schema induction. However, the instance graphs are extracted via Information Extraction (IB) tools (Rui et al., 2022), whose knowledge coverage is very limited. For example, as the representative IB tool, RESIN (Wen et al., 2021) is trained on fixed datasets and can only extract predefined types of entities and events. Besides, the extraction per- formance of RESIN is also unsatisfactory, which only achieves approximately 64% of F1-score for event detection the ACE dataset. It indicates that the IE tool is difficult extract complete in- stance information, even for predefined event types. Therefore, how to improve the knowledge coverage of instance graphs is an important problem.
Error Propagation of Auto-regressive Decod- ing: Previous graph-based approaches are based the auto-regressive generation manner (Li et al., 2021; Jin al., 2022), generating the entire event schema graph node by node, which may lead to error accumulation over time and therefore degrade the generation performance. For example, in Fig- ure 1, the model may mistakenly generate "Injure" instead of "Detonate" leading to the omission of subsequent events such as "Damage" and "Inves- tigate" in the generated schema or resulting in in- correct nodes being generated in the next. The final generated event schema graph will consist of dozens of nodes and edges at the minimum, as each instance graph used for training contains an average 11 event nodes and 246 temporary links accord-
The contributions of our work include: (1) We propose Knowledge-Enriched discrete Diffusion Model (KDM) for complex event schema induc- tion task. To the best of our knowledge, we are the first simultaneously utilize LLMs and dif- fusion models to accomplish the task. (2) To improve knowledge coverage of instance graphs, we propose an Instance Graph Expansion module, which distillates the event knowledge in LLMs with python code-style prompt. To solve the error prop- agation problem, we design event skeleton gen- eration module, which predicts all nodes and links simultancously. (3) We conduct extensive experi- ments on three widely used datasets. Experimental result indicates that our proposed method outper- forms sfate-of-the-eeeeeeoo baselines.

---

Ulnsten Exucution
Investigat
Investigele
Arrest
Unitorm distribution
Transport
Forward
Desonase
Danag
Detonate
Transport
interrgatee
Reverse Pracess Taanporrt
Injure
Eextution
Transport
Injare
Evacuation
Amest
Pa (6 (o"))
6
Neural network
(G*)
Figure 2: The discrete diffusion process. In forward process, the noise changes the types of nodes and edges.
2 Preliminaries and Problem Formulation
define the distribution of graph as (N, E), where node set N is sampled from node feature distribution N E Ruxa and edge set & is sam- pled from edge feature distribution E E Ruxxxb Here, c(p) represents the one-hot vector (category scalar) sampled based on probability distribution p. At time I, the instance graph is defined g (W", - (c(NN), af ()))).
2.1 Preliminaries
Discrete diffusion model preserves the discrete characteristics of each element in the training data To. introduces noise to each element z æo to into the uniform distribution and reverses them by removing the noise (Austin et al., 2021). Fig- ure shows the process of graph-based discrete diffusion.
The objective of this task to learn an event schema Sy from a set of instance graphs Dy = ((m)) that belong to the same specific topic.
The forward process. This process progres- sively adds noise to Zy by transition probability matrix Qi step.
Our Approach
g( (0 xz IQ:
(1)
To solve the complex event schema induction task, we propose a Knowledge-Enree Enriched Discrete Diffu- sion Model, as shown in Figure 3. Our method mainly consists of three modules: (1) Instance Graph Expansion, which expands the instance graphs using the complex event knowledge ob- tained from LLMs atomically. (2) Event Skeleton Induction, which summarizes the event evolution skeleton using discrete diffusion model. (3) Entity Relation Prediction, which decorates the arguments to the event skeleton and then use simple graph transformer to predict the entity relations. We will illustrate each component in detail.
where 1Qrly- = qx=; = x1 = i) indicates the probability of transition from 2t-1 i to
j. The forward process gradually converts each ab æ0 uniform distribution when T is large enough.
The reverse process. Reverse process Po with learnable parameters aims to convert the noise distribution æy back to the original æp:
Po (xtorr) po (xrr) II ( (mx)
Po (z) (1:-1/1))) g(wr-) w., 3e (xal
3.1 Instance Graph Expansion
and according to Bayes formula as follows:
In this section, we will illustrate how to obtain knowledge about event schemas from LLMs and inject them into instance graphs. Complex event schemas involve intricate graph structures, while LLMs are good at processing unstructured lan- guage tasks. To retain structured information of instance graphs, need LLMs to be able han- dle structured inputs and outputs. Considering the powerful coding capabilities of LLMs, we treat events as Python objects. detail, events, enti- ties, and their intricate relations can correspond to classes, attributes, and instances the object-
a(x,-1|a,, vo) g( 24-- 100 (x))) (oo)
(4)
q< (Fa)
Therefore, the task becomes predicting Po (o mx) using neural network.
2.2 Problem Formulation
In the instance graphs about specific topic y (e.g.. Bombing), nodes represent events and en- tities, while edges have three types: the temporal link, the argument link and the entity relation link. The instance graph is denoted as g = (W, E), We

---

schema
Complete Arguments
Instance graphs
Graph Transformer
Reverse Process
LLM
Uniform Distribution
Hidden node representation
Forward Process
MLP
Extract
Expanded Instance graphs
Event Skeleton
Instance Graph Expansion
Event Skeleton Generation
Entity Relation Prediction
Figure The model structure. The data passes through Instance Graph Expansion module, Event Skeleton Generation module, and Entity Relation Prediction module in sequence to obtain the final schema.
oriented paradigm, respectively. This module in- cludes three aspects: event knowledge expansion, temporal relation expansion, and entity relation ex- pansion.
time as Gt.
The forward diffusion process. In this process, we apply noise separately on each node and edge. This is achieved by multiplying the node and edge distributions with the transition probability matrix Q. By doing we can obtain the graph G. from the previous graph Gt-1. Mathematically, this can be expressed as:
In event knowledge expansion, we select fre- quently occurring event sequences from the train- ing instance graphs and write them into Python classes. Then we ask the LLM to enrich Python code. In this way. we will obtain new classes which represent new events that have a high correlation with the scenario. We filter out new events that occur less frequently than a hyperparameter K and are not in the predefined event categories. In tempo- ral relation expansion, write the obtained events as multiple-choice questions to establish their tem- poral relation with existing event sequences. In entity relation expansion, we obtain the argument connection relation between the new and existing events by encoding new events into Python code and instantiating the class. By effectively lever- aging the complex event knowledge contained in LLMs, our approach enhances the event schema generation process. The details and examples refer to Appendix E.
g(G.|G.-1) IN..IQ..E..I (5)
-NNQ." EQ.")
Where Q: - QQQQ1 Qt The transition probability matrix Qi is defined as a'I + (1 a' d')1(1)²/KK where is a column vector of all ones, and a² varies from to O (Austin et al., 2021). This formulation ensures that the distribu- tion (G Go) consistent with uniform distribution when time / becomes sufficiently large.
Next, we sample the node and edge types from these probability distributions to obtain a discrete graph:
G. = =(=[((GG)G---))
(6)
The reverse diffusion process. We aim to re- move the noise from the graphs using a parameter- ized reverse process Po. Following the formulation presented by Austin et al.(2)))), we express the posterior po po GG Gi) as:
3.2
Event Skeleton Generation
In this section, will introduce the forward and reverse processes of discrete diffusion based on the instance graphs through Instance Graph Expansion module. We have adopted the diffusion framework Vignac et (202)) and made improvements based on it. Here, we denote the distribution G at
Po GG.
To predict the clean graph distribution G Po (G G.) at time given the noisy input G,, we

---

train graph transformer de that outputs the clean graph representation:
where Kn is the node type number, and Ke is edge type number. Before the next reverse process, we will get the discrete graph G--1 from its distribution by probability sampling.
G = (N, E.) (G.)
(8)
Our model @0 adopt transformer structure (Vaswani et al., 2017). Previous graph transformer model (Ying et al., 2021) is appropriate for encoding directed graphs based on time series, be- cause the relative position information between nodes is lost during the noise adding process. For instance, the self- -tttenntioo mechanism module in the transformer cannot differentiate two "transport" events that occur in different time periods. To ad- dress this issue, we encode the depth information of event nodes fixed-feature embedding Vedey into the model. Before inputting the graph into the transformer, we add the depth feature to the corresponding node feature.
Our model obtains the final event schema G through T-step reversing process in non-AR man- ner. For further algorithm and derivation details, please refer to Appendix C.
Conditional Generation. Previous approaches (Li et al., 2020, 2021; Jin et al., 2022) need to train separate models for each scenario to ensure accurate generation. However, order to generate event schemas for various scenarios using single model and improve the model's 's generalization ca- pabilities, we also propose the conditional diffusion model named as KDMall
supplement.
We incorporate the category information y of the instance graphs as an additional attribute control the training process of the model (Ho and Salimans, 2022; Dhariwal and Nichol, 2021). This allows us to influence the category of the generated schema. The formulation is as follows:
The depth fixed-feature embedding is encoded as follows:
sin(wx- na), i 2k
Deee
cos(ux ma). i 2 +
(9)
where w/ 1 1/1000024/ma and nd is the average depth of node n, the index of the depth embed- ding.
Po(Gi-I/G = / (GG-i/GG Go)po(Go|Ge: y)
(15)
Therefore, we only need to encode the category information y into the neural network. We sim- ply concatenate it into temporal features, enabling the conditional diffusion model to generate event schemas of different categories:
Inspired by Vignac et (2022),, our transformer model comprises several layers, each of which con- sists of a self-attention module and feed-forward network. For layer I, the self-attention module takes input time features f, node features N edge features Ej. and updates their representation as follows:
G² = (N₂, E,) (o(().
(16)
Atl: E W. + E. O N/Wo(NEWx>"
(10)
3.3 Entity Relation Prediction
N MLP(HW... + Att.NWwv)
(11)
In this module, we have developed a simple ar- chitecture that combines a graph transformer for obtaining node representations with an MLP layer for relation prediction. This module takes the event skeleton, expanded with event argument roles, input and generates the complete event schema by predicting the relations between entities.
1 = MLP(t + Att.)
(12)
where @ denotes the pairwise multiplication. W..
W m: W ins
W te: Wo, WK. Wy are trainable pa-
rameters.
To optimize our model, we use the cross-entropy loss LCE weighted by X:
While previous models have primarily focused on entity types in the classification process, neglect- ing the significance of events and event roles, we address this limitation by artificially aggregating them together. We initialize the node features us- ing BERT model (Devlin et al., 2018). Specifically, for each event or entity node n., BERT (T4) rep- resents its type embedding encoded by BERT. For entity nodes, nf indicates the event node that entity node n; belongs to, and nł is fixed embedding representing the role played by entity node n; in
Lee(G, G) > C)(NN,
E) (13)
GED
Once obtain the clean graph distribution G
we can infer the node distribution pe(n4-1 (me) and edge distribution po(er-1le) using the equations:
pe pe(m-11/m1) =
Σ q(u man
(14)
pu(e:-1ler) = Σ g(e:-1le:, clee-nles.co)puceuue (coee()))))

---

event n The encoding formula of embeddings v is the same as that of equation 9.
adds the sampled edge into the schema graph. The procedure is repeated until FBS detects a cycle in the schema graph after adding a new edge.
ni = concar (K))) BERT(
(17)
Double Graph Auto-encoders Model (Double- GAE) (Jin et al., 2022); the state-of-the-an schema induction model which designs a variational di- rected acyclic graph auto-encoder to extract the event skeleton. Then it uses another GCN based auto-encoder to reconstruct entity-entity relations. Large Language Model (LLM) have strong un- derstanding and generation abilities, We ask the large language model (ChatGPT) to directly gener- ate the event schema and use it as the baseline.
Our transformer encoder is the same as the model used for Event Skeleton Generation, except that it lacks the time feature. The graph transformer outputs nu corresponding to the input n,, which is then passed to the MLP predictor.
The predicted relation type Tij of entity node n; and nj is then computed as:
= (L + ni. ú, + m.]))
(18)
It is worth noting that the classification problem is highly unbalanced. To address this issue, we set different weights for different categories of the loss function, defined as:
4.3 Evaluation Metrics
To evaluate the quality of the generated schema, we compare the schema with test instance graphs in terms of the following metrics to see how well the schema match real world instance graphs, the following evaluation metrics employed: (1) Event type match. calculate the Fl score between the event types present the schema graph and test instance graphs.
L.
(19)
endiity
where Piy denotes the true relation between entity and j and H.)) is scalar function that assigns balanced weights to different relationships, with each relationship corresponding to a specific value.
(2) Event sequence match. A good schema is able to track events through a timeline. we calculate the Fl score between the event sequences of length 2 or 3 present in the schema graph and the test instance graphs
4 Experiments
4.1
Datasets
We conduct experiments using the IED Schema Learning Corpus released by Li et al. (2021). The dataset utilizes the DARPA KAIROS ontology. The corpus specifically focuses on three sub-types of complex events related to Improvised Explosive Devices (IEDs): General-IED, IED, and Suicide-IED.
(3) Node/edge type distribution. we compare the Kullback Leibler (KL) divergence of the node and edge type distributions between the schema graph and each test instance graph.
(4) Event Argument Connection Match (CM), Complex event graph schema includes entities and their relations, representing how events are con- neeted through arguments. Because there is rious long tail issue with the data, calculate the macro Fl Fl-score for every pair of relationships between entities.
However, the test data in the corpus has data quality issues since it is also extracted through IE tools. To address this, we manually modify the test data, generating golden test event schemas based on the modified data. Additionally, to ensure the objective evaluation of our model's effectiveness, we record the test results using the original, unmod- ified data, which are provided in Appendix 7.
4.4 Overall Results
As shown in Table the result demonstrates the ef- fectiveness of KDM in capturing important events and their relationships. Specifically, our approach outperforms the baseline methods in terms of event sequence matching, particularly for longer path lengths (1=3).
4.2 Baselines
In this work, we compare the proposed event schema induction model with two baselines: Frequency-Based Sampling (FBS) model which constructs the event schema according to frequency distributions of temporal links in the training data. At each timestamp, FBS samples a pair of event types according to their frequency and
These improvements can be attributed to the discrete diffusion process employed in our model. This process allows our model to simultaneously predict the categories of all nodes and edges, mak- ing it well-suited for graph generation. Addition-

---

Event type Mat
Event sequence match
KL divergence
Model
CM
Dataset
1=2
1=3
Node-type
Edge-type
FBS
0.614
0.199
0.064
2.98
6.13
DoubleGAE
0.627
0.266
0.093
2.43
5.57
0.046
General-IED
LLM
0.520
0.176
0.041
2.72
5.84
KDM (ours)
0.704
0.380
0.181
2.32
4.53
0.185
FBS
0.650
0,198
0.065
1.86
5.85
DoubleGAE
0.654
0.285
0.107
2.12
5.71
0.044
Car-IED
LLM
0.515
0.150
0.031
2.70
6.34
KDM (ours)
0.701
0.395
0.207
1.91
4.16
0.176
FBS
0.626
0.210
0.061
2.11
5.59
DoubleGAE
0.624
0.272
0.096
2.19
5.33
0.046
Suicide-IED
LLM
0.493
0.174
0.053
2.75
5.65
KDM (ours)
0.713
0.462
0.268
1.91
3.45
0.176
Table I: Schema matching score (%) is calculated by checking the intersection of the induced schemas and the manually checked test schemas.
Event type Match(F1)
Event sequence match
we find that KDMall shows improved general-
Model
Dataset
I=2
I=3
ization capabilities and better understanding of event relationships, particularly in the "General-
KDM
0.704
0.380
0.181
General-IBD
KDMall
0.723
0.413
0.202
IED" scenario. Additionally, in other datasets, KD- Mall demonstrates comparable results to the model trained on single dataset, indicating the potential of our conditional generation process. The incorpo- ration diverse training data enables the model to learn common patterns and associations across dif- ferent scenarios, leading to improved performance and broader applicability (Sastry et al., 2023; Kim et al., 2022).
KDM
0.701
0.395
0,207
Car-IED
KDMall
0.703
0.389
0.198
KDM
0.714
0.462
0,268
Suicide-IED
KDMall
0.714
0.455
0.251
Table 2: Schema matching score (%) (6)bb by checking the in- tersection of induced schemas and manual checked test schemas. KDMall is the conditional diffusion model trained on three IED datasets.
4.6 Ablation Experiment
ally, the Transformer architecture leveraged in our model effectively utilizes global features through the self-attention mechanism, resulting in improved prediction accuracy.
To demonstrate the effectiveness of our approach, we conduct ablation studies on the "Suicide-IED dataset. (1) IGE Module Ablation Experiment: To prove the effectiveness of our approach, we conduct experiments as shown in Table 3. JSON is prevalent format for representing structured data. We encode the data in JSON format and instruct the LLMs perform expansion, while maintaining the rest of the process consistent with the Python prompt approach. As shown in the Ta- ble, the results obtained through the use of Python prompts are noticeably better than those achieved with JSON prompts. And after filtering, the event types generated by the Python prompt are signifi- cantly more numerous than those generated by the JSON prompt. This observation underscores the effectiveness of the Python prompt approach. (2) Diffusion Model Ablation Experiment: Ta- ble 4, comparing our KDM model with a variant
Furthermore, our model shows remarkable im- provements in the Connection Match evaluation, in- dicating the effectiveness our graph transformer model than GCN graph auto-encoder Double- GAE.
4.5 Conditional Generation Results
Building upon the aforementioned diffusion model, in order to improve the possibility of the model's generalization ability, we present an extension in the form of conditional diffusion model as sup- plement. This model enables the generation of event schemas for various scenarios using a single model.
As shown in Table 2, when comparing KD- Mall with our model trained on a specific-dataset,

---

Event type Match(FI)
Event sequence match(FI)
KL. divergence
Model
EN
1=2
I=3
Node-type
Edge-rype
without IGE
0.672
0.411
0.216
1.96
3.67
IGR with ISON prompt
0.685
0.418
0.221
1.94
3.89
IGE with Python prompt
0.713
0,462
0.268
1.91
3.45
9
Table 3: Results of different prompts for the IGE Module on Suicide-IED dataset. EN is the number of effective events generated by the LLM after filtering, which are used for Instance Graph Expansion.
Event type Match(F1)
Event sequence match(F1)
KL divergence Node-type
Model KDM
1=2
1=3
Edge-type
0.713
0.462
0.268
1.91
3,45
w/o depth
0.709
0.429
0.209
1.92
3.83
w/o IGE
0.672
0.411
0.216
1.96
3.67
Table 4: The diffusion model ablation experiment on Suicide-IED dataset. "wlo depth" denoted as trained the model without depth features in graph transformer; "w/o IGE" denoted as trained the model on the dataset without Instance Graph Expansion module.
that removes the Instance Graph Expansion mod- ule, Our model achieves a 1% increase node matching accuracy, proving the effectiveness of
MACRO
KDM
KDM w/o RE
KDM w/o Weight
KDM w/o IGE
Instance Graph Expansion module. Additionally, by incorporating depth information, we observe a notable 5.9% improvement in sequence match- ing. These results demonstrate that the inclusion of depth information enhances our model's ability capture the structural characteristics of graph, proving the effectiveness adding depth features. (3) Entity Predictor Ablation Experiment: In Figure 4, Compared to not setting weight hyperpa- rameters, our model achieves significant 5.57% improvement in the macro F1 index, demonstrat- ing that our weight scalar function significantly addresses the long-tail data problem. Moreover, when comparing the results of "w/o RE" and "w/o IGE", the improvements highlight the effective- ness of adding role and event features and Instance Graph Expansion module.
16.53
12.10
Figure 4: The entity predictor ablation experiment. "w/o RE" denoted as trained without fixed role embedding features and event embedding features; "w/o weight" de- noted as trained without hyperparameter weight; "w/o IGE" denoted as trained without Instance Graph Expan- sion module.
Iransport
Thangor
Detonate
Dte
START
Anest
Dvacuution
4.7 Case Study
(a) the event schema skeleton without Instance Graph Ex- pansion module
Figure 5, we observe that our Instance Graph Expansion module successfully generates a schema that encompasses broader range of events and ex- hibits more comprehensive temporal relationships within complex events. This outcome support the effectiveness of leveraging object-oriented coding distill knowledge from LLMs. Additionally, provide case study showcasing the diffusion pro- cess on the "Suicide-IED dataset in Figure 6 in Appendix.
Attack
Die
Traspori
Deconage
Imvustige
Evacuation
Intervationn
START
Injar
Attack
Arrest
Transport
(b) the event sehema skeleton with Instance Graph Expan- sion module
Figure 5: Case Study on event schema skeleton

---

5 Related Work
tiple diverse topics. We achieved the best results among multiple different evaluation indicators.
Event Schema Event schema induction is com- prehensive graphical pattern composed of temporal and multi hop argument relationships (Li et al., 2020; Jin et al., 2022). It actually a combina- tion of atomic schema induction (Chambers, 2013; Yuan et al., 2018; Du and Ji, 2022; Wang et al., 2021) and script learning (Rudinger et al., 2015; Granroth- Cranroth-Wilding and Clark, 2016; Weber et al.. 2018). Clearly, the event schema induction has broad application significance. For example, the event schema facilitates analysis and prediction of future events, aiding in the development of reac- tion plans for relevant scenarios (Li al., 2021; Dror et al., 2023; Pan et al., 2021). Event schemas can be used as guidance information in informa- tion extraction, which helps people understand the internal logic of events (Wen et al., 2021).
Limitations
We only consider the temporal relationship be- tween events here and do not consider the hier- archical structure of the event schema, which may result in not perfect event schemas generated by
Due to the limited availability of datasets, our conditional diffusion model KDMall has only un- dergone unified training and testing on three highly related explosive events, requiring more categories and quantities of data for the comprehensive ability testing of the model.
Ethics Statement
We use a discrete diffusion model to generate event skeletons and design an entity relationship predic- tor. At the same time, we have fully explored the potential rich knowledge in LLM for knowledge ex- pansion. Our work has improved the effectiveness of event schema induction, helping people better summarize the logic and ontology knowledge of events, making contributions to this field.
Diffusion models Diffusion model (Sohl- Dickstein et al., 2015; Ho et al., 2020) has achieved impressive results image, text and audio generation (Rombach et al., 2022; Shen al., 2023; Li ct al., 2022; Gong et al., 2022; Kong et al., 2020; Yuan et al., 2022). Recently, Vignac et al. (2022) have shown great potential in graph generation field. Previous graph diffusion models embedded graphs in a continuous space by adding Gaussian noise to the nodes and edges feature (Niu et al., 2020; Jo et al., 2022). However, this approach destroys the graph's sparsity and makes it hard capture the node connections (Vignac et al., 2022). Discrete diffusion model (Austin et al., 2021; Yang al., 2023; Vignac et al., 2022; Johnson et al., 2021) overcomes this problem by utilizing Markov process that can occur independently on each node or edge.
Acknowledgements
This work is supported by the National Key Re- search and Development Program of China (No. 2020AAA01004000), the National Natural Science Foundation of China (No. 62176257, 61976211). This work also supported by the Strategic Pri- ority Research Program of Chinese Academy Sciences (Grant No.XDA27020100 ), the Youth In- novation Promotion Association CAS, and Yunnan Provincial Major Science and Technology Special Plan Projects NN0.202202AD0800044,
6 Conclusion
In this work, we identify the limitations of previ- ous works and proposed a Knowledge-enriched dis- crete diffusion model. To enhance the quality and coherence of the generated schemas, we harness the potential and rich knowledge present in LLMs by utilizing them for Instance Graph Expansion. Our model leverages discrete diffusson process learn and generate event skeletons, while incor- porating an entity relationship predictor to predict the relationships between event arguments. Addi- tionally, we propose a conditional diffusion model with the purpose of generating schemas for mul-
References
Jacob Austin, Daniel Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems. 34: 7988 1-991117999.
Nathanael Chambers. 2013. Event schema induction with probabilistic entity-driven model. Proceed- ings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807, Seattle, Washington, USA. Association for Computa- tional Linguistics.

---

Nathanael Chambers and Dan Jurafsky. 2008. Unsuper- vised learning of narrative event chains. In Proceed- ings of ACI-08: HIT. pages 789-797, Columbus, Ohio. Association for Computational Linguisties.
Jonathan Ho and Tim Salimans. 2022. Classifier- arXiv preprint
free diffusion guidance. arXiv:2207.1255
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, 2021. Argmax flows and multinomial diffusion: Learning categorical dis- tributions. Advances in Neural Information Process-
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
ing Systems, 34:124 14:12454-22465.
Xiaomeng Jin, Manling Li, and Heng Ji. 2022. Event schema induction with double graph autoencoders. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- rional Linguistics: Human Language Technologies, pages 2013-2022 Seattle, United States. Association for Computational Linguistics.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lee, and Kristina Toutanova, 2018, Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint
Prafulla Dhariwal and Alexander Nichol, 2021. Diffu- sion models beat gans on image synthesis. Advances neurul information processing systems, 34:8780- 8794.
Jachyeong Jo. Scul Lee, and Sung Ju Hwang. 2022. Score-hased generative modeling of graphs via the system of stochastic differential equations. In Inter- national Conference on Machine Learning, pages 10362-10383, PMLR.
Rotem Dror, Haoyu Wang, and Dan Roth. 2023. Zero- shot on-the-fly fly event schema induction. In Findings of the Association for Computational Linguistics: EACL 2023, pages 705-725, Dubrovnik, Croatia. As- sociation for Computational Linguistics.
Daniel Johnson, Jacob Austin, Rianne van den Berg. and Daniel Tarlow. 2021. Beyond in-place corrup- tion: Insertion and deletion in denoising probabilistic models. arXiv preprint (xXiv:2107007655
Xinya Du and Heng Ji. 2022. Retrieval-augmeet generative question answering for event argument extraction. arXiv preprint arXiv:2211,07067
Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. 2022. Diffusionclip:: Text-guided diffusioo models for robust image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2426-2435.
Xinya Du, Zixuan Zhang. Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang. Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Han- nan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng. Qing Lyu. Charles Yu, Carl Edwards. Xiaomeng Jin, Yizhu Jiao, Ghaza- lch Kazeminejad, Zhenhailong Wang, Chris Callison- Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang. Martha Palmer, and Heng Ji. 2022. RESIN-11: Schema- guided event predic- tion for 11 newsworthy scenarios. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies: System Demonstra- tions, pages 54-63, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. 2020. Diffwave: A versatile dif- fusion model for audio synthesis. arXiv preprint arXiv:2009.088 09760.99997......
Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han, and Clare Voss. 2021. The future is not one-dimensional: Com- plex event schema induction by graph modeling for event prediction. arXiv preprint 1xXiv2204.06344
Manling Li, Qi Zeng. Ying Lin, Kyunghyun Cho. Heng Ji, Jonathan May, Nathanael Chambers. and Clare Voss. 2020. Connecting the dots: Event graph schema induction with path language modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 684-695, Online. Association for Computa- tional Linguistics.
Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. 2022. Diffuseq: Sequence to se- quence text generation with diffusion models. arXiv preprint arXiv:2210.08933.
Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori Hashimoto. 2022. Diffusion- Im improves controllable text generation. Advances in Neural Information Processing Systems, 35:4328- 4343.
Mark Granroth-Wilddng Wilding and Stephen Clark. 2016. What happens next? event prediction using composi- tional neural network model. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI'16 16, page 2727-2733. AAAI Press.
Kiem- Hieu Nguyen, Xavier Tannier, Olivier Ferret, and Romaric Besançon. 2015. Generative event schema induction with entity disambiguation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. De- noising diffusion probabilistic models. Advances neural information processing systems, 33:6840- 6851.

---

Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 197.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances neural information processing systems, 30.
Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. 2020. Permuta- tion invariant graph generation via score-based gen- crative modeling. In International Conference on Ar- sificial Intelligence and Statistics, pages 4474-4484. PMIRR
Clement Vignac, Igor Krawezuk, Antoine Siraudin, Bo- han Wang. Volkan Cevher, and Pascal Frossard. 2022. Digress: Discrete denoising diffusion for graph gen- eration. arXiv preprint arXiv:2209.11 14734.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models follow instruc- tions with human feedhack. Advances Neural Information Processing Systems, 15:27730-227744
Haitao Wang, Tong Zhu, Mingtao Wang, Guoliang Zhang, and Wenliang Chen. 2021. A prior informa- tion enhanced extraction framework for document- level financial event extraction. Data Intelligence, 3(3):460-476.
Xiao Wang. Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Yong Wei, Yaowei Wang. Yonghong Tian, and Wen Gao. 2023. Large-scale multi-modal pre-trained models: A comprehensive survey. Machine Intelligence Research, pages 1-36.
Jeff Z. Pan, Elspeth Edelstein, Patrik Bansky, and Adam Wyner. 2021. A knowledge graph based approach social science surveys. Data Intelligence, 3(4):477- 506.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. 2022. High- resolution image synthesis with latent diffusion mod- cls. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695.
Noah Weber, Niranjan Balasubramanian, and Nathanael Chambers. 2018, Event representations with tensor- based compositions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang. Xiaodong Yu, Alexander Dong, Zhenhailong Wang. Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís. Brian Chen. Susan Windisch Brown, Martha Palmer, Chris Callison- Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang. and Heng Ji. 2021. RESIN: A dockerized schema- guided cross-documed cross- moss-linguul cross- media in- formation extraction and event tracking system. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies: Demonstrations, pages 133-143, Online. Association for Computational Linguistics.
Rachel Rudinger, Pushpendre Rastogi. Francis Ferraro. and Benjamin Van Durme. 2015. Script iuction as language modeling. In Proceedings of the 2015 Con- ference on Empirical Methods in Natural Language Processing, pages 1681-1686, Lisbon, Portugal. As- sociation for Computational Linguistics.
Carl
Yong Rui, Vicente Ivan Sanchez Carmona, Mohsen Pourvali, Yun Xing, Wei-Wen Yi, Hui-Bin Ruan, and Yu Zhang. 2022. Knowledge mining: a cross- disciplinary survey. Machine Intelligence Research, 1942).991114.
Chandramouli Sastry, Sri Harsha Dumpala, and Sageev Oore. 2023. Training diffusion classifiers with de- noising assistance. arXiv preprint 0rXiv:2306.09192
Donghhao Yang, Jianwei Yu, Helin Wang, Wen Wang. Chao Weng, Yuexian Zou, and Dong Yu. 2023. Diffsound: Discrete diffusion model for text-to- sound generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Diffusion- ner: Boundary diffuusion for named entity recognition. arXiv preprint arXiv:2305. 13298.
C Ying. T Cai, S Luo, S Zheng, G Ke, D He, Y Shen, and TY Liu. 2021. Do transformers really perform bad for graph representation? arxiv 2021. arXiv preprint arXiv:2106.0523
Jascha Sohl-Diekstein, Eric Weiss, Niru Mah- eswaranathan, and Surya Ganguli. 2015. Deep un- supervised learning using nonequilibrium thermo- dynamics. In International conference on machine learning, pages
Hongyi Yuan, Zheng Yuan. Chuanqi Tan, Fei Huang. and Songfang Huang. 2022. Seqdiffuseq: Text dif- fusion with encoder-decoder transformers. arXiv preprint arXiv:2212. 10325.
PMLR.
Hugo Touvron, Thibaut Lavril. Gautier Izacard, Xavier Martinet, Manee-Aune Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Fric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint DXXi:2302.13971.
Quan Yuan, Xiang Ren, Wenqi He, Chao Zhang, Xinhe Geng, Lifu Huang, Heng Ji, Chin-Yeu Lin, and Ji- awei Han. 2018. Open-schema event profiling for massive news corpora. In Proceedings of the 27th ACM International Conference on Information and

---

Knowledge Management, CIKM 18, page 587-596, New York, NY, USA. Association for Computing Machinery.
A Data Preprocessing
In the data preprocessing stage. Firstly, for each complex event, we constructed an instance graph by merging corefcrential events entities. Isolated events were excluded from the instance graphs dur- ing the graph construction process.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang. Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303. 18223.
followed the cleaning strat- egy outlined in Jin et al. (2022). We deleted links with the same start and end types, as well as event-event links such as (DIE, INJURE). (ARRESTIAILDETANE ATTACK), (ENDPO- SITION, STARTPOSITION), (DEFEAT, EX- CHANGEBUYSELLI (SENTENCE, DIE), (END- POSITION, SENTENCE), and (THREATENCO- ERCE, RELEASEPAROLE) from the instance graphs. The maximum number of graph nodes m is set to 50.
Specifically,
B Training And Evaluation Details
In our event skeleton induction process, we utilize I 12-layer Transformer model. Additionally, employ 3-layer Transformer our entity relation predictor. To balance the trade-off between nodes and edges, we set to The learning rate is set to le-4, and the number of diffusion training epochs is set to 2500. The sealar function H (fig)) is set to 0.1 if riy indicates "No-Relation", otherwise the function is set to 0.9. We conduct evaluations using 500 randomly generated event schemas for each performance metric. The node number is sampled from a range of 25 to 35. We choose the model checkpoint from the last epoch for evaluation.
In the Instance Graph Expansion process, we select the top 10 frequently occurring event se- quences from the training data as inputs for Chat- GPT. Each event sequence is input to ChatGPT 10 times to obtain the final result. Furthermore, we use a hyperparameter K of to filter out events generated by ChatGPT that oceur less frequently. In the Entity Relation Prediction module, each event has a predetermined set of argument roles. For example, the "Injure" event may have the ar- gument role "Victim" limited to entity types "PER" and "AML". We count the occurrences of entity categories for each role in all instance graphs. The entity category with the highest occurrence in the corresponding role is then inserted into the event skeleton.
To modify the test data, we made the following modifications: I. Merge the same path: For all subsequent nodes of each event node, merge event

---

nodes with the same type, starting from the START node and merging in the order of the BFS algorithm. Supplementary event nodes: Based on human judgment, randomly add possible missing events that may occur in the schema.
2021):
((22/or 1, 20 1120)
(21111)21,00)
xz xQ((i) o 00 To (z+)
Conditional Discrete Diffusion Model
To train the discrete diffusion process, mini- mize the negative logarithmic likelihood of the pre- dicted distribution of the model using variational lower bound (VLB), We use Gg here represent G:
Transition probability matrix in forward pro- cess. In discrete diffusion model, transition probability matrix Q is defined to corrupt data for each step. Here Q: a'I + (1 (1)((1)) /K, where 1 is column vector of all ones, a' varies from to 0 making sure the node and edge fea- ture sampled from is a uniform distribution time (Hoogeboom et al., 2021; Yang et al., 2023). 3 = (1 /²)/ / K and the transition matrix can be represent as:
C = - EatGai Logpo (Ga)]
S LVLB
= EalGaaa flog (Go)
= ExGrr) ( Gu) (r
Σ D Go) l (o(
Tath
gr
gr
gt
B
a²
+
ge
ge
Pog
Q: =
EgGor [Dz(q pe G.))
8
gr
a' + 18
loggo(G.., Ga)
can calculate ((<;)<0)) according to follow-
ing formula:
DRaloGG-- Go) (o(G-1
2o Q: = a xo 3
(20)
log po (G
as / is enough large, a' is close to 0, the graph distribution G3 is confirm to uniform distribution. Reverse discrete diffuuson process, we convert the noise GT into G, whose joint probability hav- ing a Markovian structure follows (Vignac et al., 2022):
Please note that G, is sampled from the node
number distribution G,, and the corresponding depth distribution G.. Therefore, the proba- bility pe neGTT can be expressed pa pGr) Po ( Ga) (G,,, Ga).
The terms Lr and Lt-1 represent the Kullback- Leibler (KL) divergences between graph categori- cal distributions, while Lo represent the predicted probabilities of the graph Go based on the noisy graph G1. The algorithms and 2 is the training and generating algorithms about KDM.
T
log PoppooGrrr = log (Gr) log po (G (.)))
where po is the process of the reverse with learnable parameters f. and for each discrete elements x in graph Go.r posterior probability is
D Supplement Experiment
po(2z-1/21)
Σ
As presented in Table 5, we evaluate our model the original testing data used by (Jin al., 2022) and observe consistent outperformance of our model compared to the baselines specially in sequence match. This result highlights the strong capability of our discrete diffusion model in gener- ating high-quality event schemas. Interestingly, we
Σ a 2o 2000 100
20
The posterior q(It-1/20, 2o) can be derived accord- ing to Bayes formula as follows (Austin et al.,

---

that models without knowledge expansion ex- hibited better performance in this evaluation. This finding also suggests that the original test data may not effectively measure the quality of the generated schemas.
Algorithm 1: Training KDM
1 for G in
2 sample t from Uniform(T);
3 sample G, from distribution
(NQ) EQI);
estimate distribution G. = áo
5 calculate loss Lo p G);
update network Φ9
7 end
(a) 500
Algorithm 2: Sampling from KDM
1 sample Gr from Uniform distribution;
2 for / Too / do
3 convert Gr to distribution G
G &g (Gt);
5 estimate distribution po G; G/G.);
6 sample G from distribution;
end
E Instance Graph Expansion Details
(b) [=250
The process of the Instance Graph Expansion is shown in Figure 7, we also present our event knowl- edge expansion prompt in Figure 8, temporal lation expansion prompt in Figure 10 and entity relation expansion prompt in Figure 9.
(c)
Figure 6: Case Study diffusion process. During the reverse process, gradually changes from 500 and the corresponding schema for at different times corresponds to the above.

---

Event type Match(F1)
Event sequence match(F1)
KL. divergence
Model
Dataset
|=2
1=3
Node-type
Edge-type
FBS
0.617
0.149
0.064
1.88
4.32
DoubleGAE
0.697
0.273
0.128
1.66
4.96
General-IED
KDM
0.663
0.318
0.132
3.01
5.08
KDM w/ IGE
0.698
0.327
0.138
2.52
4.87
FBS
0,542
0.126
0,038
4.12
6,37
DoubleGAR
0.674
0.259
0.081
2.14
5.42
Car-IED
KDM
0,702
0,371
0,180
2.07
4,47
KDM w/o IGE
0.757
0.411
0.185
1.42
4.24
FBS
0.642
0.164
0.048
2.39
6.36
DoubleGAE
0.709
0.290
0.095
1.76
5.91
Suicide-IED
KDM
0.688
0.415
0.217
2.13
3.75
KDM w/ IGE
0.738
0.467
0.268
1.61
3.50
Table 5: Schema matching score (%) by checking the intersection of induced schemas and test schemas. both the training and testing instances are not through Knowledge Expansion. The baseline's results are provided by previous work.
(Attack, Detonarekxploaee Injure)
Filter
class Atterk(eevnn):
complex bombing scenario, 'Attcck, DetonateExplooe! 'Ijuure' happoned in time order, what the most 'evecuation' in this scesario:
closs Altack(even):
def (self_ sttackr,, target, instrument,
placey:
class Exsooaaeelllldeeeeeent
sell target
cless Injure(event):
self.instrmmet insenament
A:Artack
place = place
B-DetonateExplode
class evacsatiioneevenl)
Cinjuure
Jetomattdxploob(eeennt
possible python
bombing event
relation between your choice and
sceaaro we know is above, please enrich the instantiating class relation between class
class injure(event):
lvvccaaion in bombing
secnario:
Astemporal; eevacuation' happereed
Attack, Detonnttexppilode Injure and evacuation, you should only generate instantlate above class.
some python
hombing event
secnario we know is above, please enrich the demo
Estemporal: levacuationl happened
'named entity' you should not repeat the python demo above!
yython
Cattribute
ChatGPT
class Rescue(event):
DetonateExplode
sell rescure rescuer
(self, rescuer, victim, location):
DetonateExplode
anacker
attacker
Injure
Instrument
selEvictim vizim
Injure
selt location locationclass
Amtak
Largett
Attaek
Evacuation
Evacuntioneeven):
Evacuation
doggeooation
method
Anjure
def Init [selt, location, methody:
selElocation location
sellmethod method
event knowledge expansion
temporal relationship expansion
entity relationship expansion
Figure 7: This figure illustrates the Instance Graph Expansion process. The event knowledge expansion process write selected event sequences into Python code and complete them by ChatGPT to generate new events: The temporal relationship expansion process utilize choice question link new event to sequence; The entity relationship expansion process instantize the event class to get argument relations.

---

Input:
class Attack(event):
def Init (self, attacker, target, instrument, place):
self.attacker attacker
self.target = target
self.instrument instrument
self.place place
class DetomnttExxloooleveen):
def _init ____iii_____,,.______________ attacker, target, instrument, explosive_device place):
self.attacker attacker
self.target = target
delf.instrmmen- instrument
self. enffeexplsivee...mm device explosive device
self,place place
class Injure(event):
def Def_init__ [_e_ii victim, injurer, Instrument):
self.victim victim
self.injurer = injurer
self.instrmmnt= instrument
some python demo model bombing event scenario we know is above, please enrich the demo with new 'named entity' and 'event' python class. should not repeat the python demo abovelyou should not repeat the python demo abovelyou should not repeat the python demo above!
Answer:
class Bombing(event):
def init ineitnseef,,,..._._.illlliii attacker, target, instrument, place, casualties):
self.attacker attacker
self.target = target
self. self.instrmmntt instrument
self,place place
self.casmalties casualties
class Evacuation(event);
def _init [self location, evacuees, reason):
self.location = location
eifeevaceees evacuees
self.reason reason
class Investigation(event)
def L_____nnt_______ location, investigators, date):
self.location location
self intestigttiissss investigators
self.start date start date
Figure 8: Instance of event knowledge expansion process
Input:
a complex bombing event scenario, 'Attack, FDeronateExplode' 'Injure' happened in time order, what is the most relative choice class 'evacuation' in this scenario, please choose your answer from:
A:Attack
B:DeeonateExplode
C:Injure
then choose what the possible relation between your choice and 'evacuuttonn' bombing scenario, please choose your from:
Astemporal; 'evaacuation' happened first
Rstemporal; 'evacuation' happened after
Cattribute
(attention you can only answer the above two questions only with
Answer:
B: DetonateExplode
B: Temporal; "evacuation' happened after
Figure 9: Instance of temporal relation expansion process

---

Input:
class Attack(event):
def init__sell attacker, target, instrument, place):
self.attacker attacker
self.target target
self.instrument instrument
self.place = place
class DetonnareExpoodeeevent]:
def [self__sselff attackes, target, instrument, explosive_devicc, place):
self.attacker attacker
self.target target
self.instrument instrument
self.place place
class Injure(event):
def (self, victim, injurer, instrument):
self.victim
self.injurer injurer
self.instrument instrument
class Evacuation(event)):
def (self, location, method):
self.location location
self.method method
some python demo model hombiig event scenario we know is above, please enrich the demo by instantlating the class to explain the relation between class Deoncc and Bombing, you should only generate demo to instantlate the above class.
one demo example is below:
Instantiate Transport event
transport cersport_eeenttt event Transport
(ransport(Vchcllee Vehicle -"car", Origin ""purk", Destination -BBuilding)
Instantiate DetonateExplode event
detonate_explodee
explode_eveed event DetommeExploakkataakkeemmmmma target="Building, instrumen=="xxpllssvv"""
explosive device="C4", place="City Center")
Instantiate Investigate event
investigate event Invessiaate(laattion""CCig Center", invettigato=="eeectiiee Smith")
Answer:
Instantiate Attack event
attack_event Attack(atacker="]oll "Jhhn, turget=""illiing" "isposment"" "Cace="City Center")
Instantlate DetonareFxplon event
detonate_ese
aetooate_explode_cco explode_event Detonarexxpooo@@ttoee""""" target="Building, instrument="Explosvvee,
explosive device="C44 place="Citt Center")
Instantiate Injure event
injure_ed event eeentt Injure(victim=
injurer= John", instrument= instrument""xxplosive"
Instantiate Evacuation event
evacuation_eveee event Evacuation(lccation="Ciy Center", method=" method=="mergenncy evacuation")
Figure 10: Instance of entity relation expansion process


Reviews:
[{"review_id": "tJpOPq9Cp8", "reviewer": "Reviewer_XvjL", "summary": "", "questions": "Question A: For Reject 1, why choose code style to build prompt?\nQuestion B: What is the experimental result of Connection Match evaluation in table 1?\nQuestion C: In the experiments, why was there no direct comparison with the predicted results of LLM?\nQuestion D: Has there been any filtering for biases and sensitive information that may arise from large models?\nQuestion E: In the diffusion model, how do you choose the initial random seed, which is very important for the results?\nQuestion F: The experimental results of the generative methods are usually very variable. Is there any exploration of the robustness of the model?\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your guidance and suggestions. We will use these as a foundation to further enhance our work. Below is a point-by-point response to your feedback.\n\n**Response to Rejection 1: Why did the paper choose the Python-style object-oriented prompt for the IGE module without providing reasoning or experimental validation?**\n\nApologize for not explaining the reasons and experiments behind our use of the Python style object-oriented prompt in the IGE module. Based on your advice, we've added the following explanations:\n\nComplex event schemas involve intricate graph structures, while LLMs are good at processing unstructured language tasks. To retain structured information of instance graphs, we need LLMs to be able to handle structured inputs and outputs. Considering the powerful coding capabilities of LLMs, we treat events as Python objects. In detail, events, entities, and their intricate relations can correspond to classes, attributes, and instances in the object-oriented paradigm, respectively. To prove the effectiveness of our approach, we conduct experiments as shown in Table.\n\n\n\n| IGE module             | Event type match(F1) | Event sequence match(l=2) | Event sequence match(l=3) | KL divergence(Node type) | KL divergence(Edge type) | EN    |\n| :--------------------- | -------------------- | ------------------------- | ------------------------- | ------------------------ | ------------------------ | ----- |\n| without IGE            | 0.672                | 0.411                     | 0.216                     | 1.96                     | 3.67                     | -     |\n| IGE with JSON prompt   | 0.685                | 0.418                     | 0.221                     | 1.94                     | 3.89                     | 2     |\n| IGE with Python prompt | **0.713**            | **0.462**                 | **0.268**                 | **1.91**                 | **3.45**                 | **9** |\n\n\n\n\nThe table express the results of different prompts for the IGE Module on Suicide-IED dataset. **EN** is the number of effective events generated by the LLM after filtering, which are used for instance graph expansion.\n\nJSON is a prevalent format for representing structured data. We encode the data in JSON format and instruct the LLMs to perform expansion, while maintaining the rest of the process consistent with the Python prompt approach. As shown in the Table, the results obtained through the use of Python prompts are noticeably better than those achieved with JSON prompts. And after filtering, the event types generated by the Python prompt are significantly more numerous than those generated by the JSON prompt. This observation underscores the effectiveness of the Python prompt approach.\n\n**Response to Rejection 2: What are the missing baselines like LLM direct predictions?  The ablation experiment about IGE and ESG modules is insufficient**\n\n1. Given the unsatisfactory outcomes of using LLMs directly for the task, we opted not to establish it as our baseline. As elaborated in the previous response, we recognize that LLMs struggle with processing structured graph data. To facilitate a more balanced comparison, we add the LLMs' results, as presented in the table below.\n\n   | Dataset     | Model     | Event type match(F1) | Event sequence match(l=2) | Event sequence match(l=3) | KL divergence(Edge type) | KL divergence(Edge type) | CM        |\n   | ----------- | --------- | -------------------- | ------------------------- | ------------------------- | ------------------------ | ------------------------ | --------- |\n   | General-IED | FBS       | 0.614                | 0.199                     | 0.064                     | 2.98                     | 6.13                     | -         |\n   | General-IED | DoubleGAE | 0.627                | 0.266                     | 0.093                     | 2.72                     | 5.84                     | 0.046     |\n   | General-IED | LLM       | 0.520                | 0.176                     | 0.041                     | 2.72                     | 5.84                     | -         |\n   | General-IED | KDM(ours) | **0.704(0.004)**     | **0.380(0.002)**          | **0.181(0.003)**          | **2.32(0.000)**          | **4.53(0.007)**          | **0.185** |\n   | Car-IED     | FBS       | 0.650                | 0.198                     | 0.065                     | 1.86                     | 5.85                     | -         |\n   | Car-IED     | DoubleGAE | 0.654                | 0.285                     | 0.107                     | 2.70                     | 5.71                     | 0.044     |\n   | Car-IED     | LLM       | 0.515                | 0.150                     | 0.031                     | 2.70                     | 6.34                     | -         |\n   | Car-IED     | KDM(ours) | **0.701(0.000)**     | **0.395(0.001)**          | **0.297(0.000)**          | **1.91(0.000)**          | **4.16(0.002)**          | **0.176** |\n   | General-IED | FBS       | 0.626                | 0.210                     | 0.061                     | 2.11                     | 5.59                     | -         |\n   | General-IED | DoubleGAE | 0.624                | 0.210                     | 0.096                     | 2.19                     | 5.33                     | 0.046     |\n   | General-IED | LLM       | 0.493                | 0.174                     | 0.053                     | 2.75                     | 5.65                     | -         |\n   | General-IED | KDM(ours) | **0.713(0.000)**     | **0.462(0.001)**          | **0.268(0.000)**          | **1.91(0.000)**          | **3.45(0.000)**          | **0.176** |\n\n\nIn the table, schema matching score (\\%) is calculated by checking the intersection of the induced schemas and the manually checked test schemas. The variance of our experiments on three different random number seed is in the brackets.\n\n\n2.\tAs illustrated in Figure 7, the IGE module involves a sequential integration of event knowledge expansion, temporal relation expansion, and entity relation expansion. Each subsequent module builds upon the foundation of the preceding one, culminating in the acquisition of a comprehensive and usable event knowledge base. Thus every module is indispensable, rendering it unfeasible to conduct ablation experiments. \n\n**Response to Rejection 3: provide more detailed hyperparameters and improve reproducibility**\n\nActually, we provide an in-depth exposition of all our implementation details and hyperparameters in Appendix B (i.e., between Line 828 and Line 857). Simultaneously, we are committed to open-sourcing our code for reproducibility and further research.\n\n**Response to Question A: why choose code style to build prompt?**\n\nPlease see the response to Rejection 1.\n\n**Response to Question B: What is the experimental result of Connection Match evaluation in table 1?**\n\nActually, we have elucidated the definition of \"CM\" between Line 438 and Line 444.\n\n**Response to Question C:  In the experiments, why was there no direct comparison with the predicted results of LLM?**\n\nPlease see the response to Rejection 2.\n\n**Response to Question D:  Has there been any filtering for biases and sensitive information that may arise from large models?**\n\nIn our experiments, we do not encounter any biases or sensitive information. Therefore, we do not apply filtering process.\n\n**Response to Question E: In the diffusion model, how do you choose the initial random seed, which is very important for the results?**\n\nIn our experiments, we utilize three distinct sets of random seeds, all of which ultimately yield converged diffusion models. To underscore the stability of the model, we supplement the experimental variance, as depicted in the Table above. From the result, we can observe that the impact of random seed on the results is relatively minor. \n\n**Response to Question F: The experimental results of the generative methods are usually very variable. Is there any exploration of the robustness of the model?**\n\nPlease see the response to Question E.\n\n**Response to Typos Grammar Style And Presentation Improvements**\n\nThank you very much for pointing out the spelling, grammar, and expression issues. We will correct these errors in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper utilizes the LLMs to enhance the knowledge coverage of instance graphs, which is simple and effective. \n- With the help of the ability of large models and generative methods, this article seems to make good use of the advantages of both.\n- This paper is well structured, but some details are still worth improving.\n", "reasons_to_reject": "- This paper proposes some effective techniques but not provides why choose them, such as IGE module use a Python style object-oriented prompt to extract knowledge from LLMs, but this is not the only way, obviously. So why the authors choses this way needed to be explained or be proved it experimentally?\n-  The experimental setting lacks some important baselines, such as LLM direct prediction results; The ablation experiment is insufficient, and the important submodules in the IGE and ESG modules are not explored, such as: event knowledge expansion, temporal relation expansion, and entity relation expansion, et al.\n- The details of the experiment are not detailed enough, some hyperparameters cannot find the corresponding settings, and the reproducibility is weak.\n", "questions_to_authors": "Question A: For Reject 1, why choose code style to build prompt?\nQuestion B: What is the experimental result of Connection Match evaluation in table 1?\nQuestion C: In the experiments, why was there no direct comparison with the predicted results of LLM?\nQuestion D: Has there been any filtering for biases and sensitive information that may arise from large models?\nQuestion E: In the diffusion model, how do you choose the initial random seed, which is very important for the results?\nQuestion F: The experimental results of the generative methods are usually very variable. Is there any exploration of the robustness of the model?\n", "paper_topic_and_main_contributions": "To address the complex event schema induction task, this paper leverages the LLMs to inject knowledge into instance graphs to improve the knowledge coverage of instance graphs, and utilizes discrete diffusion model to tackle error propagation of auto-regressive decoding. Experimental result indicates the proposed KDM outperforms two baselines.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "-\tIn section 4.4, the data lists 4 evaluation metrics, but only discusses sequence event match and connection match evaluation. Entity type match and KL-div of nodes and relations should also be discussed appropriately.\n-\tLine 492: In Table 3, comparing our KDM model with a variant that removes … There are two variants in the table, which do not seem to correspond.\n-\tLine 502: There is a grammar error.\n-\tIn Section 4.6, Entity Predictor Ablation Experiment doesn't seem to be very important to explain the contribution of the paper.\n"}}, {"review_id": "4wE4P06lFE", "reviewer": "Reviewer_WC1S", "summary": "", "questions": "Have you designed any experiment to inspect and validate auto-generated codes from LLM?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful and insightful reviews. We will utilize these insights as a foundational framework to augment and refine our works. Here, we've outlined a detailed response addressing each of your feedback points:\n\n**Response to Rejection 1: Authors need to elaborate their ablation study regarding instance graph expansion module. The article did not tackle the problem of reliability and robustness of such auto-generated code snippets from LLMs.**\n\nI apologize for not adequately conveying the details and innovative aspects of our IGE (instance graph expansion) module. Based on your advice, we've added the following explanation:\n\nComplex event schemas involve intricate graph structures and structured knowledge representation. LLMs are good at processing unstructured language tasks, but there isn't a well-established method to make structured graph schema understandable to LLMs. When we had LLMs do IGE directly, they generated extensive unstructured natural language text. While this text contain knowledge, converting it into usable structured knowledge that we can use is challenging. To solve this problem we need LLMS to be able to handle structured inputs and outputs. \n\nTo retain structured instance graph information and use LLMs' coding power, we treated events as Python objects. We recognize that events, entities, and their intricate relationships can correspond, respectively, to classes, attributes, and instances in the object-oriented paradigm. \n\nTo better elucidate the reliability of the LLM-generated code approach, we devised experiments as illustrated in the table below.\n\n\n\n| IGE module             | Event type match(F1) | Event sequence match(l=2) | Event sequence match(l=3) | KL divergence(Node type) | KL divergence(Edge type) | EN    |\n| ---------------------- | -------------------- | ------------------------- | ------------------------- | ------------------------ | ------------------------ | ----- |\n| without IGE            | 0.672                | 0.411                     | 0.216                     | 1.96                     | 3.67                     | -     |\n| IGE with JSON prompt   | 0.685                | 0.418                     | 0.221                     | 1.94                     | 3.89                     | 2     |\n| IGE with Python prompt | **0.713**            | **0.462**                 | **0.268**                 | **1.91**                 | **3.45**                 | **9** |\n\nThe table express the results of different prompts for the IGE Module on Suicide-IED dataset. **EN** is the number of effective events generated by the LLM after filtering, which are used for instance graph expansion.\n\nJSON is a prevalent format for representing structured data. We encoded the data in JSON format and instructed the large model to perform expansion, while maintaining the rest of the process consistent with the Python prompt approach. \n\nAs shown in the Table, the results obtained through the use of Python prompts are noticeably better than those achieved with JSON prompts. And after filtering, the event types generated by the Python prompt are significantly more numerous than those generated by the JSON prompt. This observation underscores the effectiveness of the Python prompt approach.\n\nTo ensure the robustness of the generated code, as mentioned in line 226 of the text: \"We filter out new events that occur less frequently than a hyperparameter K and are not in the predefined event category.\" This is done to ensure that the event knowledge extracted from LLMs is grounded in common sense and remains stable. \n\nPertaining to the ablation experiments conducted within the IGE module, as illustrated in Figure 7 within the text, our approach in the IGE module involves a sequential integration of event knowledge expansion, temporal relation expansion, and entity relation expansion. Each subsequent module builds upon the foundation of the preceding one, culminating in the acquisition of a comprehensive and usable event knowledge base. Every module is indispensable, rendering it unfeasible to conduct ablation experiments.\n\n**Response to Rejection 2: Authors' experiments were conducted on just one specific event scenario.**\n\nThanks for your suggestion. Following previous studies, we utilize the same dataset and evaluation metrics for fair comparison. Since no other datasets related to this task is available, we cannot explore model assessment in another scenarios. We acknowledge the importance of comprehensively evaluating our model across different datasets and plan to do so in the future.\n\n**Response to Question 1: Have you designed any experiment to inspect and validate auto-generated codes from LLM?**\n\nPlease see the response to Rejection 1.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The motivation and contribution of the paper is clear.\n\nVarious experiments and analysis are carried out on a hot topic.", "reasons_to_reject": "Authors need to elaborate their ablation study regarding instance graph expansion module. Even though authors presented some discussions in table 3 and in section 4.7, they did not tackle the problem of reliability and robustness of such auto-generated code snippets from LLMs.\n\nAuthors' experiments were conducted on just one specific event scenario. They should have evaluated their methods on a set of different event scenario such as disease outbreak, coup, riots or any others. It's hard to say the proposed method is superior to the existing methods, seeing the performance on a single scenario.\n", "questions_to_authors": "Have you designed any experiment to inspect and validate auto-generated codes from LLM?", "paper_topic_and_main_contributions": "The motivation the paper is to overcome the challenges such as error propagation and data quality issues in event schema induction. Existing methods just induce the atomic schema without considering correlations between events and struggle with extracting complete instance information.\n\nThe paper proposes a knowledge-enriched discrete diffusion model for complex event schema induction task. They mainly propose an instance graph expansion module to improve knowledge coverage of instance graphs and an event skeleton generation module which predict all nodes and links altogether.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KwTlbCLLhI", "reviewer": "Reviewer_VM43", "summary": "", "questions": "1. What different ChatGPT prompting formats were explored, and why is the adopted prompting better?\n2. What are the sizes of the datasets in Sec 4?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful and insightful reviews.\n\n**Response to Rejection 1: Problem formulation (Sec 2) is confusing and unclear.Sec 2.2 is really the problem formulation but this is not at all sufficient. It does not formulate the schema induction problem, nor define the concepts involved.....**\n\nThank you very much for your suggestions. We will make adjustments to the placement of the definition section of the diffusion model. \nAs for the Problem formulation, in lines 180 to 196 of our paper, particularly in lines 193 to 196, we provided a formal and conceptual definition of our problem.\n\n**Response to Rejection 2: -   Sec 4 Experiments need lot more details, eg. what are the sizes of the datasets. Also, given that 'we manually modify the test data, generating golden text event schemas based on the modified data', the unmodified results should also be included......**\n\nI apologize that we do not provide detailed information about the dataset. The training data sizes in the IED dataset are as follows: suicide\\_ied:105, car\\_bombing\\_ied:38, general\\_ied:59.\n\nIn the main text (i.e., Section 4), we conduct experiments using modified data. To ensure a fair and comprehensive comparison, we include supplementary experiments in the Appendix D. The results of evaluating on unmodified data are presented in Table 4, and the analysis of these results is provided between Line 904 and Line 916.\n\nWe will make our modified data and the source code publicly available for reproducibility and further research.\n\n**Response to Rejection 3: the metrics are defined rather informally. In addition to F1-measure, I would like to see precision and recall numbers, to check if there is any tradeoffs.**\n\nI apologize for the oversight. Actually, our evaluation metrics are in line with previous studies. Due to space limitations, precision and recall are not included in the presentation of our experimental results. However, we have provided a more comprehensive set of outcomes, as illustrated in the Table below.\n\n| Dataset     | Event type match prec | Event type match rec | seq match(l=2) prec | seq match(l=2) rec | seq match(l=3) prec | seq match(l=3) rec |\n| ----------- | --------------------- | -------------------- | ------------------- | ------------------ | ------------------- | ------------------ |\n| General-IED | 0.738                 | 0.739                | 0.449               | 0.438              | 0.233               | 0.228              |\n| Car-IED     | 0.681                 | 0.751                | 0.374               | 0.452              | 0.193               | 0.267              |\n| Suicide-IED | 0.703                 | 0.765                | 0.462               | 0.531              | 0.279               | 0.350              |\n\nWe observe that our model's recall is higher than the precision. We analyze that there could be several reasons for this disparity:\n\n(1) The discrete diffusion model relies on probability sampling, which introduces a chance of selecting nodes and edges with relatively lower probabilities. This phenomenon can lead to a higher recall, as it encompasses a broader range of possibilities.\n\n(2) The training dataset includes a more diverse set of sample instances, whereas the testing dataset is comparatively limited. Consequently, certain scenarios present in the training phase might not be adequately represented during testing, contributing to the observed difference.\n\n**Response to Rejection 4:  ChatGPT prompting methodology is not explained well. What different formats were explored, and why is the adopted prompting better?**\n\nI apologize for not adequately conveying the details and innovative aspects of our IGE module.\n\nComplex event schemas involve intricate graph structures and structured knowledge representation. LLMs are good at processing unstructured language tasks, but there isn't a well-established method to make structured graph schema understandable to LLMs. When we had LLMs do IGE directly, they generated extensive unstructured natural language text. While this text contain knowledge, converting it into usable structured knowledge that we can use is challenging. To solve this problem we need LLMS to be able to handle structured inputs and outputs. \n\nTo retain structured instance graph information and use LLMs' coding power, we treated events as Python objects. We recognize that events, entities, and their intricate relationships can correspond, respectively, to classes, attributes, and instances in the object-oriented paradigm. To prove our approach works, we designed comparative experiments as shown in the table below.\n\n\n| IGE module             | Event type match(F1) | Event sequence match(l=2) | Event sequence match(l=3) | KL divergence(Node type) | KL divergence(Edge type) | EN    |\n| :--------------------- | -------------------- | ------------------------- | ------------------------- | ------------------------ | ------------------------ | ----- |\n| without IGE            | 0.672                | 0.411                     | 0.216                     | 1.96                     | 3.67                     | -     |\n| IGE with JSON prompt   | 0.685                | 0.418                     | 0.221                     | 1.94                     | 3.89                     | 2     |\n| IGE with Python prompt | **0.713**            | **0.462**                 | **0.268**                 | **1.91**                 | **3.45**                 | **9** |\n\n\n\nThe table express the results of different prompts for the IGE Module on Suicide-IED dataset. **EN** is the number of effective events generated by the LLM after filtering, which are used for instance graph expansion.\n\nJSON is a prevalent format for representing structured data. We encoded the data in JSON format and instructed the large model to perform expansion, while maintaining the rest of the process consistent with the Python prompt approach. As shown in the Table, the results obtained through the use of Python prompts are noticeably better than those achieved with JSON prompts. And after filtering, the event types generated by the Python prompt are significantly more numerous than those generated by the JSON prompt. This observation underscores the effectiveness of the Python prompt approach.\n\n**Response to Rejection 5:  Sec 4.7 case study is too brief to be worthwhile.**\n\nThe \"Sec 4.7 Case Study\" is included to showcase the effectiveness of our IGE module. This segment aim to provide a more intuitive illustration of how event schemas that undergo knowledge expansion through LLMs are not only more efficient but also more refined, thereby emphasizing the validity of our IGE module.\n\n**Response to Rejection 6: I think there's too much space devoted for covering the theory behind diffusion model, but much less space for explaining the main problem statement, technical contributions and evaluation details.**\n\nRecognizing that the diffusion model serves as a fundamental pillar of our approach, we acknowledge the significance of clarifying its operational mechanisms to enhance the understanding of our contributions and innovations.  Accordingly, we have allocated a substantial portion of our work to comprehensively expound on the theory underpinning the diffusion model, and this content is mainly housed within the appendix.\n\nWe have delineated the principal challenges in Introduction sections spanning Line 70 to Line 87 and Line 88 to Line 107, while also providing a more visually intuitive representation of the task through Figure 1.  A concise encapsulation of our technical contributions can be found within Line 135 to Line 150.  Furthermore, detailed elucidation of training and evaluation specifics has been presented in Line 420 to Line 444 and Line 828 to Line 857. \n\n**Response to Question A: What different ChatGPT prompting formats were explored, and why is the adopted prompting better?**\n\nPlease see the response to Rejection 4.\n\n**Response to Question B: What are the sizes of the datasets in Sec 4?**\n\nPlease see the response to Rejection 2.\n\n**Response to Typos Grammar Style And Presentation Improvements**\n\nThank you very much for pointing out the spelling, grammar, and expression issues. We will correct these errors in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Complex event schema induction is an important problem.  \n- Specific ideas to address key issues with existing approaches, and propose an improved method. \n- Better performance reported in empirical studies. ", "reasons_to_reject": "Though the proposed method has been demonstrated empirically, I think the paper is not very sound.\n\n- Problem formulation (Sec 2) is confusing and unclear. It starts by defining the diffusion model which I think should come later. Sec 2.2 is really the problem formulation but this is not at all sufficient. It does not formulate  the schema induction problem, nor define the concepts involved. Without this, I think it is difficult to understand the contributions in the rest of the paper. \n- Sec 4 Experiments need lot more details, eg. what are the sizes of the datasets. Also, given that 'we manually modify the test data, generating golden text event schemas based on the modified data', the unmodified results should also be included. Also I think the modified data should be released publicly. \n- In Sec 4.3, the metrics are defined rather informally. In addition to F1-measure, I would like to see precision and recall numbers, to check if there is any tradeoffs. \n- ChatGPT prompting methodology is not explained well. What different formats were explored, and why is the adopted prompting better?  \n- Sec 4.7 case study is too brief to be worthwhile. \n- I think there's too much space devoted for covering the theory behind diffusion model, but much less space for explaining the main problem statement, technical contributions and evaluation details.  This paper needs major revisions to address this.", "questions_to_authors": "1. What different ChatGPT prompting formats were explored, and why is the adopted prompting better?\n2. What are the sizes of the datasets in Sec 4?", "paper_topic_and_main_contributions": "This paper addresses the problem of inducing event schemas that are more complex than atomic event representations (eg. event tuples) and may have correlations, which is an important problem. The authors propose an improved method to address two major issues in existing schema induction approaches i) below-par coverage of instance graphs ii) error propagation in auto-regressive schema generation. The key ideas to resolve the issues are respectively i) use zero-shot ChatGPT prompting to extract related entities and events, and expand the instance graphs ii) employ non auto-regressive approach based on discrete diffusion model to predict the schema graph nodes and edges. Experiments have been done on 3 datasets for performance analysis in terms of F1-scores. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "I found several grammar issues, eg.\n\n- [Page 2] 'Therefore, how to improve the knowledge coverage of instance graphs is an important problem.'\n- [Page 2] \"..predicts all nodes and links simultaneously in non-auto-regressive manner but not generates individually based on time series\""}}]

Author rebuttal:

Meta-review: This paper introduces a knowledge-enriched discrete diffusion model for event schema induction. Further, the authors use an OpenAI LLM  to generate training data. The technologies used here are combined in an interesting and novel way, and the paper robustly demonstrates that the strategy leads to improved performance on several benchmarks. 

The model and results presented are interesting. As identified by multiple reviewers, the main flaw of this paper is clarity – the original version left out many details, and important sections were underspecified. This has been partially addressed by authors in responses – i.e., the authors have added motivation for their choice to prompt through code generation. When the paper is updated to reflect the rebuttals, it will be in a much stronger state.
Decision: Accept-Findings