CDEP: Cross-domain Event Prediction Based on Continuous
Learning
Yuxin Zhang

Deqiang Yin

Zhenqian Wu

Southeast University
NanJing, China
zzyx_cs@seu.edu.cn

Southeast University
NanJing, China
gmrqiang@gmail.com

Jiangsu Automation ResearchInstitute
Lianyun Gang, China
254088544@qq.com

Yike Wang

Shengning Chen

Guilin Qi∗

Jiangsu Automation ResearchInstitute
Lianyun Gang, China
upcwyk@163.com

Jiangsu Automation ResearchInstitute
NanJing, China
352284858@qq.com

gqi@seu.edu.cn
School of Computer Science andEngineering, Southeast University
Nanjing, China

ABSTRACTKEYWORDS

Event prediction aims to extrapolate potential future events byanalyzing patterns and trends inherent in historical events. The internet is replete with a plethora of event information, some of whichpertain to events with profound ramifications on both the naturalenvironment and society at large, such as natural calamities, actsof terrorism, the proliferation of infectious diseases, and criminal
incidents. Proactively predicting the emergence of these events, or
forecasting their subsequent trajectories, can serve as an invaluablestrategy in attenuating their detrimental impacts on human civilization. Existing event prediction methods are inherently crafted tocater to specific application realms, rendering models adept in one
domain ineffective in forecasting events across disparate domains.
Nevertheless, irrespective of the semantic or structural divergencescharacterizing events across these domains, there exists an underlying commonality in event development trends. To bridge thisresearch gap, this article presents a novel event prediction model,
dubbed CDEP that amplifies the precision of event forecasting byidentifying parameters quintessential to domain-specific event prediction, and restricting the update of parameters related to event
prediction commonalities, facilitating their adaptation or seamless
transition across previously uncharted domains. Experimental results across two public benchmark datasets unequivocally highlightthe substantial enhancement brought about by continual learning,
CDEP could outperform the state-of-the-art model by a considerable
margin.

event prediction, continual learning, graph neural networks

CCS CONCEPTS• Computing methodologies → Lifelong machine learning.
∗ Corresponding author.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-XXXX-X/18/06. . . $15.00
https://doi.org/XXXXXXX.XXXXXXXACM Reference Format:
Yuxin Zhang, Deqiang Yin, Zhenqian Wu, Yike Wang, Shengning Chen,and Guilin Qi. 2018. CDEP: Cross-domain Event Prediction Based on Continuous Learning. In Proceedings of the International Joint Conference on
Knowledge Graphs (IJCKG’23). ACM, New York, NY, USA, 9 pages. https://doi.org/XXXXXXX.XXXXXXX

1INTRODUCTION

Event prediction aims to extrapolate potential future events byanalyzing patterns and trends inherent in historical events. Theinternet is replete with a plethora of event information, some ofwhich pertain to events with profound ramifications on both thenatural environment and society at large, such as natural calamities, acts of terrorism, the proliferation of infectious diseases, andcriminal incidents. Proactively predicting the emergence of these
events, or forecasting their subsequent trajectories, can serve as
an invaluable strategy in attenuating their detrimental impacts onhuman civilization.
The significance of event prediction has led to the proliferationof various prediction models across different domains. For instance,
meteorologists employ specialized models to forecast weather patterns, while law enforcement agencies utilize distinct algorithms
to anticipate criminal activities. However, a glaring challenge facedby the current landscape of event prediction is the confinement ofthese models within their specific realms of application. A model
adept at predicting earthquakes might falter when it comes to forecasting a pandemic’s spread. This limitation stems from the factthat these models, although designed for specific applications, do
not capitalize on the fundamental similarities underlying event
development trends across diverse domains. Despite the evident semantic differences in events across varied domains, the underlying
patterns governing their emergence and progression bear certain
commonalities. Leveraging these commonalities can pave the way
for a more universal, cross-domain event prediction model that canadapt and perform proficiently across different domain scenarios.
In recent years, there has been notable progress in continual
learning methods, particularly in addressing the challenging issueof catastrophic forgetting in neural network models. Catastrophic

IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan

forgetting occurs when neural networks tend to "forget" previously
acquired knowledge when learning new tasks or domains. The integration of continual learning techniques into event prediction models has emerged as a promising approach to mitigate this problem.
By incorporating continual learning strategies, event prediction
models can retain and leverage common knowledge across various
domains, enhancing their adaptability to cross-domain event prediction tasks. In our observations, textual descriptions of the same
event often exhibit redundancy and noise, which can inadvertently
introduce misleading into predictive models. For a comprehensive understanding of events, it’s paramount that models adeptly
discern the sophisticated structural nuances of events as well as
their intrinsic evolutionary trend patterns. To address this challenge, we advocate for the transformation of event-related textual
data into a structured graphical representation. Nevertheless, continual learning methods based on graph-structured data are still
immature. Existing graph-based continual learning methods can be
categorized as regularization methods[1, 3, 11, 20], which restrict
the update of key parameters; parameter isolation methods[28],which assign different parameters to different tasks; and memory
replay methods[29], which store representative nodes. These aforementioned methods are still defective in practical applications andcannot be directly applied to downstream tasks.
In light of the challenges previously highlighted, we present
an innovative Cross-Domain Event Prediction (CDEP) method anchored in graph continual learning. CDEP takes gnn as the backboneof event prediction models and improves GNN to model complex
entity and event heterogeneous knowledge in event graphs in orderto facilitate the extraction of embedded node representations. Building upon this foundation, we integrate a graph continual learning
methodology into the event prediction framework. This integration
empowers the model to seamlessly encode shared knowledge spanning multiple domains[4, 10], thereby enhancing its adaptability forcross-domain event prediction tasks. In addition, design a memory
playback sample sampling strategy for cross-domain event prediction. Experimental results across two public benchmark datasets
unequivocally highlight the substantial enhancement brought about
by continual learning, CDEP could outperform the state-of-the-art
model by a considerable margin.

2 RELATED WORK
2.1 Event Prediction
Event prediction methods can be broadly classified into two primarycategories based on their utilization of historical event data and theirapproach to forecasting future events: rule-based and sequencebased methods.
Rule-based methods typically mine associations between eventsand their precursors using methods based on frequent set [9]. Specifically, for a new input 𝑥 ′ , the typical strategy is to compute theset of events triggered by any association rule starting from 𝑥 ′ asproposed by Han et al.[9], and then rank them according to theconfidence level, retaining the most important ones.
Sequence-based methods are mainly categorized into completesequence classification and event generation. Complete sequenceclassification methods treat event prediction as a multi-classification
problem, where each class represents a semantic class of possible

Yuxin Zhang, Deqiang Yin, Zhenqian Wu et al.

next events[7]. Typically, as proposed by Tama et al.[21], this isdone by comparing the input sequence with the features of each category thereby identifying the category with the higher probability
value as the next event.
Event generation methods usually fall into two categories, attributebased methods and description-based methods. The attribute-based
method generates a vector of events at the 𝑡 +1 moment by encoding
attributes such as location, subject, and semantic category, whichis then decoded by a decoder to finally obtain various attributes
for the next event [6]. Su et al.[19] state that the description-based
methods typically encode a textual sequence of events into a vector
representation of events using a recurrent neural network. The
vector representation of each event of the sequence of events isthen fed into a higher-level recurrent neural network to generatepredicted event vectors[25].

2.2

Continual Learning

Unlike traditional deep learning models that can only acquire static
knowledge, continuous learning is able to continuously acquire,
update, memorize, and utilize knowledge in an ever-changing environment. Two approaches to continuous learning are introduced:
weight regularization and replay-based methods.
Weight regularization methods aim to impose constraints on the
update range of the weight parameters in a neural network. Amongthem, the EWC(Elastic Weight Consolidation)[13] method is a typicalweight regularization method that preserves the knowledge of theold task by restricting the difference between the old weights andthe new ones.
The main idea of memory replay is to store old training samples
by storing a smaller subset of the old task that can be used to train
together with new data or to constrain the generative network. The
biggest difficulty of memory replay lies in the need to repeatedly
sample, augment, and replace the stored training samples during
construction. Early sampling methods mostly used random sampling such as the reservoir sampling algorithm [17] and the ringbuffer algorithm [16]. The drawback of the memory replay method
is that there is a risk of overfitting when dealing with old training
samples. This is due to the fact that only a small portion of the old
training samples are stored in the memory buffer, which affects thegeneralization ability of the model[23].

3 PRELIMINARIES
3.1 Problem Statement
In practical scenarios, real-world events often comprise multiple
atomic sub-events intricately interwoven through relations, such as
causal and temporal relations. Such aggregations of atomic eventsand their interrelations are termed complex events. The event prediction task defined in this paper is a narrow semantic-based event
prediction. Given a set of documents, denoted as D, that portrays asingular complex event, the model predicts the semantics of possible
future events, i.e., event types. This is formalized by the expression:𝐶ˆ = argmax𝐶𝑖 ∈𝐶 𝑃 (𝐶𝑖 |𝑠 1, 𝑠 2, · · · , 𝑠𝑇 ), where 𝑠𝑇 denotes the event
type of the atomic event 𝑒𝑇 in the complex event, and 𝐶ˆ is the optimal prediction in the candidate set of event types 𝐶 = {𝐶 1, · · · , 𝐶 𝑁 },𝑁 is the number of atomic event types included in the complex
event. To accurately comprehend complex events, it is imperativeCDEP: Cross-domain Event Prediction Based on Continuous Learning

IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan

Entity
node
实体节点

υ0

υ1

e0

e1

for models to not only internalize representations of atomic events
but also to discern the intricate structural characteristics of complex
events and their inherent evolutionary trajectories. Consequently,
a pragmatic approach involves converting these complex events
into a graph-structured format.
For a given document set D, that delineates a specific complex
event, we employ advanced information extraction methodologies
to construct an instance graph G. This graph encompasses a set of
event nodes 𝐸, and a distinct set of entity nodes 𝑉 . Interconnecting
these nodes, we introduce a set of relational edges R, which articulate the intricate relationships and interactions within the event
structure. In the context of continual learning, the model undergoessequential training across a succession of event prediction tasks
T = T1, · · · , T𝐾 that are distinctly domain-disjointed. Every taskT𝑘 encompasses a set of complex event instance graphs designatedfor both training 𝐺𝑘𝑡𝑟 and evaluation 𝐺𝑘𝑡𝑒 , accompanied by theirrespective event prediction labels Y𝑘 = 𝑦1, · · · , 𝑦𝑘 𝑁 . Here, 𝑘 𝑁 represents the number of atomic event types present within the task
T𝑘 . This article aims to develop an event prediction model tailored
for a sequence of distinct domain-specific tasks. The overarching
objective is to ensure the model’s proficiency in forecasting events
within novel domains while preserving its predictive accuracy forpreviously encountered domains.

rate of key parameters affecting the prediction performance, thus
preserving the predictive capability of the event prediction modelon historic domains. Complementing this, our uniquely crafted
memory replay strategy, optimized for graph-structured data, provides an additional module to counteract the model’s catastrophic
forgetting tendencies.

3.2

4.1

Event Graph Construction

To effectively capture the intricate relationships among events andthe inherent structural attributes of complex events, this section
advocates for converting a document collection from a corpus,
which delineates the same complex event, into a corresponding
graph-structured representation. We employ the state-of-the-art
information extraction system OneIE[15] to extract entity, relation,and event information. Subsequently, for a collection of documentsthat describe a singular complex event, co-reference disambiguation
is executed across the extracted entities and events. Upon acquiring
the discrete event nodes, we ascertain both temporal and causal
interrelations among events utilizing the derivative prompt joint
learning event relations extraction method. The instance graphof the constructed complex event is denoted as G. This graph G,
encompasses a set of event nodes 𝐸, along with a set of entity nodes
𝑉 . Further, it incorporates a set of relational edges R linking thesenodes. The relation edges R can be classified into three distinct
categories: (1) event-event edges R𝑒𝑣𝑒 −𝑒𝑣𝑒 connecting the directed
temporal or causal edges between two events; (2) event-entity edges
R𝑒𝑣𝑒 −𝑒𝑛𝑡 connecting attribute edges between events and entities;and (3) entity-entity edges R𝑒𝑛𝑡 −𝑒𝑛𝑡 connecting relationship edges
between entities.

4

METHODOLOGY

The proposed CDEP model is articulated around three core modules: an event prediction backbone, a topology-aware continual
learning mechanism, and a memory replay strategy tailored forgraph-structured data. Given an instance graph of complex events,
the backbone module forecasts potential future event types. To
maintain prediction efficacy across historic domains, CDEP adopts
a nuanced continual learning mechanism. CDEP utilizes a topologyaware continual learning-based approach to slow down the update

事件节点
Event
node
事件-实体
Event
– Entity ℛReve-ent
实体-实体Entity– Entity ℛRent-ent
事件-事件
Event
– Evente2？

ℛ

Reve-eve

υ3

υ2

υ4

信息传递机制Information
Transfer Mechanism
层→ +1层
𝑙 𝑙𝑎𝑦𝑒𝑟 → 𝑙 + 1 𝑙𝑎𝑦𝑒𝑟−1层𝑙 − 1→
𝑙𝑎𝑦𝑒𝑟层
→ 𝑙 𝑙𝑎𝑦𝑒𝑟e1, ,= ReLU−υ0, ,= ReLU

−

× −1

υ1

, ,

= ReLU−υ2, ,= ReLU

−

× −1

υ1

…

e0

, ,

= ReLU

−

norm

e1

,

e1

Figure 1: Event Prediction message passing mechanism

Event Prediction Model

The CDEP model integrates a relational graph convolutional network
as its foundational architecture for event prediction. To adeptly unearth underlying patterns in potential event evolution, a bespoke
message-passing mechanism is designed for each distinct relationship type, as visually depicted in Figure 1. CDEP initializes the semantic representation of entities in the complex event instancegraph by pre-training the BERT model and employs a linear tensor operation to aggregate the argument information of the event
nodes, the initialized semantic encoding of the atomic event 𝑒𝑖 is:∑︁
𝒆𝑖0 = MLP(
BERT(𝑤 𝑗 )),(1)𝑖𝑗 ∈ N𝑒𝑛𝑡

𝑖 denotes the set of argument nodes neighboring thewhere N𝑒𝑛𝑡
atomic event 𝑒𝑖 and 𝑤 𝑗 denotes the semantic text of the argument
entity. Conventional relational graph convolutional networks adopt
unique edge representations for each type of edge. However, this approach can cause an exponential surge in model parameters as theedge types increase, rendering it unsuitable for cross-domain contexts. Addressing this, CDEP conceptualizes edges within the complex event instance graphs into three distinct categories: R𝑒𝑣𝑒 −𝑒𝑛𝑡 ,R𝑒𝑣𝑒 −𝑒𝑣𝑒 , and R𝑒𝑛𝑡 −𝑒𝑛𝑡 . Specifically, R𝑒𝑣𝑒 −𝑒𝑛𝑡 represents a unidirectional edge from an event node to its corresponding argument
node; R𝑒𝑣𝑒 −𝑒𝑣𝑒 signifies a unidirectional edge from one event nodeto another, and R𝑒𝑛𝑡 −𝑒𝑛𝑡 corresponds to the undirected edge interconnecting entities. In tandem with this, CDEP introduces specialized messaging mechanisms for each edge type, ensuring a morerefined aggregation of the semantic and structural attributes inherent to atomic events. The passing information of the three edges in
the 𝑙 layer of the graph neural network is defined as follows:



𝑖,𝑗,𝑙
𝒎𝑒𝑣𝑒 −𝑒𝑛𝑡 = ReLU 𝑾𝑎𝑙 𝒆𝑖𝑙 − 𝒗𝑙𝑗(2)

IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan

Yuxin Zhang, Deqiang Yin, Zhenqian Wu et al.



 
𝑖,𝑗,𝑙
𝒎𝑒𝑛𝑡 −𝑒𝑛𝑡 = ReLU 𝑾𝑟𝑙 𝒗𝑖𝑙 − 𝒗𝑙𝑗 ||𝒓

(3)




𝑖,𝑗,𝑙
𝒎𝑒𝑣𝑒 −𝑒𝑣𝑒 = ReLU 𝑾𝑡𝑙 𝒆𝑖𝑙 − 𝒆𝑙𝑗(4)where 𝑾𝑎 , 𝑾𝑟 , 𝑾𝑡 denote the linear variation matrices of the edges
R𝑒𝑣𝑒 −𝑒𝑛𝑡 , R𝑒𝑛𝑡 −𝑒𝑛𝑡 , and R𝑒𝑣𝑒 −𝑒𝑣𝑒 , respectively, and || denotes vector concatenation operations; In addition, CDEP learns a specificvector representation 𝒓 for R𝑒𝑛𝑡 −𝑒𝑛𝑡 . Entities directly aggregate
neighborhood information to update node representations. Atomic
events need to consider not only neighboring entities and events
but also the degree of association of neighboring events. Therefore,
this section uses edge-aware attention to aggregate information
between events and events or entities:
𝛼𝑖,𝑗 = 𝜎 MLP 𝒆𝑖 − 𝒆 𝑗
(5)

∑︁
∑︁
©
ª
𝑖,𝑗,𝑙
®
𝒆𝑖𝑙+1 = 𝑛𝑜𝑟𝑚 ­­𝒆𝑖𝑙 +𝛼𝑖,𝑗 𝒎𝑒𝑣𝑒 −𝑒𝑣𝑒 +𝒎𝑖,𝑘,𝑙𝑒𝑣𝑒 −𝑒𝑛𝑡 ®
𝑖𝑖𝑗 ∈ N𝑒𝑣𝑒𝑘 ∈ N𝑒𝑛𝑡
«
¬
∑︁
𝑗,𝑖,𝑙
𝒗𝑖𝑙+1 = 𝒗𝑖𝑙 +𝒎 𝒆𝒗𝒆−𝒆𝒏𝒕(6)𝑎𝑡𝑡𝑒𝑒𝑣𝑒 −𝑒𝑛𝑡 = (𝒆𝑖𝑙 −1𝑊𝑎𝑙 )𝑇 tanh(𝒗𝑙𝑗 −1𝑊𝑎𝑙 )
𝑖,𝑗,𝑙

(10)

𝑙
= 𝑓𝑒𝑣𝑒 −𝑒𝑛𝑡 (𝒆𝑖𝑙 −1 ; 𝒗𝑙𝑗 −1 ; 𝜃 𝑒𝑣𝑒−𝑒𝑛𝑡 )𝑙𝑙𝑙
where 𝜃 𝑒𝑣𝑒−𝑒𝑛𝑡 = 𝑊𝑎 and the attention coefficient 𝑎𝑡𝑡𝑒𝑒𝑣𝑒 −𝑒𝑛𝑡,𝑖,𝑗 isrepresented by the distance between 𝑒𝑖 and 𝑣 𝑗 . During continual
learning, the model only focuses on the topological informationof the event nodes, so only the parameters related to the R𝑒𝑣𝑒 −𝑒𝑣𝑒 ,R𝑒𝑣𝑒 −𝑒𝑛𝑡 edges are considered. The model loss for the domain task
T𝑘 is denoted as L (𝐺𝑘𝑡𝑟 ; 𝜃 ),𝜃 = {𝜃 𝑒𝑣𝑒 −𝑒𝑣𝑒 , 𝜃 𝑒𝑣𝑒 −𝑒𝑛𝑡 } = {W𝑚+𝑛 }contains all the parameters of the neural network. When a smallchange occurs in one of the parameters of the neural network,
the change in the structural characteristics of the model can beapproximated as:




𝑙
𝑙
𝑙 −1 𝑙 −1 𝑙𝑓 𝒆𝑖𝑙 −1 ; 𝒆𝑙𝑗−1 ; 𝜃 𝑒𝑣𝑒−𝑒𝑣𝑒 + ΔW𝑚 − 𝑓 𝒆𝑖 ; 𝒆 𝑗 ; 𝜃 𝑒𝑣𝑒 −𝑒𝑣𝑒


∑︁
𝑙
≈
𝑔𝑚 𝒆𝑖𝑙 −1 ; 𝒆𝑙𝑗−1 ΔW𝑚
,
𝑚

(11)

(7)

𝑖
𝑗 ∈ E𝑒𝑣𝑒𝑖N𝑒𝑛𝑡
denotes the atomic event 𝑒𝑖 neighboring argument nodes,
𝑖 denotes the set of neighboring event nodes for which there
N𝑒𝑣𝑒
𝑖 denotes the entity node
exist directed edges pointing to 𝑒𝑖 , and E𝑒𝑣𝑒𝑣𝑖 of the set of neighboring event nodes 𝑛𝑜𝑟𝑚(•) is the regularization function. CDEP pools all atomic event representations in thecomplex event graph to obtain the graph representation 𝑔, by which
𝑔 predicts the event type 𝑐 of future event nodes:




𝒄 = Softmax MLP Pooling 𝒆 1, · · · , 𝒆 |𝐸 |(8)

Pooling operations can be alternatively average pooling, maximum
pooling, etc., after which future event types are predicted by amultilayer perceptron.

4.2

mechanism. Therefore CDEP uses the distance between nodes as
the attention coefficient, which is calculated as follows:

Continual Learning based on Topology
Awareness

Upon completing the training process for domain task T𝑘 , the model
acquires a set of optimized parameters denoted as 𝜃𝑘∗ . These parameters are fine-tuned to minimize the event prediction loss specific tothe domain. However, it’s important to recognize that not all parameters within the GNN contribute equally to the model’s performance.
Inspired by the TWP method [11], CDEP models the attention coefficients between the central node and its first-order neighboring
nodes as topological features of the center node. Thus, according
to Eqs. 4, 5, and 6, the attention coefficients of the R𝑒𝑣𝑒 −𝑒𝑣𝑒 edges
can be rewritten as:


𝑖,𝑗,𝑙
𝑙
𝑎𝑡𝑡𝑒𝑒𝑣𝑒 −𝑒𝑣𝑒 = 𝑓𝑒𝑣𝑒 −𝑒𝑣𝑒 𝒆𝑖𝑙 −1 ; 𝒆𝑙𝑗−1 ; 𝜃 𝑒𝑣𝑒(9)−𝑒𝑣𝑒
where 𝑓 (•) denotes the neural network projection of the atten𝑙tion coefficients obtained from the node representations; 𝜃 𝑒𝑣𝑒−𝑒𝑣𝑒
denotes the parameter associated with the computation of the attention coefficients of the R𝑒𝑣𝑒 −𝑒𝑣𝑒 edges; 𝒆𝑖𝑙 −1 , 𝒆𝑖𝑙 −1 , 𝒆𝑙𝑗−1 thendenote the feature embedding of nodes 𝑒𝑖 and 𝑒 𝑗 in layer 𝑙, respectively. Unlike R𝑒𝑣𝑒 −𝑒𝑣𝑒 , R𝑒𝑣𝑒 −𝑒𝑛𝑡 edges do not have an attention


𝑓

𝑙
𝑙
𝒆𝑖𝑙 −1 ; 𝒗𝑙𝑗 −1 ; 𝜃 𝑒𝑣𝑒−𝑒𝑛𝑡 + ΔW𝑛𝑙𝒆𝑖𝑙 −1 ; 𝒗𝑙𝑗 −1 ; 𝜃 𝑒𝑣𝑒
−𝑒𝑛𝑡



−𝑓

∑︁ 
(12)
≈
𝑔𝑛 𝒆𝑖𝑙 −1 ; 𝒗𝑙𝑗 −1 ΔW𝑛𝑙 ,
𝑛

∑︁
𝑚



𝑔𝑚 𝒆𝑖𝑙 −1 ; 𝒆𝑙𝑗−1𝑔𝑛 𝒆𝑖𝑙 −1 ; 𝒗𝑙𝑗 −1 ΔW𝑛𝑙
𝑛


∑︁
𝑙 −1
𝑙
ΔW𝑚+𝑛
,
=
𝑔𝑚+𝑛 𝐻𝑖,𝑗

𝑙
ΔW𝑚
+

∑︁

(13)

𝑚+𝑛



𝜕𝑓
𝑙 −1 =
where 𝑔𝑚+𝑛 𝐻𝑖,𝑗
𝜕W𝑚+𝑛 denotes the partial derivatives of
the attention coefficients with respect to the parameter W𝑚+𝑛 , and𝑙 −1 is the node representation of the output of the 𝑙 − 1 layer. The
𝐻𝑖,𝑗attention coefficients of the atomic event node 𝑒𝑖 as the center nodein layer 𝑙 of the event prediction model form a multidimensional
vector of:𝑖,𝑙𝑖,𝑙𝑎𝑡𝑡𝑒𝑖𝑙 = 𝑎𝑡𝑡𝑒𝑒𝑣𝑒−𝑒𝑣𝑒 ∪ 𝑎𝑡𝑡𝑒𝑒𝑣𝑒 −𝑒𝑛𝑡𝑖𝑖𝑖, | N𝑒𝑣𝑒𝑖, | N𝑒𝑛𝑡
|,𝑙
|,𝑙
𝑖,1,𝑙
𝑖,1,𝑙
= 𝑎𝑡𝑡𝑒𝑒𝑣𝑒
,
·
·
·
,
𝑎𝑡𝑡𝑒
,
𝑎𝑡𝑡𝑒
,
·
·
·
,
𝑎𝑡𝑡𝑒
−𝑒𝑣𝑒
𝑒𝑣𝑒 −𝑒𝑣𝑒
𝑒𝑣𝑒 −𝑒𝑛𝑡𝑒𝑣𝑒 −𝑒𝑛𝑡(14)
CDEP defines the structural feature loss as the square of the 2paradigm of the multidimensional vector of attention coefficientsfor all atomic event nodes in the complex event instance graph, andcomputes each parameter bias:
hi 2
𝑙 , · · · , 𝑎𝑡𝑡𝑒 𝑙
𝜕
𝑎𝑡𝑡𝑒


1
|𝐸 | 2(15)
𝜕𝑓𝑙 −1
𝑔𝑚+𝑛 𝐻𝑖,𝑗
=
=
.
𝜕W𝑚+𝑛
𝜕𝑤𝑚+𝑛
𝑙 −1 )||] for all paThis leads to an importance score 𝐼𝑘 = [||𝑔𝑚+𝑛 (𝐻𝑖,𝑗rameters in the task T𝑘 with respect to the topology of atomic event
nodes. Catastrophic forgetting of event prediction tasks in crossdomain prediction tasks can be effectively mitigated by penalizing
parameter changes that are critical to the impact of the old task,
as shown in Figure 2. While maintaining the stability of importantparameters allows the model to remember the learned task, it also
results in a less malleable model that is unable to learn subsequentCDEP: Cross-domain Event Prediction Based on Continuous Learning

input
输入

1"#$%&"#$%&
1层 →→ "−−11层

IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan

Importance
Score
重要性分数

层 " "#$%&-,/,0, ,#))%*+*,*+*(equation
9)
(公式4
− 9)

1

Task

…

2

1

…

2

3

…

3

…

…

Task

拓扑结构重要性得分
Topology
Importance Score

-,3,0
, ,
#))%*+*,*12(equation
10)
(公式4
− 10)
-,/,0
-,3,0
, ,
, ,
#))%-0 = [#))%(equation
10)
[
− 14)
*+*,*+* , #))%*+*,*12]](公式4…

…

ℒ ( )=ℒ

( )+

−1
ℎ=1

⊗( −

∗)

+

Figure 2: Continual Learning based on Topology Awarenessnew tasks. In order to retain sufficient model capacity for futuredomains, CDEP promotes minimization of parameter importancescores by adding one-paradigm numbers as regularization terms to
the importance scores. Thus, the total loss in training the current
task T𝑘 is:
!
𝑘∑︁
−1
𝑛𝑒𝑤
∗ 2
(16)
L𝑘 (𝜃 ) = L𝑘 (𝜃 ) +𝐼ℎ ⊗ (𝜃 − 𝜃ℎ ) + 𝛽 ||𝐼𝑘 || 1ℎ=1

where ⊗ denotes elemental multiplication; L𝑘𝑛𝑒𝑤 (𝜃 ) denotes theprediction loss (cross-entropy loss) of the event prediction model attask T𝑘 ; 𝐼ℎ represents the parameter on old task Tℎ importance scorematrix; 𝛽 is a hyperparameter controlling the degree of influenceof the importance score.

4.3

Graph Matching Based Memory Replay
Strategy

A substantial body of prior research on continual learning has
demonstrated the efficacy of memory-playback-based strategies in
mitigating catastrophic forgetting. Therefore, this paper proposes agraph-matching-based memory playback strategy to further alleviate the catastrophic forgetting problem of event prediction models
in cross-domain tasks by playing back old tasks.
According to the objective of the memory replay strategy, CDEP
needs to find the subset of data that best represents a task, andthis subset of data needs to contain as many common features of
the task as possible. For event prediction, CDEP selects the mostrepresentative complex event instance graphs in each domain andstores them in the memory playback module. Specifically, thesefiltered complex event instance graphs should be able to represent
as many instance graphs of the same domain as possible, with thehighest similarity between them and other instance graphs. Thus,
the selection problem for memory can be converted into a problemof maximizing the total graph matching score.
When comparing two complex event instance graphs, denoted
as G𝑁 and G𝑀 , CDEP’s attention is primarily directed towards

the event types associated with the atomic event nodes withinthese graphs. Consequently, the specific atomic event nodes are
abstracted into their respective event types, while the entity nodes
are disregarded in this context. The assessment of graph similarity
in CDEP involves a two-fold perspective: event type similarity andrelation edge similarity. These two dimensions are pivotal in evaluating the likeness between complex event instance graphs. The
calculation of these similarities is outlined as follows:∑︁∑︁
Sim (G𝑁 , G𝑀 ) =
1+
1,
(17)
𝑋𝑖 𝑗 =1

𝑋𝑖𝑘 =1,𝑋 𝑗𝑙 =1

where 𝑋 ∈ {0, 1} | G𝑁 |,| G𝑀 | denotes the binary matching matrix, and𝑋𝑖 𝑗 = 1 when, and only when, 𝑖 ∈ G𝑁 and 𝑗 ∈ G𝑀 matches. CDEP
calculates the matching scores of any two complex event instancegraphs in the dataset, and filters the complex event instance graphwith the highest total matching score as the playback "memory".

5 EXPERIMENT
5.1 Datasets
In the present study, we engage in a systematic experimental evaluation of the proposed methodologies. To ensure robustness andgeneralizability, we utilize two prominent event prediction datasets:
the Improved Explosive Device (IED) dataset[14] and the Chinese
Emergency dataset.
The IED dataset is on improvised explosive device attacks, with
corpus content taken from Wikipedia and relevant news. It contains
4 task types, namely car bomb, drone bomb, suicide bomber, andgeneral IED attack. Its event schema definitions include 24 entity
types, 48 relation types (including entity and event relation), 67
event types, and 85 argument types. For an in-depth statisticalbreakdown of the dataset, readers are directed to Table 1.
The Chinese Emergency Incident Dataset is a dataset about social
security incidents from the Internet news corpus. It is small in sizeand contains five task types, namely "earthquake", "fire", "traffic
accident", "terrorist attack", and "food poisoning". Its event schemaIJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan

Yuxin Zhang, Deqiang Yin, Zhenqian Wu et al.

Table 1: Detailed description of the event prediction experiment dataset
Datasets

Number of tasks

IED

4

the Chinese Emergency

5

Subsets

Documents

Event subsets

EventsArguments

Relations

Train
Dev
Test
Train
Dev
Test

5247
575
577
265
33
34

343
42
45
265
33
34

41672
4661
5089
4763
545
596

136894
15404
16721
15670
1793
1963

122846
13320
14054
14579
1689
1881

definition includes 35 event types and 25 argument types. Detailed
statistics of its content are shown in Table 1.

5.2

Evaluation Metrics

Our evaluative measures for event prediction encompass Precision, Recall, and F1 score in this paper. To provide a more granular
evaluation of the proposed method’s performance after incremental task training, we introduce the performance matrix, denoted
as 𝑀 𝑝 ∈ R𝑇 ×𝑇 . This matrix is on the basis of F1 scores, with 𝑇𝑝symbolizing the aggregate number of tasks. Specifically, The 𝑀i,jdenotes the F1 score of the model on the jth task after training
from the 1st task to the ith task. In seeking a comprehensive metricthat encapsulates the holistic performance of the event prediction
model, particularly in the realm of continual learning across varied
domains, we adopted the AP (average performance) and AF (average forgetting) metrics, as conceptualized by Lopez et al.[16]. The
foundational definitions for AP and AF are presented as follows:𝑝j=1 𝑀i,jÍ𝑖
𝐴𝑃 = {
𝐴𝐹 = {iÍi−1|i = 1, . . . ,𝑇 }𝑝𝑝
j=1 𝑀i,j − 𝑀j,j

i−1

(18)
|i = 2, . . . ,𝑇 }Specifically, for the metric AP, after the model has been trained𝑝on a series of tasks from 1 to i, the first i line of 𝑀 𝑝 : {𝑀i,j |j =1, . . . , i} covers the model’s performance on each of the previous
tasks. Therefore the average of the i row is the model’s average𝑝𝑝performance on all previous tasks. For the metric AF, {𝑀i,j −𝑀j,j |j =1, . . . , i − 1} denotes how much the model forgets on the task j
after the training task i. Its average value is the average amount offorgetting for each trained task after the model has been trained
from task 1 to task i.

5.3

Comparison Method

In this paper, we hope to analyze and validate the accuracy of theevent prediction method we proposed in predicting the types of
events that may occur in the future by comparing it with the baseline model, and the impact of a topology-aware continual learning
module on the catastrophic forgetting problem in event prediction
models which for cross-domain scenarios. Notably, this researchstands as a pioneering effort in the realm of event prediction predicated on complex event subgraphs. Given this innovative approach,we have selected the advanced heterogeneous graph neural networkas the benchmark for our event prediction method. Our baselinemodel bifurcates into two primary components: (1) the heterogeneous graph neural network and (2) the graph continual learning
methodology.
5.3.1 Heterogeneous Graph Neural Network. The selected graphneural networks for our comparative evaluation are intricately
designed to handle the heterogeneity of events and entities in complex event subgraphs. What follows is a concise overview of thesophisticated graph neural network models incorporated in ourstudy:(1) GraphSAGE[8]: A model for updating the embedding representation of each node using the embeddings of neighboring
nodes.
(2) GAT[22](Graph Attention Networks): A graph neural network that uses a self-attention mechanism to compute theweights of neighboring nodes to update the representationof each node.
(3) GCN[12](Graph Convolutional Networks): A graph neuralnetwork for updating per-node representation using aggregated information from local neighbor nodes.
(4) R-GCN[18](Relational Graph Convolution Network):A graphneural network for modeling relational data.
(5) HAN[24](Heterogeneous Graph Attention Network): HAN canhandle heterogeneous graphs consisting of multiple types ofnodes and edges, and it uses different attention mechanisms
to capture the relations between different types of nodes.
(6) HetGNN[26](Heterogeneous Graph Neural Network): HetGNN
use a cross-type adjacency matrix to make connections between nodes of different types and aggregates them to generate feature representations.
5.3.2 Graph Continual Learning Method. This paper compares
state-of-the-art graph continual learning methods based on the
CGLB[27] platform. The realized graph continual learning methods
are as follows:(1) Bare model: Denotes a CDEP model without continual learning method, which can be considered as a lower bound forcontinual learning performance.
(2) EWC[13](Elastic Weight Consolidation): Based on the importance of the model weight to previous tasks, a secondary
penalty was been added to the model weights to maintainits performance on existing tasks.
(3) MAS[2](Memory Aware Synapses): MAS assesses the importance of a parameter based on the sensitivity of the prediction
to the parameter.

CDEP: Cross-domain Event Prediction Based on Continuous Learning(4) EMR[5](Episodic Memory Replay): The method trains themodel based on data from the current task and a small number of in-memory instances.
(5) GEM[16](Gradient Episodic Memory):The method stores representative data in scene memory. During learning, GEM uses
the gradient computed from the stored data to modify thegradient of the current task in order to prevent an increasein the forgetting of previous tasks.
(6) Oracle Model: Does not follow the continual learning setting
and trains all tasks simultaneously. Thus, the Oracle Model
has no forgetting problem and its performance can be seen
as an upper bound for continual learning.

IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, JapanFor the event prediction model, a 3-layer graph neural network isused as the main structure with a uniform embedding dimension of
768. The AdamW optimizer is chosen to use, the initial learning rate
is set to 1e-4, and the graph representation is uniformly operatedusing average pooling. In the event prediction model performance
evaluation process does not consider the continual learning scene
and only compares the prediction performance of heterogeneousgraph neural networks. Therefore, in this experiment, the event
prediction model performance needs to be evaluated by integrating
data from all domains into a complete dataset, including training
and testing data.
Within the scope of our graph continual learning experiments,
we’ve established specific hyperparameters for the benchmarks
under consideration. For both the EWC and MAS baselines, the regularization hyperparameter is determined at 10,000. For the topologyaware continual learning methodology, we’ve calibrated the hyperparameter within a range of 0.1 to 0.01. Furthermore, whenemploying the graph-matching-based memory playback approach,
we typically select a memory capacity of 10, i.e., the memory module will store 10 items of training data under this task. For enhanced
robustness and accuracy in our evaluations, each experiment within
this study was executed five times, leveraging varied random seeds.
The mean value across these iterations is then adopted as the definitive outcome for experimental appraisal.

in predictive accuracy. For our event prediction model presented
in this study, we introduce a particular information transfer mechanism catering to diverse inter-node relationships. By leveraging
edge-aware attention, our model discerns trends in inter-event
progression based on the relational magnitude between adjacentatomic events. Consequently, our proposed event prediction model
registers state-of-the-art (SOTA) performance across both datasets,achieving an enhancement of 5.3% in the F1 score over other leadingedge methods.
Table 3 elucidates the performance matrix for cross-domain continual learning, detailing the prediction accuracy across each task
within the event prediction continual learning scene. When juxtaposed against the Bare model, all continual learning methods exhibit
notable enhancements, affirming the presence of a pronounced catastrophic forgetting challenge in the cross-domain event predictiontask. Note that the AF metrics are not applicable to the Oracle Model,
given its training regimen that concurrently utilizes the aggregateddata from all tasks, without following the continual learning framework. Evaluating the resultant AP and AF metrics, each methodology surpasses the Bare Model, with the regularization-centric EWC,
MAS, and TWP frameworks being particularly commendable. Suchobservations underscore the efficacy of regularization strategies inalleviating the catastrophic forgetting conundrum. Furthermore,
the GEM framework stores representative data in situational memory and uses this to adjust the updating gradient for the current
task.
The GEM framework exhibits a marked dependency on situational memory selection, leading to fluctuating performance outcomes. Nonetheless, we observed that the EMR methodology offers
an elegantly simple yet effective solution for cross-domain continual learning via a straightforward memory playback strategy. Intriguingly, when benchmarked against established baseline models,
the topology-aware continual learning strategy emerges superior,securing the first position in AF performance and outperforming
all except the Oracle Model in AP metrics. This superior performance underlines the capability of the topology-aware approachto counteract catastrophic forgetting, achieved by slowing down
the rate of change of topology-related important parameters.

5.5

5.6

5.4

Experimental Setup

Comparative Experimental Results andAnalysisTable 2 delineates a comparative evaluation of the performance ofvarious graph neural networks on the event prediction task. An
insightful scrutiny of these results elucidates that certain graphneural networks, namely GraphSAGE, GAT, and GCN, which lack thecapability to model edge features, manifest suboptimal prediction
efficacy. This limitation arises from their inability to distinguishbetween entities and event nodes, rendering them ineffective indiscerning potential trajectories in event progression. In juxtaposition, advanced models such as R-GCN, HAN, and HetGNN, which are
adept at handling heterogeneous graphs, exhibit superior prediction metrics. These models are distinguished by their proficiency
in modeling features from diverse edges and have some ability toidentify the trend of event evolution.
Notably, the R-GCN model, while being the most straightforward
and simple in its design, outperforms the likes of HAN and HetGNN

Results and Analysis of Ablation
Experiments

To elucidate the contributions of individual modules in our continual learning-based cross-domain event prediction model, this paper
provides an in-depth exploration through ablation experiments.
We employ varied subscripts to demarcate distinct experimental
module configurations. Broadly, our ablation study pivots on two
primary aspects: the event prediction module and the continual
learning module.
For the event prediction model, the subscripts ’rgcn’ and ’gat’
denote the adoption of R-GCN and GAT models, respectively. The
continual learning module will be appropriately adjusted according
to the type of graph neural network adopted by the event prediction
model, subscript ’nocl’ indicates that the continual learning module
is not used; ’tsacl’ subscript indicates that only topology-aware
based continual learning method is used; subscript ’gm’ indicatesthat only graph-matching based memory playback based continualIJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan

Yuxin Zhang, Deqiang Yin, Zhenqian Wu et al.

Table 2: Experimental Results of Heterogeneous Graph Neural Networks
MethodGrapthSAGE
GAT
GCN
R-GCN
HAN
HetGNN
CDEP

Precision
54.69
63.01
56.41
73.19
71.94
75.12
77.45

IED
Recall
90.33
81.71
92.77
76.13
76.12
73.40
82.57

Table 3: Experiment Results of Graph Continual Learning
MethodBare model
EWC
MAS
EMR
GEM
CDEP
Oracle Model

IED
AP
AF
36.33 -44.76
49.72 -31.37
71.54 -9.55
69.12 -11.97
70.36 -10.73
73.62 -7.47
79.93-

Chinese Emergency
AP
AF
40.59
-45.2453.98-31.8575.80-10.0373.38-12.4574.62-11.21
77.88
-7.95
84.11
-

learning method is used; subscript full indicates that the continual
learning based continual learning cross-domain event prediction inthis paper model optimally. Table 4 demonstrates the results of theablation experiments.
Table 4: Results of Ablation Experiments
Method𝐶𝐷𝐸𝑃𝑟𝑔𝑐𝑛
𝐶𝐷𝐸𝑃𝑔𝑎𝑡
𝐶𝐷𝐸𝑃𝑛𝑜𝑐𝑙
𝐶𝐷𝐸𝑃𝑡𝑠𝑎𝑐𝑙
𝐶𝐷𝐸𝑃𝑔𝑚𝐶𝐷𝐸𝑃 𝑓 𝑢𝑙𝑙

IED
AP
AF
69.51 -10.92
51.45 -29.94
36.33 -44.76
68.61 -13.56
58.26 -19.78
73.62 -7.47

Chinese Emergency
AP
AF
73.25
-12.1555.74-29.3740.59-45.2467.38-13.61
57.89-20.01
77.88-7.95

5.6.1 Event Predict Module. Ablation experiment results underscore the significance of information feature aggregation methods
in graph neural networks for the performance of event prediction
models. Of the models evaluated, the R-GCN emerges as the most direct and unadorned, yet it exhibits superior predictive performanceover the GAT. This superior performance could be attributed to thechallenges GAT faces in differentiating between the three types ofedges present in complex event subgraphs. Notably, R-GCN’s performance wanes in Dataset A, which encompasses more intricate
event types and entity interrelations. Furthermore, as extended
neighborhoods proliferate, the catastrophic forgetting challenge
for R-GCN becomes more pronounced, revealing its limited scalability. In contrast, our proposed event prediction model introduces
specialized information transfer methodologies tailored for distinct

F1
68.13
71.15
70.16
74.63
73.97
74.25
79.93

Chinese Emergency
Precision Recall
F1
62.15
87.83
72.79
70.47
78.27
74.17
63.87
83.12
72.23
80.65
74.80
77.62
74.01
78.45
76.17
82.58
70.93
76.31
84.91
83.32 84.11

inter-node associations. By leveraging edge-aware attention mechanisms, the model discerns evolutionary trends of inter-events,
grounded on the relational intensity of neighboring atomic events.
Consequently, our model stands out, registering optimal results
both in event prediction accuracy and in resistance to forgetting.
5.6.2 Continual Learning Module. Experimental outcomes revealthat both the topology-aware continual learning and the graphmatching memory-playback-based continual learning method significantly mitigate the catastrophic forgetting dilemma in event
prediction models. Notably, the topology-aware continual learning method can maintain 58.63% prediction F1 on the first domaineven after training the last domain task. This suggests that there
are part of common structural features in the event prediction
tasks of different domains, which imply the evolutionary patternof events. Of significant interest is the observation that the graphmatching-based memory playback approach, despite its conceptual
simplicity, delivers exceptionally promising results. Crucially, thesetwo continual learning strategies are not conflicting, suggestingthat their combined application could offer synergistic benefits. By
harnessing them jointly, one can more robustly counteract model
forgetfulness and elevate cross-domain event prediction accuracy.

6

CONCLUSION

In this paper, we propose a cross-domain event prediction method
based on graph continual learning, CDEP. The method uses a graphneural network structure based on relation graph convolutional
networks to model complex heterogeneous knowledge of entities
and events in the event graph in order to obtain embedded representations of event graph nodes. Based on this by introducing a
topology-aware continual learning approach in the event prediction model, the model encodes the common knowledge between
different domains, thus better adapting to the cross-domain event
prediction task. In addition, a memory playback sample samplingstrategy for cross-domain event prediction is proposed. Comparison and ablation experiments for cross-domain event prediction are
designed and implemented, and the experimental results verify theeffectiveness of the event prediction method and continual learning
approach in this paper.

7

ACKNOWLEDGMENTS

We thank the reviewers for their valuable comments. This work is
partially supported by the National Nature Science Foundation ofChina under No. U21A20488.

CDEP: Cross-domain Event Prediction Based on Continuous Learning

REFERENCES[1] Kian Ahrabian, Yishi Xu, Yingxue Zhang, Jiapeng Wu, Yuening Wang, and Mark
Coates. 2021. Structure Aware Experience Replay for Incremental Learning inGraph-based Recommender Systems. In CIKM ’21: The 30th ACM International
Conference on Information and Knowledge Management, Virtual Event, Queensland,
Australia, November 1 - 5, 2021, Gianluca Demartini, Guido Zuccon, J. Shane
Culpepper, Zi Huang, and Hanghang Tong (Eds.). ACM, 2832–2836. https://doi.org/10.1145/3459637.3482193[2] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, andTinne Tuytelaars. 2018. Memory Aware Synapses: Learning What (not) to Forget.
In Proceedings of the 3rd European Conference on Computer Vision (Lecture Notes
in Computer Science, Vol. 11207). 144–161.
[3] Jie Cai, Xin Wang, Chaoyu Guan, Yateng Tang, Jin Xu, Bin Zhong, and Wenwu
Zhu. 2022. Multimodal Continual Graph Learning with Neural Architecture
Search. In WWW ’22: The ACM Web Conference 2022, Virtual Event, Lyon, France,
April 25 - 29, 2022, Frédérique Laforest, Raphaël Troncy, Elena Simperl, Deepak
Agarwal, Aristides Gionis, Ivan Herman, and Lionel Médini (Eds.). ACM, 1292–
1300. https://doi.org/10.1145/3485447.3512176[4] Zhibin Chen, Yuting Wu, Yansong Feng, and Dongyan Zhao. 2022. Integrating
Manifold Knowledge for Global Entity Linking with Heterogeneous Graphs. DataIntell. 4, 1 (2022), 20–40. https://doi.org/10.1162/DINT_A_00116[5] Cyprien de Masson d’Autume, Sebastian Ruder, Lingpeng Kong, and Dani
Yogatama. 2019. Episodic Memory in Lifelong Language Learning. CoRR
abs/1906.01076 (2019).
[6] Huilong Duan, Zhoujian Sun, Wei Dong, Kunlun He, and Zhengxing Huang. 2020.
On Clinical Event Prediction in Patient Treatment Trajectory Using Longitudinal
Electronic Health Records. IEEE Journal of Biomedical and Health Informatics 24,
7 (2020), 2053–2063.
[7] Mark Granroth-Wilding and Stephen Clark. 2016. What Happens Next? Event
Prediction Using a Compositional Neural Network Model. Proceedings of the
AAAI Conference on Artificial Intelligence 30, 1 (Mar. 2016).
[8] William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive Representation
Learning on Large Graphs. In Proceedings of the 30th International Conference on
Neural Information Processing Systems, NIPS.
[9] Jiawei Han, Micheline Kamber, and Jian Pei. 2011. Data Mining: Concepts andTechniques (3 ed.). Morgan Kaufmann.
[10] Danyang Hu, Meng Wang, Feng Gao, Fangfang Xu, and Jinguang Gu. 2022.
Knowledge Representation and Reasoning for Complex Time Expression in
Clinical Text. Data Intell. 4, 3 (2022), 573–598. https://doi.org/10.1162/DINT_A_
00152[11] Yiding Yang Huihui Liu and Xinchao Wang. 2021. Overcoming Catastrophic
Forgetting in Graph Neural Networks. In Proceedings of the 35th Annual Meetingof the American Association for Artificial Intelligence, AAAI.
[12] Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with
Graph Convolutional Networks. CoRR abs/1609.02907 (2016).
[13] James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume
Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka
Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, andRaia Hadsell. 2016. Overcoming catastrophic forgetting in neural networks. CoRR
abs/1612.00796 (2016). arXiv:1612.00796[14] Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, HengJi, Jiawei Han, and Clare R. Voss. 2021. Future is not One-dimensional: Graph
Modeling based Complex Event Schema Induction for Event Prediction. CoRR
abs/2104.06344 (2021).
[15] Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A Joint Neural Model for
Information Extraction with Global Features. In Proceedings of The 58th Annual
Meeting of the Association for Computational Linguistics, ACL.
[16] David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient Episodic Memory for
Continual Learning. In Proceedings of the 31st International Conference on Neural
Information Processing Systems, NIPS. 6470–6479.
[17] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu,
and Gerald Tesauro. 2019. Learning to Learn without Forgetting by Maximizing
Transfer and Minimizing Interference. In Proceedings of the 7th International
Conference on Learning Representations, ICLR.
[18] Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg,
Ivan Titov, and Max Welling. 2017. Modeling Relational Data with Graph Convolutional Networks. CoRR abs/1703.06103 (2017).
[19] Zichun Su and Jialin Jiang. 2020. Hierarchical Gated Recurrent Unit with Semantic
Attention for Event Prediction. Future Internet 12 (02 2020), 39.
[20] Sagar Sunkle, Deepak Jain, Krati Saxena, Ashwini Patil, Tushita Singh, Beena Rai,
and Vinay Kulkarni. 2021. Integrated "Generate, Make, and Test" for Formulated
Products using Knowledge Graphs. Data Intell. 3, 3 (2021), 340–375. https://doi.org/10.1162/DINT_A_00096[21] Bayu Adhi Tama and Marco Comuzzi. 2019. An empirical comparison of classification techniques for next event prediction using business process event logs.
Expert Systems with Applications 129 (2019), 233–245.

IJCKG’23, December 8 to 9, 2023, Miraikan, Tokyo, Japan[22] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, PietroLiò, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of the
3rd International Conference on Learning Representations, ICLR.
[23] Eli Verwimp, Matthias De Lange, and Tinne Tuytelaars. 2021. Rehearsal Revealed:
The Limits and Merits of Revisiting Samples in Continual Learning. In Proceedingsof the IEEE/CVF International Conference on Computer Vision, ICCV. 9385–9394.
[24] Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S.
Yu. 2019. Heterogeneous Graph Attention Network. In WWW. ACM, 2022–2032.
[25] Shuqi Yu, Linmei Hu, and Bin Wu. 2019. DRAM: A Deep Reinforced IntraAttentive Model for Event Prediction. In Knowledge Science, Engineering andManagement: 12th International Conference, KSEM. 701–713.
[26] Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V.
Chawla. 2019. Heterogeneous Graph Neural Network. In Proceedings of the 25th
SIGKDD Conference on Knowledge Discovery and Data Mining, KDD. 793–803.
[27] Xikun Zhang, Dongjin Song, and Dacheng Tao. 2022. CGLB: Benchmark
Tasks for Continual Graph Learning. In NeurIPS.
http://papers.nips.cc/
paper_files/paper/2022/hash/548a41b9cac6f50dccf7e63e9e1b1b9b-AbstractDatasets_and_Benchmarks.html[28] Xikun Zhang, Dongjin Song, and Dacheng Tao. 2023. Hierarchical Prototype
Networks for Continual Graph Representation Learning. IEEE Trans. Pattern Anal.
Mach. Intell. 45, 4 (2023), 4622–4636. https://doi.org/10.1109/TPAMI.2022.3186909[29] Fan Zhou and Chengtai Cao. 2021. Overcoming Catastrophic Forgetting in Graph
Neural Networks with Experience Replay. In Thirty-Fifth AAAI Conference on
Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational
Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021.
AAAI Press, 4714–4722. https://doi.org/10.1609/aaai.v35i5.16602

Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009