Temporal Extrapolation and Knowledge Transfer for Lifelong Temporal Knowledge Graph Reasoning
Authors: Zhongwu Chen
Chengjin Xu
Fenglong Su
Zhen Huang
Yong Dou
Venue: EMNLP 2023 Findings (2023)
Keywords: Temporal Knowledge Graph
Lifelong Reasoning
Temporal Extrapolation
Knowledge Transfer.

Abstract: Real-world Temporal Knowledge Graphs keep growing with time and new entities and facts emerge continually, necessitating a model that can extrapolate to future timestamps and transfer knowledge for new components. Therefore, our work first dives into this more realistic issue, lifelong TKG reasoning, where existing methods can only address part of the challenges. Specifically, we formulate lifelong TKG reasoning as a temporal-path-based reinforcement learning (RL) framework. Then, we add temporal displacement into the action space of RL to extrapolate for the future and further propose a temporal-rule-based reward shaping to guide the training. To transfer and update knowledge, we design a new edge-aware message passing module, where the embeddings of new entities and edges are inductive. We conduct extensive experiments on three newly constructed benchmarks for lifelong TKG reasoning. Experimental results show the outperforming effectiveness of our model against all well-adapted baselines.

Temporal Extrapolation and Knowledge Transfer for Lifelong Temporal Knowledge Graph Reasoning
Zhongwu Chen', Chengjin Xu²*, Fenglong Su Zhen Huang',, Yong Dou'
Naationl Key Laboratory of Parallel and Distributed Computing. College of Computer, National University of Defense Technology;
International Digital Economy Academy (IDEA);
National University of Defense Technology.
[cheezzhongwu22..sfffnngggogggg htapr//innooonnnnnunnnnnnnnnne .huangzhen,douyong | @ nuddt.edu..n..xucheeee @weee.iideee......
Abstract
(- dayl day30
(- day:3 - dayS8 Japan
{- day59 89
South Kirow
Japan
North Korea
Draka
North
Real-world Temporal Knowledge Graphs keep growing with time and new entities and facts emerge continually, necessitating a model that can extrapolate to future timestamps and trans- fer knowledge for new components. Therefore, our work first dives into this more realistic is- sue, lifelong TKG reasoning, where existing methods can only address part the challenges. Specifically, we formulate lifelong TKG rea- reinforcement
OLE 114/01/22
host
20 1o000771
Philippines
Jupant
North Koroa
Chiine
heat visk
hres visr
China
coosult
2014/6t/8
2014/02/00
consult
0014/12220
2014/07/21
Degoolute
Kaveaa
make statementt
Foanee
China
2014/03//0 Framers
Seen Eaiiiy
Emerging Emeiyy
Relation
time
Figure 1: A sample sequence of three TKG snapshots in lifelong TKG reasoning. New entities (blue nodes) and new facts (all edges) in each snapshot emerge with time.
soning as a camppora--pth-heer learning (RL) framework. Then, we add tempo- ral displacement into the action space of RL to extrapolate for the future and further propose a temporal-rule-baser reward shaping to guide the training. To transfer and update knowledge, we design new edge-aware message passing module, where the embeddings of new entities and edges are inductive. We conduct extensive experiments on three newly constructed bench- marks for lifelong TKG reasoming. Experimen- tal results show the outperforming effectiveness of our model against all edlltad baselines.
TKG reasoning, Figure is an example in dataset ICEWS14 for temporally growing TKGs.
However, SKG reasoning methods (Trouillon et al., 2016) lack the consideration of temporal changing: conventional transductive TKG reason- ing methods (Lacroix et al., 2020) need re-train for their cloled-world assumption; and the latest induc- tive TKG reasoning methods (Chen et al., 2023b) treat emerging entities simultaneous, oversim- plifying the real scenario and thereby lcading their genuine performance to be questionable. Hence, our proposed lifelong TKG reasoning issue more challenging and realistic.
1 Introduction
Knowledge Graphs (KGs) are constructed to store structured facts about human knowledge or the objective world, and formalize facts entities (nodes) and relations (links) between them. Static Knowledge Graphs (SKGs) and Temporal Knowl- edge Graphs (TKGs) are two typical forms of KGs. SKGs store facts in the form of triples (es, r, eo) and TKGs extend triples to quadruples (e,, T; co, t), where L indicates the happening time. Since real- world events are usually ever-changing and associ- ated with time, TKGs are naturally confronted with issues of continually emerging entities and facts in the future timestamps throughout their whole life- cycle (Chen et al., 2023a). Therefore, this paper investigates TKG link prediction task over incom- plete TKGs in the lifelong setting, named lifelong
We formulate the lifelong TKG reasoning as pathhbaa- RL framework and design the whole pipeline for extrapolating, transferring and updating. In the following, we introduce our targeted solutions and expound on their advantages over existing methods.
First, we focus on the temporal displacement between timestamps of candidate edge and its pre- ceding edge and add temporal displacement into RL action space. TKGE models (Xu et al., 2021) rely on embeddings of absolute timestamps and are only fit for the past timestamps. Obviously, they do not meet the requirements of lifelong learning. On the contrary, our used transferable temporal dis-
*Corresponding Author

---

placement in RL can be extrapolated from known timestamps to arbitrary future timestamps based on the magnitude of the displacement. In addition, we further design a shaaing module based on temporal rules found by RL., which only have the temporal order constraints of relations. This mod- ule makes the reasoning get rid of particular entities and will still be applicable for future timestamps. Secondly, lifelong TKG reasoning can be consid- ered as multiple consecutive inductive TKG reason- ing processes. Recently, inductive TKG reasoning methods (Park et al., 2022; Xu et al., 2023) can only deal with future time, not new components, not to mention their ability to continuously learn as required in lifelong TKG reasoning. Therefore, we design a new edge-aware message passing module, which not only transfers learned relation types to initialize emerging entities, but also updates the embeddings of all entities and edges in new TKGs. We also use the embeddings of specific edges rather than immobile relation types, since we seek to ex- plore the concrete environment for each fact to counteract the influence of rapid TKG growth. We build three new benchmarks based on three popular datasets to simulate the lifelong TKG rea- soning scenario. In the experiments, we carefully adapt existing baselines by empowering them with temporal extrapolation or knowledge transfer capa- bilities. In summary, our main contributions are:
ductive setting where they are trained and tested in fixed of components. Recently, induc- tive SKG reasoning has drawn much attention. GralL (Teru et al., 2020), SE-GNN (Li et al., 2021a) and MakEr (Chen et al., 2022) are all GNN- based inductive reasoning models, from the points of view of subgraphs, data relevance and meta- learning. PathCon (Wang et al., 2021) leverages relational message passing for relation prediction, however, we aim the harder entity prediction. Moreover, MINERVA (Das et al., 2018) first in- troduces RL to search for the tail entity of cach query end-to-end. Multi-Hop (Lin et al., 2018) ad- vances MINERVA and does reward shaping based on SKGE methods.
2.2 Inductive TKG Reasoning
Inductive TKG reasoning models mainly deal with entities in the future time. xERTE (Han et al., 2021) is delicately designed to forecast future links by an iterative sampling of temporal neighbours. TGAP (Jung et al., 2021) introduces temporally relevant events in GNN for better explainability. For RL-hased TKG reasoning. CluSTeR (Li et al., 202 lb) regards RL as a clue searching stage, but it strips temporal dimension away from RL and then rearranges the clues in chronological order at the next temporal reasoning stage. However, they can not handle unseen entities emerging with time. TITer (Sun al., 2021) further defines a relative time encoding to distinguish the same entity dif- ferent timestamps and leverage the query informa- tion to represent unseen entities. Different from the above models, we introduce temporal displacement of facts in the RL and propose relation-type-bart knowledge transferring for emerging entities.
our knowledge, we are the first to pose and ex- plore the more challenging and realistic lifelong TKG reasoning issue, which simulates growing TKGs in terms of timestamps, entities and facts, and we formulate it as a RL-based framework.
To solve the challenges of temporal extrapolation and knowledge transfer in lifelong TKG reason- ing, we propose the targeted solutions: temporal displacement, temporal-rule-bat based reward shap- ing and an edge-aware message passing module.
2.3 Lifelong KG Reasoning
Recently, how to retain and reuse previous knowl- edge in a new environment has become research highlight (Wang et al., 2019). MBE (Cui et al., 2022) explores inductive SKG reasoning under the multi-bateh emergence scenario, which is similar to the concept of lifelong KG reasoning without considering time. Next, LKGE (Cui et al., 2023) first formally studies lifelong SKG reasoning via transferring knowledge and using TransE (Bordes et al.. 2013) the base model. However, they did not pay attention the crucial temporal factor in TKGs. To this end, we raise lifelong TKG reason- ing, which involves both unseen components and future timestamps, making this issue challenging
We build three new benchmarks to evaluate our model. Experiments on temporal link prediction show that our model not only achieves the best average performance but also has progressively improving results on growing TKG snapshots.
2 Related Works
2.1 Inductive SKG Reasoning
Traditional SKG reasoning models, such as SKG Embedding (SKGE) methods, focus on the trans-

---

transfer
update
Gi-1
querymleg.a F ?.
5,,,,,
U
9M
action
reward
eq
shaping
selection
M-g
Section3.5
Tav
Section3.4
Section3.3
update
update
conf(R)
Seew Eusly
Rvioiine Type
G31
temporal rule R
Yemporat Displacomerr
transfer
Figure 2: Model overview, e, is candidate entity and öt As is the corresponding temporal displacement (i 1, 2).
and realistic.
achieve knowledge transferring and updating, we inject embeddings of relation types into emerging entities Crew- and then update all the embeddings of entities and edges g in proposed edge-awer aware message passing module (Section 3.4). Section 3.5 describes our designed lemporal-rule-bast reward shaping.
3 Methodology
3.1 Preliminaries
Growing TKGs in lifelong TKG reasoning can be viewed sequence of snapshots: G (G1. G2, ...,Gp), each of which, Gū. contains collection of fact quadruples over continuous time period and 9; = {E,, R,, Tr, Dj}. &i, R., Tr, D, are entity. relation, timestamp and fact sets, and & & E1+1. R2 = R1+1. 4 < 1,41 (4 C TT,441 C 1441), and D, n Di = 0. We use Eai+1 = Ei+1 & and Dii to denote the emerging entities and facts. The TKG link prediction task asks to predict the miss- ing entities of the query edge in incomplete TKGs. For lifelong TKG reasoning, leverage the TKG link prediction task above train new model Mi+1 by transferring and updating knowledge in M; to fit Eai+1 and D++1- Specifically, D, in Gi is divided into training set Fi, validation set V; and test set Q, After finishing the training on F; and validation on Ve, the model M; is evaluated the accumulated test sets 2 for overall assessments.
3.3 Reinforcement Learning Framework
For each edge, we add its reversed edge to 9ú, mak- ing the reasoming traceable and controllable. For each entity at every timestamp to Gi which allows the agent to stay in place, and they work stop actions.
also add self-loop temporal edges
3.3.1 Environment Setting
Our environment can be formulated as a Markov Decision Process (MDP) over TKGs and has the following components. We take G; as an example. States. Let S and (qs (.g) denote all possi- ble states of G3 and the query. At step E {0,, M), the agent locates at entity Cm and timestamp t.m the state sm (em- bm Eg. Jq. (qg E S, Specifi- cally, the initial state is 80 = (Cgs lups Cqp rqs ln). Time-constrained Actions. Let Ar denote the action space of G,. Let AT denote the set of op- tional actions of sm in Gj- Compared with SKGs, the time dimension causes the action space of RL in TKGs extremely large. Hence, add two tem- poral constraints prune the action space, since facts closer to bm in the state 8m under considera- tion are more likely to be directly relevant the prediction:
3.2 Model Overview
Figure 2 is the architecture of our model. Along a sequence of growing TKGs, our model trans- fers and updates knowledge iteratively from the previous TKG snapshot to the next one, avoiding re-training. Inside each Gi, regard the reasoning walk-based action selecting process. An agent starts from the query entity, constantly takes ac- tions through temporal edges based on temporal displacement, and expects to reach the target entity within a limited number of steps (Section 3.3). To
AT = {(6 tm tt) l go= (em, r ,', Gi
fm - S T, I' i' tm S tq) (1)

---

E ); where o' is candidate edge, 1m- -in-iii is the temporal displacement, T is hyperparameterr A/t naturally considers reversed and self-loop temporal edges. Transitions. The transition function S; x A, + S; is deterministic under 93 and updates the states depending on the selected actions.
Eg ReLU (Wg q [uey ungs Toly, hmm
A - σ (Wx Lue, uy' Tal', Dey Dry They: hmm
(3)
where We, W9. Wg.
q: Wa are learnable matrices, (,)) is vector dot product. After scoring all actions, policy network (e Sm) is obtained through softmax.
Rewards. In the default formulation, agents - = Cans), where SM (eM, tM, Eq. rg: (q) is the final state, eans is the answer to the query and is binary indicator function. Our designed reward R(sM) shaped by temporal rules will be described in Section 3.5.
receive reward
3.3.3 Training and Optimization We fix the search path length M. In lifelong TKG reasoning, the policy network is trained by maximizing the expected reward over growing TKG snapshots G1,92, S2, Gp. Hence, our model
3.3.2 Policy Network
is required to train over F2, Fp in turn:
First, the temporal displacement between times- tamps of current state fm (fo 14) and its sub- sequent action tm can integrally capture the time- related dynamics. The temporal displacement is
j(Ø) = Been [B [B
(4)
[R(sar radlll
where E []. Then, use the REINFORCE algorithm iteratively optimize our model:
donated as 6tm bm-1- im S T. Secondly, because of temporally evolving TKGs, even the relation types of two edges in G; and Gi+1 are the same, their semantics can change considerably due to different surrounding environments. It is only by taking surrounding edges into account that we can understand their contextual semantics. Moreover, the above operations are also in line with the foun- dation of RL, which is constant interaction with the environment.
VoJ(0) V Σ R(SM (gsPgnhg)) logTe- (5)
mE(0,M]
3.4 Embedding Transfer and Update
SKGs with static entity properties can be seen as long-term valid knowledge and are helpful to gener- ate accurate evolutional embeddings of entities (Li et al., 2021c; Niu and Li, 2023). Therefore, for each TKG snapshot G. the timestamps masked to convert G; its corresponding SKG snapshot 9
Therefore, the input of policy network has three parts: Uem Ugm Töt... E Rd i.e., the embeddings of entity Cm, edge Im (cm-1, I'm Cm), and tem- poral displacement 8lm (ue.. Ug... are obtained from edge-aware aware message passing module in Sec- tion 3.4, Tetm is transferred from Gi 1 and updated in G.). Path history hm in G3 is encoded as follows:
We adopt relation-type transferring layer over Gj since the relation types of connected edges provide valuable clues about their natures. Our in- troduced transferring layer injects learned knowl- edge about relation types into new entities. For- mally, for new entity in Gi, we generate its beginning embedding u&((c) Rd:
hm = LSTM (hm-1, [uem> Ugm: Totml))
(2)
he = LSTM (0, [uey: Uro Tat,
(e(()) = tanh Σ (e Win
where hm E R2d, ur Rd is the embedding of the special starting relation ro, and 8t, 0. For a candidate next action a' = (e', g'. St') ATT (öt' = Im (') in Eq. 1, we calculate the probability of its state transition based on the correlation of the action and the query in terms of entities and edges:
+E E(endyenn
W Wout up
(6)
where (T =
Wout (6)=((e)r,, (() (e,,,) E g Wi. W Waut E dddd are two learnabe weight matrices. u, E Rd. serving as the embedding of relation type r. is learnable throughout the whole lifelong TKG reasoning process. Furthermore, to avoid recalculating for seen entities, we only generate embeddings for emerging entities and inherit the
2 (d', 8m) X(uue, Tsel WeEg)
+ (1 X) (aug,, Tael W Eg);;

---

embeddings for seen ones from the preceding TKG snapshot Gi-1-
introduce lemporal-nile-asse based reward shaping to guide the training of the agent:
In order to update embeddings for all compo- nents in GT we propose a new edge-aware message passing module via bidirectional communication between edges and nodes. This module enables our model to better adapt to the rapidly changing envi- ronment in lifelong TKG learning, For each edge in 9 the links connected to its two endpoints act as relevant semantic environment.
R(sM) R&(SM) + con con.
(9)
con ( is obtained by dividing the rule support by the body support.
Constructed Benchmarks
To conduct evaluation for lifelong TKG reasoning, construct three new TKG benchmarks based three datasets ICEWS14, ICEWS05-15 (García- Durán et al., 2018) and ICEWS18 (Jin al., 2020) to simulate their growth situation of entities, named as ICEWS14-lifelong ICEWS05-15- 1illeeloog and ICEWS18-lifelle Table shows the statistics of the new benchmarks.
Therefore, we alternately pass edge-aware messages between nodes and edges to aggregate unique environment knowledge for each edge as follows. (e) and (g) are embeddings of entity e and edge g at f-th layer:
u (e) tanh (w ((ee)
Σ
of (=(e W (((c)))u(())
1. Counting. More uniform entity distribution makes the changing of G more concentrated filter entities that oceur
E o' (=(oeo)) W v (u((e),
and significant, less than 10 times and count the remaining entities, relations, timestamps as [8], [R], [T]
" (0)
a <Ww [u(()) -f (cloft) (Cright) +b²
((
2. Lifelong Simulating. New entities emerge almost all the timestamps. Hence, we accu- mulate the number of entities in chronological order. First, we define the set of entities at to = 0 as the initial & (i = 1,..., 5). Sec- ondly, we iteratively add the set of entities at the next timestamp to expand & Once [& 2 TO 41 we stop expanding E,, record the current timestamp t, and then start the searching for 4 in the same way. Thirdly, after obtaining five timestamps t ~ Is. we denote the union of TKGs from 4-1 to 4 as TKG snapshot G.- Since the relations are dense, we can ensure the number of relations in all Gi [Ry), is equal to |R]. For the last TKG snapshot G6. we set 45 = [T] to ensure all facts in D are covered.
(8)
where = = ue). -e( u (g) u, (o') up, r(r') is the relation type of g(g'), Steft and eright are two endpoints of edge g. W self W W. IRaxd, we E Rdx3d and by e Rdd are learnable weight matrices. Message transition function ş (e'),"), uf environment knowledge by calculating the corre- lation between connected entities and edges. is hadamard product.
(g')) stores
After L-layer updating, the final representations of each entity and edge g are (c) and of (g). In the absence of ambiguity, RL (Section 3.3) u, and ug. respectively.
abbreviate them in
3.5 Temporal-RRle-Basee Reward Shaping
3. Dividing. For each TKG snapshot Gi we randomly divide Gi into training set F,, vali- dation set V, and test set Q with ratio 3:1:1.
For a query (eg, Tg. rg: ?, tg) with answer cans- RL gives a reasoning trajectory (leg: F1, (1) r2, 62, (e) (eM-1, TM, eM, tM)), where tq 2 t 2 12 2 2 tM- Then, we can extract temporal rule R. ('M,...,,7y descending temporal constraints and denote the confidence of R as cR). According Sec- tion 3.3.1, since the agent receives a binary reward only based on whether eas is equal to cans, regard- less of the quality of the reasoning temporal paths,
12, T1)
+ Tq with non-

---

92
Benchmarks
[P]
172
16
[Pg]
17g
12
[2.]
18
17g
[6g]
Tifeot4-Jfelonn
3,609
19,690
83
4,333
9,282
39
5,075
11,328
45
5,803
14,937
62
0,533
17,531
67
7.128
17,962
69
SEEWSOS-I5-lifclong 15-lifelong
1,065
9,197
457
1.157
3,843
172 1,249
5,296
229
1,342
8,255
394
1,434
19,064
917
1,520
33,201
1,848
ICEWS1S-Bifelog
1,782 5,418
1,950
1,601
2,107
2,661
2,265
3,854
2,422
8,832
2,519 38,207
Table 1: Statistics of the constructed benchmarks.
[E,), [P:]. [7%] are numbers of entities, facts and timestamps in G,.
5 Experiments
5.2 Results on Lifelong TKG Reasoning
We run 10-seed experiments for all models on our three new benchmarks and report average results on the six TKG snapshots. The MRR, Hits@3 and @its@ 10 results are shown in Table 2. Although the TransE used in LKGE is efficient to model static relationships of entities, it is not sensitive to temporal change. Therefore, LKGE does not deal with the temporal dimension in lifelong TKG reasoning and confuses the timestamps in growing TKGs. As a result, LKGE only has comparable Hits 10 results. Similar to our model, the adapted L-MBE uses a RL framework, benefiting its rea- soning process. The static link augmentation in L-MBE is also proven to be effective in TKG rea- soning (Niu and Li, 2023). Even so, the results of L-MBE are still worse than ours. The reason is that our model specifically constructs a more targeted RL environment for lifelong TKG reasoning and further takes temporal displacement into account for temporally extrapolating.
5.1 Experimental Setup
Baselines.
We compare model against the SOTA lifelong SKG reasoning model LKGE (Cui al., 2023). LKGE uses Transl the knowledge transfer mod- ule and can be well-adapted to lifelong TKG rea- soning. MBE (Cui et al., 2022) is designed for multi-batch emergence scenario and is also a pow- erful baseline since it leverages walk-based reason- ing and has an inductive entity encoding module. We name MBE adapted for lifelong TKG reasoning L-MBE.
Based on the framework of LKGE, we leverage the SOTA TKGE model TeRo (Xu et al., 2020) for L-TeRo, by defining the timestamp embed- dings as a linear function. L-TTTer, L-TGAP and L-TLogic were modified from TITer (Sun et al., 2021). TGAP (Jung et al., 2021) and TLogic (Liu et 2022). TITer is encapsulated according the requirements of lifelong TKG reasoning, forming L-TITer. TGAP can handle future timestamps, but is not inductive, so L-PG randomly initializes the emerging entities. TLogic is based on entity- independent temporal logical rules, so L--TLogic can be used for lifelong TKG reasoning by trans- ferring its found rules. Evaluation Metries.
However, the frameworks of LKGE and L-TeRo are based on traditional KGE and TKGE models, limiting their performances in lifelong TKG reason- ing. L-TITer, L-TGAP and L-TLogic are adapted from recent powerful TKG link prediction models, but there is a certain degree of reduction in their original performances when used in lifelong TKG reasoning. The reasons are that the IM module in TTTer is pareeeter-free, so the knowledge can not be transferred and updated; TGAP is in the interpo- lation setting, the embeddings of future times- tamps can not be well initialized: TLogic relies on temporal rules, but the transferred old rules may conflict with the new ones in future TKG snapshots. Therefore, these existing baselines can not perform perfectly in the lifelong TKG reasoning. On the contrary, our model consistently outperforms all the baselines across the three benchmarks. Some results of our model are twice that of basclines. This is because our model comprehensively con- siders all the requirements of lifelong TKG reason- ing and the proposed three targeted solutions solve the problems of temporal extrapolation, knowledge transfer and knowledge update.
Following the convention, we conduct the exper- iments on the temporal link prediction task and report Mean Reciprocal Rank (MRR) and Hits @ (k: 3, 10). Then, we evaluate the knowledge lifelong learning capability using forward transfer (FWT) and backward transfer (BWT) (Lopez-Paz and Ranzato, 2017).
1
PWT= p- A p (ha. - ha.))
where P is the number of TKG snapshots, hag is the MRR scores on Q, after training the model Mj on F,,

---

ICEWSO5-15-lifclong
MRR
Hits@3
Hits@ 10
MRR
Hits@3
Hits 10
MRR
Hits
Hits@ 10
LKGE
0.178
22.73
44.31
0.214
34.91
53.73
0.132
17.78
31.77
L-MBE
0.349
37.32
46.93
0.287
32.39
44.80
0.155
16.97
25.03
L-TeRo
0.182
27.34
47.03
0.280
36.95
55.20
0.196
26.14
34.94
L--TTTer 0.193
21.95
33.26
0.275
31.87
48.25
0.249
31.42
40.35
L-TGAP 0.220
24.84
36.92
0.309
34.71
51.45
0.293
37.52
43.67
L-TLogic 0.323
36.83
47.28
0,296
33.73
45.67
0.251
37.40
39.09
Ours 0.377
39.64
49.79
0.323
38.25
57.35
0.385
46.56
55.26
Table 2: Average lifelong TKG reasoning results on the six TKG snapshots (% for @3 and Hits@ 10). Bold numbers denote the best results.
A4
M1g
Ads
A44
Ats
.Afs
ICEWSt4-Hilleg
ICEWSI8-liitoon
0.7
0.2
our model
LKGE
828
BLKGP EI-ITIP
620
BLKGE
MAMI
BI-TT
MRR
818
EL T88 EOTT moTe
BL Taka Borr modal
8.33
0.11
698
650
4010
L-MBE
L-TGAP
-0.0
PWT
BWT
FWT
BWT
MRR
0.3
Figure 4: FWT and BWT of MRR results on bench- marks: ICEWS14-lifelong and
036
0.25
Figure 3: MRR changes on -o-ffooo- of our model and three baselines: LKGE, L-MBE, L-TGAP.
update knowledge for new TKGs. Empirically, per- formance on G1 is the upper bound of our model since G1 injects knowledge to our model and knowl- edge transfer and update start when G1 grows to G2-
5.3 Performance Evolution
To demonstrate the performance evolution of our model and three baselines during lifelong TKG reasoning, the MRR results on all TKG snapshots are reported in Figure 3.
5,4 Knowledge Lifelong Learning Capability
To accurately quantify the knowledge lifelong learning capability of all models lifelong TKG reasoning, we report the FWT and BWT scores of MRR results Figure 4. Since we specifically de- sign the whole reasoning pipeline for extrapolating and transferring, the FWT scores of our model are the best among all baselines. LKGE and L-MBE originally suitable for lifelong KG reasoning, so they work relatively well over SKGs. The FWT scores of L-TLogic are also poor because its rule- based strategy is too restricted by symbolic rela- tions to reason over future TKGs.
We find:
(i) For the same G; as our model learns over growing TKGs, the performance M; MM on G; remains relatively steady. This suggests that our model has a strong ability to transfer embeddings to emerging components continuously and avoids forgetting previous knowledge. However, the other three baselines all experience rapid degradation in performance and suffer from catastrophic forget- ting.
(ii) For the same Mj, we observe its perfor- mance changing on different TKG snapshots. Since old knowledge is not always suitable for new facts, need to update learned knowledge, otherwise the performance of Mr will drop. The inductive embedding layer in L-MBE sometimes succeeds in updating (from G2 to 93). L.KGE and I-TGAP clearly fail to update knowledge for the decreasing MRR. This demonstrates our powerful ability to
The BWT scores of baselines are all negative due to the overwriting of relation types embeddings the result of their update of learned knowledge. On the contrary, proposed edge-awarc message passing module focuses on updating the unique en- vironment of each fact rather than individual rela- tion types, which makes our model introduce richer information for more accurate predictions. The low BWT scores of L-TeRo show the inefficiency

---

Query
(Sudan, sign formal agreement, Ethiopia, 2014/12/25)
Confidence
(Sudan, praise or endorse, Ethiopia, 2014/1 1/26)
(Ethiopia, provide military aid, Sudan, 2014/10/19)
0.75
(Sudan, cooperate economically, Ethiopia, 2014/09/15)
(Sudan, sign formal agreement, South Sudan, 2014/11/26)
0.64
(South Sudan, express to meet, Ethiopia, 2014/10/22)
(Sudan, Criticize or denounce, Barack Obama, 2014/12/12)
0.29
(Barack Obama, sign formal agreement, China, 2014/12/01)
Table
Reasoning trajectories and confidence of corresponding temporal rules. Target entities are underlined.
Benchmarks
Ours
w/o td
w/o mp
w/o IS
MRR results decrease by 18.07% on average over three benchmarks, indicating the significance of considering temporal displacement in RL, because be temporally extrapolated the future time. Secondly, we remove our proposed edge-aware message passing module (w/o mp) and randomly initialize new entities. In this case, our model can not transfer update knowledge and the agent not capture the specific environment of each edge in RL. This leads to a drop of 10.86% on average on MRR, implying the importance of this module.
(CEWS14-lifelon)
0.377
0.321
0.335
0.349
ICEWS05- Iiflelon
0.323
0.268
0.284
0.291
ICEWS18-Lifelong
0.385
0.299
0.349
0.362
Table 4: MRR results of ablation study for our model the three benchmarks.
TKGE in lifelong TKG reasoning. L-TITer and L-TGAP are well-adapted baselines by combining their original TKG reasoning ability with the de- mands of lifelong TKG reasoning, their BWT scores are better than other baselines.
Finally, we leverage the original binary reward to replace the temporal-rule-based reward shaping in our model (w/o rs) and obverse 7.77% per- formance degradation. This phenomenon means, by the training guidance of temporal rules inde- pendent of particular entities, our model obtains comprehensively enhanced ability to select reliable actions.
5.5 Case Study
Reasoning Trajectories and Temporal Rules. To demonstrate the reasoning ability of our model over the temporal edges, we perform a study in Table which shows two positive reasoning jectories and a negative one for the target query (Su- dan, sign formal agreement, Ethiopia, 2014/12/25). Then we further give the scores of confidence of the corresponding temporal rules extracted from the reasoning trajectories. The length of paths is set to 3 and we remove the self-loop actions for clarity, It can be seen that our model succeeds in telling the reasonable temporal rules with high confidence from weak ones with low confidence and thereby guides the action selection stably and efficiently.
Conclusion
In this work, the lifelong TKG reasoning issue in- volves continually emerging entities and facts in the future timestamps. To address this new prob- lem, we propose our model under the framework of RL and our model uses temporal displacement the action space to extrapolate to the future times- tamps; uses a new edge-aware message passing module to inductively transfer and update learned knowledge to new entities and facts; and uses temporal-rule-bon reward shaping to guide the training. The experimental results on three newly constructed benchmarks illustrate that our model has the best performance for lifelong TKG reason- ing and the strongest knowledge lifelong learning capability.
5.6 Ablation Study
To further examine the effect of the three proposed solutions for lifelong TKG reasoning in our model, we conduct an ablation study as shown in Table 4, First, we replace temporal displacement in RI. with timestamps (w/o td), i.c., rely on embeddings explicit timestamps while selecting actions. The

---

Limitations
Yuanning Cui, Yuxin Wang, Zequn Sun. Wenqiang Liu, Yiqiao Jiang, Kexin Han, and Wei Hu. 2023. Life- long embedding learning and transfer for growing knowledge graphs. In AAAI.
This paper mainly focuses on lifelong TKG rea- soning, where we consider emerging entities and facts in the future timestamps as TKG grows, and do not consider the changing case of relations. In most cases, the number of entities in TKGs is much larger than that of relations and the emergence of entities lasts longer and is more common than that of relations. For instance, ICEWS14 has 7128 en- tities and 230 relations; the accumulated number of relation types in ICEWS14 increases rapidly to 115, half of the total number, in 10 days, on the contrary, the accumulated number of entities is steadily increasing over the entire time period of ICEWS14. Therefore, study the more severe case of emerging entities and leave the research for emerging relations in lifelong TKG reasoning to future works.
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy. Alex Smola, and Andrew McCallum. 2018. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning.
Alberto García-Durána Sehastijan Dumancic, and Math- ias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Confer- ence on Empirical Methods in Natural Language Processing.
Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. 2021. Explainable subgraph reasoning for forecast- ing on temporal knowledge graphs. In Proc. of ICLR.
Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. 2020. Recurrent event network: Autoregressive struc- ture inferenceover temporal knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6669- 6683. Association for Computational Linguistics.
References
Antoine Bordes, Nicolas Usunier, Alberto Gareía- Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Pro- ceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 2787-2795.
Jaehun Jung, Jinhong Jung. and U Kang. 2021. Learn- ing to walk across time for interpretable temporal knowledge graph completion. In Proc. of KDD.
Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view.net.
Mingyang Chen, Wen Zhang, Zhen Yao. Xiangnan Chen, Mengxiao Ding, Fei Huang, and Huajun Chen. 2022. Meta-learning based knowledge extrapolation for knowledge graphs in the federated setting. In Pro- ceedings of the Thirty-Firss International Joint Con- ference on Artificial Intelligence, IJCAI-22, pages 1966-1972. International Joint Conferences on Arti- ficial Intelligence Organization. Main Track.
Ren Li, Yanan Cao, Qiannan Zhu, Guanqun Bi, Fang Fang, Yi Liu, and Qian Li. 2021a. How does knowl- edge graph embedding extrapolate to unseen data: a semantic evidence view. In AAAI.
Zhongwu Chen, Chengjin Xu, Fenglong Su, Zhen Huang, and Yong Dou. 2023a. Incorporating struc- tured sentences with time-enhanced bert for fully- inductive temporal relation prediction. In Proceed- ings of the 46th International ACM SIGIR Confer- ence on Research and Development in Information Retrieval, SIGIR '23, page 889-899, New York, NY, USA. Association for Computing Machinery.
Zixuan Li, Xiaolong Jin, Saiping Guan, Wei Li, Jiafeng Guo, Yuanzhuo Wang, and Xueqi Cheng. 202 Search from history and reason for future: Two-stage reasoning temporal knowledge graphs In Pro- ceedings of the 59th Annual Meeting of the Associa- tion for Computational Linguistics and the I Itth Inter- national Joint Conference on Natural Language Pro- cessing (Volume Long Papers). pages 4732-4743, Online. Association for Computational Linguistics.
Zhongwu Chen, Chengjin Xu, Fenglong Su, Zhen Huang. and Yong Dou. 2023b. Meta-learning based knowledge extrapolation for temporal knowledge graph. In Proceedings of the ACM Web Conference 2023, WWW '23, page 1433-2443 New York, NY, USA. Association for Computing Machinery.
Zixuan Li. Xiaolong Jin, Wei Li. Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. 2021c. Temporal knowledge graph reason- ing based on evolutional representation learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, SIGIR '21, page 408-417, New York, NY, USA. Association for Computing Machin- ery.
Yuanning Cui, Yuxin Wang, Zequn Sun, Wenqiang Liu, Yiqiao Jiang, Kexin Han, and Wei Hu. 2022. In- ductive knowledge graph reasoning for multi-bater emerging entities. In CIKM.

---

Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2018. Multi-hop knowledge graph reasoning with reward shaping. In Proceedings of the 2018 Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2018, Brussels, Belgium, Octo- ber 31-November 4, 2018.
Chengjin Xu, Yung- Yu Chen, Mojtaba Nayyeri, and Jens Lehmann. 2021. Temporal knowledge graph completion using a linear temporal regularizer and multivector embeddings. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2569-2578, Online. Association for Computational Linguistics.
Yushan Liu, Yumpu Ma, Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. 2022. Tlogic: Temporal logical rules for explainable link forecasting on tem- poral knowledge graphs. AAAI.
Chengjin Xu, Mojtaba Nayyeri, Fouad Alkhoury, Hamed Shariat Yazdi, and Jens Lehmann. 2020. TeRo: A time-aware knowledge graph embedding via temporal rotation. In Proceedings of the 28th International Conference on Computational Linguis- rics, pages 1583-1592 Barcelona, Spain (Online). International Committee on Computational Linguis- tics.
David Lopez-Paz and Marc' Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In NPP.
Guanglin Niu and Bo Li. 2023. Logic and commonsense-guinn guidedd temporal knowledge graph completion. In AAAI.
Yi Xu, Junjie Ou, Hui Xu, and Luoyi Fu. 2023. Tem- poral knowledge graph reasoning with historical con- trastive learning. AAAI.
Namyong Park, Fuchen Liu, Purvanshi Mehta, Dana Cristofor, Christos Faloutsos, and Yuxiao Dong. 2022. Evokg: Jointly modeling event time and network structure for reasoning over temporal knowledge graphs. In Proceedings the Fifteenth ACM Interna- tional Conference on Web Search and Data Mining, WSDM '22, page 794-803, New York, NY, USA. Association for Computing Machinery.
A Appendix
A.1 Implementation Details.
we tune the hyperparameters of our model us- ing grid-search: learning rate in (0.0005, 0.0001, 0.001 batch size in (4096, 8192), embedding dimension in ( (100, 200, 300). The length of the walking path of the agent in RI, M, is tuned in (3. 4, 5). The layer of our proposed edge-aware message passing module, L,, is in a range of (1, 2, } The hyperparameter T in Section 3.3.1 is 40 in ICEWSI4
Haohai Sun, Jialun Zhong, Yunpu Ma, Zhen Han, and Kun He. 2021. Travelr::: Reinforcement learn- ing for temporal knowledge graph forecasting. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8306 8319, Online and Punta Cana, Dominican Re- public, Association for Computational I Linguistics.
300 in ICEWS05- 1----00----iiflong.. 29 in ICEWS18 88 1--lifloong. While training. The dis- count factor of REINFORCE is 0.95. We clip gradi- ents greater than 20 to avoid the gradient explosion. While testing, we use beam search to obtain a list of predicted entities with the corresponding scores. The beam size is set to 100. The re-run baselines are based on their public codes and further adapted for our constructed benchmarks for lifelong TKG reasoning. All the experiments are carried out on one A100 GPU.
Komal Teru, Etienne Demis, and Will Hamilton. 2020. Inductive relation prediction by subgraph reasoning. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 9448-9457. PMLR
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Interna- tional Conference on Machine Learning.
Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. 2019. Sen- tence embedding aligmment for lifelong relation ex- traction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, WAACL-HLT 2019, Minneapolis, MN, USA. June 2-7, 2019, Volume I (Long and Short Papers), pages 796-806. Association for Computational Lin- guistics.
A.2 Computational Complexity Analysis.
To see the efficiency of our proposed model, we analyze the computational complexity of the pro- cess of knowledge transferring and updating. There are two parts in the edge-aware message passing module, one is for initializing the embeddings for new entities, and the other is for updating all com- ponents. First, for the first part, the expected time complexity of Eq. 6 in each iteration is ((E)), where |E| is the maximum number of new entities in the six TKG snapshots. Secondly, for the next
Hongwei Wang, Hongyu Ren, and Jure Leskovec. 2021. Relational message passing for knowledge graph completion. KDD '21, page 1697-1707, New York, NY, USA. Association for Computing Machinery.

---

part, if there are X emerging entities and edges connected new entities, each emerging entity takes [[d] 2Y elements as input expectation, X
where [[d] is the expected node degree. The cost aggregation is X [R[d] 2Y. Therefore, Eq. 7 has the time complexity (22)D)) For Eq. 8, it is performed for [DI times and each update takes 3 elements input. Therefore, the cost of update in each iteration is ((3 We iterate these two updating functions for L times, the overall time complexity (LL)D) where [D] is the maximum number of edges the six TKG snapshots. Finally, the time complexity the process of knowledge transferring and updating is [[ + L(D)). where [E] is the maximum number of new entities, [D] is the maximum number of edges in the six TKG snapshots; L is the iteration time.


Reviews:
[{"review_id": "Eaa2oo5Mpd", "reviewer": "Reviewer_NDHZ", "summary": "", "questions": "A. In Line 136, ‘the harder entity prediction’, the expression is too vague and needs more explanation. \\\nB. In the abstract, ‘lifelong TKG reasoning’ is the task, not the issue. The issue may indicate the unseen entities and future timestamps.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Response to the Concerns of Reviewer NDHZ.__\n\nWe sincerely thank you for your valuable suggestions. We respond to your comments and provide clarifications as follows.\n\n> Question 1.  According to the introduction, this method is actually to calculate the relative time of facts, and the idea of using the relative time to assist TKG reasoning has been proposed in TITer. What’s the difference between temporal displacement and relative time in TITer?\n\n__Response to Question 1__: Thanks for your comment. We would like to clarify that the concept of calculating the relative time of facts in tasks related to TKGs has been widely adopted for a long time. Our contribution lies in the way our model leverages this concept (in various perspectives from TITer), rather than proposing it.\n\n> Question 2. The proposed Temporal-Rule-Based Reward Shaping module seems an incremental method based on previous methods.\n\n__Response to Question 2__: Thanks for your comment. We would like to make the following clarification.\n\nThis module is one of several proposed modules in our model. First, we have accurately identified the research gaps in the technical weaknesses of existing RL-based KG reasoning works through a comprehensive examination. Secondly, we propose a new Temporal-Rule-Based Reward Shaping module that specifically addresses these research gaps. Thirdly, the experiments demonstrate the superiority of our proposed module, affirming the novelty of our contributions.\n\n\n> Question 3. In Line 136, ‘the harder entity prediction’, the expression is too vague and needs more explanation.\n\n__Response to Question 3__: Thanks for your comment. In line 136, we wrote \"PathCon leverages relational message passing for relation prediction, however, we aim at the harder entity prediction.\", which means that compared with the relation prediction task, the entity prediction task is more challenging. For better understanding, we will revise the writing to \"PathCon leverages relational message passing for the relation prediction task, however, we aim at the entity prediction task, which is more challenging. Because there are more entities than relations.\"\n\n> Question 4. In the abstract, ‘lifelong TKG reasoning’ is the task, not the issue. The issue may indicate the unseen entities and future timestamps.\n\n__Response to Question 4__: Thanks for your suggestions. \n\n\"Unseen entities and future timestamps\" are the challenges in our proposed \"lifelong TKG reasoning\", which is a new issue. The task is TKGC.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A. This paper introduces a new setting for temporal knowledge graph reasoning, i.e., lifelong. \\\nB. The code is available.", "reasons_to_reject": "A. The introduction to Temporal-Rule-Based Reward Shaping in Section 3.5 is not clear enough, and the authors are expected to expand on Eq. 9 further. \\\nB. According to the introduction, this method is actually to calculate the relative time of facts, and the idea of using the relative time to assist TKG reasoning has been proposed in TITer. What’s the difference between temporal displacement and relative time in TITer? \\\nC. The proposed temporal-rule-based reward shaping module seems an incremental method based on previous methods. \\\nD. About the first case study in Sec 5.5, it’s more like motivation and illustration, not a case study of your experimental result.", "questions_to_authors": "A. In Line 136, ‘the harder entity prediction’, the expression is too vague and needs more explanation. \\\nB. In the abstract, ‘lifelong TKG reasoning’ is the task, not the issue. The issue may indicate the unseen entities and future timestamps.", "paper_topic_and_main_contributions": "This paper proposes the temporal displacement, temporal-rule-based reward shaping and an edge-aware message passing module to solve the challenge of lifelong TKG extrapolation task. The authors extract three dedicated datasets suitable for lifelong TKG extrapolation tasks from existing datasets and the proposed method achieves state-of-the-art performance on these datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eNDrY8PE5g", "reviewer": "Reviewer_dRYt", "summary": "", "questions": "The authors provided the code for reproduction but I couldn't access the link.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Response to the Concerns of Reviewer dRYt.__\n\nWe sincerely thank you for your valuable suggestions. We respond to your comments as follows\n\n> Question 1.  The proposed method outperforms other methods, but more experiments are required. First, experiments on new entities should be conducted to evaluate the proposed method. One of the contributions is on inductive TKG reasoning. Detailed analysis on inductive reasoning is required.\n\n__Response to Question 1__: Thanks for your comment. \n\nFirst, we clarify the distinctions and connections between our proposed lifelong TKG reasoning and the inductive TKG setting. Inductive setting treats new entities as simultaneous. Lifelong TKG reasoning is a multiple consecutive inductive TKG reasoning processes (as explained in lines 34-41 and illustrated in Figure 1) and thus naturally requires to consideration of new entities in experiments. Therefore, all of our experiments involve new entities. Figure 3 proves the validity of our model for lifelong TKG reasoning at the same time as it demonstrates our ability for inductive TKG setting.\n\n> Question 2.  To evaluate the proposed method, experiments on standard TKG setting would be helpful. The proposed method shows good results and how strong it is on standard TKG setting substantiates the superiority of the proposed method.\n\n__Response to Question 2__: Thanks for your suggestions. We clarify two reasons for not including experiments on the standard TKG setting.\n\n+ Reason 1: Our proposed lifelong TKG reasoning is essentially different from the standard transductive setting.\n\n+ Reason 2: It is also worth noting that there are many papers on inductive reasoning, such as RMPI[2], MorsE[3], SE-GNN[4], MBE[5], LKGE[6] and MaKEr[7], also do not conduct experiments in the standard transductive setting, since our model and all these works do not focus on the standard transductive setting. \n\n\n> Question 3. The authors provided the code for reproduction but I couldn't access the link.\n\n__Response to Question 3__: As the reviewer NDHZ said, \"The code is available\". Please copy \"https://anonymous.4open.science/r/EMNLP2023_ID520\" to a browser.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposed a new task setup, the lifelong TKG reasoning, and construct datasets for the task.\n2. The proposed method outperforms other competitors.\n3. The paper is well written and easy to follow.", "reasons_to_reject": "1. The proposed method outperforms other method, but more experiments are required. First, experiments on new entities should be conducted to evaluate the proposed method. One of the contributions is on inductive TKG reasoning. Detailed analysis on inductive reasoning is required.\n2. To evaluate the proposed method, experiment on standard TKG setting would be helpful. The proposed method shows good results and how strong it is on standard TKG setting substantiates the superiority of the proposed method\n3. It would be better to include other baselines such as RE-NET [Jin et al., 2019], RE-GCN [Li et al., 2021].", "questions_to_authors": "The authors provided the code for reproduction but I couldn't access the link.", "paper_topic_and_main_contributions": "The paper first explored the lifelong TKG reasoning and proposed a model under the framework of RL. In the experiments, they illustrate that their model shows the best performance.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i8kXsArNly", "reviewer": "Reviewer_kB2c", "summary": "", "questions": "In Section 5.3, the author mentions that the proposed model has a strong ability to avoids forgetting previous knowledge. Are there any potential explanation besides the empirical results?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "__Response to the Concerns of Reviewer kB2c.__\n\nWe sincerely thank you for providing supportive comments and acknowledging the technical soundness of our work. We respond to your comments as follows.\n\n> Question 1.  The author only evaluates the model on ICEWS datasets but not on large-scale datasets such as GDELT, which contains several millions quadruples and more realistic.\n\n__Response to Question 1__:Thanks for your suggestions. Our experiments ask to reprocess the datasets and GDELT is very large. We plan to conduct the experiments for GDELT in the future.\n\n\n\n> Question 2.  In Section 5.3, the author mentions that the proposed model has a strong ability to avoid forgetting previous knowledge. Is there any potential explanation besides the empirical results?\n\n__Response to question 2__: This is a natural and good question. \n\nThe learned knowledge can be inherited from the preceding TKG snapshot to the subsequent TKG snapshot. \nSo, the previous knowledge is well remembered and utilized in our model to avoid forgetting.\n\nWe will cite the missing papers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is the first to explore the lifelong TKG reasoning problem, which simulates growing TKGs in terms of unseen timestamps, new emerging entities and events. The approach inject embeddings of relation types into emerging entities for knowledge transfer and update.   \n\nBesides, the paper proposed a reinforcement learning framework for solving the lifelong TKG reasoning task, where an agent starts from the query entity, constantly takes actions through temporal edges based on temporal displacement, and to search the target entity.   \n\nMoreover, the author also introduced three new TKG benchmarks for lifelong reasoning based on three datasets ICEWS14, ICEWS05-15, and ICEWS18, which could foster future research. In particular, the author simulate the reality by accumulating the number of entities in chronological order, and thus, let new entities emerge at almost all the timestamps. Empirical results on the datasets show the superiority of the proposed method compared to other baselines.  \n\nThe author also investigated the performance evolution of the proposed model, demonstrating that the model’s performance remains relatively steady over growing temporal knowledge graphs.", "reasons_to_reject": "The author only evaluates the model on ICEWS datasets but not on large-scale datasets such as GDELT, which contains several millions quadruples and more realistic.", "questions_to_authors": "In Section 5.3, the author mentions that the proposed model has a strong ability to avoids forgetting previous knowledge. Are there any potential explanation besides the empirical results?", "paper_topic_and_main_contributions": "The paper studies the lifelong temporal knowledge graph reasoning problem, where a model is able to extrapolate missing links at future unseen timestamps and transfer learned graph embeddings to new emerging entities. The author formulate the lifelong TKG reasoning as a temporal-path-based RL framework. Specifically, the author focus on the temporal displacement between timestamps of candidate edge and its preceding edge and add temporal displacement into RL action space. Besides, the paper introduced a new edge-aware message passing module, which transfers learned relation types to initialize emerging entities. The author constructs three new datasets based on three benchmark datasets to simulate the lifelong TKG reasoning scenario. Experiments on temporal link prediction demonstrate the effectiveness of the proposed approach.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "There are some missing references on temporal knowledge graph forecasting:\n\nTrivedi, Rakshit, Hanjun Dai, Yichen Wang, and Le Song. \"Know-evolve: Deep temporal reasoning for dynamic knowledge graphs.\" In international conference on machine learning, pp. 3462-3471. PMLR, 2017.\n\nHan, Zhen, Zifeng Ding, Yunpu Ma, Yujia Gu, and Volker Tresp. \"Learning neural ordinary equations for forecasting future links on temporal knowledge graphs.\" In Proceedings of the 2021 conference on empirical methods in natural language processing, pp. 8352-8364. 2021.\n\nBesides, there are also some missing reference on learning unseen entities on TKG reasoning such as\n\nDing, Zifeng, Jingpei Wu, Bailan He, Yunpu Ma, Zhen Han, and Volker Tresp. \"Few-Shot Inductive Learning on Temporal Knowledge Graphs using Concept-Aware Information.\" AKBC 2022."}}]

Author rebuttal:

Meta-review: The paper studied the lifelong temporal knowledge graph reasoning problem. The authors first constructed datasets for this task and proposed a model under the framework of RL. 

All reviewers agreed that the lifelong TKG task is important and novel. Nevertheless, the reviewers also raised some concerns on the experiments setup and comparison. Most of these concerns were addressed by the authors during the rebuttal phase.
Decision: Accept-Findings