INFER: A Neural-symbolic Model For Extrapolation Reasoning on Temporal Knowledge Graph
Authors: Ningyuan Li
Haihong E
Tianyu Yao
Tianyi Hu
Yuhan Li
Haoran Luo
Meina Song
Yifan Zhu
Venue: ICLR 2025 Poster (2025)
Keywords: Knowledge Graph
Temporal Knowledge Graph
Temporal Rules
Temporal Validity
Primary area: learning on graphs and other geometries & topologies

Abstract: Temporal Knowledge Graph(TKG) serves as an efficacious way to store dynamic facts in real-world. Extrapolation reasoning on TKGs, which aims at predicting possible future events, has attracted consistent research interest. Recently, some rule-based methods have been proposed, which are considered more interpretable compared with embedding-based methods. Existing rule-based methods apply rules through path matching or subgraph extraction, which falls short in inference ability and suffers from missing facts in TKGs. Besides, during rule application period, these methods consider the standing of facts as a binary 0 or 1 problem and ignores the validity as well as frequency of historical facts under temporal settings.
In this paper, by designing a novel paradigm for rule application, we propose INFER, a neural-symbolic model for TKG extrapolation. With the introduction of Temporal Validity Function, INFER firstly considers the frequency and validity of historical facts and extends the truth value of facts into continuous real number to better adapt for temporal settings. INFER builds Temporal Weight Matrices with a pre-trained static KG embedding model to enhance its inference ability. Moreover, to facilitates potential integration with existing embedding-based methods, INFER adopts a rule projection module which enables it apply rules through conducting matrices operation on GPU. This feature also improves the efficiency of rule application. 
Experimental results show that INFER achieves state-of-the-art performance on various TKG datasets and significantly outperforms existing rule-based models on our modified, more sparse TKG datasets, which demonstrates the superiority of our model in inference ability.

Published as a conference paper at ICLR 2025
INFER:
A NEURAL-SYMBOLLE MODEL FOR EXTRAP- OLATION REASONING ON TEMPORAL KNOWLEDGE GRAPH
Ningyuan Li', Haihong E Tianyu Yao', Tianyi Hu', Yuhan Li', Haoran Luo',
Meina Song', Yifan Zhu
Beijing University of Posts and Telecommunications
ABSTRACT
Temporal Knowledge Graph(TKG) serves as an efficacious way to store dynamic facts in real-world. Extrapolation reasoning on TKGs, which aims predicting possible future events, has attracted consistent research interest. Recently, some rule-based methods have been proposed, which considered more interpretable compared with embedding-basee methods. Existing bile-basd methods apply rules through path matching subgraph extraction, which falls short in inference ability and suffers from missing facts in TKGs. Besides, during rule application period, these methods consider the standing of facts binary or 1 problem and ignores the validity as well as frequency of historical facts under temporal settings. In this paper. by designing a novel paradigm for rule application, we propose IN- FER, neural-symbolic model for TKG extrapolation. With the introduction of Temporal Validity Function, INFER firstly considers the frequency and validity of historical facts and extends the truth value of facts into continuous real number to better adapt for temporal settings. INFER builds Temporal Weight Matrices with pre-trained static KG embedding model to enhance its inference ability. More- over, to facilitates potential integration with existing embedding- based methods, INFER adopts a rule projection module which enables it apply rules through con- ducting matrices operation on GPU. This feature also improves the efficiency of rule application. Experimental results show that INFER achieves state-of-the-art performance on various TKG datasets and significantly outperforms existing rule- based models on our modified, more sparse TKG datasets, which demonstrates the superiority of our model in inference ability.
INTRODUCTION
Knowledge Graphs(KGs) play key roles in multiple downstream applications (Hildebrandt et al. 2019 Lan & Jiang] 2020). which store knowledge in the form of triples (s, o) representing subject entity and object entity are linked by the relation r. Traditional KGs appear to be static snapshots of real-world facts. However, in practice, facts evolve over time and may not be true in perpetual way. Thus, TKGs are proposed to model the dynamic properties of facts in which each fact is represented quadruple (s, r, o, t) where t denotes the timestamp of the fact.
Extrapolation reasoning on TKGs focuses on predicting potential facts at future timestamps based on the observed historical facts. Some effective embedding based models including RE-Net (Jin jet al. 2019), CyGNet ZZhu et al. (021a), RE-GCN (Li et al. 2021), CENET (Xu et al. 2022) etc. have been proposed for TKG extrapolation, which embed entities and relations into vectors and utilize neural networks to capture structural information, temporal dependency and historical information. Embedding- substantial performance. However, due to the "black box problem" of neural networks, embedding- based methods are deemed to be lacking interpretablity and reliability. Recently, several rule-baser methods like TLogic (Liu et al. 2022) and TR-Rule, (Li et al. 2023) have been proposed for better
models exhibit strong learning and reasoning abilities and yield
"Corresponding author.

---

Published as a conference paper at ICLR 2025
Rule: (X, Unsatis fied_With, Ii) A Trade_ Z, 22) - (X, Plays_ Z, T)
Unsatisfied With
35
Express
THUNDER
Negotiate- with
THUNDER
Interest
Plays For ?
LIKE
35
LIKS
Trade with
Defeated
(Missed)
T3
WARNIONK
by
T1
T2
Figure 1: Illustration of how missing facts in TKGs affect the rule application procedure.
Query (Germany, Praise. I.T.)
Role
[X. C...hhh Y.J) AV (Y. Praine, Z. -2 (Xt, Praise, 2,3)
(Rloou phane, 0.4)
Prrate,
MAk
Dican plone
(Camali 6.1)
0.12
Crnnal
CHICIR
greabe
Makae statearnt
(Praios 8.4) (Crsleiss, 1)
To
Ti
Tn-1
time
(b)
Figure 2: (a) Illustration of TKGs and previous rule-based inference. (b) Our transformed weighted graphs and the performed temporal validity aware inference.
interpretability. These models can generate human-readable rules and provide visible reasoning chains, which also achieve competitive performance.
Nevertheless, when applying rules, previous rule- based methods consider the standing of facts as a binary 0 or problem (have been true or not) for path matching, which ignores the validity as well as frequency of historical facts under temporal settings. Although previous models try add bias in the score function to increase the score of candidates derived by recent edges paths, it does not completely solve this issue especially when rule length is greater than one.
In Figure 2 (a), we can see typical TKG and the process of rule application of previous work. The red edges denote matched rule body instances. As we can see, Germany consulted the U.S. at To and the U.S. praised Japan and Korea at Ti and T respectively. Given the query and the rule, previous models will consider both entities correct and give them the same score. However, intuitively, these two "Praise" edges should not be treated equally because of the time difference and in fact, the U.S. even criticized Japan at Tx-
Additionally, existing rule-based models apply rules through purely symbolic ways such as: path matching or subgraph extractions which do not have the ability to infer missing facts in TKGs and make the application of rules conditioned on the quality of datasets. Because of the incompleteness of TKGs, we believe this drawback brings a bottleneck to previous rule-basee methods. In Figure 1, we can see that although the model mines a plausible rule and it could have derived the correct an- swer, the missing facts at T2 makes the reasoning failed. However, existing rule-based methods need to traverse and match rule body instances in TKGs on CPUs, which brings efficiency bottlenecks and hinders integration with embedding- based neural methods.
To address these issues, we propose INFER, a neural-symbolic model for TKG extrapolation, which introduces novel rule application paradigm. INFER first learns rules in TKGs with existing random walk based algorithm. Then, in our paradigm, INFER builds Temporal Weight Matrices with the help of static KG embedding models, which store the probability of each possible fact. Notably, this procedure enables the assignment of initial approximate probabilities to even missing facts, thereby enhancing the inference capabilities of our model.
2

---

Published as a conference paper at ICLR 2025
We argue that the binary truth value utilized by previous works for modeling the standing of facts not suitable enough for temporal settings. To capture the effects of the frequency and validity of historical facts, we design Temporal Validity Function that maps the frequency and the time span since the last appearance of a fact into a continuous real number. When it comes to a new timestamp, the temporal validity function is used update the temporal weight matrices which store the temporal validity of each historical fact and probability of potential unseen facts. If we consider the temporal weight matrices as adjacent matrices, this process continuously transforms the graphs of historical facts into weighted graph. Figure 2 (b) shows an example of the weighted graph computed by INFER.
As for rule application, INFER introduces a rule projection module. The rule projection module computes scores of all candidates through conducting efficient matrices calculation on GPU ac- cording to the mined rules rather than costly path matching on graph. Hence, INFER is able to apply rules more efficiently than previous models can and makes the application of longer rules affordable. This design facilitates potential integration with existing embedding-basee based reasoning methods. Moreover, INFER is able to display the scores of all entities at each intermediate step of rule application, which maintains the interpretability of rule-based methods. Code is available at https:
Saann/JaonLeee 22/INFER
Our contributions can be summarized as follows:
We argue that continuous value rather than binary value reflects the truth value of historical facts more precisely under temporal settings. We introduce a Temporal Validity Funetion which consid- ers the time span and frequency of facts together to calculate continuous truth values of temporal facts.
We propose INFER, a neural neural-symbolic model for TKG extrapolation, which adopts a novel paradigm for rule application. INFER constructs and dynamically updates temporal weight ma- trices to augment the ability to infer missing facts and capture historical clues. By performing calculations on these matrices GPU, INFER enables more efficient rule application, making possible to combine with smbedding-basee methods and maintain the interpretablity meanwhile.
Experimental results on multiple datasets show that INFER achieves state-of-the-as performance. And INFER also gives more robust performance compared with existing rule-based models on our modified datasets where a certain proportion of facts at each timestamp are removed.
2 RELATED WORK
Embedding- based reasoning is mainstream methodology for TKG extrapolation, which embeds entities, relations and timestamps into low-dimensional vector space and then utilizes neural net- works to learn the structural information, temporal dependency and historical patterns in TKGs. RE-NET (Jin et al., 2019) uses RNN and RGCN to learn representations of subgraphs which en- code structural and sequential information. CyGNet (Zhu et al. 2021a) introduces copy-generation mechanism to make the model focus more on the repetitive historical facts. TITer (Sun et al.) 2021)) uses reinforcement learning to search candidates in historical TKGs, which is guided by reward based on Dirichlet distribution. CENET (Xu et al.. 2022) argues that unseen facts are also critical in prediction and introduces contrastive learning to identify whether the current timestamp relies more on historical facts or non-historical facts. DaeMon (Dong et al. (023)) models the temporal path information between pair of nodes based on the NBFNet (Zhu al. (2221b) graph neural network and introduces a memory passing strategy to update the path information.
For better interpretability, xERTE (Han al., 202I) performs reasoning on query-based based subgraphs and captures structural as well as temporal information. TLogic (Liu et al., 2022) is the first rule- based TKG extrapolation model which mines rules in TKGs through temporal random walk and applies rules to predict future events via matching rule body instances. TR-Rules (Li et al., (023) proposes a new and appropriate algorithm to calculate the contidence rules under temporal set- tings. It also manages to mine and apply acyclic rules in TKGs which are proven to be effective. ALRE-IR (Mei et al. 2022) combines embeddings with rules by learning rule representations, which is further utilized for confidence assessment. Although embeddings for estimating confidence of rules and ALRE-IR still can not model the frequency and time validity of historical facts.
introduced, it is just used
3

---

Published as a conference paper at ICLR 2025
3 PRELIMINARIES
3.1 TKG EXTRAPOLATION
In this paper, & R. and T denote the set of all the entities, relations and timestamps respectively. We use [8| and [ to represent the number of entities and relations. TKG can be viewed sequence of graphs i.e. G [G....G,].. Each graph G; consists of all the facts that occur at timestamp 4, i.e. G; = {(s, r, o, t)]t = 4 where 8, &, r E R and to I t; S tn. Interpolation Reasoning TKGs attempts to answer queries whose timestamps range from to to bn. However, TKG extrapolation aims at predicting the answer for a future query (s, ?, ?,t") or (7, o, t') where tn <t', based on the historical facts {G]...G,).
3.2
TEMPORAL RULES
Cyclic rules under temporal settings can be defined as:
(Ei, E Tn+1) + (FE Thi, F
where F; and Ti denote variables of entity and timestamps and E R represents specific relation. The left part of the rule is called the rule head, while the right part is called the rule body. In some cases, it restricted that the same entity need to appear repetitively so that the rule body is satisfied. We call this phenomenon variable constraint. Corresponding examples and illustration can be found in Appendix A. During the symbolic rule learning procedure, each rule will be associated with corresponding confidence which is calculated by sampling on the graphs. Normally, rules do not always hold and sometimes are violated, thus confidence rule represents probability of this rule being correct.
METHOD
Rule Projection
WeighsMatriees
To
Historical Facts
Update T
s-th
Query (8, ?,Tr)
Rules
Conf
Rule
(Krr, (..)
Library
( 8.33 (.77.
(Krr. (X.r,
Rule Projection Module
Rule Learning Algorithm
Figure 3: Overview of INFER. INFER first mines rules in TKGs with existing rule learning algo- rithm and builds temporal weight matrices with pre-trained static KG embedding model. When it comes a new timestamp. INFER update the temporal weight matrices according to the historical facts. Then the rule projection module selects corresponding rules and applies rules by performing matrices operation for rule projection.
4

---

Published as a conference paper at ICLR 2025
4.1 OVERVIEW
Figure 3 illustrates the overall framework of INFER. Inspired by QTO (Bai et al. 2023). INFER builds temporal weight matrices with static KG embedding model which assigns probabilities to all possible facts to enable the model to infer missing facts during rule application. INFER first uses the rule learning algorithm of previous symbolic methods build a rule library. Then INFER introduces a Temporal Validity Function which maps the frequency and time span of historical fact into a continuous value between O and The Temporal Validity Function is used to update the temporal weight matrices at each new timestamp. Given query, the rule projection module selects corresponding matrices or vectors for projection according to the relevant rules stored in rule library and finally gives the scores of all candidates. In most cases, more than one rule can be applied for query, INFER leverages Noisy-OR to aggregate scores given by multiple rules for final prediction.
4.2 RULE LEARNING
We first mine temporal rules in TKGs and calculates their confidence to build a rule library for rule application afterwards. We do not extensively search for the optimal rule learning algorithm (including rule confidence calculation) for INFER but adopt the rule learning algorithm from TR- Rules (Li al., 2023) across all experiments. We believe better rule learning algorithm with high- quality rules and more accurate confidence will further improve the performance of INFER, We leave the exploration of other rule learning methods such as the newly released (Huang et al., 2024) for future work. We only utilize cyclic rules for extrapolation reasoning and filter out those acyclic rules and store all the mined cyclic rules with their corresponding confidence.
4.3 RULE PROJECTION MODULE
4.3.1 TEMPORAL WEIGHT MATRICES
We pre-train a static KG embedding model by ignoring the timestamp in each quadruple. In INFER, we select ComplEx (Trouillon al. 2016) as the static KG embedding model. While other KG embedding models can be used as alternatives, we believe that a suitable KG embedding model can improve the overall performance of INFER, although not a definitive factor.
Then, for each relation R, we define a temporal weight matrix M, RIÉl-EE, The i-th row in Mr records the weights of the edges of type that link entity to all other entities, while the j-th column denotes the weights of edges linking each entity to entity e, with r. This intuition guides the rule projection in the following section. The treitrained KG embedding model can score each possible non-temporal facts. However, the scores are real numbers. we need to re-scale the scores to [0.1] resulting in valid probability. Given head entity e E E and relation we apply the softmax function to the scores of all possible candidates and the corresponding values are stored in the temporal weight matrix of relation
M,(i.) Eisksjoj explf(e,r,eg))
where I is the score function of the KG embedding model, i and j are the indices of entities.
There are two reasons why we utilize static embedding models instead of temporal models. The first one is that if we use temporal embedding models then the size of the matrix for relation is M, E |1))[())117 which costs too much space. The other reason is that since temporal models score the same fact at various timestamps differently and we do not consider specific timestamps during application, static embedding models which capture the overall structural information is enough for INFER. For resource savings, following QTO (Bai et al. 2023) those values smaller than the threshold.
we set threshold value filter out
4.3.2 TEMPORAL VALIDITY FUNCTION
The rule application algorithm of previous symbolic methods is to match instanced rule bodies in observed graphs. This process can be viewed as first assigning binary truth values to historical facts or assigning binary weights to edges in graphs: I for facts have appeared, 0 for unseen facts, i.e. no
5

---

Published as a conference paper at ICLR 2025
edges. Then the model traverses the graph to search whether there are instanced paths matching the rule body and finally gives binary results of whether the instanced rule heads hold true. However, the temporal validity and frequency of facts cannot be modeled in this way. To be specific. the facts that occurred recently should not be treated as equally as the facts that held true long ago (Liu et al.) 2022). And facts that frequently appeared should be more significant and trustworthy.
Hence, we attempt to extend the truth value to continuous real number via considering time validity and frequency of historical facts and propose the following Temporal Validity Function:
1
Vi,,r, (c)
+ x VF -Mr(s.0)
(2)
Vlc
Hast
(3)
where te denotes the current timestamp, flast represents the latest timestamp when fact (s, o) stands, F is the frequency of (s, o) counted from O to fc-1, (-) is the sigmoid function, 7 is hyper-parameter for smoothing and M, is the temporal weight matrix introduced above. It should be noticed that fast should satisfy tast fc-1, since query.
not observe facts concurring with the
As we see that the Temporal Validity Function consists of two parts: the time span and the frequency. The time span part which the first item in the formula means that the credibility of a fact weakens as time goes on as long as it does not recur. The square root ensures the attenuation is neither too fast nor insensitive for measuring the validity. The motivation of designing the second item which is the frequency part is that for those frequently appearing facts, we believe they are more consistent and robust. The frequency serves to amplify the temporal weight of fact when it recurrent. While À the weight of the frequency part ranging from 0 to 0.5. It gets bigger as the fact repeats more times in history, which helps to alleviate the attenuation of time span part when frequent fact does not oceur temporarily.
When it comes to new timestamp & E T, given rs) E H; which is a historical fact, we use the above function to update the temporal weight matrices:
(s, min min,V((,,,,,)))11)
To certain extent, this design of Temporal Validity Function is an empirical result. We provide more details and analysis about this part in Appendix D.
4.3.3 RULE PROJECTION
When answering query (s, (g, 2, t'), INFER finds relevant rules Sr, with respect to ry from the rule library. For each rule in Sr,, we traverse its rule body A, n [E Ri, E Ti) to get all the relations in order that the temporal causality is maintained RR R2.. R.)- Then, inspired by the multi-hop reasoning operation proposed in QTO (Bai et al. 2023), the rule is projected iteratively:
M R. [s,:]
i i=1
Ans;
(5)
maz col
Ans x| C MR.) 2 25i5n
Firstly, we select the s-th row of MR, which denotes the scores of each entity serving as object entity in facts with subject entity s and relation rlj. If the rule length is more than 1, we transpose the row vector and repeat it along the row direction for []| times which results in a matrix of [g] |E). Then we calculate the Hadamard product of the obtained matrix and the corresponding temporal weight matrix. Finally, we take the maximal value of each column to obtain a new row vector.
This procedure will be repeated given by the rule. The row vector can be viewed as a fuzzy set representing the scores of all entities at the intermediate step of reasoning. Then, the Hadamard product computes the scores of all entities with a fuzzy set rather than a specific entity, thus giving a matrix. The column-wiss max operator selects the highest value among all possible paths for each candidate. This process can be considered as another way to traverse and search in graphs which is based on operating on the generalized adjacent matrices of weighted graphs.
1 times and the final row vector is the scores of all candidates

---

Published as a conference paper at ICLR 2025
Algorithm Pseudocode of Rule Projection Without Variable Constraints
Require: (g. Tg)): Query; {R1, Rm): Corresponding rule bodies; Mr: Temporal Weight
Matrices;
1: set Ans MR, [s,:] Select the s-th row of MR,
2: for in (2, + 1) do
#Repeat the transposed vector along row direction 4: Ans Ans MR, #Hadamard Product
3:
Ans Ans" T repeat((1,)))
5: Ans = max(Ans dim = -)) #Column wiiee max operation giving row vector
return Ans
However, in some cases, there variable constraints rules longer than As can see in (6), the instantiated rule bodies must comply with the constraint that the entity serving as E2 must be the same.
(E1, drw.ndds.oorrhhheeee Shhnngge TA) ist.intttttoochhaae E2,
(6)
A (Ez, avvssi, Es. T2) A Es. Make.a..isiiee Eg, Ts)
In INFER, we also design the projection algorithms for these rules. The detailed pseudocode and illustration are provided in Appendix A. Notably, when processing rules with variable constraints, additional indexing operations and calculations are required, leading to delay in inference speed.
4.4 INFERENCE
Normally, more than one rule can be used answer query. In INFER, we aggregate the scores of multiple rules with the Noisy-OR function (Meilicke et al. 2019a):
score(q)
II (1 (1-6) G. Ans,)
Ans,ERg
where y denotes the query, Rg represents the set of scores given by rules, 6, the confidence of the rule and I is all-one vector. Apart from the rule confidence, the Noisy-OR aggregation is equivalent to the union of the fuzzy sets generated by the rules.
In some cases, there might be proper rules for answering the query q. In INFER, for query (Sn; Tn, 2, t.) which does not have corresponding rules, we directly predict it based on the tempo- ral weight matrix by considering the query itself as rule of length 1. For those queries that have corresponding rules, we also make each of the queries itself as a rule of length and use it for predic- tion. Because consider the prediction based this "generated rule" represents the distribution implied by pure temporal information and frequent interactions in historical facts. We assign the average confidence of all rules to this "generated rule".
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets We evaluate INFER on five TKG datasets: ICEWS14, ICEWS18, ICEWS0515, YAGO and WIKI. The first three datasets are subsets of Integrated Crisis Early Warning System (Boschee) ctt al. 2015). These datasets record international facts that occurred in 2014, 2018 and from 2005 to 2015 respectively. Table 3 in Appendix B shows the statistics of these five datasets. To satisfy the extrapolation reasoning settings, facts are sorted ascending order based on timestamps and then split into train, valid and test.
Metrics and Implementation Details We utilize mean reciprocal rank (MRR) and Hits @k as metrics to evaluate INFER. MRR avg( rank. and Hits@k = 1/2 Σ, I(rank; k). where I is the indicator function. Higher values of both metrics indicate better performance of models. We adopt the time-aware filtering protocol proposed in XERTE (Han et al.) (021I. which turns out to be more reasonable in temporal settings. We implement INFER with Py Torch and all experiments conducted on a single 48GB NVIDIA A40 GPU. More details about implementation are reported in Appedndix E.

---

Published as a conference paper at ICLR 2025
ICEWS14
ICEWS08-15
ICRWS18
Model
MRR
Hiss11
3
Hits@ 10
MRR
Hits@
H
Hits@10
MRR
Hits@1
Hits@3
10
Dismult
27.67
18.15
31.15
46,56
28.73
19.33
12.19
47.54
10.17
4.52
10.33
21.25
Complex
34.48
49.58
31.69
21.44
35.74
52.04
21.01
11.87
23.47
39.87
TTrans
13.43
3.11
17.32
34.35
15.71
5,00
19.72
38.02
8.31
1.922
5,56
21,89
Dismiit
26.47
17.09
30.22
45.41
24.31
14.58
27.92
44.21
16.75
8.61
18.41
$3.59
DE. Si-Simpli
25.91
52.75
19.30
34,80
TN
52.12
23.35
36.115
49. 13
27.54
19.52
30.80
42, 86
21.25
13:28
24,02
36.91
RE-Net
58,28
28.68
11.34
51.52
42.97
31.26
46.85
63.47
28.81
19.05
32.44
47.51
CyGNet
36.31
42.61
XERTE
40.79
32,70
45.67
57.30
46.02 022
37. 844
52. 31
63.92
29.31 31
21.03
33.51
46.48
CENET
37.16
27.78
41.16
$5.49
27.14
18.58
29.99
44,15
Teler
46,45
22.05
83
RE-GCN
40.36
30.73
41.98
58.81
16.91
36.33
52.65
67.24
30.82
21:11
34,71
49.83
DacMon
31.85
22.67
35,92
49.80
TECHS
43.88
34.59
49.36
61.95
48.38
38.34
54.69
68.92
30.85
21.81
35.39
49.82
AnyBURL
29.67
21:26
33.35
46.73
32.05
23.72
35:45
50.46
22.17
13:10
25,44
38.91
TLogic
43.04
33.56
48.27
61.23
46.97
36.21
53.13
67.43
29.82
20.54
33,95
48.53
TR-Rules
43.32
33.96
48.55
61.17
47.64
37.06
$3,80
67.57
30.41
21.10
34.58
48.92
INFER
44.09
34.52
48.92
62.14
48.27
37.61
$4.30
68.32
31.68
21.94
35.64
50.88
INFER(With uc)
44.08
34.82
08.84
61.74
48.34
38.07
54.16
67.82
32.11
22.34
36.26
$1.19
INFER-60(with nc)
44.46
35.03
49.37
62.31
48.73
38.32
54.61
68.45
32.22
22.39
36.41
51,52
Table I: Performance of INFER on ICEWS14, ICEWS05-15 and ICEWS18. Best results are in bold and second best results are underlined.
Baseline Methods We compare INFER with static embedding, temporal embedding and rule-
based state-of- ofate-of-thh-arr art models. As for static embedding models we select Dismult (Yang et al.) 2014)
and ComplEx (Trouillon et al., 2016). Temporal embedding models include TA-DistMult (Garcfa-
Durán et al.
2018), DE-SimplE (Goel et al.
2020), TNTComplEx (Lacroix et al. 2020)) RE-Net (Jin
ct al.
2.19 CyGNet (Zhu et al. 2021a) XERTE (Han et al.) 2021). TiTer (Sun et al. 202I). RE-
GCN
Li et al. 2021). CENET (Xu et al. 2022). DacMon (Dong al. (023) and TECHS (Lin ct al.
2023
The rule-based methods we select are AnyBURL (Meilicke et al. 2019b). TLogic (Liu et al.,
2022
and TR-Rules (Li et al. (023).. We reproduce RE-GCN and remove their utilization of static
information for fairness and all of the rest results are taken from (Li al. 2023), (Dong et al. 2023
and (Lin al. 2023).
5.2
MAIN RESULTS
Table displays the results of INFER compared with selected baselines, where INFER vc) denotes we utilize rules with and without variable constraints to infer instead of only rules without variable constraints and INFER-60N (it(hthh vc) represents we use at most 60 rules for single query. As we can see, INFER-60(with vc) achieves ofte-of-tthe-an art performance on all datasets with respect to MRR, Hits @1 103. Specifically, compared with the best rule-based model TR-Rules, INFER obtains average improvements of 1 1.35% and 1.55% in MRR and Hits 10 on three datasets, which proves the effectiveness of introducing embedding models and considering the frequency and time validity of historical facts for rule application. Mcanwhile INFER maintains the interpretability of rale-based methods. We also provide exemplary CASE in Appendix C to better illustrate the process of INFER and show the interpretability of INFER. Results of INFER on YAGO and WIKI are presented in Appendix F, Table 5.
ICEWS14
ICEWS18
Model INFER
MRR
Hits@
Hits@3
Hitx@ 10
MRR
Hits@
Hits@3
Hitx@ 10
44.09
34.52
48.92
62.14
31.68
21.94
35.40
50.88
INFER(P+B)
37.67
27.34
42.76
$7.77
25.87
16.16
29.31
45,57
INFER(P)
11.70
INFERTTeme Temp Val)
41.99
33.75
47.00
57.31
30.90
21.59
35.02
49.37
INFER(B)
37.41
27.58
42.52
56.43
25.73
16.31
29.26
44,90
Table 2: Ablation study of INFER on ICEWS14, and ICEWS18. Best results are in bold.
5.3 ABLATION STUDY
We conduct ablation study on ICEWS14 and ICEWS18 and the results are reported in Table 2 where P denotes the utilization of pre- pre-trained static KG embedding models, B denotes use the traditional binary truth values for historical facts and Temp Val represents we use the truth values calculated by Temporal Validity Functions.
8

---

Published as a conference paper at ICLR 2025
315
1981 Thruaes
MR2
Hitel
Hitsoi
275
Completezens of Lotaeds (ICEWS18)
Complrizzeti of Canzes (JCEW 315)
Cumpleiaes of Cataces (ICEWS14)
Complsenes dakace (CEWS144
Figure 4: Performance of INFER and other rule-based models on our modified incomplete ICEWS14 and ICEWS18 where certain proportion of facts at each timestamp are randomly re- moved.
As we can see from the results of INFER(P+B), when using the binary truth value, the performance of INFER drops dramatically on both datasets with up to 6.42% in MRR and 4.37% Hits@ 10. This phenomenon strongly verifies motivation and proves the necessity of extending truth values to continuous number via modeling temporal validity and frequency. As we do not use the historical information and only project rules based the matrices given by the static embedding model, the results (INFER(P)) get even worse, which demonstrates the importance of historical facts for rule-based models. INFER(Temp Val) and INFER(B) report the performance when the pre-trained model is removed. Again, compared with the binary settings, the introduction of our Temporal Validity Function significantly improves the performance. The results of INFER(Temp Val) also prove that the introduction of embedding model does contribute to the overall performance, but the the capability of embedding models not decisive. Besides, by comparing (B) with (P+B) and (Temp Val) with (INFER) itself, we discover that the utilization of Temporal Validity Function amplifies the influence the pre-trained KG embedding model. We speculate the probable reason that some of the re-scaled values given by the static embedding model are relatively small compared with 1 when the truth values are binary. Hence, candidates inferred through potential missing facts surely rank after those candidates inferred by existing historical facts. As result, the effects of the pre-trained model are impaired when binary truth values are utilized.
5.4 ANALYSIS
Performance on Incomplete datatsets To demonstrate the capability in inference missing facts and robustness of reasoning incomplete datasets of INFER, we conduct experiments on modified ICEWS14 and ICEWS18 and compare INFER with previous rule-based methods. We randomly remove 30% and 50% facts at each timestamp and then pre-train the static embedding model and mine temporal rules with these incomplete datasets. Meanwhile, the historical facts used for tem- poral validity calculation are also cut to 50% and 70%. The results are shown in figure 4. As we can see, the differences between INFER and the other two models generally increase as the datasets become more incomplete. Moreover, the slope of the blue line representing INFER is smaller than that of the other two lines, which indicates that the performance of INFER is less affected as the datasets become sparse. This suggests that INFER exhibits robustness sparse datasets, which can be attributed to the introduction of the static embedding model that enhances the ability to infer missing facts during rule application, thereby improving the overall performance of INFER.
Inference Efficiency We also test TLogic (Liu et al. 2022) and our method on ICEWS 14 to calculate the time consumption, the quantity of processed rules and the number of candidates derived from each rules which serve to evaluate the efficiency and capability of models. In our reproduced experiments, the rule application time of TLogic on ICEWS14 is 2000 seconds with 8 CPUs. The average number of rules TLogic applies for single query is 32.96 and the number of candidates covers per rule at each intermediate step is 13.30. In comparison, it takes INFER 2500 seconds to apply rules ICEWS14 with single GPU. However, the average number of rules INFER uses for a single query is 38.73 and the average number of candidates per rule that INFER considers and assign positive scores each step is 1385.49.

---

Published as a conference paper at ICLR 2025
Rules per Second
Candidates per Second
200
250000
Cands/sec
Rules/sec
150
200000
150000
100
100000
50
50000
INFER
TLogic
INFER
TLoiic
Model
Model
(a)
(b)
Figure 5: Comparison on inference efficiency: (a) Comparison rules/sec, INFER gives similar performance to TLogic. (b) Comparison on candidates/sec, INFER shows significantly superiority which means that INFER achieve efficient traversal on graphs.
In contrast, INFER can apply rules more efficiently under our proposed paradigm and significantly enlarge the range of candidate entities which surely contributes to the final performance of INFER since more considered intermediate candidates denote that our model traverse more possible paths during application.
Variance Generally, INFER be viewed statistical learning method. Thus, long the mined rules and corresponding confidence are the same, when running the model with the same hyper-parameter configuration, it will yield identical results. The only possible variance comes from the pre-trained static KG embedding model. To eliminate this concern, we pretrain the Com- plEx times on ICEWS14 14 and subsequently run INFER. The results of the three experiments are completely identical. The possible reason is that the scores given by the KG embedding model are normalized by a softmax function and filtered with a threshold, which effectively avoid any obvi- ous fluctuation. Thus, we can conclude that the performance of INFER is stable and the gained improvements are not occasional issues.
Variable Constraints As we can see in table 1, when the limit of the quantity of rules set 40, the utilization of rules with variable constraints does bring significant improvements. We speculate that rules with variable constraints require more strict temporal restriction which can be satisfied by current design. Meanwhile, the inference speed of applying rules with variable constraints decreases due to extra indexing and calculations. Detailed process of applying rules with variable constraints shown Appendix A.
CONCLUSION
In this paper, propose INFER, interpretable neural-symbolic model for TKG extrapolation. INFER proposes a novel paradigm for integrating temporal rules with embedding- -bsed methods which enhances the robustness of rule-based methods missing facts. INFER is the first model to extend the truth value of facts into continuous number and designs Temporal Validity Function to model the time validity and frequency historical facts. We introduce rule projection module which manages apply rules by conducting faster matrices operations GPUs instead of previous costly path matching graphs. Experimental results show that INFER achieves state-of-the-ar performance three TKG datasets. Meanwhile, INFER also remarkably improves the efficiency of rule application.
Reproducibility Statement. We provide our code at htttttpss//gghhhii com/ hatn/JJsonnLeee22 INFER to enhance reproducibility. And the implementation details are provided in section 5.1 and Appendix E.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (No. 62473271) and ( (No. 62176026).
REFERENCES
Yushi Bai, Xin Lv, Juanzi Li, and Lei Hou. Answering complex logical queries on knowledge graphs via query computation tree optimization. In International Conference on Machine Learning, pp- 1472-1491. PMLR, 2023.
Elizabeth Boschee, Jennifer Lautenschlager, Sean O'Brien, Steve Shellman, James Starz, and Michael Ward. Icews coded event data. Harvard Dataverse, 12, 2015.
Hao Dong, Zhiyuan Ning, Pengyang Wang. Ziyue Qiao, Pengfei Wang, Yuanchun Zhou, and Yanjie Fu. Adaptive path-memory network for temporal knowledge graph reasoning. arXiv preprint arXiv:2304. 126044, 2023.
Alberto García-Durán, Sebastijan Dumančić, and Mathias Niepert. Learning sequence encoders for temporal knowledge graph completion. arXiv preprint 1rXiv:1809.000 03202, 2018.
Rishab Goel, Seyed Mehran Kazemi, Marcus Brubaker, and Pascal Poupart. Diachronic embedding for temporal knowledge graph completion. Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 3988-3992 2020.
Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp. Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In International Conference on Learming Representations, 2021.
Mareel Hildebrandt, Swathi Shyam Sunder, Serghei Mogoreanu, Mitchell Joblin, Akhil Mehta, Ingo Thon, and Volker Tresp. A recommender system for complex real-world applications with non- linear dependencies and knowledge graph context. In The Semantic Web: 16th International Con- ference, ESWC 2019, Portorož, Slovenia, June 2-6, 2019, Proceedings 16, pp. 179-193. Springer, 2019.
Rikui Huang, Wei Wei, Xiaoye Qu, Shengzhe Zhang, Dangyang Chen, and Yu Cheng. Confidence is not timeless: Modeling temporal validity for rule-based temporal knowledge graph forecasting. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 10783-10794, 2024.
Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. Recurrent event network: Autoregressive structure inference over temporal knowledge graphs. arXiv preprint
2019.
Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. Tensor decompositions for temporal knowledge base completion. arXiv preprint arXiv:2004.04926. 2020.
Yunshi Lan and Jing Jiang. Query graph generation for answering multi-hop complex questions from knowledge bases. Association for Computational Linguistics, 2020.
Ningyuan Li, E Haihong, Shi Li, Mingzhi Sun, Tianyu Yao, Meina Song, Yong Wang, and Haoran Luo. Tr-rules: Rule-based model for link forecasting on temporal knowledge graph considering temporal redundancy. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7895,7794, 2023.
Zixuan Li, Xiaolong Jin, Wei Li, Saiping Guan, Jiafeng Guo, Huawei Shen, Yuanzhuo Wang, and Xueqi Cheng. Temporal knowledge graph reasoning based on evolutional representation learning. In Proceedings of the 44th international ACM SIGIR conference research and development in information retrieval, pp. 408-417, 2021.
Qika Lin. Jun Liu, Rui Mao, Fangzhi Xu, and Erik Cambria. Techs: Temporal logical graph net- works for explainable extrapolation reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 1281-1293, 2023.

---

Published as a conference paper at ICLR 2025
Yushan Liu, Yunpu Ma, Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 4120-4127, 2022.
Xin Mei, Libin Yang. Xiaoyan Cai, and Zuowei Jiang. An adaptive logical rule embedding model for inductive reasoning over temporal knowledge graphs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 7304-7316, 2022.
Christian Meilicke, Melisachew Wudage Chekol, Daniel Ruffinelli, and Heiner Stuckenschmidt. Anytime bottom-up rule learning for knowledge graph completion. In International Joint Con- ference on Artificial Intelligence, 2019a. URL httpps:/api...mmatiiccioooo .seanntccscoolar.co
CorpusID:26387760
Christian Meilicke, Melisachew Wudage Chekol, Daniel Ruffinelli, and Heiner Stuckenschmidt. Anytime bottom-up rule learning for knowledge graph completion. In IJCAI, pp. 3137-3143, 2019b.
Haohai Sun, Jialun Zhong, Yunpu Ma, Zhen Han, and Kun He. Timetraveler: Reinforcement learn- ing for temporal knowledge graph forecasting, arXiv preprint arXiv:2109.04101, 2021.
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Com- plex embeddings for simple link prediction. In International conference on machine learning, pp- 2071-2000 PMLR, 2016.
Yi Xu, Junjie Ou, Hui Xu, and Luoyi Fu. Temporal knowledge graph reasoning with historical contrastive learning. arXiv preprint arXiv:2211.10900 2022.
Bishan Yang. Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv: 1xXi.:/41..6575. 2014.
Cunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng, and Yan Zhang. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 4732-47444 2021a.
Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang. Neural bellman-ford net- works: general graph neural network framework for link prediction. Advances Neural Infor- mation Processing Systems, 34:29476-29490 2021b.
A RULE PROJECTION
A.1 VARIABLE CONSTRAINTS
As we mentioned in section 3.2, in some rules, entity variables in the rule bodies do not have to be unique since the same entity might be involved in multiple facts different timestamps. This phe- nomenon which is called Variable Constraints brings constraints to the rule application procedure. Here some examples:
(A, (A, support, C,T). riol, B,T)) A(B make stalement, A, A2) (AA resort, C, Ta)
The above example rule with variable constraints. As we can see in the body of the rule, in the second hop, the entity which B make statement to has to be the same with the entity that riot with B at Ti.
However, the below case is rule without variable constraints which is different with the rule given above. Because the entity which make statement to can be or can not be the same with the entity riot with B at Tı.
(A, support, C,T), B,T:) ALB make statement, D,Ta) AD, C,Ts)
mm.make
12

---

Published as a conference paper at ICLR 2025
A.2 RULE PROJECTION WITH VARIABLE CONSTRAINTS
Algorithms and 3 present the pseudocode for rule projection with two types of variable constraints: [1.3] and [0.2], assuming rule length of 3, where the numbers in the list indicate that the corre- sponding entity variables in the rule body must refer to the same entity. In our experiments, we also mine rules with variable constraints [[1.3],[0.2]] and rules of length from ICEWS0515 with variable constraints [0.1] and [1,2]. Here, we provide the projection algorithms for the two most common types of variable constraints. Algorithms for these variable constraints can be easily derived following Algorithms 2 and 3 and our provided code can serve reference.
Algorithm 2 Pseudocode of Rule Projection With Variable Constraints [1,3]
Require: (8, Tg. ?, Ty): Query; {Rı, R.... Corresponding rule bodies; MR: Temporal Weight
Matrices;
1: set Ans = MR, [s. Select the s-th row of MR.
2: Ans = Ans" "repeat(1,1]))
#Repeat the transposed vector along row direction
3: Ans Ans @ MR, #Hadamard Product gives matrix of R where Ans[i]]y] denotes
the score of entity ej when entity e serves as the constrained variable.
4: zeros(1, |
5: for in (0, (0) do
6: row Ans[i.:]
7: row row" T .repeat(1,
8: temp- row MR,
9: temp templ:, i]
10: res[i] = max(temp, dim 1)
return res
Algorithm Pseudocode of Rule Projection With Variable Constraints [0,2]
Require: (s, Tg. ?, ?,Tg): Query; {R1, R2, Ra}: Corresponding rule bodics; MR: Temporal Weight
Matrices;
1: set Ans = MR, [s Select the s-th row of MR,
2: Ans = Ans reeeeareeat #Repeat the transposed vector along row direction
3: Ans Ans @ MR, #Hadamard Product
4: Ans = sn) #Only select the s-th column to satisfy the variable constraints.
5: Ans = Ans. Ans.repeat(1,[E]
6: Ans Ans @ MR, #Hadamard Product
7: Ans = max dim 1) #Column-wies max operation giving row vector return Ans
In our preliminary conception, variable constraints of longer rule are solvable without having to tra- verse every possible scenario for algorithm design. In this conception, we can treat the algorithms for variable constraints that have been designed for lengths of 2 or 3 as algorithm units. The calcu- lations for variable constraints in longer rules can broken down into combinations of these units. For example, for rule of length 5 with variable constraints [[0,2,4]]]
(A, RI. B) A (B, R2, A) A (A, R3, C) A (C, R4, A) A (A, Rs. D)
We can first use existing algorithms to compute the results under the [0.2] constraint when reaching variable#2 (A). This intermediate result can then serve the initial condition for the third relation (R3) calculation in the rule. At this point, the subsequent calculation can be seen as computation for a rule of length with variable constraints [0.2] ([2,4) > [0,2]) (A, R3, C) A (C, R4, A) A (A, RG, D). This is again problem that can be solved by our algorithm units.
This approach allows us to tackle longer rule variable constraints by leveraging the solutions for shorter ones, effectively avoiding exhaustive searches through all possible scenarios. By decompos- ing complex problems into simpler, already-solved components, we can design more efficient and scalable algorithms for handling variable constraints in various rule lengths. We leave the detailed design of a more flexible and automatic algorithm for handling variable constraints in long rules to the future work.
13

---

Published as a conference paper at ICLR 2025
B STATISTICS OF DATASETS
Table shows the statistics of these five datasets. To satisfy the extrapolation reasoning settings, facts are sorted in ascending order based on timestamps and then split into train, valid and test.
Train
Valid
Test
18
[
[7]
ICEWST4
63685
13825
13221
7128
230
365
SCSSS
522958
69234
69147
10488
251
4017
ICEWS18
373018
43995
49545
23033
304
WIKI
539286
67538
63110
12554
24
252
YAGO
Table 3: Statisties of ICEWS14, ICEWS05-15, ICEWS18, WIKI and YAGO.
C EXEMPLARY INFERENCE CASE
Figure 6 displays an inference case of INFER. Given the query (AshrafGh ani Ahmadzai, Make Visit, ?, 201 2014/11/11 1/l whose answer is China and corresponding rule of length 3, list top candidates at each step of rule application provided by INFER. As for the result, INFER successfully ranks China the second place. Moreover, we discover that the other 3 entities: Afghanistan, Jens Stoltenberg and North Atlantic Treaty Organization also serve as the object entity for this query although at other timestamps, which demonstrates that INFER is indeed capable of obtaining relevant candidates.
AshrafGh ani Ahmadzu
Intend to cooperate
Host visit
a
Rank
Cands
Score
Rank
Caad
Score
Raak
Cand
Score
Abdullah.
Sesttenberg
Afghanlstan
J. Kerry.
0,943
NATU
0JM643
Afghamistan
8.0909
A.G.Almuadzai
0.6525
Stolteaberg
Sharif
IUI555
Curacao
0.0122
Veternss UK
0,0262
NATO
0.0122
Figure 6: Exemplary interpretable inference procedure of INFER.
D TEMPORAL VALIDITY FUNCTION
Table 4 reports the performance of various forms of Temporal Validity Function on ICEWS14. We mainly focus on the main time span part, where Square root, Cube root, Reci and Exp represent the functions I
V Vr and exp(ø) used for the time span part respectively and Binary denotes the previous binary truth value settings for historical facts. As we can see, following our intuition, by proposing a function which is negative to the time span values and ranging in [0.1], INFER can properly model the temporal validity of historical facts. However, Exp gives worse performance compared with Binary, getting bigger leading to the "forget" historical facts. From this point of view, when using the Binary settings, the temporal validity of facts do not decay, i.e. the model strongly memorize every fact. Thus, Binary and Exp are two extreme cases, we try to explore a more suitable form to model the temporal validity and select the Square root in INFER, which is still primary attempt and we believe more sophisticated Temporal Validity Function can surely improve the performance of INFER.
natural speculate that is that exp(a)
_1 drops too fast as the time span
14

---

Published as a conference paper at ICLR 2025
ICEWS14
Model
MRR
Hits@1
Hits@3
Hits@10
INFER
44.09
34.52
48.92
6214
Square
44.09
34.52
48.92
62.14
Cube root
43.76
34.18
48.54
62.01
Reci
41.36
31.52
45.94
60.61
Exp
33.08
23.58
36.87
52.18
Binary
37.67
27.34
42.76
51.77
Table 4: Results of various forms of Temporal Valid Function in INFER on ICEWS14. Best results are bold.
E IMPLEMENTATION DETAILS
When mining rules, the configuration given by TR-Rules et al. 2023). We select Com- pIEXTrouilon et al. (2016) as the static KG embedding model, which is traindd with N3 regularizor and relation prediction task. T in equation (2) is set 15 for ICEWS14 and ICEWS18. While for ICEWS0515 which covers a time span of ten years, we set it to 150. For some queries, there might be too many rules be utilized for inference. However, for simplicity and resource saving, set a limitation on the most rules can be used for a single query which is 40 unless specified and the threshold for building temporal weight matrices is 0.0005.
F RESULTS ON YAGO AND WIKI
Table 5 reports the results of INFER on WIKI and YAGO, as we can see INFER still outperforms existing methods and significantly improves the rule-based baselines WIKI. As for YAGO, our model and other rule-based methods fall short compared with embedding-based methods. As men- tioned in (Huang al. 2024), some relations YAGO can not be modeled by cyclic rules which take to 10% of the test set. Since our model only uses cyclic rules, it is affected to a certain extent. However, INFER still gain improvements compared with TLogic. should be noted that during experiments, we notice that the introduction of "generated rule" mentioned in section 4.4 causes bias on the YAGO dataset, which drastically boosts the performance. Thus, we do not lever- age the "generated rules" for evaluation on YAGO for fair comparison. We speculate the possible reason is that there are more repetitive facts YAGO which amplifies the impact the "generated rules" too much especially when given the fact that less patterns can be modeled with cyclic rules. We also conduct experiments on the other four datasets to investigate the bias issue and the results show that only yields slight fluctuation to the overall performance on the other four datasets.
WIKI
YAGO
Model INFER
MRR
Hits@1
Hits@B
Hits
Hits@1
Hits@3
Hils
86.48
85.09
87.38
88.67
83.74
83.54
83.72
84.31
TLogic
78.93
73.05
84.97
86.91
78.76
74.31
83,38
83.72
TECHS
75.98
82.39
89.24
92.39
TITER
73.91
71.70
76.96
REGCN
78.53
74.50
81.59
84,70
82.30
78.83
84.27
88.58
Table 5: Results of INFER on WIKI, and YAGO. Best results are in bold.
ICEWS14
Model
Hits@R3
INFER-1
43,83
34.15
49.01
61.99
40.59
30.90
INFER-2
20.49
10.61
22.97
42,38
TR-Rules-2
15.28
6.40
16.82
35.29
INFER-3
40.69
32.48
45.40
55.99
TR-Rules-3
40.45
31.46
45.52
57.35
Table 6: Results of INFER with rules of different lengths on ICEWS14. (Model-X, X denotes only use rules of length X.)
Best results are bold.
15

---

Published as a conference paper at ICLR 2025
G LIMITATIONS
Although INFER obtains significant improvements on three datasets, there are still limitations. As mentioned section the main limitation of INFER is that can explicitly model the exact order of historical facts which might be crucial for improving the performance of longer rules with variable constraints. We provide the performance of INFER using rules of different lengths with comparison to TR-Rules in Table 6. It can be observed that, INFER greatly improves the effect of applying rules of length of and 2 compared to TR-Rules. As for length of INFER still performs better MRR and Hits @ I but the overall improvements are not that significant compared with shorter rules. Besides, INFER can not leverage acyelic rules for inference, which have been demonstrated crucial in KG completion.
16


Reviews:
[{"review_id": "5jaXQvsboh", "reviewer": "Reviewer_agps", "summary": "The paper introduces INFER, a neural-symbolic approach to temporal knowledge graph extrapolation.  INFER  uses a Temporal Validity Function that captures how frequently facts occur, as well as their validity over time using continuous values. INFER uses pre-trained static knowledge graph embeddings to construct Temporal Weight Matrices. A rule projection module that reformulates rule application as matrix operations. \nThe paper evaluates INFER's effectiveness using three ICEWS datasets. When tested on modified sparse temporal knowledge graph datasets, INFER shows promising inference capabilities. These results highlight that INFER's combination of continuous temporal validity scoring and GPU-optimized rule application offer useful techniques for temporal knowledge graph reasoning.", "questions": "1. How does your acceleration method differ in nature from TensorLog and Neural-LP?\n2. Many methods have developed from the Neural-LP approach.  How does your method compare to those?\n3. Why do you not compare to TimePlex as the state of the art method that has been demonstrated on these ICEWS datasets?\n4. Can the gap in performance with TimePlex and rule-based sysetsms be explained in terms of the differences between timestamp and interval-time objectives of different temporal knowledge graph methods?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper introduces a new scoring mechanism for temporal rule validity.  Alternate techniques in the literature appear to be more complicated.\n\n2. The paper evaluates differentiable rule-based inference systems across several ICEWS datasets, providing a demonstration of these systems on temporal knowledge graphs that represent time in event \"timestamp\" form.", "weaknesses": "1. The author's claimed novelty rests on acceleration of rule-based processing using matrix operations on a GPU.  This is common in differentiable rule-learning systems, apparently first introduced as TensorLog, with associated inductive learning system Neural-LP, see e.g. [1].\n\n2. The paper claims: \"Experimental results show that INFER achieves state-of-the-art performance on three datasets and significantly outperforms existing rule-based models on our modified, more sparse TKG datasets, which demonstrates the superiority of our model in inference ability.\" The authors should consider that both embedding based and rule based systems can perform quite well relative to the methods they compare against.  For instance, consider the following comparison with TimePlex [2]:\n\n| | ICEWS14 | ICEWS14 | ICEWS14 | ICEWS05-15 | ICEWS05-15 | ICEWS05-15 |\n|--------|---------|-----|---------|----------|---------|-----|\n| Method | MRR | HITS@1 | HITS@10 | MRR | HITS@1 | HITS@10 |\n| TimePlex | **60.40** | **51.50** | **77.11** | **63.99** | **54.51** | **81.81** |\n| INFER | 44.09 | 34.52 | 62.14 | 48.27 | 37.61 | 68.52 | \n\nThe table below illustrates the methods similar to those compared in this paper evaluated on wikidata and yago data sub-sets.  The table also includes a rule-based method (TILP [3]) that is demonstrated to perform on par with Timeplex, illustrating that the performance gap between TimePlex and INFER show above may not be limited to embedding based methods.                                                                                   \n\n| | WIKIDATA12k | WIKIDATA12k | WIKIDATA12k | YAGO11k | YAGO11k | YAGO11k |\n|--------|---------|-----|---------|----------|---------|-----|\n| Method | MRR | HITS@1 | HITS@10 | MRR | HITS@1 | HITS@10 |\n| TLogic | 0.2536 | 0.1754 | 0.4424 | 0.1545 | 0.1180 | 0.2309 |\n| ComplEx | 0.2482 | 0.1430 | 0.4890 | 0.1814 | 0.1146 | 0.3111 |\n| TA-ComplEx | 0.2278 | 0.1269 | 0.4600 | 0.1524 | 0.0936 | 0.2626 |\n| DE-SimplE | 0.2529 | 0.1468 | 0.4905 | 0.1512 | 0.0875 | 0.2674 |\n| TimePlex | **0.3335** | 0.2278 | **0.5320** | 0.2364 | **0.1692** | 0.3671 |\n| TILP | 0.3328 | **0.2342** | 0.5289 | **0.2411** | 0.1667 | **0.4149** |\n\n\n[1] @inproceedings{10.5555/3294771.3294992,\nauthor = {Yang, Fan and Yang, Zhilin and Cohen, William W.},\ntitle = {Differentiable learning of logical rules for knowledge base reasoning},\nyear = {2017},\nisbn = {9781510860964},\npublisher = {Curran Associates Inc.},\naddress = {Red Hook, NY, USA},\nabstract = {We study the problem of learning probabilistic first-order logical rules for knowledge base reasoning. This learning problem is difficult because it requires learning the parameters in a continuous space as well as the structure in a discrete space. We propose a framework, Neural Logic Programming, that combines the parameter and structure learning of first-order logical rules in an end-to-end differentiable model. This approach is inspired by a recently-developed differentiable logic called TensorLog [5], where inference tasks can be compiled into sequences of differentiable operations. We design a neural controller system that learns to compose these operations. Empirically, our method outperforms prior work on multiple knowledge base benchmark datasets, including Freebase and WikiMovies.},\nbooktitle = {Proceedings of the 31st International Conference on Neural Information Processing Systems},\npages = {2316–2325},\nnumpages = {10},\nlocation = {Long Beach, California, USA},\nseries = {NIPS'17}\n}\n\n@inproceedings{jain-etal-2020-temporal,\n    title = \"{T}emporal {K}nowledge {B}ase {C}ompletion: {N}ew {A}lgorithms and {E}valuation {P}rotocols\",\n    author = \"Jain, Prachi  and\n      Rathi, Sushant  and\n      {Mausam}  and\n      Chakrabarti, Soumen\",\n    editor = \"Webber, Bonnie  and\n      Cohn, Trevor  and\n      He, Yulan  and\n      Liu, Yang\",\n    booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)\",\n    month = nov,\n    year = \"2020\",\n    address = \"Online\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2020.emnlp-main.305\",\n    doi = \"10.18653/v1/2020.emnlp-main.305\",\n    pages = \"3733--3747\",\n    abstract = \"Research on temporal knowledge bases, which associate a relational fact (s,r,o) with a validity time period (or time instant), is in its early days. Our work considers predicting missing entities (link prediction) and missing time intervals (time prediction) as joint Temporal Knowledge Base Completion (TKBC) tasks, and presents TIMEPLEX, a novel TKBC method, in which entities, relations and, time are all embedded in a uniform, compatible space. TIMEPLEX exploits the recurrent nature of some facts/events and temporal interactions between pairs of relations, yielding state-of-the-art results on both prediction tasks. We also find that existing TKBC models heavily overestimate link prediction performance due to imperfect evaluation mechanisms. In response, we propose improved TKBC evaluation protocols for both link and time prediction tasks, dealing with subtle issues that arise from the partial overlap of time intervals in gold instances and system predictions.\",\n}\n\n[3] @inproceedings{xiongtilp,\n  title={TILP: Differentiable Learning of Temporal Logical Rules on Knowledge Graphs},\n  author={Xiong, Siheng and Yang, Yuan and Fekri, Faramarz and Kerce, James Clayton},\n  booktitle={The Eleventh International Conference on Learning Representations}\n}", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XI0hRPSFTA", "reviewer": "Reviewer_cyuQ", "summary": "The article presents a detailed neuro-symbolic model addressing the task of Temporal Knowledge Graph Completion, particularly focusing on the challenge of extrapolation. This task aims to infer knowledge for a given Knowledge Graph at time T using only past data. The authors propose a well-structured solution, providing an extensive evaluation that compares their model to current state-of-the-art symbolic and neural methods.", "questions": "1. In Section 4.3.2, the function $V(s, r, o, t_c)$ is introduced. However, it is unclear how the model handles $t_{last}$ if the fact $<s, r, o>$ was never previously observed. Is the value set to 0, potentially conflicting with timestamp 0, or is another value assigned?\n2. Could the authors elaborate on the benefits of examining a significantly higher number of candidates (100x), especially given the notable increase in runtime (an additional 500 seconds)? Understanding the trade-offs would be helpful.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Comprehensive Evaluation: The paper includes a thorough evaluation of the proposed model, supplemented by an ablation study and an efficiency analysis, making it easy to understand the model's effectiveness.\n2. Efficient Model Design: The model circumvents the common complexity issue of creating separate matrices per timestamp, resulting in a compact, efficient design.\n3. Clear and Reproducible Description: The methodology is well-articulated, allowing for straightforward reproduction of the results.", "weaknesses": "1. Minor Typos and Inconsistencies:\n   - Line 451/452: The term \"INFER(Temp)\" is used instead of \"INFER(Temp Val).\"\n   - Line 515: The rule quantity is stated as 40, while Table 1 lists it as 60, which could lead to some confusion.\n2. Efficiency Study Observations: The conclusions drawn in the efficiency study are somewhat unclear. Specifically, a competing model achieving a similar score while exploring fewer candidates might suggest that the alternative approach is better optimized in its candidate selection process. Clarifying these efficiency aspects would strengthen the study.\n3. Completeness of Graph Argument: The argument concerning graph completeness and the slope behavior appears overstated, as the trend lines for the three methods are quite similar. Revising this interpretation could enhance clarity.\n4. Ambiguity in Section 4.3.3: The fourth paragraph lacks clarity regarding \"variable constraints in rules,\" which limits the reader's understanding of the approach. A clearer explanation or example would be beneficial here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B08ROKLMpg", "reviewer": "Reviewer_EcyD", "summary": "To solve the problem that rule-based methods in the field of temporal knowledge graph inference have insufficient reasoning ability when graph facts are missing, and the fact state is simply considered as binary, ignoring the validity and frequency of historical facts, this paper proposes the neural symbol model of INFER, which quantifies fact credibility in time dimension by introducing a time validity function. At the same time, the time weight matrix is introduced so that the model can infer the missing facts and deal with the incompleteness of the map. And to improve the efficiency of rule reasoning, a rule projection module is proposed, which uses GPU-based matrix operation instead of traditional path matching.", "questions": "(1) Is the use of the square root form of the time decay term in the time validity function based on the conclusions obtained from experiments?\n(2) What is the additional overhead if you replace the static embedded model with a time-embedded model? And What's the performance boost?\n(3) When calculating the rule confidence, the INFER model adopts the rule learning algorithm in TRRules. The rule base retains only cyclic rules and filters out acyclic rules. TR-Rules uses acyclic rules and proves its effectiveness. What is the reason for filtering acyclic rules here? Is there any experimental support?\n(4) Does the static embedding model provide the same level of confidence for the same fact at different time points?\n(5) How does INFER's rule projection module perform when dealing with rules of different lengths?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "(1) This paper introduces effective means such as the time validity function to quantify fact credibility in the time dimension and proves its effectiveness through experiments, which provides ideas for how to improve the reasoning ability of rule-based methods in dealing with incomplete graphs.\n(2) The average performance of the experiment is good on multiple datasets, and the self-made TKG data with sparse facts is obtained. The experimental results show that the INFER model is significantly better than other methods when the facts are sparse.\n(3) The chart is clear, the paper is completed with a high degree, and it is easy to interpret.", "weaknesses": "(1) The Design of the time validity function: The time validity function proposed in this paper calculates the time weight of historical facts based on the time interval and frequency of fact occurrence. Although the above two terms are considered at the same time, the function form is relatively fixed and more dependent on experience. Adapting the attenuation rate using data-driven methods may enhance the model's adaptability. \n(2) The performance of the model on the ICEWS05-15 dataset does not exceed that of TECHS, and there is no detailed analysis of the results in this paper.\n(3) The rule projection module used by INFER loses the ability to directly model the sequence of facts to a certain extent, which may affect the accuracy in scenarios requiring strict time order or multi-jump reasoning. Additional experiments are needed for evaluation, especially for long rule samples with variable constraints. \n(4) INFER introduces neural network embedding and complex matrix operations, and although the authors show entity scores when the rules are applied, it still damages the interpretability of the traditional rule model to some extent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D74IxPNGIS", "reviewer": "Reviewer_qjZA", "summary": "This paper introduces INFER, a neural-symbolic model designed for Temporal Knowledge Graph (TKG) extrapolation reasoning. Traditional rule-based methods for TKGs, though interpretable, struggle with temporal reasoning as they treat facts as binary and ignore temporal frequency and validity. NFER addresses these issues through a Temporal Validity Function, which enables continuous truth values and models the frequency and validity of historical facts for better temporal adaptation. Additionally, INFER incorporates Temporal Weight Matrices with a pre-trained static KG embedding model to improve inference quality. A rule projection module enhances computational efficiency by leveraging GPU-optimized matrix operations, allowing INFER to scale effectively and integrate with embedding-based approaches. Experiments show that INFER achieves state-of-the-art results on several datasets, demonstrating enhanced inference capabilities over existing models, particularly in sparse TKG settings.", "questions": "See Weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.   Temporal Knowledge Graph (TKG) reasoning is an important  research topic.\n 2.   The proposed method shows better results compared to existing rule-based methods.", "weaknesses": "1.  The motivation behind the method design is unclear, and the description lacks clarity. For example, in Section 4.3.3, the rationale for the rule projection strategy is not well-explained, and the meanings of terms like “Ans_i” and “Ans” are not clarified.\n2.  Limited novelty. The techniques used in the proposed method do not introduce significant innovations.\n3.  The temporal and spatial complexity of the inference process appears high, which might impact practical applicability. It would be beneficial to provide an analysis of the computational complexity.\n4.  Some experimental details are insufficiently explained. For example, in Line 426, the phrase “traditional binary truth values for historical facts” needs further clarification. It is unclear what specific criteria or methods are used to assign these binary truth values to historical facts.\n5.  The dataset coverage is insufficient. The paper only uses the ICEWS dataset, which belongs to a specific category of TKG data. It would be valuable to include additional datasets like WIKI or GDELT to demonstrate the method’s generalizability across different data types.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)