TILP: Differentiable Learning of Temporal Logical Rules on Knowledge Graphs
Authors: Siheng Xiong
Yuan Yang
Faramarz Fekri
James Clayton Kerce
Venue: ICLR 2023 poster (2023)

Abstract: Compared with static knowledge graphs, temporal knowledge graphs (tKG), which can capture the evolution and change of information over time, are more realistic and general. However, due to the complexity that the notion of time introduces to the learning of the rules, an accurate graph reasoning, e.g., predicting new links between entities, is still a difficult problem. In this paper, we propose TILP, a differentiable framework for temporal logical rules learning. By designing a constrained random walk mechanism and the introduction of temporal operators, we ensure the efficiency of our model. We present temporal features modeling in tKG, e.g., recurrence, temporal order, interval between pair of relations, and duration, and incorporate it into our learning process. We compare TILP with state-of-the-art methods on two benchmark datasets. We show that our proposed framework can improve upon the performance of baseline methods while providing interpretable results. In particular, we consider various scenarios in which training samples are limited, data is biased, and the time range between training and inference are different. In all these cases, TILP works much better than the state-of-the-art methods.

Published as a conference paper at ICLR 2023
TILP: DIFFERENTIABLE LEARNING OF TEMPORAL LOGICAL RULES ON KNOWLEDGE GRAPHS
Siheng Xiong, Yuan Yang, Faramarz Fekri & James Clayton Kerce Georgia Institute of Technology Atlanta, GA 30332. USA {sxiong45, yyang75 eeaaeeh..eeeeeeeeeeeeeeeeeeee faramarz. fekri pekrideece @ece. fekri@ece.gatech... www.ch.etchh.... clayton. pertenntereeegt:: gatech.edu
ABSTRACT
Compared with static knowledge graphs, temporal knowledge graphs (tKG), which can capture the evolution and change of information over time, are more realistic and general. However, due to the complexity that the notion of time in- troduces to the learning of the rules, an accurate graph reasoning, c.g., predicting new links between entities, is still a difficult problem. In this paper, we propose TILP, differentiable framework for temporal logical rules learning. By designing a constrained random walk mechanism and the introduction of temporal operators, we ensure the efficiency of our model. We present temporal features modeling in tKG, e.g., recurrence, temporal order, interval between pair of relations, and duration, and incorporate into our learning process. We compare TILP with ttate-of-the-ar art methods on two benchmark datasets. We show that our proposed framework can improve upon the performance of baseline methods while provid- ing interpretable results. In particular, we consider various scenarios in which training samples are limited, data biased, and the time range between training and inference are different. In all these cases, TILP works much better than the state-of oftth-oft methods.
INTRODUCTION
Knowledge graphs (KGs) contain facts r,e)),)) representing relation r between subject entity es and object entity c,, e.g., (David Beckham, plays for, Real Madrid). In real world, many relations are time-dependent, c.g., a player joining a team for a season, a politician holding a position for certain period of time, and two persons' marriage lasting for decades. To represent the evolution and change information, temporal knowledge graphs (tKGs) have been introduced. An interval I, indicating the valid period of the fact, is utilized by tKGs to extend the triples (es, T, co) into quadruples (ea, r, co I), e.g., (David Beckham, plays for, Real Madrid, [2003, 2007).
KGs such link predication, i.e., inferring missing facts using isting facts, is a common task for real-world applications. However, the introduction of temporal information makes this task more difficult. The important dynamic interactions between entities can not be captured by learning methods developed for static KGs. Recently, a few embedding-basee frameworks have been proposed address the above limitation, e.g., НyTE (Dasgupta al. (2018), )
Automatically reasoning
TNTComplEx (Lacroix et al. (2020)), and DE-SimplE (Goel et al. (2019). The common principle adopted by these models is to create dime-dependent embeddings for entities and relations.
Alternatively, first-order inductive logical reasoning methods have some desirable features relative to embedding methods when applied to KGs, as they provide interpretable and robust inference re- sults. Since the resulting logical rules contain temporal information in tKGs, we call them temporal logical rules. Some recent works, e.g., StreamLearner (Omran et al. (2019). and TLogic (Liu et al. 22021), )) have introduced framework for temporal KG reasoning. However, there are still several unaddressed issues. First, these statistical methods count from graph the number of paths that sup- port given rule its confidence estimation. As such, this independent rule learning ignores the interactions between different rules from the same positive example. For instance, given certain rules, the confidence of some nules might be enhanced, while that of others can be diminished. Sec-

---

Published as a conference paper at ICLR 2023
ond, these methods cannot deal with the similarity between different rules. Given reliable rule, reasonable to believe that the confidence another similar rule, egg. with the same predi- cates but slightly different temporal patterns, is also high. However, its estimated confidence with these methods can he quite low if is infrequent in the dataset. Finally, the performance timestamp- -aannddd--ddd----- methods interval-based tKGs is not demonstrated. It should be noted that the temporal relations between intervals are more complex than those of timestamps. All these problems are solved by our poural-network-basee framework.
these
In this paper, we propose TILP, a differentiable inductive learning framework. TILP benefits from a novel mechanism of constrained random walk and an extended module for temporal features mod- eling. We achieve comparable performance to the state-of ofrthe-r- methods, while providing logical explanations for the inference results. More specifically, our main contributions are summarized as follows:
TILP, novel differentiable and temporal inductive logic framework, is introduced based on constrained random walks on temporal knowledge graphs and temporal features modeling. is the first differentiable approach that can learn temporal logical rules from tKGs without restrictions.
Experiments on two benchmark datasets, i.e., WIKIDATAI2K and YAGO1 are con- ducted, where our framework shows comparable or improved performance relative to the state-of-the-an art methods. For test queries, our framework has the advantage that provides both the ranked list of candidates and explanations for the prediction.
The superiority of our method compared to existing methods is demonstrated in several scenarios such as when training samples are limited, data biased, and time range of training and testing are different.
2 RELATED WORKS
Embedding- methods. Recently, embedding- based methods for tKGs started emerging for more accurate link prediction. The common principle of these methods is to create time-dependent embeddings for entities and relations, e.g.. HyTE (Dasgupta al. (2018). TA-ComplEx (García- Durán et al. (2018)) TNTComplEx (Lacroix et al. (2020). ) and DE SimplE (Goel et al. (0199). These embeddings are plugged into standard scoring functions in most cases. Further, other works, e.g. TAF-IIP (Jiang et al. (2016)), and TimePlex (Jain et al. (0200), have investigated the explicit temporal feature modeling, and merged it into the embedding algorithms. The main weakness of embedding- baaedd methods is lack of interpretability.
well as their failure when previously unob- served entities, relations, or timestamps present during inference.
Logical-rule-based methods. Logical-rule-based methods for link prediction tKGs is mainly based on random walks. Although these works show the ability of learning temporal rules, they perform random walks in very restricted manner. which impairs the quality of learned rules. For example, Dvee (Mahdavi et al. (2018)) and Change2vec (Bian et al. (2019) both process tKGs set of graph snapshots different times where random walks performed separately. DynNeeEmbedd ( (Nguyen et al. (2018) requires the edges in walks to be forward in time. Stream- Learner OOmrn
(2019) first extracts rules from the static random walk, and then extend them separately into time domain. The consequence is that all body atoms in the extended rules have the same timestamp. TLogic (Liu et al. (202T) is the most recent work which extracts temporal logi- cal rules from the defined temporal random walks. The temporal constraints for temporal random walks are built on timestamps instead of intervals, and are fixed during the learning. This inflex- ibility impairs its ability in temporal constraints learning. Furthermore, both Streaml Learnrr and TLogic, which can truly learn temporal logical rules, are statistical methods which estimates the rule confidence by counting the number of rule groundings and body groundings.
Differentiable rule learning. Several works utilize neural network architectures for rule learning, e.g., Ne (Yang et al. (2017)) NTP (Rocktäschel & Riedel (2017)) DeepProblog (Manhaeve et al. (2018) 1 ), AILP (Evans& & Grefenstette 2. RuLES (Ho et al. (011)), IterE (Zhang et al. (2019)) ), dNL-ILP (Payani& & Fekri (2019) and NLProlog (Weber al. (019)). These works mainly focus on static KGs, lacking the ability capturing temporal patterns. Converting from static KGs to temporal KGs is not trivial extension. For example, Neural-L.P reduces the rule learning problem to
2

---

Published as a conference paper at ICLR 2023
matrix multiplication. By defining the operators and neural control system, this framework realizes logical rule learning in an end-to-end fashion. However, the extra temporal constraints in logical rules break down the Markovian property of random walks which serves the foundation of the past frameworks.
PRELIMINARIES
Temporal knowledge graph. A temporal knowledge graph (tKG) & is a collection of facts repre- sented by quadruple (e.. r, co. 1). This fact, also called edge or link, implies a relation from the subject entity es the object entity 6, during interval We define an interval 7 with its start time to and end time te, i.e., I = [t.. fe). To allow bidirectional random walks, we imagine the existence of inverse edges, i.c., (E.. r Es, I). The set of entities, relations, timestamps and intervals are denoted by E, R. T and Z, respectively.
Link prediction. This task to predict missing links with observed facts from the same tKG. To be specific, given query (e,, 7,, 2, and the observed facts from the same tKG g, ranked list of candidates for the missing object is required. For subject prediction, the query is formulated as (e.rr_, 7) Compared with static link predication, the temporal link prediction is much harder: Even given the same subject (or object) and relation, the correct answer can change with different query intervals.
Temporal relation. Temporal relation (TR) between two timestamps and t' can take the form be brre, equal, a.feer, which are denoted by / < L', ( i', and (', respectively. For TR between two intervals and 1', there are 13 different relations in total given by Allen's interval algebra (Allen I983)). For example, == [t., te] is before I' := [ty, t iff te <, However, the resulting temporal logical rules would become too specific by directly using all these 13 types. Thus, we group them into classes: TR E (before, touching, after), where TR denotes the possible TR between two intervals, be fre is defined as previously mentioned, after is the converse of be ore, and touching is the group of all the other 11 types. In other words, touching is used when two intervals have overlap with each other. It should be noted that timestamp can be considered special kind of interval with equal start time and end time. Thus, our definition of temporal relation TR can be also used to describe TR between timestamps.
Temporal logical rule. A temporal logical Rule of length I E N is defined as
Pu+1 (E1, 4+11 + A (E, Es+1, 4.) M/=1 (N TTt (4.
(I)
where F, E & denotes variables of entities, I, E I denotes variables of intervals, P, E R denotes predicates, which are grounded relations in logical rules, and TR E {before, touching, after) denotes grounded temporal relations between intervals.
The left arrow in Rule is called "entails", i.e., the rule body on the right entails the rule head on the left. The rule head contains a head predicate P4+1. also called the target predicate. For the link prediction task, target predicates T+1 is called query interval. Similarly, P; and I, for E [1,7] are called body predicates and body intervals, respectively. The rule is also called "chain-like" because the rule body corresponds walk from Ej to E
given. Thus, use Pa. to denote it in the following sections.
A rule is grounded by substituting the variables E and I with constants. For example, grounding of the following temporal logical rule
ReceiveAward (E1, E2, 12) NominatedFor (E1, E2, touching(41.
given by the edges (Alice Bradley Sheldon, receive award, Nebula Award for Best Novelette, [1977, 1977]) and (Alice Bradley Sheldon, nominated for, Nebula Award for Best Novelette, [1977, 1977]) in WIKIDATA 12k dataset. Since logical rules can be violated, rule confidence, the probabil- ity that a rule is correct, needs to be estimated.
4 CONSTRAINED RANDOM WALK
Path constraint. Temporal logical rules can be considered as constraints for random walks on IKG. Generally speaking, these constraints can be divided into classes: Markovian and non-
3

---

Published as a conference paper at ICLR 2023
Markovian. With Markovian constraints, the calculation of next state probability is only related to current state probability, ie., a random walk is performed without the consideration of previous visited edges. Otherwise, need to record the previous visited edges to ensure that non-Markovian constraints are satisfied.
For the temporal logical rule given in (1), Markovian constraints include body predicates, i.e., P, for i E [1, 1.. and TRs between query interval and every body interval, i.e. TR.,4+1 for E [1, Further, the -on-Markovian constraints include pairwise TRs between body intervals, i.e., TR2a for j E [1,1- 1] and E
[j + 1,4] Filtering operators / for these constraints are defined as
r, co. ()) = { fr r,
if r otherwise.
(2)
STR (le,, cor n), (' 11)= = {
if (I, TR, otherwise.
(3)
before
Le < 1/
9 (I,
after
if t/
(4)
touching
otherwise.
where fo and f denote filtering operators for predicate P and temporal relation TR, respectively, and g the TR evaluation function with / te] and I' [.).
Constrained random walk. Since a successful random walk should satisfy both classes of con- straints, we first perform random walk under Markovian ones, and then filter out the results accord- ing to non Mar-aano ones. To ensure the efficiency of our framework, we use matrix operators built from the filtering operators. Given query (e,, I). for every pair of entities e, Cy E E, the operator M,CMM, E {0, 1il related to step i under corresponding Markovian constraints CMM, = (P., (R,, is defined as:
(Mr.cM)aa = max fCM, (F) FEFoi
(5)
= maxx fr. (F) FIRIA (F, re,,,, FEFub.
where (M.CM))... denotes the (x, entry of MLCM,, F denotes single fact, and Fy.z denotes the set of facts from by to 6z- The essence of the operator is the adjacency matrix under Markovian constraints, and we set the entry maximum to 1. With these operators, we can actually find all the paths between any pair of entities that satisfy these Markovian constraints. Suppose we start from entity e.. After I steps of random walk under corresponding Markovian constraints, the process is written
(6) where V: E NIE is the indicator vector, e.g., for V1 only the entry related to e, is set to 1, other entries being O. When arriving some entities, the corresponding entries would be greater than O. With these indicator vectors, we can obtain every single constrained random walk Wiln) for n E N
Vi+1 = Ma..M,
for 1 sisl
given by ((e(m) (n) (m) I(")), (.....(e)"" (c(m)) (m) (n) 7(1 )))
For non non-Markovian constraints CN₁.a- (TR we apply corresponding filtering functions to these walks as:
1-1 II II fCN ,11gm) "/n"), (m) o.. I! cf
fww(w(n)) CMd
(e(") , (m) 1 {()))
[-1 1
II j=l k=j+1
II STR ((e)m) .o -e 1, 1()) (n) (n) (m) eet (()))
(n) (n) (n)
,rk
The set of filtered walks Sw: {W(()) C. fcx(W(()) 1} is the final result of our algorithm. In this process, we also remove walks that involve repeated edges. In fact, by introducing new filtering functions, this framework
have more possibilities. For example, using the numerical comparison filtering functions (Wang et al. (019)), our method is able to learn logical rules with numerical features such as a person's age, height and weight.
4

---

Published as a conference paper at ICLR 2023
5 LEARNING TEMPORAL LOGICAL RULES
The framework mainly consists of two stages: rule learning followed by rule application. In the learning stage, given set of positive examples for the target predicates, are going to find all the paths from the subject entity to the object entity in the tKG. Then we extract temporal logical rules from these paths, and estimate their confidence. The parameterised distributions of different temporal features introduced in this sections are also measured at this stage.
In the application stage, given a query and of temporal logical rules related the target pred- icate, we are going to find all the random walks that satisfy these rules. By calculating the arriving rate of each rule and aggregating all the rules, we can obtain the temporal logical rule score for all candidates. In addition, we use all the evidences related to these candidates to evaluate their temporal feature scores which, together with temporal logical rule scores, form the final scores.
Rule confidence. For the extracted temporal logical rules, we need to estimate their confidence for the rule application. Instead of estimating confidence for every single rule, we create attention vectors for every type of constraints, and re-use them in different rules. Sharing confidence of constraints creates a joint and robust learning process, and largely reduces model parameters.
In our temporal logical rules, there are two types of constraints, predicates and temporal relations. For every target predicate, we create a set of attention vectors to denote the confidence of using certain constraint. Obviously, there are many factors that can affect these attention vectors, e.g. the target predicate, the query interval, the rule length, entity properties, and so on. To make this problem tractable, some simplifications predicates and TRs are dependent on the target predicate and the rule length. Furthermore, to deal with varying lengths of rules, attention vector of the rule length is also required. Inspired by Neural-LP (Yng et al. (017)). we design a set of mapping functions based on RNN.
made here. We suppose that the attention vectors of
non-Markovian
(W Yeus (W Yeun (Dir)
constraints
Markovian constraints
(wr)ia mm (w..), (wig ae (w) (w Yaai
1
Figure 1: The RNN-based system for solving the attention vectors
Specifically, shown in Fig. 1, let WLen R& be the attention vector of rule length where I. is the maximum. For certain length / [], at each step i E [1, we calculate the attention vector of predicate (wp){ E IR [R] and its TR between query interval (WTR) E IR!T We also calculate the attention vector pairwise TR between body intervals (WTR)ţ E R|TR for j [1, and k E 1, using corresponding states.
h/ update (em (X', Ph))
(8)
(wp){ = softmax (Wph' + +p)
(9)
(wrn){141 = softmax (WrRh + brR)
(10)
(WTR))J softmax (WiRth); h// + bTR)
(11)
WLen = softmax (Wien f fembaa (x Ph)) + by.n)
(12)
where h &dd denotes the tth-step state in the 1-length rule with feature dimension d, h/ is set as
0, x E Rd denotes the embedding matrix for target predicate Pn. fembda(X, := xxx x embedding lookup function in which up is a one-hot indicator vector for input p. i.e.,
5

---

Published as a conference paper at ICLR 2023
Sembaia (X', Pr) E Rd, and Wp E R WIR E R WiR R T2244 bp E RIRi, brn, VTR E RR.RR,, bLen TR are all learnable parameters.
x2d Wen Biidd
Given these attention vectors, the confidence of a l-length Rule given by (1) is written as:
I
score(Rule) Tembaa(('Len 0) II (emmmaa((w))) P.) (e) Sembod TR44,,,,,R444444 TBa1-1)
(13)
I-1 II
II fembba((WTRi) TR2,k) J=1 k=j+1
where fembda (W 1) E R denotes the confidence of length 1, fembod((wp)) E R denotes the confidence of predicate P;, (embaa((wTR)(L. 11,44) E IR denotes the confidence of temporal TR R denotes the confidence of temporal relation
relation T T.,,14. and fembaa(( TRzA-
Temporal feature modeling. In our framework, we only focus on the connecting paths between the two entities in a query, and the TR between intervals is actually discretized, which might impair our model performance. To address these limitations, we introduce temporal features modeling where extra evidences and continuous distribution measurements are involved. Inspired by TimePlex JJain et al.] (2020), we design an extended module. The main features considered here include Recurrence, T TemporalOrder, PrelationPaii I and Duration.
Recurrence describes the distribution of recurrence of the same relation. Different from TimePlex's s description (c,, Ca. *), we measure this feature with more general form (eg, r,, K x). For each relation r. we use a parameter (Prec), to denote the probability that this relation will happen again for the same subject. It should be noted that inverse relations r-1 are also involved.
TemporalOrder describes the distribution of the temporal order between relation pairs happening for the same subject, i.e., (e,, r, *, I) (e,, r,,,, I'). This feature is implied by TRs temporal logical rules some extent. However, for two touching intervals, we still cannot tell which one happens earlier. Thus, a parameter (porder)r.r" is adopted for every pair of relations r and r' denote the probability that r happens earlier than r'.
Relation PaitiionnPPrrlllaa Interval describes the distribution of the time gaps between relation pairs happening for the same subject. Different from TimePlex, given a pair of relations r and r', we consider two types of distributions for their time gap, Gaussian and exponential, with parameters (Hpprr))r Jr,r', (Apoör)r.s )r,r and (Apair)r,r', respectively. Gaussian distribution is preferred when there is a roughly fixed interval such the birth date and death date of the same person, while exponential distribution is more suitable for two strongly correlated relations.
Duralion describes the distribution of the interval length of every relation. We suppose the duration of each relation r follows Gaussian distribution with parameters (μa)r; (az)r- It is common in large tKGs that the exact date of some facts are missing. With this feature, we can estimate these missing dates and improve our model performance.
With these temporal feature distributions, we can further evaluate the candidates of query. Similar TimePlex, a linear function of probability used as scoring function, i.c., drees Φordery Opair. However, more evidences are used by our model in the evaluation. The evidences of TimePlex only include facts happening between the known entity and every candidate, while our model ex- tends with the constrained random walks. Given query r,.r,?,11 ?, I) and a candidate e,, all the evidences used in the temporal feature modeling module include facts between e, and e,s iee, Fe,s := {Cee *, e., +)), facts e, but not 6x. i.e., Fa,s = {(e., *,*, +)} Fa,s and con- strained random walks Swe between eg and ec. The temporal feature modeling score defm of the candidate is given
(efm(ee) = (101fm,1 + 120rfm,2(ee) + 7a01fm,3(ee)
(14)
where fm, (cc), d (lem, ((ec) are the scoring functions given Fe.e. Fe.s and (wa(e, respectively, and 7n 72, a/ 2 are the corresponding weights. The details of these scoring functions shown in Appendix A.

---

Published as a conference paper at ICLR 2023
Table 1: Link prediction performance the benchmark datasets
Dataset
WIKIDATA
YAGO1IK
Model
MRR
hit@1
hit@ 10
MRR
hit@1
hit@ 10
Neural-LP
0.1823
0.0908
0.3848
0.1001
0.0401
0.1845
AnyBURL
0.1908
0.1030
0.3904
0.0908
0.0378
0.1814
TLogic
0.2536
0.1754
0.4424
0.1545
0.1180
0.2309
ComplEx
0.2482
0.1430
0.4890
0.1814
0.1146
0.3111
TA-ComplEa
0.2278
0.1269
0,4600
0.1524
0.0936
0.2626
HyTE (TransE)
0.2528
0.1470
0.4826
0.1355
0.0332
0.2981
DE-SimplE
0.2529
0.1468
0,4905
0.1512
0.0875
0.2674
TNT-Complex
0,3010
0.1973
0,5069
0.1801
0.1102
0.3128
TimePlex (base)
0.3238
0.2203
0.5279
0.1835
0.1099
0.3186
TimePlex
0.3335
0.2278
0,5320
0,2364
0.1692
0.3671
TILP (w/o tfm)
0.3114
0.2152
0,5077
0.1880
0.1336
0.3089
TILP
0.3328
0.2342
0.5289
0.2411
0.1667
0.4149
Candidate ranking. Suppose by applying temporal logical Rule, find total of N successful random walks in which Ne of them arrive at entity ec. With the assumption that each walk has equal contribution, we can calculate the arriving rate a₂ Ne/N. By aggregating all the rules learned for the target predicate, we obtain the logical rule score ΦTLR. Combining with the temporal feature modeling score f via corresponding weights TTLR:: f 2 0, the final score ØTILP becomes
ΦTIR(Ce) = Σ
(15)
Rude
(TILP(e)) = (TTLNTTLN(e) (afmOefmlee)
(16)
The training is divided into two phases. In the first phase, the attention vectors for predicates, TRs and rule length are learned by maximizing the score of correct candidates. In the second phase, all the distribution parameters temporal features are fitted with training samples. Then we train the parameters of weights for the temporal feature modeling module with frozen attention vectors, i.e, ØTLR is used for prediction in the first stage, and ØTILP adopted the second stage.
EXPERIMENTS
6.1 EXPERIMENTAL SETUP
We evaluate TILP standard tKG datasets, WIKIDATAI2K and YAGO1 (Dasgupta et al. 2018)). Detailed dataset introduction and statistics are given in Appendix B. These datasets contain temporal facts the form (e,, co, eo. e.g. (John Okell, worksAt, SOAS University of London, 1159,, 1999)). The temporal specificity of facts in these datasets can be at the year, month, day level, although month and day data are not present the majority of examples; we remove month and day information from such facts to achieve more uniform data representation. For datasets with higher granularity, we would expect improved performance due to more precise temporal relations. For the link predication task data of the form (c,, r. Co. I), we generate a list of ranked candidates for both object prediction (e., r, ?, /) and subject prediction rto,r-1,' 7). The maximum rule length is set to for both datasets. The standard metrics, mean reciprocal rank (MRR), hit@ hit@ 10 are used for comparison of the methods. Similar to Jain et al. (202)), we perform time- aware filtering which gives a more valid performance evaluation.
We compare TILP with state-of-the-art baselines in two dimensions: static v.s. temporal, logical- aule-based v.s. embedding- based. The static logical-rule-bact based methods include Neural-LP (Yang
et al. (2017)) ) and AnyBURL (Meilicke et al. (2220). The temporal logical-rule-based method is TLogic (Liu et al. (202T). The static embedding-bere base model ComplEx (Trouillon et al. (2016)). TThe temporal embedding- model include TA-ComplEx (Garefa-Durán et al. (2018). HyTE

---

Published as a conference paper at ICLR 2023
Dasgupta al. (2018)). DE- DE-SimplE (Goel et al. (2019)), TNT-Complex (Lacroix et al. (2020)). and TimePlex (Jann et al. (2020)). The results for all embedding- based models are from Jain et al. (2020). Ablation studies the temporal feature modeling module (TILP w/o tfm) are also conducted.
6.2 RESULTS ANALYSIS
The results of the experiments are shown in Table with the efficiency study given in Appendix C. 'he performance TILP is comparable with the best temporal embedding method across all metrics, with TimePlex performing slightly better than TILP against half of the evaluated metrics. Note that because of the inttrprittcrre form of the TILP predictions, its temporal logical rules provide explanatory support for predictions, while the interpretation of TimePlex embeddings would be more opaque. Some examples of the learned rules and groundings from TILP on the WIKIDATA dataset given below.
Rule
memberOf (F1, E,,, 1.) <- memberOf E2, I1) memberOf (Ez, F.z, 12)
AmemberOf [Eg, F4, I2) M (Aj=141 touching(1;, 12))
Grounding: Ej = Somalia, Eg = International Development Association, E3 = Kingdom of the Netherlands, E4 International Finance Corporation, 11 = [1962, present]. 17 = [1961, present]. 43 = =19956, present]. I4 = present.
Rule 2:
receive (F1, F., 14) nominatedFor (F₁, F2, T1) A nominatedFor (Flz, F3, I2)
Award (Ez, E4, I3) A before 12) ~2=1 after(Ts, Iạ) A3=1 before (oaiore([g:))) I4)
Grounding: E₁ ZDF, Eg International Emmy Award for best drama series, E3 DR, E₁ Peabody Awards, 11 = [2005, 2005], I2 = [2009, |2009,2009]. 73 [1997, 1997]. 1T = [2013, 2013).
Granularity of temporal relations: We observe that the well-known TLogic approach, which uses a point-in-time data representation of temporal facts, doesn't perform as well as more recently developed methods these two interval-based tKGs. Several factors likely contribute to this phe- nomenon. First, the foundation of the temporal walks TLogic is the relative order of time points. Without the introduction of temporal relation between intervals, TLogic not be simply extended for interval- -based tKGs. Second, to improve model efficiency, TLogic uses sampling strategy with the control of the total number of temporal walks. This strategy actually impairs model performance due to no guarantee for successful long-distance random walks. It is proven by the facts that a lot of temporal logical rules we found in these two datasets are of length which can be challenges for TLogic.
Temporal feature modeling: These experiments suggest that time time-aware aware learning is an important component for link prediction in tKGs since all static learning methods are out-performed by their counterparts with temporal learning abilities. This leaves us with comparison of logical-rule-based methods to embedding- based methods. The former integrates temporal relations into logical rule representation, while the latter uses time-dependent embeddings. Both approaches can be imple- mented in a variety of ways, for example previous explicit temporal features modeling approaches have used: Recurrence, TemporalOrder, Relation Interual and Duration. The alterna- tive representation of temporal intervals continuous distributions has demonstrated the greater success than these previous models, as shown by the evaluation of both TimePlex and TILP (our model). However, our extensions of the temporal feature modeling module are non-trivial since evidences of constrained random walks can not be found by any embedding- based methods.
6.3
MORE DIFFICULT PROBLEM SETTINGS
Our model uses neurally generated symbolic representations of temporal and entity relationships, while performing par with mance, symbolic representations convey several advantages in understanding the prediction quality in tKGs. To demonstrate these strengths, propose the following more difficult problem settings
embedding based methods. Looking beyond perfor-
8

---

Published as a conference paper at ICLR 2023
in consideration of data efficiency, model robustness, and transfer learning. All these scenarios important and changeling tasks in KG reasoning, and temporal information in tKGs makes the prob- lems harder. A few related works (Mirtaheri et al. (2020) Xu et al. (202I) Liu et al. (202T) try to give some solutions, but there is still room for improvements. To simplify the discussion, we restrict comparison of our model to the standard and highest performing methods for temporal link prediction: TLogic (ogical-rule-based) and TimePlex (embedding-basee
Few training samples. Training data is often expensive obtain for new scenarios, and therefore the performance of a method under limited training data is an important consideration. We paramet- rically examine the relative performance of our model in this low-data scenario, and demonstrate its data efficiency using the MRR metric. Details of the setting and the results are shown in Appendix D. observed that when the training set size decreases, TILP outperforms all the baseline meth- ods. Through constrained random walks, TILP is able to capture all the patterns related to query relation which are independent on entities. Reducing training set size only changes the frequency of different patterns. To contrast, embedding-basee methods require enough training samples to learn good embeddings entities and relations.
Biased data. Another common problem in knowledge graph learning is data imbalance. For some rare relations, it hard to collect samples. For example, in the WIKIDATAI2 12k dataset, where there are 40621 edges in total, the number of edges for relation capitalOf and residence and 80, respectively. This demonstrates that achieving good model for every relation in the dataset requires the ability to manage biased representation frequency. Details of the setting and the results are shown in Appendix D. From the results, we conclude that the attention vectors of predicates, temporal relations and rule length are relation-dependent in TILP, making it less susceptible than other methods to data imbalance. In contrast, embeddings of entities are shared among all the relations, making embedding- based method suffer more susceptible data imbalance.
only 86
Time shifting, For tKG, the problem of time shifting is also noteworthy. Intuitively, more over- lapping between the period of training set and that of test set brings the higher model performance. However, this condition can be violated in scenarios such as link forecasting. Similar to TLogic, we re-split the whole dataset according to the start time of each relation. Details of the setting are shown in Appendix D, and the results are shown in Table 2. One major limitation of most time-aware embedding- based methods is the use absolute timestamp as anchors, preventing generalization to either time shifting settings and inductive settings (Liu et al. (202I)). With such limitations in mind, TILP extracts temporal logical rules with relative temporal relations, providing greater flexibility, e.g. for transfer learning to arbitrary temporal periods.
Table 2: Link prediction performance with time shifting setting
Dataset
WIKIDATAI2 12k
YAGO1
Model
MRR
hit@1
hit@ 10
MRR
hit@ 1
hit@ 10
TimePlex (base)
0.0975
0.0647
0.1588
0.0581
0.0299
0.1085
TimePlex
0.0976
0.0651
0.1580
0.0639
0.0341
0.1146
TLogic
0.1508
0.0934
0.3201
0.1107
0.0621
0.1804
TILP
0.2657
0.1623
0.4412
0.2069
0.1194
0.3686
7 CONCLUSION
TILP, the first differentiable framework for temporal logical rules learning, has been proposed for the link predication task on temporal knowledge graphs. Experiments on two standard datasets indi- cate that TILP achieves comparable performance to the state-of-thed the--art embedding- based methods while additionally providing logical explanations for the link predictions. In addition, we consider some important learning problems in temporal knowledge graphs, where TILP outperforms most baselines. An interesting direction for future work is to predict event intervals with temporal logical rules. In such task, the learned rules must contain both numerical values and temporal relations, situation which should further benefit from the expressive power of the logical rules considered in the TILP framework.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported by sponsored research award by Cisco Research.
REFERENCES
James F Allen. Maintaining knowledge about temporal intervals. Communications of the ACM, 26 (11):832 843, 1983.
Ranran Bian, Yun Sing Koh, Gillian Dobbie, and Anna Divoli. Network embedding and change
modeling in dynamic heterogeneous networks. In Proceedings of the 42nd International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 861- 11-8664, 2019.
Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. HyTE: Hyperplane-based tem- porally aware knowledge graph embedding. In Proceedings of the 2018 Conference on Em- pirical Methods in Natural Language Processing, pp. 2001-2011, Brussels, Belgium, October- November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D188 1225. URL 1225
Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1-64, 2018.
Alberto Gareía-Durán, Sebastijan Dumančić, and Mathias Niepert. Learning sequence encoders for temporal knowledge graph completion. arXiv preprint (nXiv:1809.0833 2018.
Rishab Goel, Seyed Mehran Kazemi, Mareus Brubaker, and Pascal Poupart. Diachronic embedding for temporal knowledge graph completion, 2019. URL UBL./ttpps///arrii.orr////////
03143
Vinh Thinh Ho, Daria Stepanova, Mohamed Gad-Elrab, Evgeny Kharlamov, and Gerhard Weikum. Rule learning from knowledge graphs guided by embedding models. In International Semantic Web Conference, pp. 72-90. Springer, 2018.
Prachi Jain, Sushant Rathi, Soumen Chakrabarti, et al. Temporal knowledge base completion: New algorithms and evaluation protocols. arXiv preprint arXiv:2005.0505: 2020.
Tingsong Jiang, Tianyu Liu, Tao Ge, Lei Sha, Baobao Chang, Sujian Li, and Zhifang Sui. Towards time-aware knowledge graph completion. In Proceedings of COLING 2016, the 26th Interna- fional Conference on Computctional Linguistics: Technical Papers, pp. 1715-1724, 2016.
Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. Tensor decooposiiion for temporal knowledge base completion, 2020. URL httL/httss///axittt http://aaavvv..g////////////0/
Jennifer Lautenschlager, Steve Shellman, and Michael Ward. ICEWS Event Aggregations, 2015.
URL httpp:::/dii..rrgg..otot 7910
Julien Leblay and Melisachew Wudage Chekol. Deriving validity time in knowledge graph. In Companion Proceedings of the The Web Conference 2018, pp. 1771-1776, 2018.
Kalev Leetaru and Philip A Schrodt. Gdelt: Global data on events, location, and tone, 1979-2012.
In ISA annual convention, volume pp. 1-49. Citeseer, 2013.
Yushan Liu, Yunpu Ma. Marcel Hildebrandt, Mitchell Joblin, and Volker Tresp. Tlogic: Temporal logical rules for explainable link forecasting temporal knowledge graphs, 2021. http:
Sedigheh Mahdavi, Shima Khoshraftar, and Aijun An. dynnode2vec: Scalable dynamic network embedding. In 2018 IEEE international conference on big data (Big Data), pp. 3762-3765. IEEE, 2018.
Farzanch Mahdisoltani, Joanna Biega, and Fabian Suchanek. Yago3: A knowledge base from mul- tilingual wikipedias. 7th biennial conference on innovative data systems research. CIDR Con- ference, 2014.
10

---

Published as a conference paper at ICLR 2023
Robin Manhaeve, Sebastijan Dumancie, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic programming. Advances Neural Information Process- ing Systems, 31, 2018.
Christian Meilicke, Melisachew Wudage Chekol, Manuel Fink, and Heiner Stuckenschmidt. Re- inforced anytime bottom up rule learning for knowledge graph completion. arXiv preprint 0441:2 2020.
Mchrnoosh Mirtaheri, Mohammad Rostami, Xiang Ren, Fred Morstatter, and Aram Galstyan. One- shot learning for temporal knowledge graphs. arXiv preprint 1xXv::0010.11144, 2020.
Giang H. Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Hunyee Koh, and Sungchul Kim. Dynamic network embeddings: From random walks to temporal random walks. 20/8
IEEE International Conference Big Data (Big Data), pp. 1085-1092. IEEE, 2018.
Pouya Ghiasnezhad Omran, Kewen Wang, and Zhe Wang. Learning temporal rules from knowl- edge graph streams. In AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering, 2019.
Ali Payani and Faramarz Fekri. Inductive logic programming via differentiable deep neural logic networks, 2019. URL
httpg//bs//19006 03523
Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving. Advances in neural information processing systems, 30, 2017.
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Com- plex embeddings for simple link prediction. In International conference on machine learning, pp. 2071-2080. PMLR, 2016.
Rei Wang, Daria Stepanova, Csaba Domokos, and Zico Kolter. Differentiable learning of numerical rules in knowledge graphs. International Conference on Learning Representations, 2019.
Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, and Tim Rocktäschel. Niprolog: Reasoning with weak unification for question answering in natural language. arXiv preprint 1811117 2019.
Yonghui Xu, Shengjie Sun, Huiguo Zhang, Chang' Yi. Yuan Miao, Dong Yang. Xiaonan Meng, Yi Hu, Ke Wang, Huaqing Min, et al. Time-aware graph embedding: A temporal smoothness and task-oriented approach. ACM Transactions on Knowledge Discovery from Data (TKDD), 16(3): 1-23, 2021.
Fan Yang, Zhilin Yang, and William W. Cohen. Differentiable learning of logical rules for knowl- edge base reasoning, 2017. URL htLttttss://aaat wwwpiiivv.g///bbbb//e/ http/abs/1110 08367
Wen Zhang. Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang. Abraham Bersstein, and Huajun Chen. Iteratively learning embeddings and rules for knowledge graph reasoning. In The World Wide Web Conference, pp. 2366-2377, 2019.
A DETAILS OF TEMPORAL FEATURE MODELING
Given a query (e., r, f) and a candidate e., the facts used in this module include three parts: the set of edges from ea to e, denoted by Fe,s. the set of edges from e, other entities denoted by Fe,s, and the set of paths from e, to e, denoted by Swefer..)) Let R(1), R(22, R(3) C denote the set of relations cxisting in Fc... Fo.s and Swefee,e.) respectively. Given a relation r'. let (T.),. (T,),, (T&)3, E R be the closest start time, existing in Fe... Fe.a and Swa(e,.ea) to the start time of the query interval denoted by L,, respectively. In addition, we introduce scoring function á to integrate different probabilities:
o(ec; h, w. EMER(E.) exp(wx (sp(wr,))) (Hrr,," + by,r"
(17)
EXTER(ed) exp(wxpp (Wr."!)
11

---

Published as a conference paper at ICLR 2023
where o(ec) IR denotes the score of ec. the probability h E R(||[ is corresponding with the query relation r and another relation r', i.c., hy,"' E R, R(ee) S R denotes the set of relations related to Cz. and w. b E TR. [ [ are learnable parameters, i.e, Wr,r", E IR.
In model, given temporal feature x R related to candidate cc. query relation r, and an- other relation r', its probability by." may follow three types of distributions: 1) Bemoulli with parameter E IR R| i.c., hpr (Prr)*(E pr.r:)===; 2) Gaussian with parameters 1. E | i.e., har = (2; Mr,",, 3) Exponential with parameter À E R| i.e. burr" = Ar,y exp( --r,y'
For example, with a yearly resolution in YAGOI dataset, we found the RelationnPirInteev between a person's ''s birth and graduation is normally distributed with mean 22 and standard devia- tion 1, and that between a person's birth and death is also normally distributed with mean 70 and standard deviation 6. Let wasBornin, r' = graduatedFrom, r" diedIn, we have Mrrr'' = 22, Frr'' 1. Mr.-" = 70, Fr,"" 6. Given a query (?, wasBornin, Nashville, [1872, 1872)) and a candidate e, = Cass Canfield, we first check the known facts related to e. which include (Cass Canfield, graduatedFrom, Harvard University, [1919, 1919) and (Cass Canfield, diedIn, New York City, [1986, 1986)). Then the temporal feature, i.e., x = 11872 - 1919] 47. Thus, Hy,," = N(x; pr,r', (r.r') = N47: 22, 1) 7.65 x e e-13 Similarly, hy, ( 1986|; 70, 6) 1.39 x e e-3, These probabilities hy hh, are integrated with (17) to form the scoring function (pair(ee). As for other temporal features, the cal- culation processes of the scoring functions are similar. The differences are: 1) Since Recurrence is only related to the query relation, we use (18) which can be considered as simplified version of (17). 2) Both Recurrence and TemporalOrder follow Berouuli distribution, i.e., = 0 or 1. More details are shown below.
Interval, related to 6or T and r'
Recurrence describes the probability distribution of recurrence of relation r on entity e and is considered in Fe.s and Fe,8- We suppose this probability follows a Bernoulli distri- bution with parameter Prec,2 E |IRl in Fe, and Fe,s, respectively. The following scoring function is defined:
(rec hrees Wrees bree) = (Wree)r(hree) + (bree)r
(18)
where the temporal feature related to and r is x = 1(r E R(ee)), its probability (hree), = ((Prec) )1 (Pr E R is based on Prec E R2 RIR) i.c. hree E R and Wrec: bree E [R(2] are learnable parameters. Given R(ex)= = R(1,, we have Dree, (ees 1 Wree,1, 1)ec,1) with parameters hyec,1, Wrec,1, brec,1 E IR RI; Given R(ec) = R(2), we have Pree:2(ext Rree.2, 2ree,2, brec.2) with parameters hrec,2, Wree,2, bree, E RIR),
T TemporalOrder describes the probability distribution of the temporal order of two re- lations r and r'. and is considered in Fe.e. Fe,z and Swo(esred) We suppose this probability follows a Bernoulli distribution with parameter Parder, R | R in Fe,. Fe.z and Swo(enee)) respectively. Given R(ee) R()), we calculate Border, (ee; Norder, Worder,1, border;)) with (17), where horder; E [ is based on Porder;1 with x [1,, < (Ta),.), and Worder,1, border;1 E [R|RR(
Porder,3 E
R2) are learnable parame- ters. Similarly, given R(ex) R(22, we calculate Φorder; 2 (Eez Morder, Worder,2 border;)) with parameters horder, Worder, border,2 ||R[ where horder;2 is based on Porder.2 with
11t < (T.)²:); Given R(E,) R((), we calculate Gorder, a (ea; horder,3 3, Worder,3, 3, border,3) with parameters horder,3, Worder, border,3 E [[][[|] where border,3 is based on Porder.3 with l(t. < (T.)..).
Relation PeriioonaPPinnreeeeee describes the distribution of the time gap between two relations and r', and is considered in Fe.., Fe.s and Swe(ts,ne)- We suppose this probabil- ity follows Gaussian or exponential distribution with parameters 1pairr, ppair, Apair, Mpair,2 pair; 2 Hpair,3, pairr,, Apair;3 E R ( in Fe,s. Fa,s and (we(e.,e)) respectively. Given R(ea) R(1), we calculate Φpair, (eej Apair, 1, Wpair, 1pair.1)
[
12

---

Published as a conference paper at ICLR 2023
with (17), where Apair,11 E ||R|I|R| is based (Mpasr,,1 pair.1) Apair, with T = 16, (T and ippair,: Upair,1 E [[R[I|R| are learnable parameters. Simi- larly, given R(Ce) = R(2), we calculate @pair;2(ee; Apair;2 upair,2 (pair:2) with param- eters 11pair,2, 2, Upain. E [R[R/ where hpain.2 is based on (Mpair poirr,2 2) or Apain,2 with I = 11, (Ta) Given R(ee) = R((3), we calculate Opair,3(ee; Apair, Wpair,3, 0pair.3) with parameters Appair,3. Wpair,3, 0pain,3 E [R|R|x|RI, where Apain, is based on (/poor,3, Jf pair,3) ) or Xpaór,3 with x = -TT)(T,)) (Ta)
Duration describes the probability distribution of interval length of every relation. We suppose this probability follows a truncated Gaussian distribution with parameters Mdd dd IR R within the interval [0, +20). It is common in large tKGs that the exact date of some events are missing. With this feature, we can estimate these missing dates and improve our model performance. For example, given a fact or query with incomplete interval = [ts, ?]. we generate a duration to ~ ((ua)rr (aa)r, O, +oo) where r is the corresponding relation, and set f = [L,, l + Ld].
With these temporal feature distributions, we can further evaluate the candidates given a query. We fisrt combine the scores in each part with (19)-(21). Then the scores from different parts combined to obtain the temporal feature modeling score Φefm given in (14).
Oefm, (Ee) = Tree, (ee) + Torder, (eo) + Ypair, 10pir, pairi,111 (eo) ((e)) ree. ((ee) + Torder. (ee) + 2ppttr (eo)
(19)
(20)
@1fm,3 Torder, 3orrdr, + 7pair, pair,,
(21)
where all 7 2 0 are learnable weights. For each part, the sum of weights is equal to 1, i.e., Tree,1 + Torder.1 ppain,1 = 1, 7ree.2 + Yorden2 + Ypparr = 1, and Torder.3 + ppair,3 1.
B DATASET DISCUSSION
There are four benchmark temporal knowledge graph datasets including ICEWS(Lautnnschageer al.] (2015)) GDELTeLeetaru & Schrodi (2013)). WIKIDATAA(eblly & Chekol (2088) and YAGO (Mahdisoltani et al. (2014)). As for the temporal representation, the first two datasets use timestamps, while the last two uses intervals. Compared with timestamp-based methods, interval- based tKGs are both more general and more difficult to learn. Thus, we mainly focus on WIKIDATA and YAGO datasets in our experiments.
WIKIDATA is a large knowledge base based on Wikipedia. To form the WIKIDATA 12k dataset, a subgraph with temporal information is extracted by Dasgupta al. (2018). It is guaranteed that ev- cry single node are related to multiple facts, and the top 24 frequent relations are selected. YAGO is another large knowledge graph built from multilingual Wikipedias. Similarly, some temporally as- sociated facts are distilled out from YAGO3 to form the YAGO1 1k dataset (Dasgupta et al. (2018). In this dataset, every single node is connected by more than one edge, and the top 10 frequent rela- tions are selected. For the WIKIDATA 12k and YAGOI 1k datasets, they contain many time-sensitive relations such as 'residence', 'position held', 'member of sports team', 'member of", 'educated at' in WIKIDATA 12k and 'worksAt', 'playsFor', 'isAffiliatedTo', "has WonPriz''' 'owns' in YAGOI 1k. These sensitive relations make the link prediction task in these two datasets more challenging. For example, in the WIKIDATAI24 datatset, person become member of different teams, hold different positions, and receive different awards in various periods. Thus, is necessary to model temporal information for link prediction tasks in tKGs. Table shows the dataset statistics with yearly time resolution, where G, E, R, T, I are the set of edges, entities, relations, timestamps and intervals, respectively.
C EFFICIENCY STUDY
For the rule learning process, the time complexity of TLR module (path search, rule extraction, and the arriving rate calculation) is ON. Nrate (L|g) + L N2Nptt)) where is the maximum rule length, N Npos is the number of positive examples, Nrute is the maximum number of rules found in
13

---

Published as a conference paper at ICLR 2023
Table 3: Dataset statistics with a yearly time resolution
Dataset
Gtrainl
[gootal
19esrl
[8]
[R]
[T]
[
WIKIDATAI2K
32,497
4.062
4,062
12,544
24
237
2,564
YAGO1Ik
16,408
2,051
2,050
10,622
10
251
6,651
single example, and Nath is the maximum number of paths for given rule that be found in a single example. The time complexity of tfm module (distribution parameters measurement) is po + [R| L.Nrule-Vpea )). where ò is the maximum node degree. Similarly, for the rule application process, the time complexity of TLR module is O(Ngry-Nyyut (Lgg N pacn)) where Ngry is the number of queries, Nuu is the maximum number of rules for given target predicate, and path is the maximum number of paths for given rule that can be found in single query. The time complexity of tfm module is O (K K + LN (aath)) )) where K is the maximum number of candidates for a single query. Since the path search for each positive example and the rule application for each query maximum rule length L = 5, the rule searching of TLR module on a 4-CPU machine takes 1740.6s on WIKIDATA 12k training set, and 571.9% 1.9s on YAGO1 training set. The distribution parameters measurement of tfm module takes 44.1s on WIKIDATAI 12 training set, and 6.7s on YAGOI Ik training set. The rule application of TLR module takes 1522.8s on WIKIDATA12K validation set, and 529.6s YAGO1 Ik validation set. The scoring of tfm module takes 1713.8s on VIKIDATA12K validation set, and 527.8s 8s on YAGO1 1k validation set.
both independent, these processes
done in parallel. Given the
D DETAILS OF THE MORE DIFFICULT PROBLEM SETTINGS
Few training samples. In experiments, we randomly reduce the number of training samples in training set and evaluate different models for the two datasets. To alleviate the effects of different data distributions, we repeat this experiments for 5 rounds. The results are shown in Fig.2, where we draw the average MRR curve with err bars. When the training set size decreases, TILP outperforms all the baseline methods. Through constrained random walks, TILP is able capture all the patterns related to a query relation which are independent on entities. Reducing training set size only changes the frequency of different patterns. To contrast, embedding- based methods require enough training samples to learn good embeddings of entities and relations. In this setting, our method is also better than TLogic, which demonstrates the advantages of the neural-network-haaee logical rule learning framework relative to the statistical methods.
Dotasct W WIDDAT11
Dusaset: YA8011K
0.25
Tere"eedaa
Treeftex
Tlogle
0.55
322
3.5
07
0.1
0.8
0.4
0.5
0.6
6.7
0.
Triiiing sire
Tratning net sirw
Figure 2: Link prediction performance with few training samples
Biased data. For the two datasets, we first obtain statistic results on the number of edges for each relation. Then for specific relation, randomly reduce 50% of the edges in the training set, and evaluate the performance changes caused by this setting. We conduct the experiments for every relation separately, and repeat for rounds to alleviate the influence of different data distributions.
14

---

Published as a conference paper at ICLR 2023
The original test set from random generation contains few queries of rare relations. To guarantee enough test queries for each relation, we adjust the distribution of queries in the test set, trying to set the number of queries of different relations to be equal. If queries of a certain relation not enough, we would randomly choose half of them. In Fig.3, the average change of MRR for different models shown, where error bars have been suppressed for readability. We conclude that the attention vectors of predicates, temporal relations and rule length are relation-dependent in TILP, making it less susceptible than other methods data imbalance. In contrast, embeddings of entities are shared among all the relations, making embedding based method suffer more susceptible to data imbalance. As a statistical method, TLogic also fails since can build dependency between different rules.
Cotosot W WIDATA1
Oatasec YAGO11K
100
93
TinePiee 2003
Treaflex base
00
Change 163)
80
UR11 Change
70
12
16
15
10
Relattoo id
Relation
Figure 3: Link prediction performance with biased data
Time shifting. In practice, we put the edges missing start time into training set, and correct edges with wrong start time (greater than the year of 2022). For WIKIDATA dataset, the start time range for training set, validation set and test set are [0, 2008), [2008, 2012), [2012, 2018), re- spectively. For YAGOLIK Ik dataset, the start time range for training set, validation set and test set are [-431, 2006], [2006, 2011], [2011, 2022]. respectively. The results are shown in Table 2. One major limitation of most time-aware embedding- -based methods is the use of absolute timestamp anchors, preventing generalization to either time shifting settings and inductive settings (Liu et al. (202T)) With such limitations in mind, TILP extracts temporal logical rules with relative temporal relations, providing greater flexibility, e.g. for transfer learning to arbitrary temporal periods, In this setting, our method still better than TLogic which builds their model on timestamps, and ignores the necessity of learning all possible temporal patterns from data.
15


Reviews:
[{"review_id": "HvbDsVNcAYC", "reviewer": "Reviewer_EsnR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThe paper proposes a novel method to model temporal logical rules for knowledge graph completion. In specific, the authors first define constrained random walk for rule learning. Then, based on learned rules, the authors design a module to model various temporal features and apply rules for prediction.\n================================================================\nI have read the response and that solved most of my concerns. I would like to raise my score to 6.", "summary_of_the_review": "see above", "strength_and_weaknesses": "Strengths:\n1. The motivation is reasonable.\n2. The paper proposes a novel framework to model various temporal logical rules.\n3. The experiments demonstrates the effectiveness of the proposed model.\n\nWeaknesses:\n\nThe presentation needs further improvements and the paper is difficult to follow. For example:\n\n 1) what is the difference between r and P? \n 2) The notations in Section 4 are not very clear, why sometimes using h and t for head and tail entities, and sometimes using x and y? \n 3) What does the operator M mean? An example may be helpful.\n 4) The task formulation is actually similar with conventional link prediction without time interval. And, in table 1, why the ComplEx performs much better than TA-ComplEx, the temporal version of it? I would suggest an explanation of the dataset, so that we can better understand the necessity of modeling temporal information for link prediction.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "AGCudyn2WsZ", "reviewer": "Reviewer_qZrk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a differentiable temporal logical rule method, TILP for temporal knowledge graph. The proposal can achieve similar performance as state-of-the-art baselines, while can find helpful logic patterns under some restricted scenario which no previous work can find.", "summary_of_the_review": "This paper proposes a differentiable temporal logical rule method for temporal knowledge graph which has no previous works. However, the empirical gain is limited compare with existing baselines. The potential of restricted learning is not persuasive for me since those restricted scenario is only on two datasets, and no related works are provided to show community interest in those scenario.", "strength_and_weaknesses": "## Strength\n\n- This method proposes a diiferentiable logic rule method on knowledge graph which can deal with time range related rules which has no previous works.\n\n## Weakness\n\nA lot of notations and explanations need to be refined for clarity.\n\n-  In section 4, why call the first kind Markovian? I think Markovian refers to the case that current state is only determined by previous state. However, in your case, current random walk step is not determined by previous random walk step, instead, it is determined by a given relation type and given starting time which never changes in a random walk.\n- Isn't Eq (6) wrong? In the left side, v is subscripted by length, while in the right side, v is subscripted by node identity.\n- You should specify the dimension of each variable and function for Eq (8-13). It is very hard for me to understand the process there.\n- What is $h_{0, l}$ in Eq (8)? Simply 0 as common RNN?\n- Why this random walk embedding  Eq (8-12) only cares the context relation $r_h$, and ignores relations in the random walk?\n- Can you provide the exact computation for all four temporal feature modeling? The definition is too vague for me to confidently understand.\n- I do not see the usage of recurrence and duration in Eq (14-17).\n- In section 5, it says that you will use all possible pathes between two nodes for both training and inference. So there is no \"random\" walk. Isn't this very costy?\n\nEmpirical result is not satisfying.\n\n- On only two datasets, TILP does not show clearly better performance than TimePlex. Given the fact that this method is not incremental (does not need to store full history of temporal graph), this performance is not impressive.\n- Although this paper shows TILP has the potential in restricted scenario, I am not sure if those hypothesis is interesting in tKG field or has no previous works, since I am unfamiliar with this field.\n- It is unclear why TLogic works extremely worse since time range logics can be expressed by timestamp logics, thus I will expect similar performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5bN44siI7s", "reviewer": "Reviewer_HUCP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for learning temporal logical rules. Temporal logical rules are naturally discrete, this paper utilizes a differentiable framework for learning such rules. Specifically, it proposes a random walk mechanism and designs several operators for the temporal relations. It conducts experiments on several real-world datasets to demonstrate its effectiveness.  ", "summary_of_the_review": "The problem of TRL is crucial, but the core advantage of this method is unclear; furthermore, I have some concerns about reproducibility due to the lack of codes. ", "strength_and_weaknesses": "### Strength\n1. The problem of learning temporal logical rules is important. \n2. It improves the random walk in existing works from based on timestamps into based on intervals, which more flexible. \n\n### Weakness\n1. The authors mention that StreamLearner and TLogic are both statistic learning methods, but what are the drawbacks of such methods? What type is the authors' method and what are the advantage?\n2. The writing needs improvement, for example, it should emphasize in the introduction more the reason why this model is better than other TRL methods such as [1]. \n3. The experiments are not adequate, lacking efficiency study. \n\n[1] Liu Y, Ma Y, Hildebrandt M, et al. Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(4): 4120-4127.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "-oelAPceJkW", "reviewer": "Reviewer_uk95", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper provides a differentiable method for learning temporal knowledge graphs in KGs. While differentiable rule-learning in the context of KG’s already exists, and rule-learning in the context of temporal KG’s already exists, this paper is the first to provide a differentiable rule learning method for temporal KGs.", "summary_of_the_review": "This is a well executed paper that straightforwardly follows up an existing method (TimePlex) with well motivated additions. The results are mixed but section 6.3 does a good job exploring the method in various settings. This paper would be a good addition to the literature.\n", "strength_and_weaknesses": "Thorough set of well-explained temporal features.\n\nThe addition of each feature is well-motivated, though the final set of features is quite complex and might benefit from more concrete running examples through the exposition.\n\nThorough set of experiments, though rather mixed results. The examples of learned rules are appreciated. Ablation with the “hard settings” in section 6.3 is also appreciated.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster