Confidence is not Timeless: Modeling Temporal Validity for Rule-based Temporal Knowledge Graph Forecasting (2024)

Abstract: Recently, Temporal Knowledge Graph Forecasting (TKGF) has emerged as a pivotal domain for forecasting future events. Unlike black-box neural network methods, rule-based approaches are lauded for their efficiency and interpretability. For this line of work, it is crucial to correctly estimate the predictive effectiveness of the rules, i.e., the confidence. However, the existing literature lacks in-depth investigation into how confidence evolves with time. Moreover, inaccurate and heuristic confidence estimation limits the performance of rule-based methods. To alleviate such issues, we propose a framework named TempValid to explicitly model the temporal validity of rules for TKGF. Specifically, we design a time function to model the interaction between temporal information with confidence. TempValid conceptualizes confidence and other coefficients as learnable parameters to avoid inaccurate estimation and combinatorial explosion. Furthermore, we introduce a rule-adversarial negative sampling and a time-aware negative sampling strategies to facilitate TempValid learning. Extensive experiments show that TempValid significantly outperforms previous state-of-theart (SOTA) rule-based methods on six TKGF datasets. Moreover, it exhibits substantial advancements in cross-domain and resourceconstrained rule learning scenarios.

heading: 1 Introduction
text: Representing, acquiring, and applying knowledge have always been hot topics in the field of artificial intelligence research (Cai et al., 2020; Ji et al., 2021; Mialon et al., 2023). Knowledge reasoning (Huang et al., 2023, 2024) is an invaluable pathway for humanity to understand and acquire new
∗ Corresponding author.
knowledge. Previous knowledge reasoning focused on static reasoning, however, temporal reasoning holds more profound implications for daily life. For example, if dark clouds are observed, one could take an umbrella to avoid catching a cold caused by the probable heavy rain. Temporal Knowledge Graph Forecasting (TKGF) is a natural scenario for predicting future events based on historical data, thus drawing substantial interest from researchers in recent years (Trivedi et al., 2017; Jin et al., 2020; Wang et al., 2023).
Compared to the neural methods obsessed with non-transparent latent vector representations, rulebased approaches are capable of discerning and implementing subtle rules for efficient and interpretable reasoning (Galárraga et al., 2013; Meilicke et al., 2019; Liu et al., 2022). However, previous efforts focused on estimating the static pre-
10783
dictive capability of rules (Galárraga et al., 2015; Pellissier Tanon et al., 2017; Ortona et al., 2018), i.e., confidence, overlooking their interaction with temporal information, which may result in temporally insensitive outcomes. As shown in Figure 1a, there are two temporal rules with different confidence, if we employ static or uniform temporal patterns in Figure 1b, then it will yield the outcome Macron at all future points in time. To predict the example in Figure 1a, it is necessary to model rule-independent temporal characteristics shown in Figure 1c to get richer temporal patterns.
In addition to the interaction of temporal information with rules, the interaction between rules is also important for reasoning. Existing methods perform confidence estimation and aggregating reasoning based on rule-independence assumptions (Meilicke et al., 2019; Liu et al., 2022; Li et al., 2023), which may lead to an overestimation of incorrect outcomes (Ott et al., 2021). However, delving into the interactions between rules may encounter potential combinatorial explosion of confidence (Betz et al., 2023). While parameterized confidence may alleviate this challenge (Ott et al., 2023), it remains an unsolved problem to model both the interaction of temporal information with rules and the interaction between temporal rules.
To address the aforementioned challenges, we propose a framework, named TempValid, which models the temporal validity of rules for TKGF. Specifically, TempValid assumes that confidence decays over time rather than being timeless. We design a time function with a learnable decay coefficient to regulate the decay rate impacting rule confidence. The confidence and decay coefficients are conceptualized as learnable parameters to avoid inaccurate heuristic estimation and potential combinatorial explosion. To effectively optimizing TempValid, we design a rule-adversarial negative sampling and a time-aware negative sampling strategies. Extensive experiments show that TempValid significantly outperforms existing rule-based TKGF methods and achieves competitive performance with the state-of-the-art baselines on six TKGF datasets. Moreover, TempValid significantly outperforms existing approaches in cross-dataset and low-resource scenarios.
In summary, our contributions are as follows: (1) We propose a framework named TempValid for modeling the temporal validity of temporal rules. To our best knowledge, this is the first study of investigating the temporal validity of tem-
poral rules in the context of temporal knowledge graphs. (2) We design a time function with controlled decay rate, conceptualize confidence and decay coefficients as learnable parameters, and propose two elaborate negative sampling strategies to facilitate TempValid training. (3) Our proposed TempValid significantly surpasses the existing rule-based TKGF methods and achieves competitive performance with the state-of-the-art baselines on six representative TKGF datasets, including ICEWS14, ICEWS18, ICEWS05-15, YAGO, WIKI and GDELT.

heading: 2 Related Work

heading: 2.1 Temporal Knowledge Graph Forecasting
text: Temporal knowledge graph reasoning can be classified into two settings: interpolation and extrapolation settings. Interpolation setting aims to predict missing historical facts within a known time frame (Leblay and Chekol, 2018; Garcia-Duran et al., 2018; Goel et al., 2020). However, these methods cannot predict future facts effectively.
To solve the deficiency of interpolation setting, extrapolation setting is proposed, i.e, TKGF. RENET (Jin et al., 2020) and REGCN (Li et al., 2021b) treat TKGs as sequences of subgraphs, employing GNNs to model graph structures and RNNs to capture sequential patterns, respectively. Inspired by the repetition of historical events (Trompf, 1979; Schlesinger, 1999), CyGNet (Zhu et al., 2021) and CENET (Xu et al., 2023) utilize copy mechanism to predict future facts. TiRGN (Li et al., 2022) considers the sequential, repetitive and cyclical patterns of historical facts simultaneously, achieving a remarkable performance.
However, due to the nature of black-box models, the above methods lack transparency and interpretability. To address this issue, xERTE (Han et al., 2020) generates interpretable reasoning paths through subgraph expansion and pruning. CluSTeR (Li et al., 2021a) and TITer (Sun et al., 2021) use reinforcement learning strategies to travel relational paths for reasoning. TECHS (Lin et al., 2023) attempt to encode the rules mined by TLogic and perform predictions using GNNs. The most relevant to our work is TLogic (Liu et al., 2022), which is a extension of AnyBURL (Meilicke et al., 2019), a rule-based reasoning method for static knowledge graphs. It designs the temporal random walk to mine temporal rules and aggregates temporal rules for reasoning. Compared to TLogic,
our proposed TempValid focuses on modeling the temporal validity of rules.

heading: 2.2 Estimation of Confidence
text: Although various metrics (Galárraga et al., 2013; Zhang et al., 2021) and assumptions (Galárraga et al., 2015; Pellissier Tanon et al., 2017; Ortona et al., 2018) have been proposed to model the quality of rules, confidence is considered the appropriate for knowledge graph reasoning. However, the above methods assume rule independence both in confidence estimation and rule aggregation reasoning. Such the strong assumption may lead to redundant reasoning, further overestimating incorrect answers (Ott et al., 2021; Betz et al., 2023).
Parameterizing confidence bridges the gap between confidence estimation and aggregated reasoning (Ott et al., 2023), avoiding the risks associated with strong assumptions and the potential explosion of parameter combinations. Nonetheless, these approaches ignore the specificity and complexity of confidence estimation in temporal scenarios. Currently, TR-Rules (Li et al., 2023) investigated the misestimation of the confidence due to temporal redundancy. Some works of differentiable rule learning also set confidence as a learnable parameter, however, they mostly focus on interpolation reasoning or interval reasoning and (Xiong et al., 2022; Singh et al., 2023).

heading: 3 PROBLEM STATEMENT

heading: 3.1 Temporal Rules
text: A TKG can be denoted as G = {(es, r, eo, t)}, where es, eo ∈ E are subject and object entity (Gu et al., 2022; Qu et al., 2023), r ∈ R represents the semantic relations and t ∈ T is timestamp. Based on timestamps, a TKG can be divided into sequential subgraphs {G1,G2, ...,Gt}. Given a query (eq, rq, ?, tq), the objective of the temporal knowledge graph forecasting task is to predict the missing entity based history fact {Gt|t < tq}.
Rule-based TKGF aims to predict future events by learning and applying temporal rules. In this paper, we focus on modeling the confidence and temporal validity of rules rather than the methodology for mining them, so we use TLogic’s rule learning approach to mine cyclic temporal logical rules (Liu et al., 2022), which is defined as follows:
(E1, rh,El+1, Tl+1)← ∧li=1(Ei, ri, Ei+1, Ti), with T1 ≤ T2 ≤ ... ≤ Tl < Tl+1 (1)
where the left of the arrow is termed the rule head H , while the right is the rule body B. Ei and Ti are replaceable variables that represent entities and timestamps, respectively. rh is the head relation and ri is a body relation. tb(tr) = T1 denotes the earliest time of the grounded rule instance.

heading: 3.2 Rule-Based TKGF
text: Let’s index quadruples and use yi ∈ {0, 1} denotes the truth value of quadruple qi = (es, r, eo, t). For a target relation r, given the learned temporal rule set TR, where the size of the rule set is K = |TR|, c(trj) is the confidence of the jth temporal rule trj ∈ TR. A feature vector ∆ti ∈ NK+ for every example ∆tij = t − tb(trj) denotes the time interval between the rule body and fact. Specially, ∆tij = +∞ if none of the relational path instance could be grounded by trj .
Usually, a heuristic confidence denotes the likelihood that the temporal rule trj predicted quadruple qi, i.e. p(yi|c(trj)). We are motivated by the belief that this predictive effectiveness should be subject to time rather than timelessness. Therefore, we need to design a time function tv(∆t) interacting with the confidence, i.e. the likelihood should be the form of p(yi|Φ(c(trj), tv(∆tij))), abbreviated as Φ(trj ,∆t), where Φ(·) is a function that couples the confidence with time function.
For prediction, a typical approach is aggregating rules with Noisy-OR (Meilicke et al., 2019; Liu et al., 2022) to score qi:
s(qi) = 1− K∏
j=1
(1− Φ(trj ,∆tij)) (2)

heading: 4 METHODOLOGY

heading: 4.1 Temporal Validity of Temporal Rules
text: Our core motivation is to model the temporal validity of temporal rules. Our approach is underpinned by two key intuitions: 1) The confidence of a temporal rule would diminish as the time interval increases; 2) The temporal sensitivity of the rules is different. Some rules experience an abrupt decline in predictive power, while others maintain efficacy over extended timeframes.
To model our intuition, there are two key points that must be satisfied: 1) A monotonically nonincreasing time function tv(∆t) ∈ [0, 1] needs to be designed for temporal decay on confidence. 2) The time function should contain a parameter indicating the decay rate of the temporal rule. To
satisfy the above two requirements, we design an exponential form of time decay function:
tv(∆tij) = e −βj∆tij (3)
where βj is the decay coefficient of trj that controls the decay rate of confidence c(trj). A large β indicates that the confidence is decaying fast, i.e., the temporal rule is sensitive to temporal information and vice versa. While β = 0, the confidence of the rule is considered efficient forever.
Subsequently, we applied the time function to the confidence so that it could decay over time. The coupling function that integrates confidence with time function is formulated as follows:
Φ(trj ,∆t) = c(trj) ∗ tv(∆tij) (4)
In this paper, we only discuss the effect of temporal decay of confidence on the TKGF. More powerful temporal functions and coupling methods may be potential research interests in the future.

heading: 4.2 Learning Decay Coefficients and Confidence of Rules for TKGF
text: To accurately estimate the confidence and temporal validity of temporal rules, we use a machine learning model to learn the confidence and decay coefficients. Straightforwardly, we can compute the scores of the quadruples using the Noisy-OR function in Equation 2:
s(qi) = 1− K∏
j=1
(1− c(trj) ∗ e−βj∆tij ) (5)
Noisy-OR is an aggregation function based on the assumption that the effectiveness of rules is independent, which implies the probability that at least one rule works. The cumulative product form of the score function may be difficult to be optimised due to potential gradient vanishing. Therefore, inspired by Relational Logistic Regression (Kazemi et al., 2014; Ott et al., 2023), we transform the Noisy-OR into a linear model through the g(z) = log(1−z), the two ends of the equation are positively correlated while z ∈ [0, 1). Rewriting Φ̃(trj ,∆tij) = log(1− Φ(trj ,∆tij)) and we will obtain a new scoring function:
s(qi) = K∑
j=1
c(trj) ∗ e−βj∆tij (6)
From a feature engineering perspective, for a query (eq, rq, ea, tq), TempValid generates a Kdimensional feature vector, K denotes mined temporal rules for rq. The feature value is the time interval between the tq and the earliest timestamp of the grounded instance of the rule. In Eq. 6, if we regard the confidence and decay coefficients as feature weights for training, then essentially we are performing an exponential decay transformation on the temporal information of all the rules and then linearly aggregating them.

heading: 4.3 Negative Sampling and Optimization
text: Whichever score function is adopted, we need to make the score for the positive sample higher than the score for the negative sample. Conventionally, for a quadruple qi = (eq, rq, ea, tq), negative samples can be generated by simply replacing entity q′i = (eq, rq, e ′ c, tq) or timestamps q′i = (eq, rq, ea, t ′) (Sun et al., 2018; Leblay and Chekol, 2018). This approach, however, is not applicable to rule-based methods, as numerous entities simply have no rules to ground, leading to the generation of nonsense negative samples. Therefore, we propose two adapted negative sampling strategies based on the idea of replacing entities and replacing timestamps, respectively.
Rule-Adversarial Negative Sampling. For a query (eq, rq, ?, tq) and a set of temporal rules for rq, when rules are applied to a query, it is possible to reach other incorrect candidate entities besides the correct one. A high-quality rule that not only provides accurate predictions but also avoids incorrect predictions will be assigned a greater weight. As a result, we select negative samples from quadruples that can be covered by the premined rules for training purposes. In pursuit of high-quality negative samples, we give precedence to those instances with high scores through TLogic (Liu et al., 2022). Nonetheless, there are still a great number of queries that are only able to generate a few negative samples. To alleviate this problem, negative samples generated by replacing timestamps are used as a supplement.
Time-Aware Negative Sampling. For TKGF, a positive sample requires not only the correct entities and relation, but also the correct timestamp. Guided by this intuition, we derive time-aware negative samples by replacing the timestamp of a positive sample. Employing a sliding window mechanism, we can readily generate a substantial amount of time-aware negative samples. For ex-
ample, for a positive sample (eq, rq, ea, tq) and its feature vector ∆t introduced in Section 3.2, if we set a sliding window of 200 with an offset time to, then we can get 200 negative samples with feature vector ∆t− to. We set all non-positive elements in the generated feature vector to +∞ to indicate the disabling of the corresponding temporal rule.
Ultimately, in this paper, we adopt a loss function similar to the negative sampling loss (Mikolov et al., 2013; Sun et al., 2018):
L = −log(s(qi)) + 1
N
N∑
n=1
log(s(qni )) (7)
where qni denotes nth negative sample generated based on qi, N is the number of negative samples.

heading: 5 EXPERIMENTS

heading: 5.1 Experiment Setup
text: Datasets. We evaluate TempValid on the entity prediction task of TKGF using six TKG datasets: ICEWS14, ICEWS05-15 (Garcia-Duran et al., 2018), ICEWS18, GDELT (Jin et al., 2020), WIKI (Leblay and Chekol, 2018), and YAGO (Mahdisoltani et al., 2013). We follow the previous works (Li et al., 2021b; Gastinger et al., 2023) to divide the dataset into training set and testing set, as well as validation set, with 8:1:1 according to the order of timestamps. The statistics are shown in Table 1. Evaluation Metrics. We report two extensively adopted metrics for evaluating the performance of our model in temporal knowledge graph reasoning: Mean Reciprocal Rank (MRR) and Hits@k (H@k) in percentage (%). For each query, there is a rank for the true entity among all candidates. MRR is the average reciprocal values of the ranks, and Hits@k denotes the proportion where the ranks fall within the top k. A query may exist several correct answers, filtering out additional correct entities provides a more robust assessment of model performance. We adopt the widely-used time-aware
filtered setting in TKGF to report the results (Han et al., 2020; Gastinger et al., 2023). Baseline. We compare several renowned TKGF baselines: RE-NET (Jin et al., 2020), REGCN (Li et al., 2021b) and TiRGN (Li et al., 2022) which are evolutionary representation learning based methods; TANGO (Han et al., 2021), which uses neural ODEs; CyGNet (Zhu et al., 2021) featuring copy mechanisms; TITer (Sun et al., 2021) based on reinforce learning; xERTE (Han et al., 2020), known for its interpretability; the purely rule-based reasoning approach, TLogic (Liu et al., 2022) and TECHS (Lin et al., 2023) which performs reasoning with encoding temporal rules. Hyperparameter Setting. Learning rate lr, batch size b and number of negative samples N are obtained by grid search based on the MRR of the validation set. Ultimately, 0.01 lr and 128 b are applied to all datasets, except ICEWS05-15 which uses 1024 batch size. For N , 50, 50, 50, 90, 70, and 60 are set for YAGO, WIKI, GDELT ICEWS14, ICEWS18 and ICEWS05-15, separately.
More details of the dataset and implementation are in the Appendix A and B.

heading: 5.2 Main Results
text: The results of the TKGF task are shown in Table 2 and 3. Notably, our approach outperforms TLogic, the method most closely related to ours, across all datasets. This suggests that the confidence and temporal validity modeled by our method surpass those derived from statistical or manually designed methods, as they can better capture complex semantic and temporal patterns in the data. Constrained by the size of the rule set and computational costs, both TLogic and TempValid focus exclusively on cyclic temporal rules which limits their ability to predict relations that require acyclic temporal rules for description. This is why TempValid and TLogic are worse than the representation-based models on YAGO, with some relations not being able to be modeled using only cyclic rules (the relevant data
makes up more than 10% of the test set). For non-rule-based methods, despite not modeling entity representations, TempValid demonstrates superiority over methods, such as RE-NET and REGCN, which are centered around entity evolving representations. Even though TiRGN draws on various strengths from different approaches, TempValid still achieves competitive results against it, which suggests that rule-based methods still hold promising competitive prospects.

heading: 5.3 Heuristic vs. Learned Confidence and Decay Coefficients
text: Previous efforts (Ott et al., 2023) have shown that confidence learned by a canonical model outperforms statistically derived confidence in static knowledge graph reasoning tasks. A question to be explored is whether TempValid’s efficacy is primarily due to its learned confidence, temporal validity, or a synergy of both aspects. To answer the question, we design three variant models of TempValid on ICEWS14, ICEWS18 and ICEWS05-15: (1) learning confidence without temporal information (LCWT) which makes ∆t ∈ {0, 1}K which
denotes whether the corresponding rule is applicable, (2) learning confidence with a uniform, fixed temporal validity (LCFT), and (3) employing the statistically derived confidence from TLogic while only learning temporal validity (LTV). We include TLogic which using heuristic confidence and uniform decay coefficients joins as well.
In Table 4, it can be observed that LCWT performs worse than TLogic, indicating the significance of temporal information in TKGF tasks. While learning confidence or temporal validity individually leads to performance surpassing that of TLogic, TempValid achieves a superior outcome by jointly optimizing both aspects. This suggests that our proposed TempValid find an effective way to couple the semantic confidence and temporal validity of temporal rules. While there may be room for
more sophisticated temporal validity modeling and coupling with confidence scores, we leave these explorations to future work.
In Figure 2, we further investigate the relative performance of LCFT against TempValid across different time decay coefficient β. It is evident that the model exhibits sensitivity to the β parameter. The intricate process of hyperparameter searching accentuates the necessity to learn rule-independent time decay coefficients.

heading: 5.4 Impact of Score Function and Negative Sampling Strategies
text: In this section, we discuss the contributions of optimization objectives and negative sampling strategies to the performance of TempValid.
Firstly, we experiment with replacing the scoring function with the Noisy-OR, which has been commonly used in previous works. Subsequently, we delve into the extent to which different negative sampling strategies contribute to the performance of TempValid. The strategy that optimizes only using positive samples in Equation 7 is abbreviated as OP. Removing the Rule-Adversarial Negative Sampling or Time-Aware Negative Sampling is abbreviated as w/o. TANS or w/o. RANS. The results are shown in Table 5.
It can be observed that the performance using the Noisy-OR model significantly lags behind that of the canonical model. There are two potential
reasons: 1) The Noisy-OR scoring function is computed under an assumption of rule independence, which may not be suitable when considering interactions among rules. 2) Under the assumption of rule independence and the Noisy-OR computation, even poor-quality rules can make a non-negligible positive contribution. Moreover, as we did not employ sophisticated techniques to filter rules such as clustering (Ott et al., 2021) or top-k rule selection (Betz et al., 2023; Liu et al., 2022), this might have allowed some detrimental rules to interfere with the prediction process (Ortona et al., 2018).
We observed that both negative sampling strategies can bring some gains, but the performance drops significantly after removing RANS. The reason is that the TANS essentially generates negative samples through a simple linear transformation of the feature vectors, which limits its performance gain. Nevertheless, due to its simple and costeffective, it still serves as a valuable alternative negative sampling approach.
In Figure 3, we illustrate the performance of TempValid under varying numbers of negative samples N . When N is zero, the model degenerates to the Only Positive variant as seen in Table 5. While N ≤ 50, the model’s performance improves as the number of negative samples increases apparently. It suggests that an appropriate amount of negative samples is beneficial in enabling the model to learn confidences and temporal decay coefficients effectively. As N > 50, the model’s performance fluctuates slightly within a range that is close to the best performance which indicates that TempValid is not burdened by the overhead typically associated with extensive negative sampling and is not sensitive to the choice of the number of negative samples.

heading: 5.5 Cross-dataset Generalization
text: Generally, representation learning-based methods struggle with cross-dataset generalization, training a model on a dataset and performing well on another dataset, due to many entities are not shared across different datasets. Rule-based methods, by contrast, inherently possess an advantage in this regard as they model relations rather than specific entities. In this paper, we follow the settings of (Liu et al., 2022) for conducting cross-dataset generalization experiments.
Specifically, we mine rules and train our model on ICEWS05-15, then perform predictions on ICEWS14. Similarly, we apply this procedure for training on ICEWS14 and predicting on ICEWS18. In Table 6, it can be seen that TempValid not only inherits TLogic’s ability to reason across datasets, but it also improves upon it. It shows that TempValid’s estimation of the confidence and decay rate of the rules is consistent across datasets.

heading: 5.6 Complexity Analysis
text: The time complexities of rule mining and feature vector generation in the TempValid framework, respectively, correspond to the training and inference time complexities within TLogic. For rule mining, the worst-case time complexity is O(|R|nlDb), where l is the length of the rule, n is the number of walk, D is the maximum node degree, and b is the number of body samples for pre-estimating the confidence. For feature vector generation, the worst-case time complexity is O(|G| + |TR|DL|E|log(k)), where L is the maximum rule length in temporal rule set |TR| and k is the minimum number of candidates. Regarding model parameters, TempValid is a lightweight model that requires learning only two parameters per rule: the confidence and the decay coefficient. The proposed TempValid model trains with fewer than 50,000 parameters for all datasets.
Generating feature vectors from the training set can often result in a significant overhead, especially
considering that the training set is typically larger than the test set. Fortunately, the number of parameters that TempValid needs to learn is relatively modest, making it possible to obtain near-optimal parameters even without extensive training data.
In Figure 4, we show the relationship between model performance and the size of training data on ICEWS14, ICEWS18 and ICEWS05-15, where |Train| denotes the maximum number of train data for a target relation. Evidently, TempValid can achieve baseline performance with as few as 16 data instances to learn the semantic confidences and decay coefficients. With just 32 data, TempValid attains results comparable to TLogic, which utilizes 500 samples for the estimation of confidence under standard settings. With just 32 data instances, TempValid attains results comparable to TLogic, which utilizes 500 samples for confidence estimation under standard settings.
The number of negative samples should also be considered as part of the training data. To this end,
we show TempValid’s performance on ICEWS14 as a function of training data size and number of negative samples in Figure 5. With the same amount of data, increasing the number of negative samples strengthen TempValid’s capacity to learn semantic confidences and decay coefficients effectively. Referring to Figure 3, when the N > 50, the model can achieve near-optimal results. It implies that for training TempValid, we only need to generate feature vectors from a small subset of historical quadruples rather than generating them for the entire vast collection of training quadruples.

heading: 5.7 Case Study
text: To facilitate the understanding of TempValid’s modeling mechanism, we provide a case study with three example rules in Table 7. It is observed that both TLogic and TempValid assigns a high confidence to the Rule 1, and TempValid learns a small decay coefficient. It indicates that Rule 1 is not sensitive to temporal information but are more focused on the semantic information. These rules are mainly composed of 3 step length rules. These types of rules typically have high confidence but low exposure (i.e., support (Galárraga et al., 2013)) and are usually composed of three-step length rules. For example, although Rule 1 showed high confidence, it only occurred 23 times in 500 samples.
Upon further observation of Rule 2 and Rule 3, TempValid assigned confidences that diverged from those generated by TLogic. For Rule 2, TempValid allocated a higher confidence and a decay coefficient, whereas for Rule 3, TempValid assigned a lower confidence and decay coefficient. This aligns with human cognition: If a person expresses an intention to visit someone, they are likely to meet within a short period. However, this probability decays very quickly because if they do not meet for a long time, the plan might be forgotten or canceled. As a comparison, cooperation is a long-term intention. Although the probability of it happening at any given moment is smaller than that of a meeting, in the long run, once this intention is formed, it will consistently drive the visit over a period of time.

heading: 6 Conclusion
text: In this paper, we propose the TempValid to model the temporal validity of temporal rules for TKGF. We believe that the predictive effectiveness (confidence) of the rules would decay over time. TempValid use a machine learning model to learn confidence and decay coefficients for TKGF. In addition, we design a rule-adversarial and a time-aware negative sampling strategies to train TempValid more efficiently, and obtain competitive results with the baselines on six classical benchmarks. We conclude that learning confidence and decay coefficients as well as bridging the gap between rule quality estimation and aggregate reasoning are necessary for rule-based TKGF. In addition, TempValid significantly outperforms existing TKGF method in the across datasets and low-resource scenarios.

heading: 7 Limitations
text: Our proposed has the following limitations: First, there are many patterns of rules in the time dimension, e.g., periodicity, randomness, etc., and our proposed TempValid models only temporal validity, i.e., the decay of the predictive effectiveness of rules. Second, more expressive time functions may exist. Without loss of generality, TempValid uses the common exponential form of the decay function inspired by (Yèche et al., 2023). Last but not least, the generation of feature vectors, i.e., rule grounding, is still time-consuming although it is a one-time process.

heading: Acknowledgements
text: This work was supported in part by the National Natural Science Foundation of China under Grant No .62276110, No .62172039 and in part by the fund of Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL). The authors would also like to thank the anonymous reviewers and meta review for their comments on improving the quality of this paper.

heading: A Details of the dataset
text: For datasets, on the one hand, Considering the existence of multiple variant datasets with identical names, we adhere to the dataset selection and processing strategies outlined by (Gastinger et al., 2023). On the other hand, TempValid needs to ground the rules from the raw training data to generate feature vectors. Subject to the temporal constraints of the temporal rules, the earlier the timestamp, the fewer rules the quadruple can be grounded, the sparser the generated feature vectors will be.
B Implementation Details
B.1 Hyperparameters search The search range for hyperparameters is as follows: learning rate lr ∈ {0.1, 0.01, 0.001, 0.0001}, batch size b ∈ {64, 128, 256, 512, 1024} and number of negative samples N ∈ {10, 20, 30, 40, 50, 60, 70, 80, 90, 100}. The maximum epochs for training is set as 3000. The optimal model parameters are selected based on the MRR of validation set and implement early stopping to prevent overfitting.
B.2 Feature vectors generation For a query (es, r, ?, t), we endeavor to ground the learned temporal rules TR. If there is a candidate could be connected by temporal rule trj , then ∆tj will be assigned as the time interval between the rule and t, otherwise ∆tj = +∞.
For rule-adversarial negative samples, we generate feature vectors for the candidate entities reachable by TLogic for training. For ICEWS14, YAGO and WIKI, we generate the feature vectors of top 100 candidate entities. For ICEWS18, ICEWS0515 and GDELT, we generate the feature vectors of top 50 candidate entities. If the number of negative samples is still insufficient, we supplement it with the time-aware negative sampling strategy.

heading: Future Work
text: While there may be room for
more sophisticated temporal validity modeling and coupling with confidence scores, we leave these explorations to future work. In Figure 2, we further investigate the relative performance of LCFT against TempValid across different time decay coefficient β. It is evident that the model exhibits sensitivity to the β parameter. The intricate process of hyperparameter searching accentuates the necessity to learn rule-independent time decay coefficients