TR -Rules: Rule-based Model for Link Forecasting on Temporal Knowledge Graph Considering Temporal Redundancy (2023)

Abstract: Temporal knowledge graph (TKG) has been proved to be an effective way for modeling dynamic facts in real world. Many efforts have been devoted into predicting future events i.e. extrapolation, on TKGs. Recently, rule-based knowledge graph completion methods which are considered to be more interpretable than embedding-based methods, have been transferred to temporal knowledge graph extrapolation. However, rule-based models suffer from temporal redundancy when leveraged under dynamic settings, which results in inaccurate rule confidence calculation. In this paper, we define the problem of temporal redundancy and propose TR-Rules which solves the temporal redundancy issues through a simple but effective strategy. Besides, to capture more information lurking in TKGs, apart from cyclic rules, TR-Rules also mines and properly leverages acyclic rules, which has not been explored by existing models. Experimental results on three benchmarks show that TR-Rules achieves state-of-the-art performance. Ablation study shows the impact of temporal redundancy and demonstrates the performance of acyclic rules is much more promising due to its higher sensitivity to the number of sampled walks during learning stage.

## introduction knowledge graphs (kgs) store various real-world facts in the form of (s,r,o) in which s and o represent subject and object real-world entities while r denotes a relation between s and o. kgs are critical in many downstream applications such as question answering @xcite , recommender systems @xcite and information retrieval @xcite . traditional knowledge graph can be viewed as a static knowledge base, however, plausibility of most real-world facts are dependent on time, i.e., facts in kgs may not stand perpetually. thus, tkgs are introduced to resolve this limitation, where each fact is represented by a quadruple (s,r,o,t) with timestamp t incorporated. link forecasting or extrapolation on tkg which aims at predicting possible links at future timestamps, has been one of the main tasks being studied. recently, embedding-based models like re-net @xcite , cygnet @xcite , have been proposed and perform well for extrapolation on tkgs. however, compared with rule-based models, embedding-based models are believed lacking interpretability which leads to limitations on their downstream applications. tlogic @xcite is the first model which leverages rulebased methods on tkg link forecasting and gains competitive results as well as more explainable predicting process since it generates human readable rules. it mines cyclic rules by performing temporal random walk on tkgs first, and then extracting and generalizing walks into symbolic rules. nevertheless, existing models only mine cyclic rules in tkgs and ignore acyclic rules which, also contain important information for link prediction. besides, previous works do not notice that under temporal settings, (s,r,o) which has to be unique in static kgs, might appears repeatedly with different timestamps. and this temporal property can cause the model to obtain inappropriate confidence of rules through conventional confidence calculation algorithm, which further leads to incorrect predictions. we call this issue temporal redundancy. in this paper, we first define temporal redundancy and give an intuitive illustration on why temporal redundancy causes inappropriate confidence calculation. then we propose tr-rules, a rulebased tkg link forecasting model which resolves temporal redundancy through a simple but effective aggregation strategy. tr-rules aggregates those matched rule bodies before any new rule heads appear as one body during the rule support counting process. unlike previous works merely mine cyclic rules on tkg, by performing temporal random walk on tkg, tr-rules also mines acyclic rules which is crucial for link prediction but unexplored by existing models. experimental results show that, without considering temporal redundancy, simply using acyclic rules together with cyclic rules gives worse performance, which is the probable reason why acyclic rules have not been involved and studied by existing models. ablation study also demonstrates that acyclic rules are more sensitive to the number of random walks sampled for rule mining, which consequently results in a more promising performance. our contributions are as follows: ## proposed method in this section, we propose tr-rules, a rule-based model for tkg extrapolation, which adopts both cyclic and acyclic rules mining and resolves temporal redundancy. first, we give the definition of temporal redundancy and illustrate how tr-rules resolves it. then we introduce the rule learning module of tr-rules where specifically we show the acyclc rules mining algorithm of tr-rules. finally, the rule application module of tr-rules is introduced to show how we combine cyclic and acyclic rules together to predict future events. ## conclusion in this paper, we define the problem of temporal redundancy and propose tr-rules, a rule-based tkg extrapolation model which solves temporal redundancy by replacing standard confidence with window confidence. tr-rules also firstly mines acyclic rules in tkgs which are proved to have more promising performance compared with cyclic rules because of their higher sensitivity to the number of sampled random walks. experimental results show that tr-rules achieves state-of-the-art performance in most metrics on three datasets. as for future work, it is promising to explore more sophisticated aggregation methods for window confidence calculation, such as employing cluster or machine learning methods.