An Adaptive Logical Rule Embedding Model for Inductive Reasoning over Temporal Knowledge Graphs (2022)

Abstract: Temporal knowledge graphs (TKGs) extrapolation reasoning predicts future events based on historical information, which has great research significance and broad application value. Existing methods can be divided into embedding-based methods and logical rule-based methods. Embedding-based methods rely on learned entity and relation embeddings to make predictions and thus lack interpretability. Logical rule-based methods bring scalability problems due to being limited by the learned logical rules. We combine the two methods to capture deep causal logic by learning rule embeddings, and propose an interpretable model for temporal knowledge graph reasoning called adaptive logical rule embedding model for inductive reasoning (ALRE-IR). ALRE-IR can adaptively extract and assess reasons contained in historical events, and make predictions based on causal logic. Furthermore, we propose a one-class augmented matching loss for optimization. When evaluated on the ICEWS14, ICEWS0515 and ICEWS18 datasets, the performance of ALRE-IR outperforms other state-of-the-art baselines. The results also demonstrate that ALRE-IR still shows outstanding performance when transferred to related dataset with common relation vocabulary, indicating our proposed model has good zero-shot reasoning ability.

## introduction knowledge graphs (kgs) are a form of structured human knowledge. they represent events as triples @xcite , where subject and object are entities. entities usually are objects and abstract concepts in the real world, and relations represent relationships between entities. kgs have caused great research both in academia and industry @xcite @xcite , and have been widely used in many real-world applications including relation extraction @xcite , entity linking @xcite , and question answering @xcite . however, most of the knowledge graphs are incomplete @xcite , which affects their effectiveness and limits the performance of kg-based applications. reasoning over kgs aims to infer new conclusions based on existing data and predict the missing events, which can effectively alleviate this problem. traditional knowledge graphs contain only static events, and there is a large amount of available event data with temporal correlations, where entities interact differently over time. therefore, many temporal knowledge graphs (tkgs) composed of entity interaction data with temporal attributes have emerged @xcite @xcite .tkgs extend static triples with timestamp to represent dynamic events in the form of quadruples @xcite , where timestamp represents valid time of static triple. compared with traditional static kgs, tkgs have complex temporal dynamic characteristics, which increase difficulty of reasoning on tkgs. reasoning over a tkg primarily has two settings, interpolation @xcite and extrapolation @xcite . given events within time interval [t 0 , t t ], interpolation attempts to infer missing events that happened in [t 0 , t t ], while extrapolation predicts future missing events for time @xmath0. extrapolation reasoning learns hidden connections between events from observed historical kgs and then predicts new events at future timestamps @xcite @xcite , which can be applied in practical scenarios such as disaster relief @xcite and financial analysis @xcite .this paper focuses on extrapolation reasoning task. recently, many research efforts have been put into extrapolation reasoning over tkgs and realize excellent prediction performance @xcite . these methods can be divided into two categories: embedding-based methods and logical rulebased symbolic methods. embedding-based methods such as re-net @xcite , cygnet @xcite , tie @xcite and re-gcn @xcite can capture complex information in tkg, but the black-box property of embeddings make them lack interpretability and are not suitable for many practical applications. some researchers propose to create logical rules for reasoning to enhance credibility and utility, such as streamlearner @xcite and tlogic @xcite . they employ statistical-based measures to assess confidence of rules and make predictions based on learned rules. however, the learned rules are limited, which makes the model have scalability problems and is not suitable for large-scale datasets in reality. to alleviate the above problems, we propose an adaptive logical rule embedding model for inductive reasoning (alre-ir) on temporal knowledge graphs. it can effectively capture deep structure of tkg and mine potential logical rules. logical rules are represented by a sequence of relations. therefore, relations are the core features we focus on when mining rules, and entities are just tools for extracting relation paths. first, we extract relation paths from historical subgraphs and learn embeddings of relation paths that contain historical semantics. we then match these relation paths with current events to obtain rules and assess confidence of the rules based on interpretable causal logic. finally, the quadruple can be scored according to confidence of the rules. we design training tasks from a coarse-grained quadruple perspective and a fine-grained rule perspective, respectively, and propose a one-class augmented matching loss to optimize our proposed adaptive logical rule embedding model. during the inference process, our model can adaptively extract and learn relation path features based on historical information, assess the confidence of corresponding rules, and predict missing entities. furthermore, our trained model can be applied to new datasets with a common relation vocabulary for zero-shot reasoning. in summary, this paper makes the following four contributions: ## preliminaries temporal knowledge graph (tkg). a tkg consists of dynamic events, an event is represented in the form of a quadruple (s, r, o, t) consisting of a subject entity @xmath1, a relation @xmath2, an object entity @xmath3. e and υ denote entity set and relation set respectively, and γ represents the set of timestamps. link prediction. given a missing temporal quadruple (event), link prediction aims to infer the missing part, such as predicting object entity given (s, r, ?, t) or predicting subject entity given (?, r, o, t) or predicting relation given (s, ?, o, t). for each quadruple, the training objective function is optimized to make the correct quadruple score higher than the incorrect quadruple, it is generally defined as a score function g(s, r, o, t) ∈ r. temporal logical rules. we predict future events by mining logical causal relationships between events. as shown in figure 1 , we take an event (e 1 , r 8 , e 5 , t) as an example, and mine temporal logical rules contained in it according to the historical information in the previous m timestamps. figure 1 (a) represents a subgraph composed of events that occurred in the previous m timestamps. based on this subgraph, we mine all rules that might lead to the current event. figure 1(b ) shows all paths from e 1 to e 5 , we can extract relations to get four possible logical rules in figure 1(c ). each rule r t (p, r) consists of a path @xmath4,o) and a relation @xmath5, p represents the historical reason and r represents the result at the current timestamp. ## method we propose an interpretable model for temporal knowledge graph reasoning called adaptive logical rule embedding model for inductive reasoning (alre-ir). it uses relation paths to represent logical rules implicit in the knowledge graph, and captures complex semantic features in the knowledge graph by learning embeddings of relation paths. it can adaptively extract possible rules, learn rules' embeddings, score rules based on interpretable causal logic, and finally make predictions based on relative confidence of rules. our inductive reasoning model is composed of three parts as follows: encoding, which walks out all historical relation paths for each input quadruple, learns embeddings of all relation paths. decoding, which scores quadruples according to all the temporal logical rules associated with them. training, which proposes a one-class augmented matching loss to optimize the model, so that the model can adaptively learn reasonable logical rules. the overall architecture of the model is shown in figure 2 , details of the model will be elaborated as follows. ## conclusion we propose an interpretable model for temporal knowledge graph reasoning called alre-ir. it can autonomously extract and assess rules based on historical features, and make prediction with rules. we design training tasks from a coarse-grained quadruple perspective and a fine-grained rule perspective, respectively, and propose a one-class augmented matching loss for optimization. alre-ir can be transferred to perform zero-shot reasoning on any new dataset with common relation vocabulary. experimental results demonstrate that our proposed alre-ir performs better than the state-of-the-art baselines.