RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs

1 INTRODUCTION . Knowledge graphs are collections of real-world facts , which are useful in various applications . Each fact is typically specified as a triplet ( h , r , t ) or equivalently r ( h , t ) , meaning entity h has relation r with entity t. For example , Bill Gates is the Co-founder of Microsoft . As it is impossible to collect all facts , knowledge graphs are incomplete . Therefore , a fundamental problem on knowledge graphs is to predict missing facts by reasoning with existing ones , a.k.a . knowledge graph reasoning . This paper studies learning logic rules for reasoning on knowledge graphs . For example , one may extract a rule ∀X , Y , Z hobby ( X , Y ) ← friend ( X , Z ) ∧ hobby ( Z , Y ) , meaning that if Z is a friend of X and Z has hobby Y , then Y is also likely the hobby of X . Then the rule can be applied to infer new hobbies of people . Such logic rules are able to improve interpretability and precision of reasoning ( Qu & Tang , 2019 ; Zhang et al. , 2020 ) . Moreover , logic rules can also be reused and generalized to other domains and data ( Teru & Hamilton , 2020 ) . However , due to the large search space , inferring high-quality logic rules for reasoning on knowledge graphs is a challenging task . Indeed , a variety of methods have been proposed for learning logic rules from knowledge graphs . Most traditional methods such as path ranking ( Lao & Cohen , 2010 ) and Markov logic networks ( Richardson & Domingos , 2006 ) enumerate relational paths on graphs as candidate logic rules , and then learn a weight for each rule as an assessment of rule qualities . There are also some recent methods based on neural logic programming ( Yang et al. , 2017 ) and neural theorem provers ( Rocktäschel & Riedel , 2017 ) , which are able to learn logic rules and their weights simultaneously in a differentiable way . Though empirically effective for prediction , the search space of these methods is exponentially large , making it hard to identify high-quality logic rules . Besides , some recent efforts ( Xiong et al. , 2017 ) formulate the problem as a sequential decision making process , and use reinforcement learning to search for logic rules , which significantly reduces search complexity . However , due to the large action space and sparse reward in training , the performance of these methods is not yet satisfying . * Equal contribution . In this paper , we propose a principled probabilistic approach called RNNLogic which overcomes the above limitations . Our approach consists of a rule generator as well as a reasoning predictor with logic rules , which are simultaneously trained to enhance each other . The rule generator provides logic rules which are used by the reasoning predictor for reasoning , while the reasoning predictor provides effective reward to train the rule generator , which helps significantly reduce the search space . Specifically , for each query-answer pair , e.g. , q = ( h , r , ? ) and a = t , we model the probability of the answer conditioned on the query and existing knowledge graph G , i.e. , p ( a|G , q ) , where a set of logic rules z 1 is treated as a latent variable . The rule generator defines a prior distribution over logic rules for each query , i.e. , p ( z|q ) , which is parameterized by a recurrent neural network . The reasoning predictor computes the likelihood of the answer conditioned on the logic rules and the existing knowledge graph G , i.e. , p ( a|G , q , z ) . At each training iteration , we first sample a few logic rules from the rule generator , and further update the reasoning predictor to try out these rules for prediction . Then an EM algorithm ( Neal & Hinton , 1998 ) is used to optimize the rule generator . In the E-step , a set of high-quality logic rules are selected from all the generated rules according to their posterior probabilities . In the M-step , the rule generator is updated to imitate the high-quality rules selected in the E-step . Extensive experimental results show that RNNLogic outperforms state-of-the-art methods for knowledge graph reasoning 2 . Besides , RNNLogic is able to generate high-quality logic rules . 2 RELATED WORK . Our work is related to existing efforts on learning logic rules for knowledge graph reasoning . Most traditional methods enumerate relational paths between query entities and answer entities as candidate logic rules , and further learn a scalar weight for each rule to assess the quality . Representative methods include Markov logic networks ( Kok & Domingos , 2005 ; Richardson & Domingos , 2006 ; Khot et al. , 2011 ) , relational dependency networks ( Neville & Jensen , 2007 ; Natarajan et al. , 2010 ) , rule mining algorithms ( Galárraga et al. , 2013 ; Meilicke et al. , 2019 ) , path ranking ( Lao & Cohen , 2010 ; Lao et al. , 2011 ) and probabilistic personalized page rank ( ProPPR ) algorithms ( Wang et al. , 2013 ; 2014a ; b ) . Some recent methods extend the idea by simultaneously learning logic rules and the weights in a differentiable way , and most of them are based on neural logic programming ( Rocktäschel & Riedel , 2017 ; Yang et al. , 2017 ; Cohen et al. , 2018 ; Sadeghian et al. , 2019 ; Yang & Song , 2020 ) or neural theorem provers ( Rocktäschel & Riedel , 2017 ; Minervini et al. , 2020 ) . These methods and our approach are similar in spirit , as they are all able to learn the weights of logic rules efficiently . However , these existing methods try to simultaneously learn logic rules and their weights , which is nontrivial in terms of optimization . The main innovation of our approach is to separate rule generation and rule weight learning by introducing a rule generator and a reasoning predictor respectively , which can mutually enhance each other . The rule generator generates a few high-quality logic rules , and the reasoning predictor only focuses on learning the weights of such high-quality rules , which significantly reduces the search space and leads to better reasoning results . Meanwhile , the reasoning predictor can in turn help identify some useful logic rules to improve the rule generator . The other kind of rule learning method is based on reinforcement learning . The general idea is to train pathfinding agents , which search for reasoning paths in knowledge graphs to answer questions , and then extract logic rules from reasoning paths ( Xiong et al. , 2017 ; Chen et al. , 2018 ; Das et al. , 2018 ; Lin et al. , 2018 ; Shen et al. , 2018 ) . However , training effective pathfinding agents is highly challenging , as the reward signal ( i.e. , whether a path ends at the correct answer ) can be extremely sparse . Although some studies ( Lin et al. , 2018 ) try to get better reward by using embedding-based methods for reward shaping , the performance is still worse than most embedding-based methods . In our approach , the rule generator has a similar role to those pathfinding agents . The major difference is that we simultaneously train the rule generator and a reasoning predictor with logic rules , which mutually enhance each other . The reasoning predictor provides effective reward for training the rule generator , and the rule generator offers high-quality rules to improve the reasoning predictor . Our work is also related to knowledge graph embedding , which solves knowledge graph reasoning by learning entity and relation embeddings in latent spaces ( Bordes et al. , 2013 ; Wang et al. , 2014c ; Yang et al. , 2015 ; Nickel et al. , 2016 ; Trouillon et al. , 2016 ; Cai & Wang , 2018 ; Dettmers et al. , 2018 ; Balazevic et al. , 2019 ; Sun et al. , 2019 ) . With proper architectures , these methods are able to learn 1More precisely , z is a multiset . In this paper , we use “ set ” to refer to “ multiset ” for conciseness . 2The codes of RNNLogic are available : https : //github.com/DeepGraphLearning/RNNLogic High-quality Logic Rules . some simple logic rules . For example , TransE ( Bordes et al. , 2013 ) can learn some composition rules . RotatE ( Sun et al. , 2019 ) can mine some composition rules , symmetric rules and inverse rules . However , these methods can only find some simple rules in an implicit way . In contrast , our approach explicitly trains a rule generator , which is able to generate more complicated logic rules . There are some works studying boosting rule-based models ( Goldberg & Eckstein , 2010 ; Eckstein et al. , 2017 ) , where they dynamically add new rules according to the rule weights learned so far . These methods have been proven effective in binary classification and regression . Compared with them , our approach shares similar ideas , as we dynamically update the rule generator with the feedback from the reasoning predictor , but we focus on a different task , i.e. , reasoning on knowledge graphs . 3 MODEL . In this section , we introduce the proposed approach RNNLogic which learns logic rules for knowledge graph reasoning . We first formally define knowledge graph reasoning and logic rules . Knowledge Graph Reasoning . Let pdata ( G , q , a ) be a training data distribution , where G is a background knowledge graph characterized by a set of ( h , r , t ) -triplets which we may also write as r ( h , t ) , q = ( h , r , ? ) is a query , and a = t is the answer . Given G and the query q , the goal is to predict the correct answer a . More formally , we aim to model the probabilistic distribution p ( a|G , q ) . Logic Rule . We perform knowledge graph reasoning by learning logic rules , where logic rules in this paper have the conjunctive form ∀ { Xi } li=0 r ( X0 , Xl ) ← r1 ( X0 , X1 ) ∧ · · · ∧ rl ( Xl−1 , Xl ) with l being the rule length . This syntactic structure naturally captures composition , and can easily express other common logic rules such as symmetric or inverse rules . For example , let r−1 denote the inverse relation of relation r , then each symmetric rule can be expressed as ∀ { X , Y } r ( X , Y ) ← r−1 ( X , Y ) . In RNNLogic , we treat a set of logic rules which could explain a query as a latent variable we have to infer . To do this , we introduce a rule generator and a reasoning predictor using logic rules . Given a query , the rule generator employs a recurrent neural network to generate a set of logic rules , which are given to the reasoning predictor for prediction . We optimize RNNLogic with an EM-based algorithm . In each iteration , we start with updating the reasoning predictor to try out some logic rules generated by the rule generator . Then in the E-step , we identify a set of high-quality rules from all generated rules via posterior inference , with the prior from the rule generator and likelihood from the reasoning predictor . Finally in the M-step , the rule generator is updated with the identified high-quality rules .

Summary: There is a lot of recent work on link-prediction in knowledge graphs. One approach is based on embedding entities and relations in a knowledge graph into vector spaces, and the other is based on finding rules that imply relations, and then using these rules to find new links or facts. This paper takes the latter approach. Within the area of rule-based methods, a number of recent papers have used neural network methods to simultaneously generate rules and to find rule-weights or other related parameters (indicating how important individual rules are). Simultaneously solving for rules and rule-weights is a difficult task. In this paper, the authors propose a method where they separate the rule generation process from the weight/parameter calculation process. More importantly, they add a feedback loop from the weight calculation routine ("reasoning predictor") to the rule generation routine ("rule generator"), which in my opinion is novel, even though there have been a few recent attempts (Xiong et. al. 2017) to use reinforcement learning to search for rules. The rule generator in this paper uses a recurrent neural network (RNN) and the parameters of this RNN are modified by the reasoning predictor. In other words, the iterative process has the feature that new rule generation is influenced by the calculated weights of previously generated rules. The authors perform numerical experiments on 4 standard knowledge graphs to demonstrate the performance of their method