RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs

1 INTRODUCTION . Knowledge graphs are collections of real-world facts , which are useful in various applications . Each fact is typically specified as a triplet ( h , r , t ) or equivalently r ( h , t ) , meaning entity h has relation r with entity t. For example , Bill Gates is the Co-founder of Microsoft . As it is impossible to collect all facts , knowledge graphs are incomplete . Therefore , a fundamental problem on knowledge graphs is to predict missing facts by reasoning with existing ones , a.k.a . knowledge graph reasoning . This paper studies learning logic rules for reasoning on knowledge graphs . For example , one may extract a rule ∀X , Y , Z hobby ( X , Y ) ← friend ( X , Z ) ∧ hobby ( Z , Y ) , meaning that if Z is a friend of X and Z has hobby Y , then Y is also likely the hobby of X . Then the rule can be applied to infer new hobbies of people . Such logic rules are able to improve interpretability and precision of reasoning ( Qu & Tang , 2019 ; Zhang et al. , 2020 ) . Moreover , logic rules can also be reused and generalized to other domains and data ( Teru & Hamilton , 2020 ) . However , due to the large search space , inferring high-quality logic rules for reasoning on knowledge graphs is a challenging task . Indeed , a variety of methods have been proposed for learning logic rules from knowledge graphs . Most traditional methods such as path ranking ( Lao & Cohen , 2010 ) and Markov logic networks ( Richardson & Domingos , 2006 ) enumerate relational paths on graphs as candidate logic rules , and then learn a weight for each rule as an assessment of rule qualities . There are also some recent methods based on neural logic programming ( Yang et al. , 2017 ) and neural theorem provers ( Rocktäschel & Riedel , 2017 ) , which are able to learn logic rules and their weights simultaneously in a differentiable way . Though empirically effective for prediction , the search space of these methods is exponentially large , making it hard to identify high-quality logic rules . Besides , some recent efforts ( Xiong et al. , 2017 ) formulate the problem as a sequential decision making process , and use reinforcement learning to search for logic rules , which significantly reduces search complexity . However , due to the large action space and sparse reward in training , the performance of these methods is not yet satisfying . * Equal contribution . In this paper , we propose a principled probabilistic approach called RNNLogic which overcomes the above limitations . Our approach consists of a rule generator as well as a reasoning predictor with logic rules , which are simultaneously trained to enhance each other . The rule generator provides logic rules which are used by the reasoning predictor for reasoning , while the reasoning predictor provides effective reward to train the rule generator , which helps significantly reduce the search space . Specifically , for each query-answer pair , e.g. , q = ( h , r , ? ) and a = t , we model the probability of the answer conditioned on the query and existing knowledge graph G , i.e. , p ( a|G , q ) , where a set of logic rules z 1 is treated as a latent variable . The rule generator defines a prior distribution over logic rules for each query , i.e. , p ( z|q ) , which is parameterized by a recurrent neural network . The reasoning predictor computes the likelihood of the answer conditioned on the logic rules and the existing knowledge graph G , i.e. , p ( a|G , q , z ) . At each training iteration , we first sample a few logic rules from the rule generator , and further update the reasoning predictor to try out these rules for prediction . Then an EM algorithm ( Neal & Hinton , 1998 ) is used to optimize the rule generator . In the E-step , a set of high-quality logic rules are selected from all the generated rules according to their posterior probabilities . In the M-step , the rule generator is updated to imitate the high-quality rules selected in the E-step . Extensive experimental results show that RNNLogic outperforms state-of-the-art methods for knowledge graph reasoning 2 . Besides , RNNLogic is able to generate high-quality logic rules . 2 RELATED WORK . Our work is related to existing efforts on learning logic rules for knowledge graph reasoning . Most traditional methods enumerate relational paths between query entities and answer entities as candidate logic rules , and further learn a scalar weight for each rule to assess the quality . Representative methods include Markov logic networks ( Kok & Domingos , 2005 ; Richardson & Domingos , 2006 ; Khot et al. , 2011 ) , relational dependency networks ( Neville & Jensen , 2007 ; Natarajan et al. , 2010 ) , rule mining algorithms ( Galárraga et al. , 2013 ; Meilicke et al. , 2019 ) , path ranking ( Lao & Cohen , 2010 ; Lao et al. , 2011 ) and probabilistic personalized page rank ( ProPPR ) algorithms ( Wang et al. , 2013 ; 2014a ; b ) . Some recent methods extend the idea by simultaneously learning logic rules and the weights in a differentiable way , and most of them are based on neural logic programming ( Rocktäschel & Riedel , 2017 ; Yang et al. , 2017 ; Cohen et al. , 2018 ; Sadeghian et al. , 2019 ; Yang & Song , 2020 ) or neural theorem provers ( Rocktäschel & Riedel , 2017 ; Minervini et al. , 2020 ) . These methods and our approach are similar in spirit , as they are all able to learn the weights of logic rules efficiently . However , these existing methods try to simultaneously learn logic rules and their weights , which is nontrivial in terms of optimization . The main innovation of our approach is to separate rule generation and rule weight learning by introducing a rule generator and a reasoning predictor respectively , which can mutually enhance each other . The rule generator generates a few high-quality logic rules , and the reasoning predictor only focuses on learning the weights of such high-quality rules , which significantly reduces the search space and leads to better reasoning results . Meanwhile , the reasoning predictor can in turn help identify some useful logic rules to improve the rule generator . The other kind of rule learning method is based on reinforcement learning . The general idea is to train pathfinding agents , which search for reasoning paths in knowledge graphs to answer questions , and then extract logic rules from reasoning paths ( Xiong et al. , 2017 ; Chen et al. , 2018 ; Das et al. , 2018 ; Lin et al. , 2018 ; Shen et al. , 2018 ) . However , training effective pathfinding agents is highly challenging , as the reward signal ( i.e. , whether a path ends at the correct answer ) can be extremely sparse . Although some studies ( Lin et al. , 2018 ) try to get better reward by using embedding-based methods for reward shaping , the performance is still worse than most embedding-based methods . In our approach , the rule generator has a similar role to those pathfinding agents . The major difference is that we simultaneously train the rule generator and a reasoning predictor with logic rules , which mutually enhance each other . The reasoning predictor provides effective reward for training the rule generator , and the rule generator offers high-quality rules to improve the reasoning predictor . Our work is also related to knowledge graph embedding , which solves knowledge graph reasoning by learning entity and relation embeddings in latent spaces ( Bordes et al. , 2013 ; Wang et al. , 2014c ; Yang et al. , 2015 ; Nickel et al. , 2016 ; Trouillon et al. , 2016 ; Cai & Wang , 2018 ; Dettmers et al. , 2018 ; Balazevic et al. , 2019 ; Sun et al. , 2019 ) . With proper architectures , these methods are able to learn 1More precisely , z is a multiset . In this paper , we use “ set ” to refer to “ multiset ” for conciseness . 2The codes of RNNLogic are available : https : //github.com/DeepGraphLearning/RNNLogic High-quality Logic Rules . some simple logic rules . For example , TransE ( Bordes et al. , 2013 ) can learn some composition rules . RotatE ( Sun et al. , 2019 ) can mine some composition rules , symmetric rules and inverse rules . However , these methods can only find some simple rules in an implicit way . In contrast , our approach explicitly trains a rule generator , which is able to generate more complicated logic rules . There are some works studying boosting rule-based models ( Goldberg & Eckstein , 2010 ; Eckstein et al. , 2017 ) , where they dynamically add new rules according to the rule weights learned so far . These methods have been proven effective in binary classification and regression . Compared with them , our approach shares similar ideas , as we dynamically update the rule generator with the feedback from the reasoning predictor , but we focus on a different task , i.e. , reasoning on knowledge graphs . 3 MODEL . In this section , we introduce the proposed approach RNNLogic which learns logic rules for knowledge graph reasoning . We first formally define knowledge graph reasoning and logic rules . Knowledge Graph Reasoning . Let pdata ( G , q , a ) be a training data distribution , where G is a background knowledge graph characterized by a set of ( h , r , t ) -triplets which we may also write as r ( h , t ) , q = ( h , r , ? ) is a query , and a = t is the answer . Given G and the query q , the goal is to predict the correct answer a . More formally , we aim to model the probabilistic distribution p ( a|G , q ) . Logic Rule . We perform knowledge graph reasoning by learning logic rules , where logic rules in this paper have the conjunctive form ∀ { Xi } li=0 r ( X0 , Xl ) ← r1 ( X0 , X1 ) ∧ · · · ∧ rl ( Xl−1 , Xl ) with l being the rule length . This syntactic structure naturally captures composition , and can easily express other common logic rules such as symmetric or inverse rules . For example , let r−1 denote the inverse relation of relation r , then each symmetric rule can be expressed as ∀ { X , Y } r ( X , Y ) ← r−1 ( X , Y ) . In RNNLogic , we treat a set of logic rules which could explain a query as a latent variable we have to infer . To do this , we introduce a rule generator and a reasoning predictor using logic rules . Given a query , the rule generator employs a recurrent neural network to generate a set of logic rules , which are given to the reasoning predictor for prediction . We optimize RNNLogic with an EM-based algorithm . In each iteration , we start with updating the reasoning predictor to try out some logic rules generated by the rule generator . Then in the E-step , we identify a set of high-quality rules from all generated rules via posterior inference , with the prior from the rule generator and likelihood from the reasoning predictor . Finally in the M-step , the rule generator is updated with the identified high-quality rules .

Summary: In this paper, the author proposes RNNLogic for learning FOL rules from the knowledge graph. The proposed method assigns embeddings for each relation type and uses RNN module to generate chain-like rule candidates. Candidates are evaluated with a separate evaluation module that computes the scores. The rule scores are then taken to update the generator module using EM.