LATENTLOGIC : Learning Logic Rules in Latent Space over Knowledge Graphs (2023)

Abstract: Learning logic rules for knowledge graph reasoning is essential as such rules provide interpretable explanations for reasoning and can be generalized to different domains. However, existing methods often face challenges such as searching in a vast search space (e.g., enumeration of relational paths or multiplication of high-dimensional matrices) and inefficient optimization (e.g., techniques based on reinforcement learning or EM algorithm). To address these limitations, this paper proposes a novel framework called LatentLogic to efficiently mine logic rules by controllable generation in the latent space. Specifically, to map the discrete relational paths into the latent space, we leverage a pre-trained VAE and employ a discriminator to establish an energy-based distribution. Additionally, we incorporate a sampler based on ordinary differential equations, enabling the efficient generation of logic rules in our approach. Extensive experiments on benchmark datasets demonstrate the effectiveness and efficiency of our proposed method.

## introduction knowledge graphs usually contain collections of real-world facts encoded in triplets with entity and relation information, and find broad applications in across multiple domains @xcite @xcite @xcite . despite some knowledge graphs holding hundreds of millions of triples, they still suffer from incompleteness, whereby many valid triples are missing since it is impractical to identify them all manually. therefore, a fundamental and essential task in knowledge graphs is to utilize existing facts to predict the missing ones. recent studies have focused on learning logic rules from knowledge graphs and utilizing these learned rules to predict absent facts. an example of such a rule is ∀x, y, z nationality(x, y ) ← classmate(x, z) ∧ nationality(z, y ), indicating that if z is the classmate of x and has a nationality of y , then x is likely to have a nationality of y . this rule can be applied to deduce the nationalities of new individuals. compared to other methods such as knowledge graph embedding approaches @xcite @xcite , the rule-based method @xcite is more interpretable and can be applied to inductive scenarios @xcite . most rule-based methods involve enumerating relational paths as candidate rules, followed by assigning weights to each rule to indicate their quality @xcite @xcite . when the scale of the knowledge graph expands, these methods face the challenge of exponentially growing search space. to overcome this problem, rnnlogic @xcite introduces a rule generator and a reasoning predictor to separate rule generation from rule weight learning. however, the optimization process based on the expectation-maximization (em) algorithm tends to have slow convergence, leading to extended training periods. another line of research utilizes reinforcement learning (rl) to search for logic rules by making sequential decisions @xcite @xcite . nevertheless, rl-based methods often encounter challenges such as large action spaces and sparse rewards during training. as a result, efficiently mining high-quality logic rules for knowledge graph reasoning remains a challenging task. in this paper, we propose a novel framework named latentlogic, which overcomes the aforementioned challenges. our approach bypasses the enumeration of relational paths by employing controllable sampling in latent space. furthermore, each component of latentlogic is trained in an end-to-end fashion, avoiding the indirect and inefficient optimization procedure. concretely, we utilize a vae-based autoencoder @xcite to map discrete relational paths into a low-dimensional latent space. we employ a discriminator to measure the semantic coherence between the latent vector and the rule head, thereby creating an energy-based distribution in the latent space. to obtain latent vector samples that correspond to the desired rule head, we employ a sampler based on ordinary differential equations @xcite . the latent vectors are used to generate rule bodies for the given rule head using the vae generator. extensive experiments demonstrated the computational efficiency of latentlogic in comparison to previous rule learning methods @xcite @xcite , indicating that latentlogic exhibits enhanced scalability for mining logic rules on largerscale knowledge graphs. furthermore, through experiments on two commonly used benchmark datasets, fb15k-237 @xcite and wn18rr @xcite , we observed that latentlogic successfully generates highquality logic rules for knowledge graph reasoning and evidently outperform the salient baseline methods. ## experimental setup datasets we conduct experiments on two widely used knowledge graph reasoning benchmark datasets: fb15k-237 @xcite and wn18rr @xcite . wn18rr comprises of 40,943 entities and 11 relations, with 86k/3k/3k instances set aside for training/validation/testing correspondingly. fb15k-237 comprises 14,541 entities and 237 relations and has 272k/17k/20k instances reserved for training/validation/testing, respectively. ## experimental results baselines we compared latentlogic with two taxonomies of methods, including: knowledge graph embedding methods: transe @xcite , distmult @xcite , tucker @xcite , and rotate @xcite . rule-based methods: pathrank @xcite , neurallp @xcite , drum @xcite , rnnlogic @xcite , and rlogic @xcite . evaluation metrics we adopt forward chaining @xcite for inferring miss-ing facts from logical rules. for every test triplet (h, r, t), two queries are created: (h, r, ?) and (?, r, t), using t and h as answers, respectively. to maintain consistency with previous studies, mean rank (mr), mean reciprocal rank (mrr), and hit@k are selected as evaluation metrics under the filtered setting @xcite . furthermore, to mitigate the effects of random sampling, we evaluate the model performance on five different random seeds, and report the average performance. overall performance as shown in tab. 1, we present the experimental results on the fb15k-237 and wn18rr datasets. firstly, we compare latentlogic with rule-based methods and observe that latentlogic outperforms the statistical learning method pathrank, neural differentiable methods neurallp and drum, as well as recent methods rnnlogic and rlogic. in particular, we obtain 2.56% and 2.59% relative increase in mrr and hits@10 on fb15k-237 against the state-of-the-art rule-based method rlogic @xcite . similarly, latentlogic achieves 1.69% and 2.98% increase in mrr and hits@10 on the wn18rr dataset against rlogic. then, we also compare latentlogic against salient knowledge graph embedding-based methods and find that latentlogic yields comparable performance to embedding-based methods, especially on wn18rr, where it outperforms selected embedding-based baselines. ## related work over the past years, learning logical rules over knowledge graphs has been an active research area. most traditional methods enumerate relational paths between query entities and answer entities as candidate logic rules, and further learn a scalar weight for each rule to assess the quality. some representative works include markov logic network (mln) @xcite , path ranking @xcite and probabilistic personalized page rank (proppr) algorithms @xcite . then some methods extend the idea by simultaneously learning logic rules and the weights in a differentiable way. for example, some works @xcite @xcite try to use neural logic programming to model rule-based reasoning and to learn high-quality logical rules. another kind of rule-learning method is based on reinforcement learning. the basic idea is to train a path-finding agent, which is used to search for reasoning paths over knowledge graphs to answer queries, and then extract logic rules from reasoning paths. some representative works include deep-path @xcite , minerva @xcite , m-walk @xcite and r5 @xcite . recently, rnnlogic @xcite solves this problem by training a generator and a predictor alternately. rlgoic @xcite learns rules in a recursive manner. ## conclusion learning logic rules for knowledge graph reasoning is crucial, as they offer interpretable explanations for the process of reasoning. we propose a novel framework for learning logic rules in latent space. concretely, we introduce a relational path autoencoder to map paths into latent space and a rule discriminator to access the consistency between rule bodies and rule heads. with the sampler based on ode, latentlogic can generate logic rules efficiently. experimental results showed that la-tentlogic outperforms strong baseline methods and is efficient for training.