R ul E : Knowledge Graph Reasoning with Rule Embedding (2024)

Abstract: Knowledge graph reasoning is an important problem for knowledge graphs. In this paper, we propose a novel and principled framework called RulE (stands for Rule Embedding) to effectively leverage logical rules to enhance KG reasoning. Unlike knowledge graph embedding methods, RulE learns rule embeddings from existing triplets and first-order rules by jointly representing entities , relations and logical rules in a unified embedding space. Based on the learned rule embeddings, a confidence score can be calculated for each rule, reflecting its consistency with the observed triplets. This allows us to perform logical rule inference in a soft way, thus alleviating the brittleness of logic. On the other hand, RulE injects prior logical rule information into the embedding space, enriching and regularizing the entity/relation embeddings. This makes KGE alone perform better too. RulE is conceptually simple and empirically effective. We conduct extensive experiments to verify each component of RulE.Results on multiple benchmarks reveal that our model outperforms the majority of existing embedding-based and rule-based approaches.

## introduction knowledge graphs (kgs) usually store millions of real-world facts and are used in a variety of applications @xcite @xcite . examples of knowledge graphs include freebase @xcite , word-net @xcite and @xcite . they represent entities as nodes and relations among entities as edges. each edge encodes a fact in the form of a triplet (head entity, relation, tail entity). however, kgs are usually highly incomplete, making their downstream tasks more challenging. knowledge graph reasoning, * corresponding authors which predicts missing facts by reasoning on existing facts, has thus become a popular research area in artificial intelligence. there are two prominent lines of work in this area: knowledge graph embedding (kge) and rulebased kg reasoning. knowledge graph embedding (kge) methods such as transe @xcite , rotate @xcite and boxe @xcite embed entities and relations into a latent space and compute the score for each triplet to quantify its plausibility. kge is efficient and robust to noise. however, it only uses zeroth-order (propositional) logic to encode existing facts (e.g., "alice is bob's wife.") without explicitly leveraging first-order (predicate) logic. first-order logic uses the universal quantifier to represent generally applicable logical rules. for instance, "∀x, y : x is y's @xmath0. those rules are not specific to particular entities (e.g., alice and bob) but are generally applicable to all entities. the other line of work, rule-based kg reasoning, in contrast, explicitly applies logic rules to infer new facts @xcite @xcite @xcite . unlike kge, logical rules can achieve interpretable reasoning and generalize to new entities. however, the brittleness of logical rules greatly harms prediction performance. consider the logical rule (x, works in, y) → (x, lives in, y) as an example. it is mostly correct. yet, if somebody works in new york but actually lives in new jersey, the rule can still only infer the wrong fact in an absolute way. considering that the aforementioned two lines of work can complement each other, addressing each other's weaknesses with their own merits, it becomes imperative to study how to integrate logical rules with kge methods in a principled manner. if we view this integration in a broader context, embedding-based reasoning can be seen as a neural method, while rule-based reasoning can be seen as a symbolic method. neural-symbolic learning has also been a focus of artificial intelligence research in recent years @xcite @xcite @xcite . in the kg domain, such efforts exist too. some works combine logical rules and kge by using rules to infer new facts as additional training data for kge @xcite or directly convert some rules into regularization terms for specific kge models @xcite . however, they both leverage logical rules merely to enhance kge training without actually using logical rules to perform reasoning. in this way, they might lose the important information contained in explicit rules, leading to empirically worse performance than state-of-the-art methods. to address the aforementioned limitations, we propose a simple and principled framework called rule, which aims to learn rule embeddings by jointly representing entities, relations and logical rules in a unified space. as illustrated in figure 1 , given a kg and logical rules, rule assigns an embedding to each entity, relation and rule, and defines respective mathematical operators between entities and relations (traditional kge part) as well as between relations and rules (rule part). it is important to note that we cannot define operators between entities and rules because rules are not specific to particular entities. by jointly optimizing entity, relation and rule embeddings in the same space, rule allows injecting prior logical rule information to enrich and regularize the embedding space. our experiments reveal that this joint embedding can boost kge methods themselves. additionally, based on the relation and rule embeddings, rule is able to give a confidence score to each rule, similar to how kge gives each triplet a confidence score. this confidence score reflects how consistent a rule is with the existing facts, and enables performing logical rule inference in a soft way by softly controlling the contribution of each rule, which alleviates the brittleness of logic. we evaluate rule on benchmark link prediction tasks and show superior performance. experimental results reveal that our model outperforms the majority of existing embedding-based and rulebased methods. we also conduct extensive ablation studies to demonstrate the effectiveness of each component of rule. all the empirical results verify that rule is a simple and effective framework for neural-symbolic kg reasoning. ## preliminaries a kg consists of a set of triplets @xmath1, r, t) | h, @xmath2, @xmath3, where e denotes the set of entities and r the set of relations. for a testing triplet (h, r, t), we define a query as @xmath4, r, ?). the knowledge graph reasoning (link prediction) task is to infer the missing entity t based on the existing facts and rules. ## method this section introduces our proposed model rule. rule is a principled framework to combine kg embedding with logical rules by learning rule embeddings. as illustrated in figure 2 , the training process of rule consists of three key components. consider a kg containing triplets and a set of logical rules automatically extracted or predefined by experts. they are: 1) joint entity/relation/rule embedding. we model the relationship between entities and relations as well as the relationship between relations and logical rules to jointly train entity, relation and rule embeddings in a continuous space, as demonstrated in figure 1 . 2) soft rule reasoning. with the rule and relation embeddings, we calculate a confidence score for each rule which is used as the weight of activated rules to output a grounding rule score. 3) finally, we integrate the kge score calculated from the entity and relation embeddings trained in the first stage and the grounding rule score obtained in the second stage to reason unknown triplets. ## experiments in this section, we empirically evaluate rule on several benchmark kgs and show superior performance to existing embedding-based, rule-based methods and hybrid approaches that combine both. additionally, we also conduct extensive ablation experiments to verify the effectiveness of each component of rule. furthermore, we provide theoretical analysis and case studies in appendix 18 to provide further insights and understanding. ## related work embedding-based methods embedding-based methods aim to learn embeddings for entities and relations and estimate the plausibility of unobserved triplets based on these learned embeddings @xcite @xcite @xcite @xcite @xcite . rule-based methods learning logical rules for knowledge graph reasoning has also been extensively studied, including inductive logic programming @xcite , markov logic networks @xcite , amie @xcite , amie+ @xcite , neural-lp @xcite , drum @xcite , rnn-logic @xcite and other methods @xcite . they almost solely use the learned logical rules for reasoning, which suffer from brittleness and are hardly competitive with embedding-based reasoning in most benchmarks. joint kge and logical rules some work tries to incorporate logical rules into kge models. they usually use logical rules to infer new facts as additional training data for kge @xcite or inject rules via regularization terms during training @xcite . however, they do not really perform reasoning with logical rules. ## conclusion we propose a simple and principled framework rule to jointly represent entities, relations and logical rules in a unified embedding space. the incorporation of rule embedding allows injecting rule information to enrich and regularize the embedding space, thus improving the generalization of kge. besides, we also demonstrate that with the learned rule embedding, rule can perform rule inference in a soft way and empirically verify that using an mlp can effectively model the complex interdependencies among rules, thus enhancing rule inference.