A Topological View of Rule Learning in Knowledge Graphs

1 INTRODUCTION . Knowledge graphs ( KGs ) are graph-structured knowledge bases that integrate human knowledge through relational triplets . In a KG , nodes represent entities and edges represent relational triplets connecting them . A relational triplet is defined as ( eh , r , et ) , where eh and et are the head and tail entities respectively , and r is the relation between them . KGs have been used in many problems such as recommendation systems ( Wang et al. , 2018 ) , question answering ( Huang et al. , 2019 ; Zhang et al. , 2018 ) , biomedical research ( Zhao et al. , 2020b ; Zhu et al. , 2020 ) , and zero-shot learning ( Kampffmeyer et al. , 2019 ) . Due to the limitation of human knowledge and data extraction algorithms , we can not thoroughly define and excavate all the entities and relations in a KG ( Chen et al. , 2020 ) . The incomplete structures and contents of KGs can significantly benefit from an automatic completion algorithm . Early works ( Bordes et al. , 2013 ; Yang et al. , 2014 ; Sun et al. , 2019 ) focus on incorporating the attributes of entities . Recent works develop models that are agnostic of entity attributes . They can handle new entities and dynamic KGs , which are quite common . These entity-agnostic works ( Yang et al. , 2017 ; Sadeghian et al. , 2019 ; Teru et al. , 2020 ) are called inductive relation prediction models . They predict missing triplets by learning logical rules in KGs . For example , from the KG shown in Figure 1 ( a ) , we can learn the rule : ∃X , ( X , part of , Y ) ∧ ( X , lives in , Z ) → ( Y , located in , Z ) . ( 1 ) Based on the learned rule , in Figure 1 ( b ) , we can induce the missing triplet ( ManchesterUnited , located in , Manchester ) due to the existence of the two-hop path ( Cristiano , lives in , Manchester ) and ( Cristiano , part of , ManchesterUnited ) .1 Existing inductive relation prediction works ( Yang et al. , 2017 ; Sadeghian et al. , 2019 ) mainly view rules as paths , i.e. , sequences of relations connecting two entities of interest . For example , to express the rule shown in Equation 1 , these methods need to enumerate all possible 2-hop paths . As Yang et al . ( 2017 ) pointed out , the number of learnable parameters of these methods is O ( |R|T ) , where |R| denotes the number of relation types , and T denotes the maximum length of rules . To avoid 1Technically speaking , these methods can only learn the “ and ” operation between relations . We will be interested in expanding to more sophisticated rules , but this is beyond the scope of this paper . the exponential-size search space of paths , we tackle the problem from a new perspective : to view logical rules as cycles , and then learn the rules in the space of cycles . In fact , any rule can be considered a cycle by including both the relation path and the target relation itself . The benefit of using cycles is that there is an intrinsic structure in the space of cycles . Based on the theory of algebraic topology ( Munkres , 2018 ) , the space of cycles is a vector space under certain assumptions . We can exploit the linear structure of cycle space for efficient rule learning . In particular , we focus on a basis , i.e. , a set of linearly independent cycles that can represent any cycle . Taking Figure 1 ( a ) as an example , if we choose the red cycle and the blue cycle as the cycle basis , then the green cycle can be represented as their sum . Here “ sum ” means modulo-2 sum ( Figure 1 ( d ) ) , thus the overlapped triplet ( Lebron , part of , Lakers ) is canceled . By focusing on the cycle basis that spans the cycle space , we decrease the parameter space from exponential to linear2 . However , the choice of a useful cycle basis is not trivial . Not all cycles/rules are “ good ” , or say , providing a valid relation reasoning . For example , the green cycle in Figure 1 ( a ) reveals the rule : ∃X1 , X2 , ( X1 , support , Y ) ∧ ( X1 , fan of , X2 ) ∧ ( X2 , lives in , Z ) → ( Y , located in , Z ) . ( 2 ) This rule is not considered good . There is no strong reason that a basketball team ( Y ) should be located in a city ( Z ) just because there is a fan ( X1 ) that supports both the team and a player ( X2 ) living in the city Z . The goal of relation learning is to efficiently comb through all cycles/rules and learn the good ones . A desirable cycle basis should facilitate this process . Intuitively , the elements of the basis should be as close to good cycles as possible . Revisiting the example of Figure 1 ( a ) , we prefer the basis consisting of the red and blue cycles ( both are good rules ) and avoid the green cycle . To find suitable cycle bases for relation learning , we propose an approach inspired by the theory of topological data analysis ( Dey et al. , 2010 ; Chen & Freedman , 2010 ; Busaryev et al. , 2012 ) . These works show that cycle bases generated from shortest path trees have favorable theoretical guarantees . In the context of relation learning , we argue that the good cycles are generally short , and can be easily learned using shortest-path-tree-based cycle bases . We also propose to sample multiple shortest path trees and collect a family of cycle bases . This provides sufficient redundancy and coverage to ensure we learn good cycles/rules efficiently , as experiments will show . To fully exploit the cycle bases to learn all the good rules , we propose Cycle Basis Graph Neural Network ( CBGNN ) . We build a GNN on a new graph whose nodes represent cycles in the chosen bases , and edges represent their interaction . Through the message passing of the GNN , we are running implicit algebraic operations over the space of cycles . Our method will efficiently learn to represent good cycles , and learn to predict relations in KGs . Experiments on popular inductive relation prediction benchmarks show the efficacy of our method . 2A cycle basis of size β spans a cycle space of size 2β . In summary , our contribution is three-fold : 1 . We propose , for the first time , to investigate the inductive relation prediction problem through a cycle-based perspective . 2 . Inspired by the theory of topological data analysis , we propose to exploit the linear structure of the cycle space , and to compute suitable cycle bases that can best express the rules . This empowers us to explore rule space more efficiently than previous path-based approaches . 3 . We propose a novel graph neural network , CBGNN . It runs implicit algebraic operations in the cycle space through the message passing of a GNN , and learns the representation of good cycles and good rules . Experiments show that CBGNN achieves state-of-the-art results on various inductive relation prediction benchmarks . 2 RELATED WORKS . Graph Learning with Topology . Graph structural information have been shown to enhance graph representation learning ( Kipf & Welling , 2016 ; You et al. , 2019 ; Ye et al. , 2019 ) . In recent years advanced topological information , i.e. , persistent homology ( Edelsbrunner et al. , 2000 ; Edelsbrunner & Harer , 2010 ) , have been applied to graph learning problem . Persistent-homology-based features encode multiscale topological information regarding graph structures in view of different filter functions . They provide additional discriminative power for various tasks such as graph classification ( Hofer et al. , 2020 ; Carrière et al. , 2020 ; Hofer et al. , 2017 ; Zhao & Wang , 2019 ) , link prediction ( Yan et al. , 2021 ; Bhatia et al. , 2019 ) , and node classification ( Zhao et al. , 2020a ) . From a different perspective , new graph neural networks have been proposed for high-order graphs , treated as simplicial or cell complexes ( Bodnar et al. , 2021b ; a ) . Unlike these approaches , we focus on a different task , inductive relation prediction . We exploit the space of cycles and its underlying algebraic structure for better graph representation learning . We believe the cycle-centric design of our graph neural network is generic and can extend to many other tasks beyond relation prediction . Finally , we note that in topological data analysis , homology localization , including computing short cycles representatives of a homology class and computing short cycle bases representing the whole homology group , is well studied theoretically ( Chambers et al. , 2009 ; Chen & Freedman , 2011 ; Dey et al. , 2011 ; Busaryev et al. , 2012 ; Dey et al. , 2010 ; Dey & Wang , 2022 ) . In recent years , new questions have been raised regarding finding short representative cycles for classes in persistent homology ( Wu et al. , 2017 ; Dey et al. , 2020 ) . Cycle information can complement persistent homology information to achieve better learning power . Inductive relation prediction methods . Inductive relation prediction methods can be divided into two categories : path-based methods , and GNN-based methods . Among path-based methods , AMIE ( Galárraga et al. , 2013 ) and RuleN ( Meilicke et al. , 2018 ) are classic rule learning methods . These two methods prune the process of rule searching based on strong assumptions on the attribute of rules , thus their performances are not satisfying . To deal with the problem , NeuralLP ( Yang et al. , 2017 ) and DRUM ( Sadeghian et al. , 2019 ) are proposed to learn the rules in a data-driven manner . However , as pointed in ( Sadeghian et al. , 2019 ) , these works can not learn all the rules correctly with a small number of learnable parameters . GNN-based methods such as GraIL ( Teru et al. , 2020 ) and CoMPILE ( Mai et al. , 2021 ) predict missing triplets with graph neural networks ( GNNs ) . These methods are appealing as GNNs are able to learn better representations and achieve state-of-the-art performance in general . To predict whether a certain triplet exists in the KG , these methods first extract the corresponding vicinity graph of the triplet and then learn the rules through message passing and GNN scoring . Therefore , they can only predict the triplets one by one , with a rather low computational efficiency . In the experiment part , we will empirically show the efficiency of our framework by comparing it with these models . 3 CYCLE SPACE , CYCLE BASIS , AND THE PURSUIT OF SUITABLE BASES . In this section , we explain how to find suitable cycle bases that can facilitate the learning of good cycles/rules . We first introduce the background of the cycle space and cycle basis . Next , we explain our choice of suitable cycle bases , which will be the foundation of our model . By no means our exposition is comprehensive . For more details , we refer the readers to textbooks on algebraic topology and computational topology ( Munkres , 2018 ; Edelsbrunner & Harer , 2010 ; Dey & Wang , 2022 ) . We focus on cycles in undirected graphs , while the definitions generalize to higher dimensions , e.g. , simplicial complexes . Furthermore , we focus on the algebraic structures over Z2 field , which has two elements , 0 and 1 , under modulo-2 addition and multiplication . Over Z2 field , the structure of the space of cycles is simpler and more friendly to computation . For the rest of the paper , regarding the input KG , we will use node , vertex , and entity interchangeably . We will also use edge and triplet interchangeably . Within this section , we temporarily ignore the relation associated with each triplet . We treat the input KG as an undirected graph G = ( V , E ) , where V and E denote the sets of vertices and edges , respectively .

Summary: This paper focuses on the problem of inductive relational prediction for KG completion. Previous works often regard the rules as paths, different from those works, this paper regards the rules as circles. Based on that, the authors propose a GCN-based method for learning rules. They conduct experiments on the public datasets of inductive relation prediction and the proposed method achieves better or similar performance compared to best baselines.