Inductive Relation Prediction with Logical Reasoning Using Contrastive Representations (2022)

Abstract: Relation prediction in knowledge graphs (KGs) aims at predicting missing relations in incomplete triples, whereas the dominant embedding paradigm has a restriction on handling unseen entities during testing. In the real-world scenario, the inductive setting is more common because entities in the training process are finite. Previous methods capture an inductive ability by implicit logic in KGs. However, it would be challenging to preciously acquire entity-independent relational semantics of compositional logic rules and to deal with the deficient supervision of logic caused by the scarcity of relational semantics. To this end, we propose a novel graph convolutional network (GCN)-based model LogCo with logical reasoning by contrastive representations. LogCo firstly extracts enclosing subgraphs and relational paths between two entities to supply the entity-independence. Then a contrastive strategy for relational path instances and the subgraph is proposed for the issue of deficient supervision. The contrastive representations are learned for a joint training regime. Finally, prediction results and logic rules for reasoning are attained. Comprehensive experiments on twelve inductive datasets show that LogCo achieves outstanding performance comparing with state-of-the-art inductive relation prediction baselines.

## introduction knowledge graphs (kgs) store plenty of facts by triples consisting of entities and relations. they have been widely used in different application scenarios, such as relation extraction @xcite , question answering @xcite and information retrieval @xcite . extracting triples from contexts consumes plenty of resources. therefore, some dominant methods aim at the kg completion or relation prediction by learning representations of relations and entities, such as transe @xcite , rescal @xcite , r-gcn @xcite and compgcn @xcite . however, the above methods predict relations assuming for a transductive setting, which means the entities are fixed during training and testing. in the application scenario, there will be new entities during testing. for the scenario in figure 1 (a), training and testing entities have no intersection, so the previous transductive methods will not accurately predict the relation livein between entities bill gates and w.a. in the test set without retraining the whole model. thus, some studies like grail @xcite focus more on models owning inductive ability, which can handle unseen entities by implicit first-order logic rules @xcite fitting the cognition of human beings. for example, by the following logic rule r 1 : workin(x, z) ∧ locatedin(z, y ) → livein(x, y ), the relation livein in the test subgraph can be inferred. however, existing models majorly take advantage of the topological structure of entities and triples in kgs. there still remain two issues of mining logic in inductive relation prediction. secondly, although logical reasoning provides inductive ability, the scarcity of relational semantics leads to deficient supervision of logic rules. dominant logical reasoning methods like @xcite indicates that in a kg owning m kinds of relations, the number of candidate rules within length n is o(m n ), whereas in the real kg, there only contain a few rules during logical reasoning. for example, in figure 1(b) , there are actually 4 rules whose length within 3 from trump to n.y., but at least 8 3 = 512 candidate rules. the detailed statistics of rules in kgs are shown in to address the above issues, we propose a model logco for inductive relation prediction with logical reasoning using contrastive representations. logco firstly extracts subgraphs among the target relation and relational paths within a preset length. relational paths will introduce the relation sequence semantics which is more important for entity-independence in logical reasoning. secondly, a contrastive strategy to address the deficient supervision of logic rules is introduced into logco by constructing positive and negative relational paths. then, logco obtains structural representations using a gcn. positive and negative samples containing both topological structure and relation sequence semantics are all sent to the model for contrastive representations. finally, logco applies a joint training regime combining the supervised and self-supervised information. the logical reasoning is illustrated by the relational path and the target relation, which are considered as the body and head respectively of a logic rule. our main contributions are three-fold: ## preliminary this section briefly introduces the first-order logic rule and its connection with the relational path. a first-order logic rule @xcite learned from kgs is a horn clause @xcite , which consists of an atom as head and a series of atoms as body. for example, here is a horn clause with length n : in which r 1 , r 2 , ..., r n , r t are relations and x, z 1 , z 2 , ..., z n -1 , y are variables generalized from entities in kgs. the body atoms are connected by a conjunction symbol ∧ and point to the head by an implication symbol →. in the rule, an atom contains two variables and adjacent atoms share the same variable. during inference, variables are instantiated to entities x, z 1 , z 2 , . . . , z n -1 , y and form a closed path, for example the blue thick paths in figure 1(b) . we use the relational path, target relation and β ∈ [0, 1] to represent the body, head and confidence of the rule. as we discuss in section 1, β can be obtained by the attention weight of a relational path. ## methodology this section illustrates our proposed approach logco with the help of figure 2 . ## experiments in this section, we firstly introduce benchmarks, baselines, experiment settings and details. secondly, to verify the effectiveness of logco, we implement comparison experiments on relation prediction task. then, we use ablation studies, weight analysis and case studies to comprehensively demonstrate the performance. ## conclusion we propose a novel inductive relation prediction model with logical reasoning using contrastive representations in kgs, named logco. we aim to solve two main issues in this task. to acquire the entity independence semantics from first-order logic rules, logco extracts relational paths in each subgraph. for the deficient supervision of logic rules caused by scarcity of relational semantics, logco introduces contrastive representations to obtain self-supervised information. the experiments on twelve fully-inductive datasets show the effectiveness of logco, and comprehensively demonstrate the impacts for relational paths and contrasts.