Multi-view Classification Model for Knowledge Graph Completion (2020)

Abstract: Most previous work on knowledge graph completion conducted single-view prediction or calculation for candidate triple evaluation, based only on the content information of the candidate triples. This paper describes a novel multi-view classification model for knowledge graph completion, where multiple classification views are performed based on both content and context information for candidate triple evaluation. Each classification view evaluates the validity of a candidate triple from a specific viewpoint, based on the content information inside the candidate triple and the context information nearby the triple. These classification views are implemented by a unified neural network and the classification predictions are weightedly integrated to obtain the final evaluation. Experiments show that, the multi-view model brings very significant improvements over previous methods, and achieves the new state-of-the-art on two representative datasets. We believe that, the flexibility and the scalability of the multi-view classification model facilitates the introduction of additional information and resources for better performance.

## introduction knowledge graph (kg) is a typical kind of graphstructured knowledge base (kb). nowdays, there exist many famous kgs such as yago @xcite , freebase @xcite and dbpedia @xcite . large-scale kgs are widely used in many applications such as semantic searching @xcite @xcite , question answering @xcite and machine reading @xcite . a kg contains a set of triples indicating facts, each of which is composed of a head entity, a tail entity, and a relation indicating the relationship between the two entities. it is nearly impossible to collect a complete set of facts or triples for a kg, especially in open domains. in fact, many valuable valid triples are missing even for the existing wellbuilt large-scale kgs such as freebase @xcite . many researchers devote their efforts to the problem of knowledge graph completion (kgc), the core operation of which is to evaluate the validity of candidate triples. previous work on kgc mainly include two groups, embedding-based methods and classification-based methods. embedding-based models learn embeddings for entities and relations, and evaluate candidate triples based on the embeddings and specific distance metrics. representative models include transe @xcite and its extensions @xcite @xcite , distmult @xcite and complex @xcite . classification-based models learn neural networks to evaluate the validity of candidate triples. representative models include conve @xcite and @xcite . the major advantage of classification-based methods is that they directly model the evaluation of the validity of candidate triples, probably leading to better performance. most of these previous work conducted single-view prediction based on content information, that is, evaluating a candidate triple according to a single distance metric or classification schema, resorting to information restricted in the scope of the candidate triple. we believe that multiple learning views for triple evaluation as well as context information of the candidate triple would contribute to better performance. in this work, we propose for kgc a novel multiview classification model, where multiple classification views are performed to estimate the validity of a candidate triple, based on both content and context information of the triple. there are four classification views for candidate triple evaluation. each of the first three views performs component prediction, where a specific component of the candidate triple is predicted according to the other two components as well as its nearby triples. the last view performs plausibility prediction, where the plausibility of the candidate triple is predicted according to its components as well as its nearby triples. the prediction conditions of these views investigate both content and context information of the candidate triple, that is, the components in the candidate triple, and the triples nearby the candidate triple. the content and context information can be represented as a sub-graph surrounding the candidate triple. these classification views are implemented by a unified neural network with shared embedding and encoding layers and separated prediction layers, and the classification predictions are integrated by a weighted integration procedure for better candidate triple evaluation. in the unified neural network, the sub-graphs indicating the content and context of the candidate triples are encoded in a sequencial manner, by converting the sub-graphs into sequential tree representations. it facilitates the utilization of advanced encoders such as bilstm or transformer. we experiment on two widely used benchmark datasets, fb15k-237 and wn18rr, specific versions of freebase and wordnet. we find that the multi-view model achieves the new state-of-the-art, significantly outperforming pervious work on kgc. we also find that we can promote the efficiency of the multi-view model in realistic applications, by a coarse-to-fine strategy where the first two views are performed to give a list of candidates, and the overall model is then performed to evaluate these candidates. we believe that, the flexibility and the scalability of the multi-view classification model facilitates the introduction of additional information and resources for better performance. ## related work most existing kgc models are based on kg embeddings, which aims at learning distributed representations for entities and relations in a kg. in these models, the candidate triples are evaluated by some specific distance metrics based on the embeddings. these models perform embedding learning with local information in individual triples, including translation-based models @xcite @xcite , semantic matching models @xcite @xcite , and neural network models @xcite @xcite . there also exist kgc models based on classification, where classifiers are learnt to evaluate the validity of candidate triples @xcite . both kinds of previous work consider only one view, with simple distance metrics and classification operations. in contrast, multi-view learning enables the incorporation of much more views that utilize internal and external information for triple evaluation. in recent years, many efforts were devoted to embedding learning based on non-local information such as multi-hop paths @xcite and k-degree neighborhoods @xcite . some researchers also investigated graph embeddings in social network and other areas @xcite @xcite . compared with these work, our method not only learns em- ## method: multi-view classification a knowledge graph kg contains a set of triples indicating facts, {(h, r, t)} ⊆ e × r × e. each triple (h, r, t) consists of two entities h and t referred to the subject and object of the triple, and a relation r referred to the relationship between the two entities. e and r indicates the possible entity set and the possible relation set, respectively. the fundamental problem for kgc is to define a candidate triple evaluation model f : e × r × @xmath1, giving each candidate triple (h, r, t) a score indicting the validity of the triple. ## realization: multi-task architecture we implement the multi-view learning in a multitask architecture, where each sub-task takes charge of a specific learning view. the multi-task learning strategy enables information sharing and interaction between the sub-tasks, thus leading to better performance. ## conclusion we propose a novel multi-view classification model for knowledge graph completion, where multiple classification views are performed based on both content and context information for candidate triple evaluation. the multi-view model is implemented with a simple and unified multi-task learning architecture where the parameters are shared across all the learning views. it achieves the new stateof-the-art although without using any optimization tricks. the multi-view model can be improve from two perspectives in the future. first, the multi-view model can leverage more kinds of information and resources for better performance, such as the descriptions of the entities and relations, as well as related information in external knowledge bases. second, the multi-task learning architecture can introduce different kinds of neural networks to better model different kinds of information, for example, sequential neural networks for sequences and graph neural networks for graphs.