Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer (2020)

Abstract: Predicting missing facts in a knowledge graph(KG) is a crucial task in knowledge base construction and reasoning, and it has been the subject of much research in recent works us-ing KG embeddings. While existing KG embedding approaches mainly learn and predict facts within a single KG, a more plausible solution would benefit from the knowledge in multiple language-specific KGs, considering that different KGs have their own strengths and limitations on data quality and coverage. This is quite challenging since the transfer of knowledge among multiple independently maintained KGs is often hindered by the insufficiency of alignment information and inconsistency of described facts. In this paper, we propose kens, a novel framework for embedding learning and ensemble knowledge transfer across a number of language-specific KGs.KEnS embeds all KGs in a shared embedding space, where the association of entities is captured based on self-learning. Then, KEnS performs ensemble inference to com-bine prediction results from multiple language-specific embeddings, for which multiple en-semble techniques are investigated. Experiments on the basis of five real-world language-specific KGs show that, by effectively identifying and leveraging complementary knowledge, KEnS consistently improves state-of-the-art methods on KG completion.

## introduction knowledge graphs (kgs) store structured representations of real-world entities and relations, constituting actionable knowledge that is crucial to various knowledge-driven applications @xcite @xcite . recently, extensive efforts have been invested in kg embedding models, which encode entities as low-dimensional vectors and capture relations as algebraic operations on entity vectors. these models provide a beneficial tool to complete kgs by discovering previously unknown knowledge from latent representations of observed facts. representative models including translational models @xcite and bilinear models @xcite have achieved satisfactory performance in predicting missing facts. existing methods mainly investigate kg completion within a single monolingual kg. as different language-specific kgs have their own strengths and limitations on data quality and coverage, we investigate a more natural solution, which seeks to combine embedding models of multiple kgs in an ensemble-like manner. this approach offers several potential benefits. first, embedding models of wellpopulated kgs (e.g. english kgs) are expected to capture richer knowledge because of better data quality and denser graph structures. therefore, they would provide ampler signals to facilitate inferring missing facts on sparser kgs. second, combining the embeddings allows exchanging complementary knowledge across different language-specific kgs. this provides a versatile way of leveraging specific knowledge that is better known in some kgs than the others. for example, consider the facts about the oldest japanese novel the tale of genji. english dbpedia @xcite only records its genre as monogatari (story), whereas japanese dbpedia identifies more genres, including love story, royal family related story, monogatari and literature-novel. similarly, it is reasonable to expect a japanese kg embedding model to offer significant advantages in inferring knowledge about other japanese cultural entities such as nintendo and mount fuji. moreover, ensemble inference provides a mechanism to assess the credibility of different knowledge sources and thus leads to a more accurate final prediction. despite the potential benefits, combining predictions from multiple kg embeddings represents a non-trivial technical challenge. on the one hand, knowledge transfer across different embeddings is hindered by the lack of reliable alignment information that bridges different kgs. recent works on multilingual kg embeddings provide support for automated entity matching @xcite @xcite . however, the performance of the state-of-the-art (sota) entity matching methods is still far from perfect @xcite , which may cause erroneous knowledge transfer between two kgs. on the other hand, independently extracted and maintained languagespecific kgs may inconsistently describe some facts, therefore causing different kg embeddings to give inconsistent predictions and raising a challenge to identifying the trustable sources. for instance, while the english dbpedia strictly distinguishes the network of a tv series (e.g. bbc) from its channel (e.g. bbc one) with two separate relations, i.e., network and channel, the greek dbpedia only uses channel to represent all of those. another example of inconsistent information is that chinese dbpedia labels the birth place of the ancient chinese poet li bai as sichuan, china, which is mistakenly recorded as chuy, kyrgyz in english dbpedia. due to the rather independent extraction process of each kg, such inconsistencies are inevitable, calling upon a reliable approach to identify credible knowledge among various sources. in this paper, we propose kens (knowledge ensemble), which, to the best of our knowledge, is the first ensemble framework of kg embedding models. fig. 1 gives a depiction showing the ensemble inference process of kens. kens seeks to improve kg completion in a multilingual setting, by combining predictions from embedding models of multiple language-specific kgs and identifying the most probable answers from those prediction results that are not necessarily consistent. experiments on five real-world language-specific kgs show that kens significantly improves sota fact prediction methods that solely rely on a single kg embedding. we also provide detailed case studies to interpret how a sparse, low-resource kg can benefit from embeddings of other kgs, and how exclusive knowledge in one kg can be broadcasted to others. ## related work we hereby discuss three lines of work that are closely related to this topic. monolingual kg embeddings. monolingual kg embedding models embed entities and relations in a low-dimensional vector space and measure triple plausibility using these vectors. translational models assess the plausibility of a triple (h, r, t) by the distance between two entity vectors h and t, after applying a relation-specific translation vector r. the representative models include transe @xcite and its extensions transd @xcite . despite their simplicity, translational models achieve satisfactory performance on kg completion and are robust against the sparsity of data @xcite . rotate @xcite employs a complex embedding space and models the relation r as the rotation instead of translation of the complex vector h toward t, which leads to the sota performance on kg embedding. there are also various methods falling into the groups of bilinear models such as rescal @xcite and distmult @xcite , as well as neural models like hole @xcite and conve @xcite . due to the large body of work in this line of research, we only provide a highly selective summary here. interested readers are referred to recent surveys @xcite ) for more information. ## method in this section, we introduce kens, an embeddingbased ensemble inference framework for multilingual kg completion. kens conducts two processes, i.e. embedding learning and ensemble inference. the embedding learning process trains the knowledge model that encodes entities and relations of every kg in a shared embedding space, as well as the alignment model that seizes the correspondence in different kgs and enables the projection of queries and answers across different kg embeddings. the ensemble inference process combines the predictions from multiple kg embeddings to improve fact prediction. particularly, to assess the confidence of predictions from each source, we introduce a boosting method to learn entity-specific weights for knowledge models. ## experiments in this section, we conduct the experiment of fact prediction by comparing kens variants with various kg embeddings. we also provide a detailed case study to help understand the principle of ensemble knowledge transfer. ## conclusion in this paper, we have proposed a new ensemble prediction framework aiming at collaboratively predicting unseen facts using embeddings of different language-specific kgs. in the embedding space, our approach jointly captures both the structured knowledge of each kg and the entity alignment that bridges the kgs. the significant performance improvements delivered by our model on the task of kg completion were demonstrated by extensive experiments. this work also suggests promising directions of future research. one is to exploit the potential of kens on completing low-resource kgs, and the other is to extend the ensemble transfer mechanism to population sparse domain knowledge in biological @xcite and medical knowledge bases @xcite . pariticularly, we also seek to ensure the global logical consistency of predicted facts in the ensemble process by incorporating probabilistic constraints @xcite .