Abstract: Knowledge graphs have emerged as an important model for studying complex multi-relational data. This has given rise to the construction of numerous large scale but incomplete knowledge graphs encoding information extracted from various resources. An effective and scalable approach to jointly learn over multiple graphs and eventually construct a unified graph is a crucial next step for the success of knowledge-based inference for many downstream applications. To this end, we propose LinkNBed, a deep relational learning framework that learns entity and relationship representations across multiple graphs. We identify entity linkage across graphs as a vital component to achieve our goal. We design a novel objective that leverage entity linkage and build an efficient multi-task training procedure. Experiments on link prediction and entity linkage demonstrate substantial improvements over the state-of-the-art relational learning approaches.

Introduction: Reasoning over multi-relational data is a key concept in Artificial Intelligence and knowledge graphs have appeared at the forefront as an effective tool to model such multi-relational data. Knowledge graphs have found increasing importance due to its wider range of important applications such as information retrieval BIBREF0 , natural language processing BIBREF1 , recommender systems BIBREF2 , question-answering BIBREF3 and many more. This has led to the increased efforts in constructing numerous large-scale Knowledge Bases (e.g. Freebase BIBREF4 , DBpedia BIBREF5 , Google's Knowledge graph BIBREF6 , Yago BIBREF7 and NELL BIBREF8 ), that can cater to these applications, by representing information available on the web in relational format. All knowledge graphs share common drawback of incompleteness and sparsity and hence most existing relational learning techniques focus on using observed triplets in an incomplete graph to infer unobserved triplets for that graph BIBREF9 . Neural embedding techniques that learn vector space representations of entities and relationships have achieved remarkable success in this task. However, these techniques only focus on learning from a single graph. In addition to incompleteness property, these knowledge graphs also share a set of overlapping entities and relationships with varying information about them. This makes a compelling case to design a technique that can learn over multiple graphs and eventually aid in constructing a unified giant graph out of them. While research on learning representations over single graph has progressed rapidly in recent years BIBREF10 , BIBREF6 , BIBREF11 , BIBREF12 , BIBREF13 , BIBREF14 , there is a conspicuous lack of principled approach to tackle the unique challenges involved in learning across multiple graphs. One approach to multi-graph representation learning could be to first solve graph alignment problem to merge the graphs and then use existing relational learning methods on merged graph. Unfortunately, graph alignment is an important but still unsolved problem and there exist several techniques addressing its challenges BIBREF15 , BIBREF16 , BIBREF17 , BIBREF18 in limited settings. The key challenges for the graph alignment problem emanate from the fact that the real world data are noisy and intricate in nature. The noisy or sparse data make it difficult to learn robust alignment features, and data abundance leads to computational challenges due to the combinatorial permutations needed for alignment. These challenges are compounded in multi-relational settings due to heterogeneous nodes and edges in such graphs. Recently, deep learning has shown significant impact in learning useful information over noisy, large-scale and heterogeneous graph data BIBREF19 . We, therefore, posit that combining graph alignment task with deep representation learning across multi-relational graphs has potential to induce a synergistic effect on both tasks. Specifically, we identify that a key component of graph alignment process—entity linkage—also plays a vital role in learning across graphs. For instance, the embeddings learned over two knowledge graphs for an actor should be closer to one another compared to the embeddings of all the other entities. Similarly, the entities that are already aligned together across the two graphs should produce better embeddings due to the shared context and data. To model this phenomenon, we propose LinkNBed, a novel deep learning framework that jointly performs representation learning and graph linkage task. To achieve this, we identify key challenges involved in the learning process and make the following contributions to address them:

Question: What accuracy score do they obtain?
Answer: the best performing model obtained an accuracy of 0.86