Joint Multilingual Knowledge Graph Completion and Alignment (2022)

Abstract: Knowledge graph (KG) alignment and completion are usually treated as two independent tasks. While recent work has leveraged entity and relation alignments from multiple KGs, such as alignments between multilingual KGs with common entities and relations, a deeper understanding of the ways in which multilingual KG completion (MKGC) can aid the creation of multilingual KG alignments (MKGA) is still limited. Motivated by the observation that structural inconsistencies – the main challenge for MKGA models – can be mitigated through KG completion methods, we propose a novel model for jointly completing and aligning knowledge graphs. The proposed model combines two components that jointly accomplish KG completion and alignment. These two components employ relation-aware graph neural networks that we propose to encode multi-hop neighborhood structures into entity and relation representations. Moreover, we also propose (i) a structural inconsistency reduction mechanism to incorporate information from the completion into the alignment component, and (ii) an alignment seed enlargement and triple transferring mechanism to enlarge alignment seeds and transfer triples during KGs alignment. Extensive experiments on a public multilingual benchmark show that our proposed model outperforms existing competitive baselines, obtaining new state-of-the-art results on both MKGC and MKGA tasks.

## introduction knowledge graphs (kgs) represent facts about real-world entities as triples of the form (head_entity, relation_type, tail_entity). kgs are widely used in numerous applications and research domains such as dialogue systems @xcite , machine translation @xcite , and electronic medical records @xcite . almost all kgs, however, even those at the scale of billions of triples, are far from being complete. this motivates kg completion (kgc) which aims to derive missing triples from an incomplete knowledge graph @xcite @xcite @xcite @xcite @xcite . most popular kgs such as yago @xcite , babelnet @xcite , and dbpedia @xcite , are multilingual, that is, they contain sets of triples constructed from sources in different languages. fortunately, these kgs often complement each other since a kg in one language might be more comprehensive in some domains compared to a kg in a different language, and vice versa, while still sharing a large number of entities and relation types @xcite . especially kgs for low-resource languages could benefit from triples contained in kgs for high-resource languages. although numerous approaches to kgc have been proposed in recent years @xcite @xcite , most of these only operate on one kg at a time. treating kgs independently, however, might lead to poor performance due to the sparseness of low-resource languages. motivated by this observation, some methods have tried to improve multilingual kg completion (mkgc) using multilingual kg alignment (mkga) @xcite @xcite . the alignment problem is challenging due to the varying levels of completeness of monolingual kgs @xcite . the resulting structural inconsistency between kgs leads to the problem of corresponding entities in two kgs having vastly different embeddings @xcite . figure 1 illustrates that, in principle, mkgc and mkga should be mutually beneficial. given the alignment seed set {(e, e * ), (d, d * ), (b, b * )}, the task here is to find corresponding entities for a and c which are a * and c * , respectively. the two kgs share a similar structure but the triple (b * , r4, a * ) in kg * has no corresponding triple in kg. this causes the embeddings of entities a and a * to differ and, thus, makes it difficult to identify the alignment between a and a * . indeed, a is more likely to be aligned to c * as they share a comparable local structure (similar degree and 1-hop neighbor set). thus, completing missing triples is crucial to improve the alignment quality. indeed, if one aligned a to a * , one could recover (b, r4, a) by transferring (b * , r4, a * ) from kg * . motivated by these observations, we propose jmac, a method for joint multilingual kg completion and alignment, consisting of two interdependent completion and alignment components. both components employ relation-aware graph neural networks (gnns) to encode multihop neighborhood information into entity and relation embeddings. the completion component is trained to reconstruct missing triples using the transe translation-based loss @xcite and an additional loss term that incorporates information about the already known alignments. while we learn separate embeddings for the alignment and completion components, the embeddings of the completion component are used within the alignment component, mitigating the aforementioned problem of structural inconsistencies. in addition, we propose a mechanism for estimating the alignment entropy which is used to adaptively and iteratively grow the alignment seed set. finally, we also propose a method for transferring triples based on the currently derived alignments. our contributions are as follows: ## problem definition and related work let @xmath0, r, t ) denote a kg, where e, r and t denote the sets of entities, relations, and triples, respectively. a triple (e h , r, e t ) ∈ t is an atomic unit, which represents some relation @xmath1. ## jmac: joint multilingual alignment and completion figure 2 illustrates the architecture of jmac which consists of the completion and alignment components, respectively. each component uses a relation-aware graph neural network (gnn) to encode multi-hop neighborhood information into entity and relation embeddings. the use of two gnn encoders is beneficial as embeddings that are suitable for the alignment might differ from those that are most beneficial for the completion problem.