Joint Completion and Alignment of Multilingual Knowledge Graphs (2022)

Abstract: Knowledge Graph Completion (KGC) predicts missing facts in an incomplete Knowledge Graph (KG). Multilingual KGs associate entities and relations with surface forms written in different languages. An entity or relation may be associated with distinct IDs in different KGs, necessitating entity alignment (EA) and relation alignment (RA). Many effective algorithms have been proposed for completion and alignment as separate tasks. Here we show that these tasks are synergistic and best solved together. Our multitask approach starts with a state-of-the-art KG embedding scheme, but adds a novel relation representation based on sets of embeddings of (subject, object) entity pairs. This representation leads to a new relation alignment loss term based on a maximal bipartite matching between two sets of embedding vectors. This loss is combined with traditional KGC loss and optionally, losses based on text embeddings of entity (and relation) names. In experiments over KGs in seven languages, we find that our system achieves large improvements in KGC compared to a strong completion model that combines known facts in all languages. It also outperforms strong EA and RA baselines, underscoring the value of joint alignment and completion.

## introduction a knowledge graph (kg) has nodes representing entities and directed edges representing relations between subject and object entities. in a kg, each entity (and relation) has a unique node (relation) id. a monolingual kg is associated with a single human language -an entity node (or relation id) is associated with one or more surface forms in that language. e.g., the id for the country usa may have aliases like "united states of america". kgs are usually very incomplete, as curators struggle to keep up with the real world. kg completion (kgc) is thus a strongly motivated problem involving the prediction of true facts unknown to the kg. traditional kgc methods @xcite @xcite follow a link prediction paradigm: represent a kg as an abstract graph without entity or relation aliases, and predict missing facts as id tuples. most kgc methods are applicable to only one kg at a time. however, different language speakers would maintain separate monolingual kgs in their own languages. independent completions of each kg may not be optimal, since information from one kg will likely help completion of the other. massive kgs like wikidata cover well-resourced and resource-poor languages. cross-lingual transfer thus acquires increased utility in the context of linked open data initiatives. different kgs will assign a different id, and often also a different surface form, to the same entity, such as "estados unidos de américa" for the usa entity in a spanish kg. this leads to id proliferation. recent work on entity alignment (ea) across kgs attempts to assign a unique id to all nodes representing the same entity @xcite @xcite @xcite @xcite . most recent ea methods use surface forms and possibly other text and attributes associated with each id. a related task is relation alignment (ra) -assigning synonymous relations in different kgs the same relation id. compared to ea, ra is under-explored, particularly in case of multilingual kgs. ra involves global evidence, because the decision to merge two relations in two languages is essentially an act of schema integration, having far-reaching consequences on many facts in both kgs. our key contribution is to recognize and exploit the synergy between multilingual kgc, ea and ra tasks, particularly in the realistically motivated scenario of transferring from multiple source language kgs to a target language kg. entity and relation alignments expose a kgc algorithm to more facts, which can lead to better completion. conversely, a high-confidence completion can give additional evidence to align entities and relation (see extended example in appendix a). we present alignkgc, a multi-task system that learns to optimize for kgc, ea and ra jointly. we develop alignkgc in two settings. our first 'graphonly' setting follows kgc literature, where no surface forms or other textual information is available. this helps in direct comparison against other kgc approaches. here, alignkgc uses a novel subject-object signature of each relation, which it represents as a bag of pairs of entity embeddings. relation pairs are compared for equivalence and implication using maximal matchings between these embedding sets. ra supervision imposes a loss over these maximal matching measurements, which can be reduced via gradient descent on entity embeddings. this ra-based loss term is combined with a state-of-the-art kgc loss. our second 'graphtext' setting additionally employs entity and relation surface forms (when available), as in the standard ea research. here, alignkgc incorporates similarity between surface forms as an additional loss term based on ea (ra) supervision. we evaluate alignkgc on slices of dbpedia collected in the dbp5l @xcite , dbp15k @xcite and openea @xcite ) datasets, covering seven languages. in both graphonly and graphtext settings, align-kgc improves substantially on kgc and ea tasks compared to recent baselines @xcite @xcite , and gives competitive ra performance. these results underscore the value of joint training for these tasks. our implementation is public. 1 a preliminary version of this work was published by @xcite . ## the design of alignkgc perhaps the most straight-forward way to apply a kgc system across multiple kgs is to compute kg @xmath0, ≡, e l )}, collapse their node pairs, and rename all equivalent r l , r l relation ids to a common new id (a transitive closure may be needed to identify equivalence classes of entities or relations). as pointed out in multiple works @xcite @xcite , "pre-aligned pair should share embedding to bridge different kgs. under this assumption, the semantic loss between equivalent entity or relation pair is zero." a standard kgc system, such as complex, can work on the resulting kg u , and we call this method kgcunion. comparing entity and relation embeddings across languages provides a first solution for ea and ra as well. how similar are two relations r l , r l in kg u ? comparing their embedding vectors r l , r l , computed by complex, gives a first estimate. however, we found that this did not yield the best results. instead, guiding the training through an auxiliary loss, computed using the following explicit relation signatures helped the model learn better embeddings. here s, o are interpreted as canonical ids (hence, 'hard'). this will be generalized shortly. definition 1 (hard so-signature). we define the ('hard') subject-object signature of a relation r as ## experiments we study these research questions: 1) does joint kgc, ea and ra (alignkgc) improve upon independent models for each of these tasks? 2) is there cross-lingual transfer of knowledge, for facts mentioned in one language, but queried in another? 3) what are the incremental contributions of soft overlap and text embeddings in the overall performance of alignkgc? 4) how does performance vary with the number of seed alignments (both entity and relation) provided at training time? ## conclusion we presented alignkgc, a system that jointly learns to complete multiple monolingual kgs in different languages and align their entities and relations. to our knowledge, these three tasks have never been unified before. alignkgc operates on the kg constructed by taking a union of all monolingual kgs, and extends kgc models to use novel ea and ra loss terms. in extensive experiments, alignkgc significantly improves kgc accuracy, as well as alignment accuracy on three datasets, underscoring the value of joint alignment and completion.