Multilingual Knowledge Graph Completion With Joint Relation and Entity Alignment

We study Multilingual KGC for KGs that associate entities and relations with surface forms from different languages . In such a setting , an entity ( or relation ) may be mentioned as separate IDs in different KGs , necessitating entity alignment ( EA ) and relation alignment ( RA ) . In addition to EA and RA being important subtasks for Multilingual KGC , we posit that high confidence fact predictions may also , in turn , add valuable information for alignment tasks , and vice versa . In response , we present the novel task of jointly training multilingual KGC , EA and RA models . Our approach , ALIGNKGC , uses mBERT-based surface form overlap for EA , and combines it with a KGC approach , extended to the multiple KG setting via a loss term that incentivizes RA . On experiments with DBPedia in five languages , we find that ALIGNKGC achieves up to 17 % absolute MRR improvements in KGC compared to a strong completion model that combines known facts in all languages . It also outperforms an mBERT-only alignment baseline for EA , underscoring the value of joint training for these tasks . 1 . Introduction . A knowledge graph ( KG ) has nodes representing entities and directed edges representing relations between subject and object entities . An entity has a unique node ( ID ) . Relations also have canonical labels such as born-in or works-at , with associated IDs . A KG is usually associated with a single human language — an entity node ( or relation ID ) is associated with one or more surface forms in that language . E.g. , the ID for the country USA may have aliases like “ United States of America '' . KGs are usually very incomplete , as curators struggle to keep up with the real world . KG completion ( KGC ) is thus a strongly motivated problem and studies the prediction of true facts unknown to the KG . While the problem has been intensively researched over the last few years [ Bordes et al. , 2013 , Trouillon et al. , 2016 , Jain et al. , 2018 , Sun et al. , 2019 ] , most KGC research is applicable to only one KG in one language at a time . However , different language speakers would maintain separate KGs in their own languages . Independent completions of each KG may not be optimal , since information from one KG will likely help completion of the other . Each KG will give a different ID , and often also a different surface form , to the same entity , such as “ Estados Unidos de América ” for the USA entity in a Spanish KG . This leads to the problem of ID proliferation . Recent work on entity alignment ( EA ) across KGs attempts to assign a unique ID to all nodes representing the same entity [ Chen et al. , 2017 , Sun et al. , 2017 , 2018 , 2020 , Cao et al. , 2019 , Chen et al. , 2021 , Tang et al. , 2020 ] . A related but under-explored task is relation alignment ( RA ) — assigning synonymous relations in different KGs the same relation ID . We note that RA involves global evidence , because the decision to merge two relations in two languages can have far-reaching consequences to many facts in both KGs . Our key contribution is to recognize and exploit the synergy between multilingual KGC , EA and RA tasks . Entity and relation alignments expose a KGC algorithm to more facts , which can lead to better completion . Conversely , a high-confidence completion can give additional evidence to align its constituent entities and relation . ( See Appendix A for an extended example . ) In this paper , we present ALIGNKGC , a multi-task system that learns to optimize for KGC , EA and RA jointly . ALIGNKGC uses state-of-the-art mBERT-based models for EA and combines it with an extension of the state-of-the-art ComplEx model [ Trouillon et al. , 2016 ] for single KGC . ALIGNKGC uses a novel subject-object signature of each relation , which it represents as a bag of embeddings , and compares them for equivalence and implication between relations . ALIGNKGC uses this information as additional term in its loss function for better RA and multilingual KGC performance . We evaluate ALIGNKGC on slices of DBPedia in five languages . We compare it against two single-KGC systems , one trained over each monolingual KG separately , and the other trained over a union of all KG facts . Compared to these strong baselines , we find that ALIGNKGC achieves substantial accuracy boost due to better entity and relation alignment , obtaining up to 17 % absolute MRR improvements in KGC across languages , around 20 % absolute HITS @ 1 gain for RA for rare relations , around 26 % absolute HITS @ 1 gain for RA for more frequent relations , and around 61 % absolute HITS @ 1 gain for EA . ALIGNKGC also achieves a 5.3 % absolute HITS @ 1 gain over an even stronger EA baseline that uses mBERT embeddings on entity names . These results underscore the value of joint training for these tasks . 2 . Notation and preliminaries . A knowledge graph ( KG ) provides a graph-view to a knowledge-base ( KB ) . A KG consists of entities E and relations ( aka relation types ) R. A KG instance is a triple ( s , r , o ) where s , o ∈ E and r ∈ R. These are all canonical IDs . Each ID is associated with one or more textual aliases . KGC task : For any single KG , training data is provided as ( s , r , o ) triples . A test instance has the form ( s , r , ? ) or ( ? , r , o ) where the system has to predict o or s. Multiple correct values are possible . The evaluation protocol takes a system-ranked list of candidate objects or subjects , and then measures MRR or hits @ K. Many KG embedding and KGC algorithms have been proposed in the last few years . ComplEx [ Trouillon et al. , 2016 ] with all negative instances ( no sampling ) is a simple model that gives near state-of-the-art predictions [ Jain et al. , 2020a ] . It has also formed the basis for many KGC extensions ( e.g. , [ Jain et al. , 2020b ] ) . So we use it as our baseline KGC gadget . ComplEx defines a triple score as f ( s , r , o ) = < ( 〈s , r , o ? 〉 ) , where c ? is complex conjugate , 〈· · · 〉 is a 3-way elementwise inner product and < ( · ) is the real part of a complex number . Using f , ComplEx defines Pr ( o|s , r ) = ef ( s , r , o ) /∑ o′ e f ( s , r , o′ ) , Pr ( s|o , r ) = ef ( s , r , o ) /∑ s′ e f ( s′ , r , o ) ( 1 ) and the log-likelihood KGC loss as LKGC = ∑ ( s , r , o ) ∈KG− log Pr ( o|s , r ) − log Pr ( s|o , r ) . Alignment tasks : We consider a set L = { l1 , l2 , . . . } of languages . KGl represents the KG supported by the language l. An entity in this KG is called el . A relation in this KG is called rl . While trying to improve KGC for target language l′ , KGs in multiple source languages l may be combined . An equivalence between entities el and el′ in KGs of two languages l and l′ is specified as el ≡ el′ , also written as the triple ( el , ≡ , el′ ) . Similarly , an equivalence between relations rl and rl′ is specified as rl ≡ rl′ . Other types of alignment may be possible between relation pairs , such as rl ⇒ rl′ , which means , for all s , o such that ( s , rl , o ) holds , so does ( s , rl′ , o ) . During training , a set of entity equivalences { ( enl , ≡ , enl′ ) : n = 1 , . . . , N } and a set of relation equivalences { ( rml , ≡ , rml′ ) : m = 1 , . . . , M } are revealed to the system between each KGl and KGl′ . The goal of the system is to infer additional entity and relation equivalences . The system is usually called upon to produce a ranked list of equivalences , which is evaluated using HITS @ K or MRR . To achieve KGC enhanced with alignment , the system has to infer additional triples in each KG , making best use of the revealed equivalences , while also inferring additional alignment triples between entities or relations across KGs of different languages . 3 . Proposed methods . 3.1 KGC baselines . When ComplEx is applied to any one KG in isolation , we call this method KGCMONO . Perhaps the most straight-forward way to apply a KGC system across multiple KGs is to compute KGU =⋃ l∈L KGl such that for all revealed entity equivalences { ( el , ≡ , el′ ) } , collapse their node pairs , and rename all equivalent rl , rl′ relation IDs to a common new ID . A standard KGC system can work on the resulting KGU , and we call this method KGCUNION . 3.2 Joint alignment and completion . For both textual and structured inputs , the problem of inferring a predicate or relation as entailed by another has been well-studied [ Lin and Pantel , 2001 , Bhagat et al. , 2007 , Nakashole et al. , 2012 ] . With that work as our point of departure , we ask : How similar are two relations rl , rl′ in KGU ? Comparing their embedding vectors rl , rl′ , as computed by a KGC algorithm such as ComplEx , gives an estimate . However , when training these embeddings , we found that a simple union of facts ( KGCMONO ) did not yield the best results . Instead , guiding the training through an auxiliary loss , computed using the following explicit relation signatures helped the model learn better embeddings . Here s , o are interpreted as canonical IDs ( hence , ‘ hard ’ ) . This will be generalized shortly . Definition 1 ( Hard SO-signature ) . The ( ‘ hard ’ ) subject-object ( SO ) signature of a relation is defined as SO ( r ) = { ( s , o ) : ( s , r , o ) ∈ KGU } . 3.2.1 HARD SO-OVERLAP & JACCARD SIMILARITY . Definition 2 ( SO-overlap ) . The ( hard ) overlap between two relations rl , rl′ is |SO ( rl ) ∩ SO ( rl′ ) | . Jaccard similarity can then be used as a standard symmetric belief that two relations IDs in different languages are equivalent : bJ ( rl ⇔ rl′ ) = |SO ( rl ) ∩ SO ( rl′ ) | |SO ( rl ) ∪ SO ( rl′ ) | . ( 2 ) If bJ ( rl ⇔ rl′ ) exceeds a threshold θ ( tuned using dev set ) , we add ( rl , rl′ ) to set AJ of ‘ silver ’ alignments . We define an additional RA loss term encouraging such embeddings to be aligned : LRA-J = ∑ ( rl , rl′ ) ∈AJ bJ ( rl ⇔ rl′ ) ∥∥rl − rl′∥∥1 . ( 3 ) We call this scheme Jaccard .

Summary: This paper explores how entity alignment (EA), relation alignment (RA), and knowledge graph completion (KGC) can benefit each other. It proposes a model which jointly optimizes the three tasks using a linear combination of their losses. A contribution within this is a method for aligning relations in the absence of gold entity alignments. KGC is evaluated on each of 5 languages for which a KG is available, and EA & RA are evaluated jointly. The paper is easy to follow throughout.