Guiding Neural Entity Alignment with Compatibility (2022)

Abstract: Entity Alignment (EA) aims to find equivalent entities between two Knowledge Graphs (KGs). While numerous neural EA models have been devised, they are mainly learned using labelled data only. In this work, we argue that different entities within one KG should have compatible counterparts in the other KG due to the potential dependencies among the entities. Making compatible predictions thus should be one of the goals of training an EA model along with fitting the labelled data: this aspect however is neglected in current methods. To power neural EA models with compatibility, we devise a training framework by addressing three problems: (1) how to measure the compatibility of an EA model; (2) how to inject the property of being compatible into an EA model; (3) how to optimise parameters of the compatibility model. Extensive experiments on widely-used datasets demonstrate the advantages of integrating compatibility within EA models. In fact, state-of-the-art neural EA models trained within our framework using just 5% of the labelled data can achieve comparable effectiveness with supervised training using 20% of the labelled data.

## introduction knowledge graphs (kgs) have been widely used across many natural language processing applications @xcite . however, most kgs suffer from incompleteness which limits their impact on downstream applications. at the same time, different kgs often contain complementary knowledge. this makes fusing complementary kgs a promising solution for building a more comprehensive kg. entity alignment (ea), which identifies equivalent entities between two kgs, is essential for kg fusion. given the two examples kgs shown in fig. 1 , ea aims to recognize two entity mappings donald trump ≡ d.j. trump and fred trump ≡ frederick christ trump. neural ea models are the current state-of-theart for entity alignment @xcite @xcite these methods use pre-aligned mappings to learn an ea model: it encodes entities into informative embeddings and then, for each source entity, selects the closest target entity in the vector space as its counterpart. though significant progress has been achieved, the dependencies between entities, which is the nature of graph data, is underexplored. in an ea task, the counterparts of different entities within one kg should be compatible w.r.t. the underlying dependencies. for example, in fig 1 , the two mappings donald trump ≡ d.j. trump and ivanka trump ≡ melania trump should not co-exist at the same time (i.e. they are incompatible) since "someone's daughter and wife cannot be the same person". on the contrary, donald trump ≡ d.j. trump and fred trump ≡ frederick christ trump are compatible mappings since equivalent entities' father entities should also be equivalent. through an experimental study, we verified that more effective ea models make more compatible predictions (see appendix a for more details). therefore, we argue that making compatible predictions should be one of the objectives of training a neural ea model, other than fitting the labelled data. unfortunately, compatibility has thus far been neglected by the existing neural ea works. to fill this gap, we propose a training framework emea , which exploits compatibility to improve existing neural ea models. few critical problems make it challenging to drive a neural ea model with compatibility: (1) a first problem is how to measure the overall compatibility of all ea predictions. we notice some reasoning rules defined in traditional reasoning-based ea works @xcite can reflect the dependencies between entities well. to inherit their merits, we devise a compatibility model which can reuse them. in this way, we contribute one mechanism of combining reasoning-based and neural ea methods. (2) the second problem is how to improve the compatibility of ea model. compatibility is measured on the counterparts (i.e. labels) sampled from the ea model, but the sampling process is not differentiable and thus the popular approach of regularizing an item in the loss is infeasible. we overcome this problem with variational inference. (3) the third problem lies in optimising the compatibility model, which has interdependencies with the unknown counterparts. we solve this problem with a variational em framework, which alternates updating the neural ea model and the compatibility model until convergence. our contributions can be summarized as: ## related work neural ea. entity alignment is an important task and has been widely studied. neural ea @xcite @xcite ) is current mainstream direction which emerges with the development of deep learning techniques. various neural architectures have been introduced to encode entities. translation-based kg encoders were explored at the start @xcite . though these models could capture the structure information, they were not capable of incorporating attribute information. graph convolutional network (gcn)-based encoders later became the mainstream method because they were flexible in combining different types of information and achieved higher performance @xcite @xcite @xcite . neural ea models rely on pre-aligned mappings for training @xcite . to improve ea effectiveness, semisupervised learning (self-training) was explored to generate pseudo mappings to enrich the training data @xcite . our work aims to complement the existing ea works regardless of their training methods. among previous neural ea methods, some were done on kgs with rich attributes and pay attention to exploiting extra information other than kg structure @xcite @xcite @xcite . alternatively, some others only focused on designing novel models to extract better features from the kg structure @xcite @xcite since structure is the most basic information and the proposed method would be more generic. we evaluate our method by applying it to models that only consider kg structure, which is a more challenging setting. reasoning-based ea. in the reasoning-based ea works @xcite @xcite , some rules are defined based on the dependencies between entities. with the rules, label-level reasoning, i.e. inferring the label of one entity according to other entities' labels instead of its own features, was performed to detect more potential mappings from the pre-aligned ones. functional relation (or attribute), which can only have one object for a certain subject, is critical for some reasoning rules @xcite . one rule example is: ∃r, e 1 , e 1 : r(e 1 , r, e 2 ), r(e 1 , r, e 2 ), r is functional, e 1 ≡ e 1 ⇒ e 2 ≡ e 2 . saïs et al. proposed to combine multiple properties instead of only using a functional property since the combination of several weak properties can also be functional. hogan et al. quantified functional as functionality in a statistical way. @xcite inherited the reasoning ideas from previous works and further transformed the logic rules into a probabilistic form in their work named paris. apart from @xcite , few recent works @xcite verified paris can achieve promising performance. in this work, we reuse one reasoning rule defined in paris because it is very effective and representative. combining reasoning-based and neural ea. one previous work named prase @xcite also explored the combination of neural and reasoning-based ea methods. it used a neural ea model to measure the similarities between entities and fed these similarities to the reasoning method of paris. our work provides a different combination mechanism of these two lines of methods. ## notations & problem definition suppose we have two kgs g and g with respective entity sets e and e . each source entity @xmath0. for simplicity, we denote the counterpart variables of a set e of entities as y e collectively, while use ŷe to represent an assignment of y e . the counterpart variables y l of labelled entities l ⊂ e are already known (i.e. ŷl ). ea aims to solve the unknown variables y u of the unlabelled entities u ⊂ e. one neural ea model measures the similarity s θ (e, e ) between each source entity @xmath1, and infers its counterpart via ŷe = arg max @xmath2, e ). here, θ represents the parameters of the ea model. figure 2 : overview of the emea framework. the two modules are trained iteratively with variational em. the neural ea model θ is then updated using the samples ŷu ∼ q * together with the labelled data. this em process repeats until θ converges. ## further analysis further analysis of emea is done on dataset zh_en. ## conclusion entity alignment is a primary step of fusing different knowledge graphs. many neural ea models have been explored and achieved sota performance. though, one nature of graph data -the dependencies between entities -is under-explored. in this work, we raise attention to one neglected aspect of ea task -different entities have compatible counterparts w.r.t. their underlying dependencies. we argue that making self-consistent predictions should be one objective of training ea model other than fitting the labelled data. towards this goal, we devise one training framework named emea , which can intervene in the update of ea model to improve its compatibility. in emea , we address three key problems: (1) measure compatibility with a graphical model which can aggregate local compatibilities;