An Industry Evaluation of Embedding-based Entity Alignment (2020)

Abstract: Embedding-based entity alignment has been widely investigated in recent years, but most proposed methods still rely on an ideal supervised learning setting with a large number of unbiased seed mappings for training and validation, which significantly limits their usage. In this study, we evaluate those state-of-the-art methods in an industrial context, where the impact of seed mappings with different sizes and different biases is explored. Besides the popular benchmarks from DBpedia and Wikidata, we contribute and evaluate a new industrial benchmark that is extracted from two heterogeneous knowledge graphs (KGs) under deployment for medical applications. The experimental results enable the analysis of the advantages and disadvantages of these alignment methods and the further discussion of suitable strategies for their industrial deployment.

## introduction knowledge graphs (kgs), such as dbpedia @xcite , wikidata @xcite and yago @xcite are playing an increasingly important role in various applications such as question answering and search engines. the construction of kgs usually includes several components, such as named entity recognition (ner) @xcite , relation extraction (re) @xcite , and knowledge correction @xcite . however, the content of an individual kg is often incomplete, leading to a limited knowledge coverage especially in supporting applications of a specific domain @xcite . one widely adopted solution is to merge multiple kgs (e.g., an enterprise kg with fine-grained knowledge of a specific domain and a general-purpose kg with an extensive coverage) with the assistance of an alignment system which discovers cross-kg mappings of entities, relations, and classes @xcite . embedding-based entity alignment has recently attracted more attention due to the popularity of kgs with big data (i.e. a large number of facts) such as wikidata. traditional alignment systems such as paris @xcite and logmap @xcite , which usually reply on lexical matching and semantic reasoning (e.g., for checking the violation of relation domain and range), are believed to be weak in utilizing the contextual semantics especially the graph structure of such large kgs. to address this problem, some novel embedding-based methods have been proposed with the employment of different kg embedding methods such as transe @xcite and graph neural networks (gnns) @xcite as well as some algorithms from active learning @xcite , multi-view learning @xcite and so forth. we find all these embedding-based entity alignment methods rely upon seed mappings for supervision or semi-supervision in training. they are usually evaluated by benchmarks extracted from dbpedia, wikidata and yago, all of which are constructed from the same source, namely wikipedia. these methods typically build their models with 30% (or even higher) of all the ground-truth mappings, and the training and validation sets are randomly extracted, sharing the same distribution as the test set.