Deep Reinforcement Learning for Entity Alignment (2022)

Abstract: Embedding-based methods have attracted increasing attention in recent entity alignment (EA) studies. Although great promise they can offer, there are still several limitations. The most notable is that they identify the aligned entities based on cosine similarity, ignoring the semantics underlying the embeddings themselves. Furthermore, these methods are shortsighted, heuristically selecting the closest entity as the target and allowing multiple entities to match the same candidate. To address these limitations, we model entity alignment as a sequential decision-making task, in which an agent sequentially decides whether two entities are matched or mismatched based on their representation vectors. The proposed reinforcement learning (RL)-based entity alignment framework can be flexibly adapted to most embedding-based EA methods. The experimental results demonstrate that it consistently advances the performance of several state-of-the-art methods, with a maximum improvement of 31.1% on Hits@1.

## introduction entity alignment (ea) is one of the most crucial tasks in knowledge graph (kg) studies. it aims to seek the potentially aligned entity pairs between two kgs, such that distributed knowledge can be linked for better supporting downstream applications. generally, a fact in a kg can be represented by a triplet (e 1 although recent eea methods @xcite @xcite @xcite @xcite have made great performance improvement, they rarely consider the evaluation process. for example, in figure 1 , all three films are directed by jules white and have similar casts. this makes the eea methods confused to discriminate the true aligned entities from other candidates. current ranking strategy heuristically chooses the nearest entities without considering that some entities have already been matched before. an entity with the largest similarity is not always the true target, especially when this candidate has been matched with other entities. in contrast, we can model entity alignment as a sequential decisionmaking task, where the agent sequentially decides whether a candidate embedding is aligned with the input one. then, the environment will exclude the matched candidates in the subsequent decisions. one issue with the sequential strategy is the accumulated errors. due to the heterogeneity of kgs, a pair of underlying aligned entities may not share an identical neighborhood. this makes their embeddings not as similar as desired with each other (e.g., "guns a poppin!" in figure 1 ). but semantics in the embeddings may still indicate the actual target. it is worth estimating the alignment score directly from their embeddings. it is also important to negate a most likely candidate for maximizing the long-term rewards. in this paper, we draw on the insights of reinforcement learning (rl) that has recently received great attention in many fields @xcite @xcite . with the trained embeddings of any existing eea models as raw input, we train an agent to find as many alignment pairs as possible to maximize the reward. meanwhile, we adopt a curriculum learning (elman, 1993) strategy for the environment to provide candidate entity pairs as observations of increasing difficulty. in sum, our contributions are three-fold: ## experiment we conducted experiments to verify the effectiveness of the proposed rl-based framework. the trained entity embeddings were obtained from the openea project foot_1 . ## conclusion and future work in this paper, we proposed an rl-based entity alignment framework, which can advance most existing eea methods without modifying their parameter settings or infrastructures. our experiments demonstrate consistent and significant improvement on all baseline methods. we plan to study how to jointly train eea methods and rlea for further improvement in future work.