M ix TEA : Semi-supervised Entity Alignment with Mixture Teaching (2023)

Abstract: Semi-supervised entity alignment (EA) is a practical and challenging task because of the lack of adequate labeled mappings as training data. Most works address this problem by generating pseudo mappings for unlabeled entities. However, they either suffer from the erroneous (noisy) pseudo mappings or largely ignore the uncertainty of pseudo mappings. In this paper, we propose a novel semi-supervised EA method, termed as MixTEA, which guides the model learning with an end-to-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings. We firstly train a student model using few labeled mappings as standard. More importantly, in pseudo mapping learning, we propose a bi-directional voting (BDV) strategy that fuses the alignment decisions in different directions to estimate the uncertainty via the joint matching confidence score. Meanwhile, we also design a matching diversity-based rectification (MDR) module to adjust the pseudo mapping learning, thus reducing the negative influence of noisy mappings. Extensive results on benchmark datasets as well as further analyses demonstrate the superiority and the effectiveness of our proposed method.

## introduction entity alignment (ea) is a task at the heart of integrating heterogeneous knowledge graphs (kgs) and facilitating knowledge-driven applications, such as question answering, recommender systems, and semantic search @xcite . embeddingbased ea methods @xcite @xcite @xcite dominate current ea research and achieve promising alignment performance. their general pipeline is to first encode the entities from different kgs as embeddings (latent representations) in a uni-space, and then find the most likely counterpart for each entity by performing all pairwise comparison. however, the pre-aligned mappings (i.e., training data) are oftentimes insufficient, which is challenging for supervised embedding-based ea methods to learn informative entity embeddings. this happens because it is time-consuming and labour-intensive for technicians to manually annotate entity mappings in the large-scale kgs. to remedy the lack of enough training data, some existing efforts explore alignment signals from the cheap and valuable unlabeled data in a semi-supervised manner. the most common semisupervised ea solution is using the self-training strategy, i.e., iteratively generating pseudo mappings and combining them with labeled mappings to augment the training data. for example, @xcite propose iptranse which involves an iterative process of predicting on unlabeled data and then treats the predictions above an elaborate threshold (confident predictions) as pseudo mappings for retraining. to further improve the accuracy of pseudo mappings, @xcite design a heuristic editing method to remove wrong alignment by considering one-to-one alignment constraint, while @xcite and @xcite utilize a bi-directional iterative strategy to determine pseudo mapping if and only if the two entities are mutually nearest neighbors of each other. despite the encouraging results, existing semisupervised ea methods still face the following problems: (1) uncertainty of pseudo mappings. prior works have largely overlooked the uncertainty of pseudo mappings during semi-supervised training. revisiting the self-training process, the generation of pseudo mappings is either black or white, i.e., an entity pair is either determined as a pseudo mapping or not. while in fact, different pseudo mappings have different uncertainties and contribute differently to model learning @xcite . (2) noisy pseudo mapping learning. the performance of semi-supervised ea methods depends heavily on the quality of pseudo mappings, while these pseudo mappings inevitably contain much noise (i.e., false positive mappings). even worse, adding them into the training data would misguide the subsequent training process, thus causing error accumulation and further hurting the alignment performance. to tackle the aforementioned limitations, in this paper, we propose a simple yet effective semisupervised ea solution, termed as mixtea. to be specific, our method is based on a teacher-student architecture @xcite , which aims to generate pseudo mappings from a gradually evolving teacher model and guides the learning of a student model with a mixture teaching of labeled mappings and pseudo mappings. we explore the uncertainty of pseudo mappings via probabilistic pseudo mapping learning rather than directly adding "reliable" pseudo mappings into the training data, which lends us to flexibly learn from pseudo mappings with different uncertainties. to achieve that, we propose a bi-directional voting (bdv) strategy that utilizes the consistency and confidence of alignment decisions in different directions to estimate the uncertainty via the joint matching confidence score (converted to matching probability after a softmax). meanwhile, a matching diversity-based rectification (mdr) module is designed to adjust the pseudo mapping learning, thus reducing the influence of noisy mappings. our contributions are summarized as follows: ## problem statement a knowledge graph (kg) is formalized as @xmath0, r, t ), where e and r refer to the set of entities and the set of relations, respectively. ## proposed method in this section, we present our proposed semisupervised ea method, called mixtea, in figure 1 . mixtea follows the teacher-student training scheme. the teacher model is performed to generate probabilistic pseudo mappings on unlabeled entities and student model is trained with an endto-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings. compared to previous methods that require filtering pseudo mappings via thresholds or constraints, the end-to-end training gradually improves the quality of pseudo mappings, and the more and more accurate pseudo mappings in turn benefit ea training. ## conclusion in this paper, we propose a novel semi-supervised ea framework, termed as mixtea, which guides the model learning with an end-to-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings. meanwhile, we propose a bi-directional voting (bdv) strategy and a matching diversity-based rectification (mdr) module to assist the probabilistic pseudo mapping learning. experimental results on benchmark datasets show the effectiveness of our proposed method.