MixTEA: Semi-supervised Entity Alignment with Mixture Teaching
Authors: Feng Xie
Xin Song
Xiang Zeng
Xuechen Zhao
Lei Tian
Bin Zhou
Yusong Tan
Venue: EMNLP 2023 Findings (2023)
Keywords: Knowledge Graph
Entity Alignment
Knowledge Representation

Abstract: Semi-supervised entity alignment (EA) is a practical and challenging task because of the lack of adequate labeled mappings as training data. Most works address this problem by generating pseudo mappings for unlabeled entities. However, they either suffer from the erroneous (noisy) pseudo mappings or largely ignore the uncertainty of pseudo mappings. In this paper, we propose a novel semi-supervised EA method, termed as MixTEA, which guides the model learning with an end-to-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings. We firstly train a student model using few labeled mappings as standard. More importantly, in pseudo mapping learning, we propose a bi-directional voting (BDV) strategy that fuses the alignment decisions in different directions to estimate the uncertainty via the joint matching confidence score. Meanwhile, we also design a matching diversity-based rectification (MDR) module to adjust the pseudo mapping learning, thus reducing the negative influence of noisy mappings. Extensive results on benchmark datasets as well as further analyses demonstrate the superiority and the effectiveness of our proposed method.

MixTEA: Semi-supervised Entity Alignment with Mixture Teaching
Feng Xie, Xin Song, Xiang Zeng, Xuechen Zhao, Lei Tian, Bin Zhou ( Yusong Tan College of Computer, National University of Defense Technology (xiefeng, songxin, zengxiang, zhaoxuechen, lei tian' 29, ystannenuddttttt. edu. songxin,zengxiing binzhou@nudt. edu. cn
Abstract
training data) are oftentimes insufficient, which is challenging for supervised embedding- EA methods to learn informative entity embeddings. This happens because it is time-consuming and labour-intensive for technicians to manually anno- tate entity mappings in the large-scale KGs.
Semi-supervised entity alignment (EA) is practical and challenging task because the lack of adequate labeled mappings training data. Most works address this problem by gen- erating pseudo mappings for unlaheled enti- ties. However, they either suffer from the er- roncous (moisy) pscudo mappings or largely ignore the uncertainty of pseudo mappings. In this paper, we propose novel semi-supervised EA method. termed as MixTEA, which guides the model learning with end-to-end mix- ture teaching of manually labeled mappings and probabilistic pseudo mappings. We firstly train student model using few labeled map- pings as standard. More importantly, in pseudo mapping learning, we propose bi-directional voting (BDV) strategy that fuses the alignment decisions different directions to estimate uncertainty via the joint matching confidence score. Mcanwhile, we also design matching diversity-bal baasei based rectification (MDR) module to adjust the pseudo mapping learning, thus reduc- ing the negative influence noisy mappings. Extensive results benchmark datasets well further analyses demonstrate the superiority and the effectiveness our proposed method.
To remedy the lack of enough training data, some existing efforts explore alignment signals from the cheap and valuable unlabeled data in a semi-supervised manner. The most common semi- supervised EA solution is using the self-training strategy, i.c., iteratively generating pseudo map- pings and combining them with labeled mappings to augment the training data. For example, Zhu et al. (2017) propose IPTransE. which involves an iterative process of predicting on unlabeled data and then treats the predictions above an elaborate threshold (confident predictions) pseudo map- pings for retraining. To further improve the curacy of pseudo mappings, Sun et al. (2018) de- sign a heuristic editing method to remove wrong alignment by considering one-to-one alignment constraint, while Mao et al. (2020) and Cai et al. (2022) utilize bi-directional iterative strategy to determine pseudo mapping if and only if the two en- tities are mutually nearest neighbors of each other. Despite the encouraging results, existing semi- supervised EA methods still face the following problems: (1) Uncertainty of pseudo mappings. Prior works have largely overlooked the uncer- tainty of pseudo mappings during semi-supervised training. Revisiting the self-training process, the generation of pseudo mappings is either black or white, i.e., an entity pair is either determined as pseudo mapping not. While in fact, different pseudo mappings have different uncertainties and contribute differently model learning (Zheng and Yang, 2021). (2) Noisy pseudo mapping learning. The performance semi-supervised EA methods depends heavily on the quality of pscudo mappings, while these pseudo mappings inevitably
1 Introduction
Entity alignment (EA) is task at the heart of inte- grating heterogeneous knowledge graphs (KGs) and facilitating knowledge-driven applications, such as question answering. recommender systems, and semantic search (Gao et al., 2018). Emhedding- based EA methods (Chen et al., 2017; Wang et al., 2018; Sun et al., 2020a; Yu et al., 2021; Xin et al., 2022a) dominate current EA research and achieve promising alignment performance. Their general pipeline first encode the entities from different KGs as embeddings (latent representations) in uni-space, and then find the most likely counterpart for each entity by performing all pairwise com- parison. However, the pre-aligned mappings (i.e.,

---

contain much noise (i.e., False Positive mappings), Even worse, adding them into the training data would misguide the subsequent training process, thus causing error accumulation and further hurting the alignment performance.
based EA approaches obtain promising results. Among them, some early studies (Chen et al., 2017; Sun et al., 2017) are based on the knowledge em- bedding methods, in which entities are embedded by exploring the fine-grained relational semantics. For example, MTTrnnEE (Chen et al., 2017) applies TransE (Bordes et al., 2013) as the KG encoder to embed different KGs into independent vector spaces and then conducts transitions via designed alignment modules. However, they need care- fully balance the weight between the encoder and alignment module in one unified optimization prob- lem. Due to the powerful structure learning capa- bility, Graph Neural Networks (GNNs) like GCN (Kipf and Welling, 2017) and GAT (Veličković et al., 2018) have been employed as the encoder with Siamese architecture (i.e., parrmmeer for many embedding- based models. GCN-Align (Wang et al., 2018) applies Graph Convolution Net- work (GCN) for the first time to capture neighbor- hood information and embed entities into a unified vector space, but it suffers from the structural het- erogeneity of different KGs. To mitigate this issue and improve the structure learning, AliNet (Sun et al., 2020a) adopts multi-hop aggregation with gating mechanism to expand neighborhood ranges for better structure modeling, and KE-GCN (Yu et al., 2021) combines GCN and knowledge em- bedding methods to jointly capture the rich struc- tural features and relation semantics of entities. More recently, IMEA (Xin et al., 2022a) designs Transformer-lik architecture to encode multiple structural contexts in KG while capturing align- ment interactions across different KGs.
To tackle the aforementioned limitations, in this paper, we propose simple yet effective semi- supervised EA solution, termed MixTEA. To be specific, our method is based Teacher-Student architecture (Tarvainen and Valpola, 2017), which aims to generate pseudo mappings from gradu- ally evolving teacher model and guides the learn- ing of student model with a mixture teaching of labeled mappings and pseudo mappings. We explore the uncertainty of pseudo mappings via probabilistic pseudo mapping learning rather than directly adding "reliable" pseudo mappings into the training data, which lends us to flexibly learn from pseudo mappings with different uncertainties. To achieve that, we propose a bi-directional voting (BDV) strategy that utilizes the consistency and confidence of alignment decisions in different di- rections to estimate the uncertainty via the joint matching confidence score (converted to matching probability after softmax). Meanwhile, a match- ing diversity-based rectification (MDR) module is designed adjust the pseudo mapping learning, thus reducing the influence of noisy mappings. Our contributions are summarized as follows:
(I) We propose a novel semi-superre supervssee EA framework, termed as MiXTEA', which guides the model's 's aligmment learning with an end-to-end mix- ture teaching of manually labeled mappings and probabilistic pseudo mappings.
(II) We introduce bi-directional voting (BDV) strategy which utilizes the alignment decisions in different directions to estimate the uncertainty of pseudo mappings and design a matching diversity- based rectification (MDR) module to adjust the pseudo mapping learning, thus reducing the nega- tive impacts of noise mappings.
In addition, some works further improve the EA performance by introducing the side information about entities, such as entity names (Zhang et al., 2019), attributes (Liu et al., 2020), and literal de- scriptions (Yang al.. 2019). Afterward, series of methods were proposed to integrate knowledge from different modalities (c.g., relational, visual, and numerical) to obtain joint entity representation for ÉA (Chen et al., 2020; Liu et al., 2021; Lin et al., 2022). However, these discriminative fea- tures are usually hard to collect, noise polluted, and privacy sensitive (Pei et 2022).
(III) We conduct extensive experiments and thor- ough analyses on benchmark datasets OpenEA (Sun et al., 2020b). The results demonstrate the su- periority and effectiveness of our proposed method.
2 Related Works
2.1
Embedding-based Entity Alignment
2.2 Semi-supervised Entity Alignment
While the recent years have witnessed the rapid de- velopment of deep learning techniques, embedding-
Since the manually labeled mappings used for train- ing are usually insufficient, many supi-vupee EA methods have been proposed to take advan-
Phttps://gitte htttpss/ggithuu..c///effegggggg

---

tage of labeled mappings and the large amount of unlabeled data for alignment, which can pro- vide a more practical solution real scenarios. The mainstream solutions focus on iteratively gen- erating pseudo mappings to compensate for the lack of training data. IPTransE (Zhu et al., 2017) applies threshold filtering-baac -ased self-training to yield pseudo mappings but it fails to obtain sat- isfactory performance since it brings much noise data, which would misguide the subsequent train- ing. Besides, it is also hard to determine an ap- propriate threshold to select "confident" pscudo mappings. KDCoE (Chen et al., 2018) performs co-training of KG embedding model and literal de- scription embedding model to gradually propose new pseudo mappings and thus enhance the super- vision of alignment learning for each other. To further improve the quality of pseudo mappings. BootEA (Sun al., 2018) designs an editable strat- egy based on the one-to-one matching rule to deal with matching conflicts and MRAEA (Mao et al., 2020) proposes a bi-directional iterative strategy which imposes a mutually nearest neighbor con- straint. Inspired by the success of self-training, ielation-based
relationship, entity alignment (EA) task pairs each source entity e; & via nearest neighbor (NN)
search to identify its corresponding target entity ej E &
ej = arg min d(ei, Ej) ofcE,
(1)
where d(-) denotes distance metrics (e.g,, Manhat- tan or Euclidean distance). Moreover, to mitigate the inadequacy of training data, semi-supervised EA methods make effort to explore more potential alignment signals over the vast unlabeled entities, i.e., &g and & which denote the unlabeled entity set of source KG and target KG, respectively.
Proposed Method
In this section, we present our proposed semi- supervised EA method, called MixTEA, in Figure 1. MixTEA follows the teacher-student training scheme. The teacher model is performed to gen- erate probabilistic pseudo mappings on unlabeled entities and student model is trained with an end- to-end mixture teaching of manually labeled map- pings and probabilistic pscudo mappings. Com- pared to previous methods that require filtering pseudo mappings via thresholds or constraints, the end-to-end training gradually improves the quality of pscudo mappings, and the more and more accu- rate pseudo mappings in turn benefit EA training.
RANM (Cai et al., 2022) proposes adaptive neighborhood matching method for entity alignment and combines a bi-directional iterative co-training strategy, making become a natural semi- supervised model. Moreover, CycTEA (Xin et al., 2022b) devises an effective ensemble framework to enable multiple alignment models (called aligners) to exchange their reliable entity mappings for more robust semi-supervised training, but it requires high complementarity among different aligners.
4.1 KG Encoder
We first introduce the KG encoder (denoted as [(; 0)) which utilizes neighborhood structures and relation semantics to embed entities from differ- ent KGs into unified vector space. We ran- domly initialize the trainable entity embeddings Hent E [[() xde and relation embeddings Hre E IR RR....ii where de and dp are the di- mension of entities and relations, respectively.
Additionally, other effective semi cemi-supervised EA methods, such as SEA (Pei et al., 2019), RAC (Zeng et al., 2021), GAEA (Xic et al., 2023), focus on introducing specific loss terms (e.g., reconstruc- tion loss, contrastive loss) via auxiliary tasks.
Structure modeling, Structural features are cru- cial since equivalent entities tend to have similar neighborhood contexts. Besides, leveraging multi- range neighborhood structures is capable of pro- viding more alignment evidence and mitigating the structural heterogeneity issue. In this work, we apply Graph Attention Network (GAT) (Veličković et al., 2018) to allow an entity to selectively ag- gregate its surrounding information via attentive mechanism and we then recursively capture multi- range structural features by stacking I. layers:
3 Problem Statement
A knowledge graph (KG) is formalized as G (E, R, T), where E and R refer to the set of entities and the set of relations, respectively. T - E x R x E = [(h,r,t)hh, E & Ar E R} is the set of triples, where h, т, and t denote head entity (subject), relation, tail entity (object), respectively. Given source KG G. (E,, R. Ta), a target KG G. = (Es, RR, Tt), and a small set of pre-aligned mappings (called training data) S = {(s,,e))ee, E E E E,ee, = et), where = means equivalence
h! (1) a (Enjexe, oiy w,nhh-)
(2)

---

Labeled Mappings
Final objective: man La + XCu
Entity embboddings
9
Student [1:8ma)
sim
Cross-Entropy
Source KG
M
G
Teacher Encoder [(:Beaa)
sim(
BDV
MDR
Target KG
M
pecudo mappings
Figure 1: The overall of our proposed MixTEA, which consists of two KG encoders, called student model and teacher model. We obtain entity embeddings via the KG encoder. Both labeled mappings and probabilistic pseudo mappings used to train the student model. The final student model is used for alignment inference.
exp(or(a be, Wob,,l)) Oijj E CN. exp(o(a [ he + W W)be.))
catenate intermediate features for entity e; to obtain the final entity representation:
exp(ww) Σ exp(w)
where T represents transposition, means con- catenation, Wg and a are the layer-specific trans- formation parameter and attention transformation vector, respectively. Ne, means the neighbor set of e; (including e; itself by adding a self-connection). and æij indicates the learned importance of entity Ej to entity 6₁. H denotes the entity embedding matrix at 1-th layer with (0) Hent (-) is the nonlinear function and use ELU here.
he
h
(6)
KEK
where K = (1), (2), and w E R is the trainable attention vector adaptively con- trol the flow of each feature. We feed to soft- max before multiplication ensure that the nor- malized weights sum 1.
4.2 Alignment Learning with Mixture Teaching
Relation modeling. Relation-leve information which carries rich semantics vital to align enti- ties in KGs because two equivalent entities may share overlapping relations. Considering that rela- tion directions, i.e., outward (e; + ej) and inward (e; Cz), have delicate impacts on characteriz- ing the given target entity e;, aggregators gather outward and inward relation semantics separately to provide supplementary fea- tures for heterogeneous KGs:
In the following, we will introduce mixture teach- ing, which reached by the supervised alignment learning and probabilistic pseudo mapping learning in an end-to-end training manner.
Teacher-student architecture. Following Mean Teacher (Tarvainen and Valpola, 2017), we build our method which consists of two KG encoders with identical structure, called student model [ O Ostu) and teacher model (( respectively. The student model constantly updates its parame- ters supervised by the manually labeled mappings as standard and the teacher model is updated via the exponential moving average (EMA) (Tarvainen and Valpola, 2017) weights of the student model. Moreover, the student model also learns from the pseudo mappings generated by the teacher model to further improve its performance, in which the uncertainty of pseudo mappings is formalized calculated matching probabilities. Specifically, we update the teacher model as follows:
two mean
Σ
hrd
(4)
NE
rENT
Σ
brs
(5)
NE
rENT
where + and N are the of outward and inward relations of entity e, respectively. Weighted concatenation. After capturing the contextual information of entities in terms neigh- borhood structures and relation semantics, we con-
Itea f mflea + (1 m m)8au m E (0,1) (7)

---

where 0 denotes model weights, and m is pre- set momentum hyperparameter that controls the teacher model to update and evolve smoothly.
unlabeled entiifies
pairwiie similarity
pocudo mappings
bi-directional voling
matching diversity-based rectification
ST
msiching coufiicss
Supervised alignment learning. In order to make cquivalent entities close to each other and unmatched entities pull away from each other in unified space, we apply margin- margin-based alignment loss (Wang et al., 2018; Mao et al., 2020; Yu et al., 2021) supervised by pre-aligned mappings:
M
0909E
innepesition
Pi
Figure 2: The illustration of the process of generating the probabilistic pseudo mapping matrix from two align- ment directions. We assume that entity pairs on the diagonal correct mappings and that the default align- ment direction for inference is from source to target.
La Σ
Σ
ll
b2
(Es,et)ES (Pg.5))E8
(8)
+ p lha ba.lla]+
where P is hyperparameter of margin, [x]]
predicted similarity in each row the pseudo map- ping and then we combine the results of the pseudo mappings different directions weighted by their last Hir@ scores on validation data to obtain the final pseudo mapping matrix:
max (0, 2) is ensure non-negative output, S de- notes the set of negative entity mappings, and ||- means L2 distance (Euclidean distance). Negative mappings are sampled according to the cosine sim- ilarity of two entities (Sun et al., 2018).
Probabilistic pseudo mapping learning. As mentioned above, the teacher model responsible for generating probabilistic pseudo mappings for the student model to provide more alignment sig- nals and thus enhance the alignment performance. Benefiting from the EMA update, the predictions of the teacher model can be seen as an ensemble ver- sion of the successive student models' predictions. Therefore it is more robust and stable for pseudo mapping generation. Moreover, bi-directional iter- ative strategy (Mao et al., 2020) reveals the asym- metric nature of aligmment directions (i.e., source- to-target and target-to-source which can produce pscudo mappings based on the mutually nearest neighbor constraint. Inspired by this, we propose a bi-directional voting (BDV) strategy which fuses alignment decisions in each direction to yield more comprehensive pscudo mappings and model their uncertainty via the joint matching confidence score, Concretely, after encoding, we can first obtain the similarity matrix by performing pairwise similarity calculation between the unlabeled source and target entities as follows:
pica = -B.g + 31 (( T (10)
valid(M
8 (M )-5 + ( tea
(11)
{
1, if argmax, M.ji
g(M) [ma,]]
O, otherwise
(12)
where g(.) is the function that converts the similar- ity matrix to one-hot matrix (i.e., the only position with value each row of the matrix indicates the pseudo mapping). In this manner, we arrive at the final pseudo mapping matrix plea generated by the teacher model, in which each pseudo-mapping is associated with a joint matching confidence score (the higher the joint matching confidence, the less the uncertainty). Different from the bi-directional iterative strategy, we use the voting consistency and matching confidence of alignment decisions in different directions facilitate uncertainty esti- mation. Specifically, given an entity pair (ê,, éÿ), its confidence plea when and only when both directions unanimously vote this entity pair as a pseudo mapping, otherwise its confidence in the interval (0.1) when only one direction votes for it and 0 when no direction votes for it (i.e., this entity pair will not be regarded In addition, the ideal predictions of EA need to satisfy the one-to-one matching constraint (Suchanek et al., 2011; Sun al., 2018), i.e., a source entity can be matched with at most one target entity, and vice versa. However, the joint
M Sim Otea) E
IR Ex &dl
where sim(-) denotes cosine similarity function. Micg and M represent similarity matrices in different directions between source and target en- tities, and M is the transposition of M > (i.e.,
pseudo mapping).
M = (M))) Next, for each matrix, we pick up the entity pair which has the maximum

---

decision voting process inevitably yields match- ing conflicts due the existence of erroneous (noisy) mappings. Inspired by Gal et al. (2016), we further propose a matching liversity-based rec- tification (MDR) module to adjust the pseudo map- ping learning, thus mitigating the influence of noisy mappings dynamically. We denote M (i.e.. M sim(êi, ēy; Ostu)) as the similarity matrix calculated based on the student model and define a Cross- Entso-y (CE) loss between M and piea rectified by matching diversity:
the entities thereof follow the degree distribution in real-world KGs. The brief information of ex- perimental data is shown Table (Appendix B). It contains two cross- loogull settings, i.e., EN-FR- I5K (Enggish-to-Frenchh) and EN-DE-15K (English- to to-German), and two monolingual settings, i.c., D-W-15K Srieddiatt-Wikiddttt and D-Y-15K (DBPedia-to-YACEe OpenEA,
Following the data splits in the same split setting where 20%, 10%, and 70% pre-aligned mappings are utilized for training, validation, and testing, respectively. Entity alignment is a typical ranking problem, where we obtain a target entity ranking list for each source entity by sorting the similarity scores in de- scending order. We use Hits@k k ()= and Mean Reciprocal Rank (MRR) as the evaluation metrics (Sun et al., 2020b; Xin al., 2022a). Hits @k k is to measure the alignment accuracy, while MRR mea- sures the average performance of ranking over all test samples. The higher the Hits and MRR, the better the alignment performance.
piee 1
tea J
(13)
Σ
piea
plea
plee
|&,1 -1==1 Cross-Entropy a
tea )
Lu
(14)
where p denotes the rectified pseudo mapping matrix. To be specific, the designed rectification term (Eq. (13)) measures how much a potential pseudo mapping deviates (in terms of joint match- ing confidence score) from other competing pseudo mappings in p!." and Pire. The larger the devia- tion, the greater the penalty for this pseudo map- ping, and vice versa. Notably, both M and plea are fed into a softmax to be converted to probability distributions before CE to implement probabilis- tie pseudo mapping learning. Besides, an illustra- tive example of generating the probabilistic pseudo mapping matrix is provided in Figure 2.
5.2 Baseline Methods
We choose the methods from the related work as baselines and divide them into two classes below:
Structure-based methods. These methods focus on capturing useful structural context to enrich entity representation, such (1) MTransE (Chen al., 2017), (2) GCN-Align (Wang et al., 2018), (3) AliNer (Sun et al., 2020a), (4) KE-GCN (Yu al., 2021) and (5) IMEA (Xin et al., 2022a).
Optimization. Finally, we minimize the follow- ing combined loss function (final objective) to op- timize the student model in an end-to-end training manner:
Semi-supervised methods. These methods aim to explore alignment signals from unlabeled enti- ties, such as (1) IPTransE (Zhu et al., 2017), (2) SEA (Pei et al., 2019). (3) KDCoE (Chen et al., 2018), (4) BootEA (Sun et al., 201 2018), (5) MRAFA
min La + XL,, 4stu
(15)
where À is a ramp-up weighting coefficient used to weight between the supervised alignment learn- ing (i.e., C.) and pscudo mappings learning (i.e., C.). In the beginning, the optimization is domi- nated by Ca and during the ramp-up period, Ca will gradually participate in the training to provide more alignment signals. The overall optimization process is outlined in Algorithm (Appendix A), where the student model and the teacher model are updated alternately, and the final student model is utilized for EA inference (Eq. (1)) on test data.
(Mao al., 2020), (6) RANM (Cai et al., 2022) and (7) GAEA (Xie et al., 2023).
As our method and the above baselines only contain single model and mainly rely on struc- tural information, for a fair comparison, we do not compare with ensemble-basee frameworks (e.g., CycTEA (Xin et al., 2022b)) and models infusing side information from multi-modality (e.g., EVA (Liu et al., 2021), RoadEA (Sun et al., 2022)). For the baseline RANM, we remove the name channel to guarantee fair comparison.
5 Experimental Setup
5.1 Data and Evaluation Metrics
5.3 Implementation Details
We evaluate our method on the 15K benchmark dataset (V1) in OpenEA (Sun et al., 2020b) since
All the experiments are performed in PyTorch on an NVIDIA GeForce RTX 3090 GPU. Following

---

EN-FR-15K 15K
EN-DE-ISK 15K
D-W-15K
D-Y-15K
Models
Hit@
H
MRR
Hit@1
Hit@5
MRR
Hit@1
Hit@s
MRR
Hit@
Hit@5
MRR
MTTranEE'
.247
.467
.351
.307
.188
.407
.259
.344
.463
.675
.559
GCN-Aliga*
.338
.589
.451
.811
.679
.571
.364
.580
.461
.465
.626
.536
AliNet
.364
.597
.467
.604
.759
.673
.440
.628
.522
.559
.690
.617
KE-GCN
.408
.670
.524
.658
.822
.730
.519
.727
.608
.560
.750
.644
IMEA
.458
.720
.574
.639
.827
.724
.527
.753
.626
.639
.804
.712
IPTraasEE
.169
.320
.243
.350
.515
.430
.232
.380
.303
.313
.456
.378
SEA
.880
.530
.397
.300
.718
.617
.572
.500
.591
KDCoET
.881
.680
.628
.229
.629
.580
.247
.412
.325
.661
.764
.710
BootEAT
-507
-718
.603
.675
.820
.740
.572
.744
.649
.739
.849
.788
MRAEA
-537
.779
.642
.681
.444
.754
.574
.772
.661
.725
.838
.769
RANM
.567*
.780*
.663"
.686"
.835"
.751
GAEA"
.486
.746
.602
.684
.854
.760
.562
.768
.654
.608
.791
.688
MiXTEA
.582
.807
.680
.724
.877
.797
.832
.748
.871
.802
std.
1..04
1.006
1.004
1.003
1.005
1.003
+.00
1.004
1.00
1.004
1.002
1.003
Table I: Entity alignment performance of different methods the cross-lingual and monolingual settings OpenEA. The results with i are retrieved from Sun et al. (2020b), and from Xin al. (2022a). Results labeled by * are reproduced using the released source codes and labeled by are reported in the corresponding references. The boldface indicates the best result of each column underlined the second-best. std. means standard deviation.
EN-PR-ISK
EN-DE-I5K
Experimental Results
Models
Hit@
Hit@
MRR
Hit@|
Hit@5
MRR
6.1
Performance Comparison
w/o rel.
.471
.732
.586
.679
.839
.749
w/o Cu.
.485
.716
.589
.671
.744
Table reports the experimental results of all the methods on the OpenEA 15K datasets. Even utiliz- ing only the structure information, KE-GCN and IMEA achieve inspiring performance by explor- ing the rich structural contexts. However, they are hard to further improve the performance because suffer from the lack of enough training data. We also observe that some semi-supervised EA meth- ods (e.g., IPTransE and SEA) fail outperform these structure- EA methods, reflecting the fact that both encoder design and semi cemi-supervised strategy are important components of facilitating high-accuracy EA. IPTransE obtains unsatisfac- tory alignment results since it produces many noise pseudo mappings during the self-training process but does not design appropriate mechanism to climinate the influence of noise. Besides, the performance of KDCoE is unstable. According to Sun et al. (2020b), this because many enti- ties lack textual descriptions, thus preventing the model from finding complementary mappings for co- co- training from the textual description embedding model. BootEA and MRAEA are competitive base- lines in semi-supervised EA domain. Nevertheless, BootEA needs carefully fine-tuned confidence threshold filter pseudo mappings, which often leads to unstability, while MRAEA and RANM still follows the data augmentation paradigm, which ignores the uncertainty of pseudo mappings and
w/o BDV
.556
.781
.656
.707
.863
.777
MDR
.560
.791
.663
.779
w/o B&M
.542
.771
.644
.694
.857
.766
MixTEA
.582
.807
.680
.724
.877
.797
D-W D-K
D-Y-15K
Models
@it@1
H5t@5
MRR
Hit
Hit@5
MRR
w/o rel.
.552
.756
.641
.700
.841
.761
w/o C..
.520
.708
.605
.541
.684
.605
w/o BDV
.629
.812
.709
.725
.856
.783
w/o MDR
.635
.822
.718
.731
.862
.788
w/o B&M
.609
.802
.693
.712
.849
.774
MixTEA
.647
.832
.731
.748
.871
.802
Table 2: Ablation test results.
OpenEA (Sun al., 2020b), we report the average results five-fold cross-validation. The embed- ding dimensions of entities de and relations d, are set to 256 and 128, respectively, the number of GAT layer I. is 2, the margin p is 2.0, and the momen- tum m is 0.9. In the EA inference phase, we use Cosine distance as the distance metric and apply Faiss² perform NN search efficiently. The de- fault alignment direction is from left to right, c.g., D-K-15K, we regard DBpedia as the source KG and seek to find the counterparts of source entities in the target KG Wikidata. The details of hyperpa- rameter settings are shown in Appendix C.

---

is prone cause error accumulation. Although GAEA learns representations of vast unseen en- tities via contrastive learning, its performance unstable. The bottom part of Table shows our method consistently achieves the best performance in all tasks with small standard deviation (std.). More precisely, our model surpasses state-of-the- art baselines averagely by 3.1%, 3.3%, and 3.5% in terms of Hir@ Hir@5, and MRR, respectively.
0u6
0.0
MATTA Encode
046
(3)
(a) EN-DE-15K
(b) D-W-15K
Figure Test Hir@1 curve throughout training epochs.
6.3 Auxiliary Experiments
6.2 Ablation Study
Training visualization. To inspect our method comprehensively, we also plot the test Hir@) curve throughout the training epochs in Figure 3. KG Encoder (th=0.9) represents the KG Encoder de- scribed in Sec. 4.1 applying the self-training with threshold=0.9 generate pscudo mappings every 20 epochs. We control the same experimental set- tings to remove the performance perturbations in- duced by different parameters. From Figure 3, we observe that our method converges quickly and achieves the best and most stable alignment perfor- mance. The performance of the KG Encoder gradu- ally decreases in the later stages since it gets stuck in overfitting the limited training data. Although self lelf-training brings some performance gains after data augmentation in the early stages, the perfor- mance drops dramatically in the later stages. This is because involves many noise pseudo mappings and causes error accumulation as the self-training continues. In the later stages, self-training has dif- ficulty in further generating existing erroneous mappings constantly misguide the model training, thus hurting the performance.
To verify the effectiveness of our method, we per- form the ablation study with the following variant settings: (1) w/o rel. removes the relation mod- eling. (2) w/o Lu removes probabílistic pseudo mapping learning. (3) w/o BDV only considers EA decisions in the default alignment direction to gen- erate pseudo mappings instead of applying the bi- directional voting strategy (i.e., piee (M x-1 (4) w/o MDR removes matching diversity-base rectification module in pseudo mappings learning. (5) w/o B&M denotes that the complete model deletes both BDV and MDR module.
The ablation results are shown in Table 2. We can observe that the complete model achieves the best experimental results, which indicates that each component in our model design contributes to the performance improvement. Removing relation modeling from entity representation causes per- formance drops, which identifies the relation se- mantics can help in enriching the expressiveness entity representations. W/o La caused the most significant performance degradation, especially in monolingual settings, showing the crucial role of the pseudo mapping learning in general. The re-
mappings while
Hy Hyperparameter analysis. We design hyper- parameter experiments to investigate the perfor- mance varies with some hyperparameters. Due to the space limitation, these experimental results and analyses listed in Appendix D.
sults of w/o BDV and w/o MDR suggest that the bi- directional voting strategy and matching diversity- based rectification module can do benefit improv- ing the quality of pseudo mapping learning. W/o B&M also demonstrates that the combination of BDV and MDR can further improve the alignment performance. Although w/o BDV only takes EA decisions in one direction into account, it still in- evitably brings matching conflicts since NN search neglects the inter-depyndency between different EA decisions. Compared to w/o B&M, the MDR in w/o BDV has certain positive effect, which indicates that our proposed MDR can be applied to other pscudo mapping generation algorithms and help the models to train better.
7 Conclusion
In this paper, we propose a novel semi-supervised EA framework, termed as MixTEA, which guides the model learning with an end-to-end mixture teaching of manually labeled mappings and proba- bilistic pscudo mappings. Mcanwhile, propose bi-directional voting (BDV) strategy and a match- ing diversity-basee rectification (MDR) module assist the probabilistic pseudo mapping learning. Experimental results on benchmark datasets show the effectiveness of our proposed method.

---

Limitations
International Conference, KSEM 2020. Hangzhou, China, August 28-30, 2020, Proceedings, Part 13, pages 134-147. Springer.
Although we have demonstrated the effectiveness of MixTEA, there are still some limitations that should be addressed in the future: (1) Currently, we only utilize structural contexts which are abun- dant and always available in KGs to embed enti- ties. However, when side information (e.g., visual contexts, literal contexts) is available, MixTEA needs to be extended into a more comprehensive EA framework and ensure that does not become over-complex in the teacher-student architecture. Therefore, how to involve this side information is our future work. (2) Vanilla self-training itera- tively generates pscudo mappings and adds them to the training data, where the technicians can per- form spot checks during model training to monitor the quality of pseudo mappings. While MixTEA computes probabilistic pseudo mapping matrix and performs end-to- to-eotodd training, thus making it hard to provide explicit entity mappings for the techni- cians to check their correctness. Therefore, it is imperative to design strategy to combine the self- training and probabilistic pseudo mapping learning to enhance the interpretability and operability.
Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, and Carlo Zaniolo. 2018. Co-training embed- dings of knowledge graphs and entity descriptions for cross-lingual entity alignment. In Proceedingg of the 27th International Joint Conference on Artificial Intelligence, pages 3998-4004.
Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. 2017. Multilingual knowledge graph em- beddings for cross-lingual knowledge alignment. In Proceedings of the 26th International Joint Confer- ence on Artificial Intelligence, pages 1511-1517.
Avigdor Gal, Haggai Roitman, and Tomer Sagi. 2016. From diversity- based prediction to better ontology & schema matching. In Proceedings of the 25th International Conference on World Wide Web, pages 1145 1155.
Yuqing Gao, Jisheng Liang, Benjamin Han, Mohamed Yakout, and Ahmed Mohamed, 2018. Building a large-scale, accurate and fresh knowledge graph. KDD-2018, Tutorial, 39: 19394491377
Thomas N Kipf and Max Welling. 2017. Semi- supervised classification with graph convolutional networks. In International Conference on Learning Representations.
Ethics Statement
This work does not involve any discrimination, so- cial bias, or private data. Therefore, we believe that our study complies with the ACL Ethics Policy.
Zhenxi Lin, Ziheng Zhang, Meng Wang. Yinghui Shi, Xian Wu, and Yefeng Zheng. 2022. Multi-modal con- trastive representation learning for entity alignment. In P Proceedings of the 29th International Conference on Computational Linguistics, pages 2572-2584.
Acknowledgments
We thank the anonymous reviewers for their com- ments. This work is supported by the National Nat- ural Science Foundation of China No. 62172428.
Fangyu Liu, Muhao Chen, Dan Roth, and Nigel Collier. 2021. Visual pivoting for (unsupervised) entity align- ment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4257-4266.
Zhiyuan Liu, Yixin Cao, Liangming Pan, Juanzi Li, and Tat-Seng Chua. 2020. Exploring and evaluating attributes, values, and structures for entity alignment. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 6355-6366 335--664.
References
Antoine Bordes, Nicolas Usunier, Alberto Garcia- Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Proceedings of the 26th Interna- tional Conference on Neural Information Processing Systems- Volume 2, pages 2787-2795.
Xin Mao, Wenting Wang, Huimin Xu, Man Lan, and Yuanbin Wu. 2020. Mraca: efficient and robust entity alignment approach for cross-lingual knowl- edge graph. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 420-
Weishan Cai, Wenjun Ma, Lina Wei, and Yuncheng Jiang. 2022. Semi-supervised entity alignment via relation-based adaptive neighborhood matching. IEEE Transactions on Knowledge and Data Engi- neering.
Shichao Pei, Lu Yu. Robert Hochndorf, and Xiangliang Zhang. 2019. Semi-supervised entity alignment via knowledge graph embedding with awareness of de- gree difference. In The World Wide Web Conference, pages 3130-3136.
Liyi Chen, Zhi Li, Yijun Wang, Tong Xu, Zhefeng Wang, and Enhong Chen. 2020. Mmea: entity align- ment for multi-modal knowledge graph. In Knowl- edge Science, Engineering and Management: 13th

---

Shichao Pei, Lu Yu, Guoxian Yu, and Xiangliang Zhang. 2022. Graph alignment with noisy supervision. In Proceedings of the ACM Web Conference 2022, pages 1104-1114.
Kexuan Xin, Zequn Sun, Wen Hua, Wei Hu, and Xi- aofang Zhou. 2022a. Informed multi-context entity alignment. In Proceedings of the I 15th ACM Interna- rional Conference on Web Search and Data Mining, pages 1197-1205.
Fabian M Suchanek, Serge Abitehoul, and Pierre Senel- lart. 2011, Paris: Probabilistic alignment of relations, instances, and schema. Proceedings of the VLDB Endowment (PVLDB), 5(3):157-168.
Kexuan Xin, Zequn Sun, Wen Hua, Bing Liu, Wei Hu, Jianfeng Qu, and Xiaofang Zhou. 2022b. Ensemble semi supervised entity alignment via sycle-teaching In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4281-4289.
Zequn Sun, Wei Hu, and Chengkai Li. 2017. Cross- lingual entity alignment via joint attribute-presee preserving embedding. In The Semantic Web-ISWC 2017: 16th International Semantic Web Conference, Vienna, Aus- tria, October 21-25, 2017, Proceedings. Part 1 16. pages 628 644. Springer.
Hsiu-We Yang. Yanyan Zou, Peng Shi, Wei Lu, Jimmy Lin, and Xu Sun. 2019, Aligning cross- cross-lingual lingual enti- ties with multi-aspect information. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4431-4441.
Zequn Sun, Wei Hu, Chengming Wang, Yuxin Wang. and Yuzhong Qu 2022. Revisiting embedding- bbaee entity alignment: A robust and adaptive method. IEEE Transactions on Knowledge and Data Engi- neering.
Donghan Yu, Yiming Yang, Ruchong Zhang, and
Yuexin Wu. 2021. Knowledge embedding based graph convolutional network. In Proceedings of the Web Conference 2021, pages 1619-1628.
Zequn Sun, Wei Hu, Qingheng Zhang, and Yuzhong Qu. 2018. Bootstrapping entity alignment with knowl- edge graph embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelli- gence, pages 4396-4402.
Weixin Zeng, Xiang Zhao, Jiuyang Tang, and Changjun Fan. 2021. Reinforced active entity aligmment. Proceedings of the 30th ACM International Con- ference on Information & Knowledge Management. pages 2477-2486
Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei Zhang, and Yuzhong Qu. 2020a. Knowledge graph alignment network with gated multi-hop neighborhood aggregation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 34, pages 222-229.
Qingheng Zhang, Zequn Sun, Wei Hu, Muhao Chen, Lingbing Guo, and Yuzhong Qu. 2019. Multi-view knowledge graph embedding for entity alignment. In Proceedings of the 28th International Joint Confer- ence on Artificial Intelligence.
Zequn Sun, Qingheng Zhang, Wei Hu, Chengming Wang. Muhao Chen, Farahnaz Akrami, and Chengkai Li. 2020b. A benchmarking study of embedding- based entity alignment for knowledge graphs. In Proceedings of the VLDB Endowment.
Zhedong Zheng and Yi Yang. 2021. Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. International Jour- nal of Computer Vision, (29(4)1106-1122)
Antti Tarvainen and Harri Valpola. 2017. Mean teach- better role models: Weight-averaged consis- tency targets improve semi-supervisee deep learning results. In Proceedings of the 31st International Con- ference on Neural Information Processing Systems, pages 1 1194-1204.
Hao Zhu, Ruobing Xie, Zhiyuan I. Liu, and Maosong Sun. 2017. Iterative entity alignment via joint knowl- edge embeddings. In Proceedings of the 26th Inter- national Joint Conference on Artificial Intelligence. pages
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In International Conference on Learning Representations.
Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. 2018. Cross-lingual knowledge graph alignment via graph convolutional networks. Pro- ceedings of the 2018 conference on empirical meth- ods in natural language processing, pages 349-357.
Feng Xie, Xiang Zeng, Bin Zhou, and Yusong Tan. 2023. Improving knowledge graph entity aligument with graph augmentation. In Advances Knowl- edge Discovery and Data Mining: 27th Pacific-Asia Conference on Knowledge Discovery and Data Min- ing. PAKDD 2023, Osaka, Japan, May 25-28, 2023, Proceedings, Part II, pages 3-14.

---

A Pseudocode of Training Procedure
839
Algorithm Training Procedure
83-
Input:
Knowledge graphs G. and Gr; pre-aligned entity map- pings S: unlabeled entity &, and Ec; momentum m;
04-
D.S-IKK DYVAK
margin p.
Output: the student encoder parameters ettu-
(a) number of GAT layers
(b) momentum parameter
Initialize entity emheddings and relation embeddings;
2: Initialize teacher model Beea and student model Baus
Figure Sensitivity analysis of our proposed method.
3: each epoch do
Encode entities using f[; Oatu) [ Osca): Calculate La by supervised learning Eq. (8):
Generate pseudo mapping matrix via BDV stralegy: Rectify pscudo mapping matrix via MDR module: Calculate C., by pseudo mapping learning in Eq. (14);
quantitative results are illustrated in Figure 4 (a). L=1 results in poor alignment performance due the limited structural modeling power. The best performance is achieved when L=2, except for the D-Y-15K task. Increasing L will not bring further performance improvement, we infer that there is overfitting or oversmoothing during neighhorhood aggregation. In D-Y-15K, the optimal performance is obtained when L=4. The possible reason is that in D-Y-15K, the two KGs have relatively sparse structure information (as shown in Table 3 in Ap- pendix B), therefore they need to capture more alignment evidence from distant neighbors.
RackProp(cc. + ACe); 9: Astu +
D Adam Update
10: Orea +- mdrea + m)0scu
11:
end
12:
return the student encoder parameters Ostu
Dataset Statistics
Datasets
#Ent.
#Rel.
#Tri.
English
15000
267
47334
EN-FR-15K
French
15000
210
40864
English
15000
215
47676
EN-DE-15K
The impact momentum parameter. We in- vestigate the momentum parameter m in [O, 0.9, 0.99] and the results in EN-DE-15K and D-W-155 tasks are presented in Figure (b). We found that our method maintains good performance under dif- ferent momentum settings, demonstrating that our method is insensitive
German
15000
131
50419
DRPedia
15000
248
38265
D-W-15K
Wikidata
15000
169
42746
DBPedia
15000
165
30292
D-Y-15K
YAGO
15000
28
26638
Table 3: Dataset statistics: #Ent., #Rel., and #Tri. means the number of entities, relations, and triples corre- sponding KG, respectively.
In addition, proper
m such 0.9 brings certain performance improve- ments, which indicates that the EMA update man- ner can facilitate the teacher model to yielding sta- ble and robust pseudo mappings.
C Hyperparameter Details
We tune the hyperparameters for our proposed Mix- TEA. The setting values and search ranges of hy- perparameters are described in Table 4.
Hyperparameters
Value/Search range
The number of GAT layer
[1. 2, 3] [0.9, 0.99, 0.999]
Momentum parameter m
Margin
[1, 2, 3]
Negative sample size
[10, 20, 30]
Embedding dimension
[128, 256]
Embedding initialization
Xavier
Learning rate
0.005
Table 4: Hyperparameter values and search ranges.
Hyperparameter Analysis
The impact of different GAT layers. We vary the number of GAT layer from I to and the


Reviews:
[{"review_id": "npmguiJVca", "reviewer": "Reviewer_8XC3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your generous and critical comments on our submitted manuscript. We give the respective explanations and responses as follows:\n1. \"l\" means \"l-th\" layer in the multi-layer graph attention network (GAT), which has been explained in Line 287. GAT can be stacked in multiple layers, and each layer can capture neighbor structure information with different hops. Eq (12) is the layer-wise propagation rule.\n2. We attempted to demonstrate that our model improves the alignment performance and pseudo mapping quality during the iteration process in section 6.3-\"Training visualization\".  In 6.3, we compared our model with a variant that uses a threshold to filter pseudo mappings. The test results on every epoch are shown in Figure 3. As we can see in Figure 3, the method applying threshold filtering (green line) will result in performance degradation in the later stages of training, which is due to the selection of too many wrong pseudo mappings in the early stage, resulting in the error accumulation phenomenon. In contrast, our model (blue line) which applies BDV and MDR to optimize the process of pseudo mapping learning was able to produce optimal and stable performance. This indicates that incorrect pseudo mappings in the early stage would gradually corrected by BDV and MDR during the end-to-end training process, thereby avoiding negative effects.\n3. Lines 329 to 333 mainly describe the pseudo mapping learning provided by the teacher model which has been drawn in the bottom right corner of Figure 1. In the subsequent sub-sections, we also provided a detailed introduction based on the bottom right corner of Figure 1.\n4. We are sorry for the confusion caused by the incomplete description of the experimental details. Our experimental setup strictly follows the semi-supervised setting in OpenEA, and we use unlabeled data provided in OpenEA as additional data for semi-supervised learning. At the same time, we have discussed some hyper-parameters (GAT number and momentum parameter) of the model in Appendix (C), and we will supplement more parameter experiments later. As for the parameter tuning, the layer number of GAT is searched in [1,2,3,4], the momentum parameter is set to [0.9,0.99,0.999], the margin is searched from [1,2,3], and the hidden dimension is selected in [128, 256]. We attach great importance to the issue of insufficient experimental details and will carefully supplement them in the manuscript.\n5. For a fair comparison, most of the baseline values are taken directly from their corresponding papers and our method is also under the same experimental settings with baselines. In this paper, we also give the standard deviation in Table 1 to demonstrate the stability of our model under five-fold cross-validation.\n6. To inspect our method comprehensively, we also plot the test Hit@1 curve throughout the training phase in Figure 3. To be specific, after every training epoch, we test the model over the test dataset and record the Hit@1 scores. So the x-axis of Figure 3 means epoch number and the y-axis means Hit@1 score. We are sorry that the figure does not clearly indicate the x-axis information, which has made you feel confused. Figure 3 shows that our model (blue line) achieves the best and most stable performance.\n7. Our proposed method does not require any threshold selection. The threshold we mentioned in 6.3 is the setting of the variant model, i.e., KG Encoder (th=0.9). The threshold selection for the compared variant model is indeed empirical, and we choose the threshold equal to 0.9, which is a relatively high confidence level to filter out low-confidence mappings. Perhaps in future work, we can take inspiration from this threshold-tuning strategy, but our currently proposed method does not require a threshold.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper addresses the challenging problem of using unlabeled data for entity alignment. It is well-motivated and clearly written.  \n- The paper proposes several novel and interesting approaches, such as BDV (bi-directional voting for more comprehensive pseudo mappings) and MDR (matching diversity-based rectification for dynamic mitigating the influence of noisy mappings). \n- The authors provide an extensive and detailed ablation study and results discussion. \n", "reasons_to_reject": "- Some inaccuracies in mathematical notation (e.g., line 278 - what is \"l\" in \"l-1\"?). \n- In Section 4, the authors claim that the end-to-end training \"...gradually improves the quality of pseudo mapping\". It would be useful to add some example/analysis/proof that this is indeed happening and the mappings are becoming more and more accurate (e.g., the results of the model after every n-th iteration?).  \n- Lines 329-333: being an important part of the algorithm, this MixTEA part is not reflected in Figure 1. \n- The experimental setup remains unclear to me. Was any additional unlabeled data used? How were the hyperparameters tuned? What was the search range?\n- No standard deviation for the baselines. Note that most of the baseline values are taken directly from the previous work, and some models have been trained in other settings. Therefore, a comparison of MixTEA with the baseline models should be done more carefully. \n- Figure 3 is not quite clear. \n- Section 6.1: the threshold values are mentioned here for the first time and are not discussed elsewhere. I guess the whole setting could be strengthened by adding more sophisticated methods for threshold tuning (e.g., a recent ACL paper [1]); apart from that, at least some discussion of the threshold values is required. \n\n[1] Sedova and Roth. 2023. \"ACTC: Active Threshold Calibration for Cold-Start Knowledge Graph Completion\"", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors propose a new end-to-end method MixTEA for a semi-supervised entity alignment task. MixTEA has a teacher-student architecture, where both student and teacher models are knowledge graph encoders. The method is claimed to overcome two significant issues of semi-supervised EA methods: uncertainty of pseudo mapping (by switching a binary pseudo mapping to a probabilistic one and estimating the matching uncertainty via confidence score) and an increased impact of noisy pseudo mappings (by a matching diversity-based rectification). \n\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- line 61, 64, 418, 545.. - unnecessary brackets in citations\n- line 533: \"enough\" should be removed -> \"the lack of training data\""}, "extra_text": {}}, {"review_id": "HLZpLyWNx7", "reviewer": "Reviewer_ce7N", "summary": "", "questions": "1. Why BSD works?\n2. Why MDR works? What is the meaning of Equation (12)?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comment. However, we fully disagree with you respectfully. \nFirst of all, the motivation of our paper is clear, which is also appreciated by other reviewers. In this paper, we focus on tackling the two problems in pseudo mapping learning of semi-supervised entity alignment, i.e., (1) noisy pseudo mapping learning and (2) modeling uncertainty of pseudo mappings, and we propose our MixTEA to handle the above problems.\n1. Why BDV works? BDV module is the key to modeling the uncertainty of pseudo mapping in this paper. Typically, previous entity alignment methods use the nearest neighbor (NN) search to obtain its counterpart for each entity. However, NN search in different directions will yield asymmetric results because the entities in an entity pair may not be each other's nearest neighbors (i.e. e_1's nearest neighbor is e_a, but e_a's nearest neighbor may be e_2). Therefore, the asymmetric nature of alignment directions in EA inspires us to weighted ensemble alignment results from different directions. In this way, we not only obtain more comprehensive alignment results (the results come from two directions), but also model the uncertainty of pseudo mappings.\n2. Why MDR works? MDR module can mitigate the problem of false (erroneous) pseudo mappings by reducing the confidence of alignment associated with hub entities. A Hub entity is an entity that aligns with many other entities, i.e., a hub entity repeatedly occurs as the nearest neighbors of others, which easily happens in the high-dimension vector space (also called hubness problem). However, the ideal prediction of entity alignment is one-to-one matching, which means a source entity can be matched with at most one target entity, and vice versa. As a result, hub entities will cause many false pseudo mappings in semi-supervised learning. MDR is a rectification strategy to punish the hub entities. In Eq (12), the numerator is the confidence of the alignment pair, and the denominator is the sum of the confidence that the entities in the entity pair align with other entities. Given an alignment pair (e_1, e_a), if e_a is the hub entity, the denominator in Eq (12) will bigger than numerator, and the original confidence of the alignment pair (e_1, e_a) will be reduced (i.e., penalized) because e_a is a hub entity. But if (e_1, e_a) is the true pseudo mapping, then the denominator and numerator are equal in Eq (12).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This authors introduce a bi-directional voting (BDV) strategy which utilizes the alignment decisions in different directions to estimate the uncertainty of pseudo mappings.\n2. This authors design a matching diversity-based rectification (MDR) module to adjust the pseudo mapping learning.\n3. Experimental results seem good.\n", "reasons_to_reject": "1. This paper is not well motivated\n2. The proposed BDV and MDR are not clearly explained\n", "questions_to_authors": "1. Why BSD works?\n2. Why MDR works? What is the meaning of Equation (12)?", "paper_topic_and_main_contributions": "This paper proposes a semi-supervised EA method, which guides the model learning with an end-to-end mix-ture teaching of manually labeled mappings and probabilistic pseudo mappings. \n", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8zUinYzpEb", "reviewer": "Reviewer_KV6A", "summary": "", "questions": "See my review.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your generous and critical comments on our submitted manuscript. We give the respective explanations and responses as follows:\n1. In this paper, we introduce probabilistic pseudo mapping modeling into semi-supervised entity alignment, which trains the model with an end-to-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings, unlike previous methods of appending pseudo mappings to the training set iteratively. Our proposed method has the theoretical basis of previous research. To be specific, we select Mean Teacher as the semi-supervised learning framework, which is a stable architecture based on a moving average strategy. Regarding BDV, which is based on the asymmetric nature of alignment directions in EA. Although relevant papers have studied bi-directional strategy before, they only use it for obtaining the final alignment results and did not consider weighted ensemble results from different directions to model the uncertainty (probability) of pseudo mapping. Meanwhile, MDR mitigates the problem of false pseudo mappings by punishing hub entities that align with many other entities and reducing the probability of aligning with these hub entities.\n2. How to apply BDV to traditional entity alignment methods is a very interesting issue. Firstly, we understand that the traditional alignment strategies you mentioned may refer to the traditional alignment method based on equivalence reasoning or symbol feature similarity. However, our method is neural embedding-based, which dominates current EA research and we have already introduced it in the first section. Traditional alignment methods are usually based on statistics or symbol similarity to obtain the final result, without involving any iterative training or learning process actually. The role of BDV is to generate probability pseudo mappings in each iteration to gradually improve the model performance. Therefore, it cannot be applied to traditional methods directly.\n3. We think the MDR module is simple and critical. Although the gains from MDR are modest, the results on all four datasets show that it could steadily make contributions. Additionally, MDR is proposed to mitigate false pseudo mappings that are easy to occur during the entity alignment process yet difficult to detect.\n4. In addition to the comparison experiments to prove the overall effectiveness, and the ablation experiments to prove the effectiveness of each module, we attempted to demonstrate that our model improves the alignment performance and pseudo mapping quality during the iteration process in section 6.3-\"Training visualization\". In 6.3, we compared our model with a variant that uses a threshold to filter pseudo mappings. The test results on every epoch are shown in Figure 3. As we can see in Figure 3, the method applying threshold filtering (green line) will result in performance degradation in the later stages of training, which is due to the selection of too many wrong pseudo mappings in the early stage, resulting in the error accumulation phenomenon. In contrast, our model (blue line) which applies BDV and MDR to optimize the process of pseudo mapping learning was able to produce optimal and stable performance. This indicates that incorrect pseudo mappings in the early stage would gradually corrected by BDV and MDR during the end-to-end training process, thereby avoiding negative effects.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This paper studies the entity alignment problem in a semi-supervised scenario which is a practical and challenging task. Existing methods mainly suffer from two problems: uncertainty of pseudo mappings and noisy pseudo-mapping learning. To alleviate the negative influence of these two problems, the author proposed a novel method that guides model learning with an end-to-end mixture teaching of manually labeled mappings and probabilistic pseudo mappings. In this part of the experiment, the author conducts extensive experiments to demonstrate the effectiveness of this method.\n\n-  author conducts extensive experiments to demonstrate the effectiveness of this method.\n-  The structure of the paper is reasonable and the content is sufficient. \n-  The schematic diagram is concise. Readers can fully understand the method proposed in the article through the schematic diagram", "reasons_to_reject": "- Paper proposes a bi-directional voting (BDV) strategy and a matching diversity-based rectification (MDR) module to assist the probabilistic pseudo-mapping learning but lacks the theoretical demonstration of the effectiveness of the BDV strategy and MDR model. \n\n- I would like to see how the BDV strategy improves conventional alignment strategies. \n\n- In addition, the effectiveness of the MDR module shown in Table 2 is not obvious, this component may be not critical.\n\n- Experiments do not concretely show the effectiveness of your method on alleviating the negative impact of uncertainty of pseudo mappings and noisy pseudo mapping learning.", "questions_to_authors": "See my review.", "paper_topic_and_main_contributions": "This paper introduces a new method for semi-supervised entity alignment based on Teacher-Student architecture, termed MixTEA. The main contribution of this paper is that the proposed method trains alignment learning with both manually labeled mappings and probabilistic pseudo mappings to alleviate the negative influence of noisy or uncertain pseudo mappings.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "No"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a method, MixTEA, for semi-supervised entity alignment based on a Teacher-Student architecture that works by training the student model using alignment signals from both manually labeled mappings and probabilistic pseudo mappings generated by the teacher model. Compared to existing works, the claimed methodological innovation mitigates issues associated with noisy pseudo mappings and/or lack of pseudo mapping uncertainty consideration in utilizing the pseudo labels.

== Quality ==
The reviewers believed the methodological innovations were clearly described and the paper was clearly written overall. This is a widely studied problem and the authors propose a conceptual improvement to account for uncertainly and operationalize it in a sensible way with a teacher-student model. Finally, the authors provide an extensive comparison relative to competing EA methods and a good ablation study -- where the results are good overall and establish a state-of-the-art solution under realistic constraints.

== Clarity ==
As stated in the 'quality' section, the paper is well-written overall. Additionally, the authors make good use of figures, etc. to better clarify methodological explanations. However, there were also some concerns regarding attributing empirical improvements to the specific methods proposed. Specifically, even after rebuttal and discussion, it still wasn't clear to the reviewers after discussion how much the empirical improvements were due to the proposed methods and how much was due to general empirical improvements in working to outperform a known target (i.e., hyperparameter optimization). This may be able to be remedied with more details of the experimental setup and further discussion of results. Clearly, additional theoretical understanding of BDV and MDR based improvements would be useful (or references to support this from other settings), but minimally a more detailed empirical analysis is necessary to establish their relative contributions to the the improvement of the models. This was partially addressed in the rebuttal, but the reviewers remained unconvinced that the authors have made a strong case regarding the specific empirical contributions of the proposed improvements.

== Originality ==
The paper proposes multiple novel, interesting, and sensible approaches, such as BDV (bi-directional voting for more comprehensive pseudo mappings) and MDR (matching diversity-based rectification for dynamic mitigating the influence of noisy mappings). For the most part, these innovations are specific to EA and build on existing baselines and datasets. However, there is more conceptual innovation than many EA papers.

== Significance ==
This is a widely studied problem where performance improvements have practical and potentially impactful implications. Within the EA-specific community, this work will very likely be used as a baseline for future comparisons. That being said, this work builds on existing experimental setups and datasets, so it is primarily a methods contribution. Additionally, the proposed methods to provide support for teacher-student models as an uncertainty modeling technique, but in general isn't clearly going to influence work in other tasks. Thus, it is a solid work within this application.
Decision: Accept-Findings