RHGN: Relation-gated Heterogeneous Graph Network for Entity Alignment in Knowledge Graphs
Year: 2023

Abstract:
Entity Alignment, which aims to identify equivalent entities from various Knowledge Graphs (KGs), is a fundamental and crucial task in knowledge graph fusion. Existing methods typically use triples or neighbor information to represent entities, and then align those entities using similarity matching. Most of them, however, fail to account for the heterogeneity among KGs and the distinction between KG entities and relations. To better solve these problems, we propose a Relation-gated Heterogeneous Graph Network (RHGN) for entity alignment in knowledge graphs. Specifically, RHGN contains a relation-gated convolutional layer to distinguish relations and entities in the KG. In addition, RHGN adopts a cross-graph embedding exchange module and a soft relation alignment module to address the neighbor heterogeneity and relation heterogeneity between different KGs, respectively. Extensive experiments on four benchmark datasets demonstrate that RHGN is superior to existing state-of-theart entity alignment methods.

heading: 1 Introduction
text: Knowledge Graphs (KGs), which are sets of triples like (head entity, relation, tail entity), have been widely constructed (Sevgili et al., 2022; Wang et al., 2023) and applied (Liu et al., 2020a; Zhang et al., 2022, 2021) in various fields in recent years, such as DBpedia (Lehmann et al., 2015) and YAGO (Rebele et al., 2016). In the real world, a single KG is usually incomplete as limited sources can be collected by one KG. From this perspective, entity alignment, which aims to determine equivalent entities from various KGs, is a crucial task of knowledge graph fusion and is being increasingly researched (Sun et al., 2020c; Chen et al., 2022).
Specifically, entity alignment is a task to find equivalent entities with the same color across two KGs, as illustrated in Figure 1. As the neighbors
∗corresponding author.
and relations of the same entity in various KGs are often different, also known as the heterogeneity problem, it is time-consuming to find aligned entities manually. To align the entities efficiently, many embedding-based methods have been proposed. Traditional methods (Chen et al., 2017; Zhu et al., 2017) follow the translational principle, such as TransE (Bordes et al., 2013), to represent entity embedding, which consider the triples but disregard the local neighbors. Recently, many methods (Wang et al., 2018; Sun et al., 2020b) have adopted the Graph Convolutional Network (GCN) and its variants to capture local neighbor information due to the GCNs’ remarkable ability (Welling and Kipf, 2016; Velickovic et al., 2017; Wu et al., 2021, 2023). Additionally, researchers have proposed some models to utilize relations as weights (Cao et al., 2019) or information (Mao et al., 2021; Yu et al., 2021) in the GCN-based framework. Despite this, the following two primary challenges have been encountered by the vast majority of prior methods when attempting to use relation information to solve KG heterogeneity:
First, relations should not be directly incorporated into entity representation, since confusing relations with entities leads to smooth entity representations. In DBpedia, there are 4,233,000 enti-
8683
ties but only 3,000 relations, making the same relation often established between various entities (e.g., Country in Figure 1(b)). To separate relations from entities, R-GCN (Schlichtkrull et al., 2018) learns relation matrices but numerous relations bring trouble for parameter optimization (Vashishth et al., 2019). Therefore, existing models (Nathani et al., 2019; Mao et al., 2020) employ vectors to represent relations and apply simple functions (e.g., subtraction and projection) as the neighbor message functions. However, these simple functions barely distinguish relations from entities and still bring much noise to entity representation.
Second, due to KG heterogeneity, it is challenging to unify the semantic representations between KGs during the alignment process. Specifically, KG heterogeneity includes (1) neighbor heterogeneity and (2) relation heterogeneity. Neighbor heterogeneity indicates that the same entity in different KGs have different neighbors. As illustrated in Figure 1, neighbor heterogeneity is reflected in that Da Vinci have different neighbors in two KGs, which may make us mistakenly match Da Vinci in KG1 with Florence Cathedral in KG2 as they have more identical neighbors. Relation heterogeneity means that the relation between the same entity pair can be expressed in various ways, even though these relations have similar intentions. As Figure 1 shows, relation heterogeneity is expressed as that the relation between Da Vinci and Italy is Nationality in KG1, while it is Citizenship in KG2, which causes trouble for aligning these triples though they have the similar meaning.
To tackle these obstacles, we propose a Relationgated Heterogeneous Graph Network (RHGN) for entity alignment. Specifically, we first propose a novel Relation Gated Convolution (RGC) to make entity representations more discriminative. RGC uses relations as signals to control the flow of neighbor information, which separates relations from entities and avoids noise flowing into entities in representation learning. Second, to tackle the neighbor heterogeneity between two KGs, we devise Crossgraph Embedding Exchange (CEE) to propagate information via aligned entities across different KGs, thereby unifying the entity semantics between two KGs. Third, we design Soft Relation Alignment (SRA) to deal with the relation heterogeneity. SRA leverages entity embedding to generate soft labels for relation alignment between KGs, hence reducing the semantic distance of similar relations across
KGs. Finally, extensive experiments on four realworld datasets demonstrate the effectiveness of our proposed method. The source code is available at https://github.com/laquabe/RGHN.

heading: 2 Related Works

heading: 2.1 Entity Alignment
text: Entity alignment is a fundamental task in knowledge graph study. It seeks to recognize identical entities from different KGs (Sun et al., 2020c; Chen et al., 2020). To efficiently find identical entities, embedding-based models have been extensively studied. Traditional models, such as MtransE (Chen et al., 2017), used translation-based models (e.g., TransE (Bordes et al., 2013)) to make the distance between aligned entities get closer. Following this thought, IPTransE (Zhu et al., 2017), JAPE (Sun et al., 2017), and BootEA (Sun et al., 2018) constrained models from semantic space, attributes, and labels, respectively. Traditional models, however, neglect neighbor structures in favor of triples.
Inspired by the great success of Graph Neural Networks (GNNs), numerous methods (e.g, GCN-Align (Wang et al., 2018), AliNet (Sun et al., 2020b)) employed the GNNs and the variants to capture local neighbor information (Zeng et al., 2021). Since the knowledge graph contains abundant relations, RDGCN (Wu et al., 2019a), RSN4EA (Guo et al., 2019), and Dual-AMN (Mao et al., 2021) utilized relations as weights, paths, and projection matrices in GNNs. RREA (Mao et al., 2020) proposed a unified framework for entity alignment using relations. IMEA (Xin et al., 2022) encoded neighbor nodes, triples, and relation paths together with transformers. Unfortunately, they have not paid enough attention to the differences between entities and relations, and ignored semantic differences between different graphs due to KG heterogeneity.
Relation alignment, meantime, greatly aids in entity alignment. MuGNN (Cao et al., 2019) and ERMC (Yang et al., 2021) directly used the relation alignment labels but relation alignment labels are scarce in the real world. RNM (Zhu et al., 2021) and IMEA (Xin et al., 2022) applied postprocessing to relation alignment with statistical features. However, post-processing can mine limited aligned relations. HGCN-JE (Wu et al., 2019b) jointly learned entity alignment and relation alignment, which incorporated neighbor relations into
entities. Unfortunately, non-aligned entities may also have similar neighbor relations, which means relation alignment and entity alignment should be separated. Therefore, effective relation alignment methods remain to be explored.

heading: 2.2 Graph Convolutional Network
text: Graph Convolutional Networks (GCNs) generalize convolution operations from traditional data (e.g., images or grids) to non-Euclidean data structures (Defferrard et al., 2016). The fundamental idea of graph convolutional networks is to enhance node self-representation by using neighbor information. Therefore, GCNs are typically expressed as a neighborhood aggregation or message-passing scheme (Gilmer et al., 2017).
In the broad application of GCNs, GCN (Welling and Kipf, 2016) and GAT (Velickovic et al., 2017) showed the powerful ability to capture neighbor information. Despite this, they performed poorly in KG representation as they ignored relations. To emphasize the essential role of relations in entity representation, R-GCN (Schlichtkrull et al., 2018) used a matrix to represent each relation. However, massive relations in the knowledge graph make it challenging for the relation matrixes to be fully learned. Thus, most follow-up works used vectors to represent relations. For example, KBGAT (Nathani et al., 2019) concentrated the neighbor triples as information. CompGCN (Vashishth et al., 2019) leveraged the entity-relation composition operations from knowledge embedding methods like TransE (Bordes et al., 2013) as message. KE-GCN (Yu et al., 2021) passed the gradient of the scoring function to the central node. Never-
theless, none of the above models takes account of the inequality of relations and entities. In contrast, our RHGN is able to make a clear distinction between relations and entities, resulting in more distinct entity representations.

heading: 3 Preliminaries
text: In this section, we formalize the problem of entity alignment and give some related definitions.

heading: 3.1 Problem Definition
text: In this paper, we formally define a KG as G = (E, R, T ), where E is the set of entities, R is the set of relations, and T = E × R × E is the set of triples like (Florence, Country, Italy) as illustrated in Figure 1. Without loss of generality, we consider the entity alignment task between two KGs, i.e., G1 = (E1, R1, T1) and G2 = (E2, R2, T2). The goal is to find the 1-to-1 alignment of entities SKG1,KG2 = {(e1, e2) ∈ E1 × E2|e1 ∼ e2}, where ∼ denotes the equivalence relation. To train the model, a small subset of the alignment S′KG1,KG2 ∈ SKG1,KG2 is given as the training data, and we call it seed alignment set.

heading: 3.2 Graph Convolutional Layers
text: Following previous works (Sun et al., 2020b; Guo et al., 2020; Xin et al., 2022), our RHGN model is built upon GCN framework (Welling and Kipf, 2016) to embed the entities E in KGs. Our model contains multiple stacked GCN layers, which enables entity embeddings to incorporate information from higher-order neighbors. The input for k-th GCN layer is an entity feature matrix, Ek =
{ek1, ek2, ..., ekn|eki ∈ G}, where n is the number of entities in G. To update the embedding of entities in layer k, the GCN layer aggregates neighbor information, which can be formally described as:
ek+1i = γ k(eki , Aggj∈N(i)ϕ(e k i , e k j , r k i,j)) (1)
where N(i) is the neighbors of entity i, γ is the transformation function like MLP, Agg is the Aggregate function like sum, mean or max, and ϕ is the score function.

heading: 4 RHGN: Relation-gated Heterogeneous Graph Network
text: In this section, we first present an overview of our RHGN. Then we introduce the technical details of RHGN.

heading: 4.1 An Overview of RHGN
text: As shown in Figure 2, our approach contains four components: (a) Graph Data Preprocessing (GDP), (b) Relation Gated Convolution (RGC), (c) Crossgraph Embedding Exchange (CEE), and (d) Soft Relation Alignment (SRA). Specifically, GDP first preprocesses graphs through two aspects: completing graphs by adding inverse relations and constructing the cross graph by exchanging aligned entities. Then, several RGC layers are devised to aggregate information in both original and cross graphs to get the representation of entities and relations. Meanwhile, CEE exchanges the embedding of original graphs and cross graphs between each RGC layer for efficient information propagation. Finally, SRA employs the embedding of entities to produce soft labels for relation alignment and the embedding of entities and relations will be sent to the model loss for optimization.

heading: 4.2 Graph Data Preprocessing
text: In order to make better use of the relations and address heterogeneity, we first perform data preprocessing on graphs to make graphs more complete. In detail, GDP contains two parts: Inverse Relation Embedding and Cross Graph Construction.

heading: 4.2.1 Inverse Relation Embedding
text: Since relations in KGs are normally unidirectional, following previous works (Sun et al., 2020b; Vashishth et al., 2019), we also add inverse relation to KGs. The inverse relation is defined as:
rinvi = Winvri, (2)
where rinvi is the inverse relation of relation ri. Winv is the weight matrix of inverse relation transformation. Therefore, we extend graphs as:
T ′ = T ∪ {(t, rinv, h)|(h, r, t) ∈ T}, (3)
where (h, r, t) is the triple in the original graph.

heading: 4.2.2 Cross Graph Construction
text: As we discussed in Section 1, to address neighbor heterogeneity, in this part, we first construct cross graphs through the aligned entities in the seed alignment set for efficient information propagation across KGs. Specifically, as Figure 2(a) shows, Cross Graph Construction generates cross graphs by exchanging the aligned entities in the seed alignment set S′KG1,KG2 . The entities E cross 1 in the cross graph Gcross1 are defined as:
ecross1 = { e2 if e1 ∈ S′KG1,KG2 and e1 ∼ e2, e1 else.
(4) Similarly, the entities Ecross2 in the cross graph
Gcross2 are defined as:
ecross2 = { e1 if e2 ∈ S′KG1,KG2 and e2 ∼ e1, e2 else.
(5) Taking Figure 1 as an example, (Da Vinci, Citizenship, Italy) will be in cross KG2 as we exchange Da Vinci in KG1 and Leonardo da Vinci in KG2.
Finally, the cross graphs Gcross1 and G cross 2
are defined as Gcross1 = (E cross 1 , R1, T cross 1 ) and Gcross2 = (E cross 2 , R2, T cross 2 ).The embeddings of entities and relations are randomly initialized.

heading: 4.3 Relation Gated Convolution
text: After getting the preprocessed graphs, in Figure 2(b), we use RGC to aggregate neighbors and relations to the central entity. As discussed in Section 1, directly incorporating relation into entity representation may introduce much noise. To tackle this, we separate the semantic space of relations and entities. Specifically, in figure 3, we use a non-linear activation function (σ2) as a gate to aggregate neighbors and relations. The gate treats relations as control signals to regulate the inflow of neighbor information. For the entity i at k-th layer eki , the embedding of entity i at k+1-th layer e k+1 i is computed as follows:
ek+1i = σ1( ∑
j∈N(i) W ke (e k j ⊗ σ2(rki,j))), (6)
where N(i) is the set of neighbors of entity i, and rki,j is the relation from entity j to entity i, W k e is the entity weight matrix of k-th layer, ⊗ denotes element-wise multiplication between vectors, σ1(·) and σ2(·) are non-linear activation functions. We use tanh(·) for σ1(·) and sigmoid(·) for σ2(·).
Moreover, inspired by (Vashishth et al., 2019), we also update the embedding of relations rki,j as:
rk+1i,j = W k r r k i,j , (7)
where W kr is the relation weight matrix of the k-th layer. In order to reduce the semantic gap between the two KGs, we share the weights of the RGCs between two graphs in each layer.

heading: 4.4 Cross-graph Embedding Exchange
text: According to Section 4.2, we build the cross graph to address neighbor heterogeneity among different KGs. In this section, to make information propagation across KGs more efficient, we introduce a cross-graph embedding exchange method on both original and cross graphs to reduce the entity semantic distance between KGs. As illustrated in Figure 2(c), we exchange entity embeddings between the original graph and the cross graph at each intermediate layer. Formally, Ek and Ekcross represent the entity embedding of original graph and cross graph in k-th layer respectively, the k+1th layer can be computed as:
Ek+1 = RGC(Ekcross, R k, Gk,W k), (8)
Ek+1cross = RGC(E k, Rkcross, G k cross,W k). (9)
Compared with previous work (Cao et al., 2019) that adds edges between aligned entities in the seed alignment set, CEE can effectively reduce the distance of information propagation across two KGs. Taking the entity Florence in Figure 1 as an example, if we assume that Italy in two KGs is aligned, the information from Florence in KG1 can propagate to Florence in KG2 only through 3 edges and 2 nodes with the help of CEE. According to Huang et al. (2020), a shorter propagation distance spreads more information across two KGs, making the two graphs’ entity semantics closer.

heading: 4.5 Soft Relation Alignment
text: As discussed in Section 1, relation heterogeneity also complicates entity alignment. Relation alignment, which seeks out mutually similar ties across KGs, is one direct method for resolving this problem. However, due to the lack of labels, we need to produce soft relation alignment labels by ourselves.
Inspired by prior works (Wu et al., 2019b; Zhu et al., 2021), we make use of entities to produce soft relation alignment labels as shown in Figure 2(d). We define relation label embedding as:
r′ = concat[ 1
Hr
∑
ei∈Hr ei,
1
Tr
∑
ej∈Tr ej ], (10)
where Hr and Tr are the sets of head entities and tail entities of relaiton r, respectively. Then, the relation alignment label is defined as:
yij = I(cos(r′i, r′j) > γ), (11)
where γ is the hyperparameter of the threshold. It is noteworthy that our method may either produce multiple alignment labels or no alignment labels for one relation since relation alignment does not obey 1-to-1 constraints. As shown in Figure 1, Nationality and Famous People in KG1 may be similar to Citizenship in KG2, while Location in KG2 has no similar relation KG1. This feature makes us decide to convert relation alignment task to a multi-label classification task in model loss.

heading: 4.6 Training
text: In this subsection, we introduce our loss components: the entity alignment loss and the relation alignment loss, which capture alignment information of entities and relations, respectively.

heading: 4.6.1 Entity Alignment Loss
text: Following previous work (Sun et al., 2020b; Xin et al., 2022), we minimize the contrastive alignment loss to make the distance between the aligned entities as close as possible, while the distance between the non-aligned entities is very far. The alignment loss is defined as: L1 = ∑
(i,j)∈A+ ||ei−ej ||+
∑
(i′,j′)∈A− α1[λ−||ei′−ej′ ||]+,
(12) where ei is the entity embedding concentration of all layers in the original graph and cross graph. A− is the set of negative samples generated by truncated-ϵ negative sampling strategy, || · || denotes L2 distance. [·]+ = max(0, x), and we hope the distance of negative samples to be larger than a margin λ. α1 is a hyperparameter to keep the balance between positive and negative samples.

heading: 4.6.2 Relation Alignment Loss
text: As we mentioned in Section 4.5, we transform relation alignment into a multi-label classification task. Consequently, we first calculate the cosine similarity of relations in the last layer between graphs:
xij = cos(ri, rj). (13)
Then, we use the soft labels produced in SRA to calculate the relation alignment loss, we adopt the multi-label soft margin loss:
L2 = − 1 |R| ∑
i
(yi · log( 1
1 + exp(−xi) )
+(1− yi) · log exp(−xi)
1 + exp(−xi) ).
(14)
Finally, RHGH combines the two losses as:
L = L1 + α2L2, (15)
where α2 is a hyperparameter to keep the balance between entity alignment and relation alignment.

heading: 5 Experiments

heading: 5.1 Dataset
text: For the reliability and authority of experimental results, we use the dataset (V1) in OpenEA (Sun et al., 2020c) for evaluation since it closely resembles the data distribution of real KGs. It contains two crosslingual settings extracted from multi-lingual DBpedia: English-French and English-German, as well as two monolingual settings among popular KGs: DBpedia-Wikidata and DBpedia-YAGO. We use the setting that datasets contain 15K pairs of reference entity alignment and no reference relation alignment. Table 1 provides further information about the datasets. We adhere to OpenEA’s dataset divisions, which use a 20% seed for training, 10% for validation, and 70% for testing.

heading: 5.2 Implementation Details
text: We implement our method through PyG (Fey and Lenssen, 2019) on Pytorch. We initialize the trainable parameters with Xavier initialization (Glorot and Bengio, 2010) and optimize loss with Adam (Kingma and Ba, 2015). As for hyper-parameters, we decide the important hyperparameters by grid search and keep them the same in all datasets. For example the number of RGCs’ layers is 4, the hidden size of each layer is 256, the batch size is 256, and the learning rate is 0.001. We set α2 = 10 to keep the balance of alignment loss and semantic loss. We randomly sample 25 negative samples for each pre-aligned entity pair. After every 25 epochs, we resample 25 negative samples based on the CSLS (Lample et al., 2018) and resample 100 head and tail entities respectively to generate soft relation alignment labels. The threshold γ is 0.5, the negative sample distance margin λ is 1.5 and the negative sample weight α1 is 0.1.
Followed the previous work (Sun et al., 2020b; Xin et al., 2022), we also use early stopping to terminate training based on Hits@1 performance on the validation set with a patient of 25 epochs, and the maximum training epochs is 1000. According to most previous work, we report the Hits@1, Hits@5 and MRR (mean reciprocal rank) results to assess entity alignment performance. We conduct the experiments with 5-fold cross-validation to ensure the unbiased evaluation.

heading: 5.3 Benchmark Methods
text: To evaluate the effectiveness of RHGN, we compare it with the state-of-the-art supervised structurebased entity alignment methods. we use codes and parameters released by the authors and display the best results among reproduced results and reported results in original articles. In general terms, we can classify them as follows.
• Triple-based Models. These models focus on triple, they usually use TransE (Bordes et al., 2013) to represent entities and relations, including MTransE (Chen et al., 2017), IPTransE (Zhu et al., 2017), AlignE (Sun et al., 2018), and SEA (Pei et al., 2019).
• Neighbor-based Models. These models emphasize neighbor information but ignore the relation information, they usually use GNNs to represent entities, including GCNAlign (Wang et al., 2018), AliNet (Sun et al., 2020b), and HyperKA (Sun et al., 2020a).
• Relation-enhanced Models. These models take into account the importance of relation information and incorporate relation information into entity representations, including
RSN4EA (Guo et al., 2019), KE-GCN (Yu et al., 2021), and IMEA (Xin et al., 2022).
Our model and the above baselines all focus on the structural information of KGs. For a fair comparison, we disregard additional models that incorporate side information (e.g., attributes, entity names and descriptions) like RDGCN (Wu et al., 2019a), KDCoE (Chen et al., 2018) and AttrGNN (Liu et al., 2020b).

heading: 5.4 Experimental Results
text: The results of all methods on OpenEA datasets are shown in Table 2. In general, the RHGN model has achieved the best performance compared with these SOTA baselines. Specifically, our method outperforms the best-performing baseline (i.e., IMEA, KE-GCN) on Hits@1 by 3%-6%, on MRR by 1%-5%, and on Hits@5 by 1%-3% (except for D_W_V1). Additionally, we discover some interesting phenomena as follows:
First, on all datasets, relation-enhanced models outperform neighbor-based models, and both outperform triple-based models. This fully demonstrates that relation information plays an important role in neighbor information aggregation. Second, our model has significant improvements on EN_DE_V1 and D_Y_V1, but the improvements of our model are relatively limited on EN_FR_V1 and D_W_V1, and we find that all baselines do not perform well on datasets EN_FR_V1 and D_W_V1. We believe that the semantic distance between the graphs in the two datasets is far apart, which makes it is hard to find aligned entities.

heading: 5.5 Ablation Study

heading: 5.5.1 RGC’s Ability to Utilize Relations
text: To compare the ability to utilize relations of various convolutions, We replace the RGC with re-tuned GNN variants GCN (Welling and Kipf, 2016), GAT (Velickovic et al., 2017), R-GCN (Schlichtkrull et al., 2018), and CompGCN (Vashishth et al., 2019) with the same parameters. The results are shown in Table 3. Among these models, our RGC also achieves the best performance, as GCN and GAT ignore the relations, while R-GCN and CompGCN can not take advantage of the relations well. Meanwhile, the result that R-GCN and CompGCN outperform GCN and GAT proves the essential role of relations in entity representation.

heading: 5.5.2 The Impact of Different Heterogeneity
text: To verify the impact of different heterogeneity, figure 5 reports the performances after removing CEE and SRA, respectively. We observe that both components contribute to performance improvement, demonstrating that each component design in our framework is reasonable. Meanwhile, the effects of the two components on different datasets are also different, implying that the impact of neighbor heterogeneity and relationship heterogeneity varies between different KGs.

heading: 5.6 The Distance of Information Propagation
text: We explore the effect of RGC’s layer number on model performance as layer numbers reflect the distance of information propagation. In Figure 6, we present the effect of RGC’s layer numbers with 1 to 5 on EN_FR_V1. Obviously, RHGN with 4 layers achieves the best performance over all three metrics. When the number of layers exceeds 4, the performance decline as adding more layers allows the model to collect more distant neighbor data and adds noise during information propagation. We also observe that RHGN with 2 layers has a huge improvement over RHGN with 1 layer. We believe that due to the lack of exchange entity embedding, RHGN with 1 layer cannot obtain information from the other KGs, resulting in poor performance.
Then we calculate the shortest path length from the test set entities to the training set entities in the EN_FR_V1 dataset. The average and median of shortest path length are 1.5 and 1 in EN, and the length is 1.6 and 2 in FR. This shows that most entities need 3 to 4 hops to pass their own information to the aligned entity of another graph with CEE module. As a matter of fact, RHGN with 3 and 4 layers achieves similar performance and is ahead of other variants, which also verifies that our CEE module is effective.

heading: 5.7 Visualization of Entity Embedding
text: For a more intuitive comparison of how our proposed model addresses heterogeneity across different KGs with other methods, we conduct visualization on the D_W_V1 datasets. Specifically, we perform dimensionality reduction on entity embedding of GCN, GAT, R-GCN, CompGCN, and RHGN with t-SNE (Van der Maaten and Hinton, 2008). Results are shown in Figure 4, where the same color means entities are in the same KG. Ideally, the entity distributions of two graphs should overlap as much as possible, and entity embeddings should be sparsely distributed.
From Figure 4, we find some phenomena as follows. First, entities represented by previous methods have obvious clustering in space, while incorporating relation can effectively alleviate the clustering. This phenomenon suggests that relations play an essential role in distinguishing entities and preventing over-smoothing. Second, all previous arts have significant space that is not aligned, which demonstrates that they are unable to bridge the semantic space gap caused by KG heterogeneity. However, our RHGN model’s entity embeddings are sparsely distributed in space and have a high degree of overlaps, making the model distinguish entities well and easily find aligned entities.

heading: 6 Limitations
text: Although we have demonstrated the superiority of our RHGH model compared to previous work on four real-world datasets, there are still two limitations that should be addressed in the future:
(1) As our RGC layer employs the whole graph to learn the embedding of entities and relations, like most GCN’s frameworks, the computational resources and time required by our framework increase linearly with the size of KG. To make our RHGH model effective on the KG with millions of entities, it is desirable to apply some graph chunking techniques, such as Cluster-GCN (Chiang et al., 2019), to reduce the size of the KG for our RHGH model to improve computational efficiency.
(2) Currently, our RHGH model treats each relation individually. However, relation paths consisting of multiple relations will contain more complex semantic information in KGs. Relation paths enable entities to obtain higher-order neighbor information, but it is also more difficult to align relational paths in different knowledge graphs. In future work, we will explore more efficient ways
to utilize the relation path in entity alignment, such as the relation path matching in different KGs.

heading: 7 Conclusion
text: In this paper, we studied the problem of entity alignment and proposed the RHGN model, which could distinguish relation and entity semantic spaces, and further address heterogeneity across different KGs. Specifically, we first designed a novel relationgated convolutional layer to regulate the flow of neighbor information through relations. Then, we proposed an innovative cross-graph embedding exchange module, which reduces the entity semantic distance between graphs to address neighbor heterogeneity. Finally, we devised a soft relation alignment module for the unsupervised relation alignment task, which solves the relation heterogeneity problem between graphs. Extensive experiments on four real-world datasets verified the effectiveness of our proposed methods. In future work, we will explore more ways to utilize the relation information in entity alignment, such as the relation path matching in different KGs.

heading: Acknowledgements
text: This research was partially supported by grants from the National Natural Science Foundation of China (Grants No. U20A20229, No. 62106244), and the University Synergy Innovation Program of Anhui Province (No. GXXT-2022-042).

heading: A Supplementary Experiments
text: We add some experimental results to demonstrate the effectiveness of our framework. Due to space limitations, we present the experimental results in detail in the appendix.
A.1 Ablation Study on Other Datasets
To verify that our various modules in the RHGH framework (including RGC, CEE, and SRA) are valid, we have presented the experimental results on datasets EN_FR_V1 and D_W_V1 in Table 3 and Figure 5. To fully verify that all our modules are also effective on datasets EN_DE_V1 and D_Y_V1, Table 4 shows the capability of our RGC compared with other GCNs, while Table 5 proves the validity of CEE and SRA.
From Table 4 and Table 5, we find that RHGN achieves the best performance among all variants on most metrics in all datasets, which is consistent with the experimental analysis in Section 5.5. These experiments prove that all components are valid and non-redundant in the model.
A.2 Sensitivity Analysis of Other Parameters
In section 5.6, we have discussed how the number of layers affects the model performance and
found that the number of layers is determined by the distance of information propagation. Meanwhile, other hyper-parameters may also affect the performance of the model, such as relation alignment loss α2 and relation alignment label threshold γ. Figure 7 reports how these hyper-parameters affect the experiment results on D_W_V1. The effect of these hyper-parameters on model performance is slight and further illustrates the robustness of our RHGN framework. For other hyper-parameters (e.g., negative sample distance margin λ and negative sample weight α1), we follow the previous works(like AliNet (Sun et al., 2020b)).
A.3 Error Analysis In order to explore the advantages of our RHGN model, Table 6 shows the results for entities with different numbers of neighbors on EN_FR_V1. We observe that with the increase of neighbor number, the performance of our model improves significantly. In more detail, for entities with various neighbors, our RGC can better avoid the noise caused by multiple relations. However, for entities with fewer neighbors, there is not enough information for them to align. We will attempt to solve this problem by acquiring further neighbors (like relation paths) in future work.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 5.4, Section 6 and Section 7
7 A2. Did you discuss any potential risks of your work? We do not use huge models and our experiments are fair.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 5.1 and Section 5.2

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 5.1 and Section 5.2
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? They are public datasets and open source codes.

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 5.1
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Date contains no information that names or uniquely identifies individual people or offensive content.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 5.1 and Section 5.2
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 5.1
C 3 Did you run computational experiments? Section 5
7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? We do not use huge models and efficiency is not our goal
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 5.2

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? ection 5.2 and Section 5.4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 5.2
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We will attempt to solve this problem by acquiring further neighbors (like relation paths) in future work. ACL 2023 Responsible NLP Checklist