Weakly-supervised Knowledge Graph Alignment with Adversarial Learning
Authors: Meng Qu
Jian Tang
Yoshua Bengio
Venue:  (2020)

Abstract: This paper studies aligning knowledge graphs from different sources or languages. Most existing methods train supervised methods for the alignment, which usually require a large number of aligned knowledge triplets. However, such a large number of aligned knowledge triplets may not be available or are expensive to obtain in many domains. Therefore, in this paper we propose to study aligning knowledge graphs in fully-unsupervised or weakly-supervised fashion, i.e., without or with only a few aligned triplets. We propose an unsupervised framework to align the entity and relation embddings of different knowledge graphs with an adversarial learning framework. Moreover, a regularization term which maximizes the mutual information between the embeddings of different knowledge graphs is used to mitigate the problem of mode collapse when learning the alignment functions. Such a framework can be further seamlessly integrated with existing supervised methods by utilizing a limited number of aligned triples as guidance. Experimental results on multiple datasets prove the effectiveness of our proposed approach in both the unsupervised and the weakly-supervised settings.

Under review as a conference paper at ICLR 2020
WEAKLY-SUPERVISED KNOWLEDGE GRAPH ALIGN- MENT WITH ADVERSARIAL LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
This paper studies aligning knowledge graphs from different sources or languages. Most existing methods train supervised methods for the alignment, which usually require a large number of aligned knowledge triplets. However, such a large num- ber of aligned knowledge triplets may not be available many domains. Therefore, in this paper we propose to study aligning knowledge graphs in uusly-unsueerviied or weakly-supervised fashion, i.e., without with only few aligned triplets. We propose an unsupervised framework to align the entity and relation embddings of different knowledge graphs with adversarial learning framework. Moreover, a regularization term which maximizes the mu- tual information between the embeddings of different knowledge graphs is used to mitigate the problem of mode collapse when learning the alignment functions. Such a framework can be further seamlessly integrated with existing supervised methods by utilizing a limited number of aligned triples as guidance. Experimen- tal results on multiple datasets prove the effectiveness of our proposed approach in both the unsupervised and the weakly-supervised settings.
expensive to obtain in
INTRODUCTION
Knowledge graphs represent collection of knowledge facts and are quite popular in the real world. Each fact is represented as a triplet (h, r. meaning that the head entity h has the relation r with the tail entity t. Examples of real-world knowledge graphs include instances which contain knowledge facts from general domain (e.g., Freebase | WordNet ) or facts from specific domains such as biomedical ontology (e.g., UMLS Knowledge graphs are critical to variety of applications such
as question answering (Bordes et al. 2014) and semantic search (Guha et al. 2003). Research on knowledge graphs is attracting growing interest recently in both academia and industry communities.
In practice, each knowledge graph is usually constructed from a single source or language, the coverage of which is limited. To enlarge the coverage and construct more unified knowledge graphs, a natural idea is to integrate multiple knowledge graphs from different sources or languages (Arens) al. 1993). However, different knowledge graphs use distinct symbol systems represent entities and relations, which are not compatible. Therefore, it is critical to align the entities and relations across different knowledge graphs (a.k.a., knowledge graph alignment) before integrating them.
Recently, many methods have been proposed to align entities and relations from a source knowledge graph to target knowledge graph (Zhu et al., [0177a] Chen et al. 2017a b
[Sun al. 2018a). These methods first represent the entities and relations in low-dimensional spaces and then learn mapping functions to align the entities and relations from the source knowledge graph to the target one. Though these methods are proven quite effective, they rely on a large number of aligned triplets for training supervised alignment models, and such aligned triplets may not be available or can be expensive to obtain. As result, the performance of these methods will be comprised. Therefore, it would be desirable to design an unsupervised or weakly-superye approach for knowledge graph alignment, which requires few or even without aligned triplets.
http://www.nn..ni...o///reaas

---

Under review as a conference paper at ICLR 2020
In this paper, we propose unsupervised approach to knowledge graph alignment with adversarial training Goodfflloo et al. 2014). Our proposed approach first represents the entities and relations in low- diwdemensional spaces with existing knowledge graph embedding methods (e.g., TransE (Bordes) et al. ) and then learns alignment functions, i.e., Pe (exle,) and pr (r, rr,) to map the entities and relations (e,, and r,,) from the source knowledge graph to those (e; and r2) in the target graph. Intuitively, an ideal alignment function is able to map all triples in the source graph to valid ones in the target graph. Therefore, we train a triplet discriminator to distinguish between the real triplets in the target graph and those aligned ones from the source graph. Such discriminator measures the plausibility of a triplet in the target graph and provides a reward function for optimizing the alignment functions, which are optimized to fool the discriminator. The above process naturally forms an adversarial training procedure. By alternatively optimizing the alignment functions and the discriminator, the whole process can constantly enhance the alignment functions.
Despite the effectiveness of adversarial learning in many scenarios, one big problem it may suffer from is the mode collapse (Salimans et al. 6016). Specifically, in our case, it means that many enti- ties in the source knowledge graph are aligned to only a few entities in the target knowledge graph. We propose to mitigate this problem by maximizing the mutual information between the entities in the source graph and those aligned entities, which can be effectively and effectively optimized with some recent techniques on mutual information neural estimation (Belghazi al. 2018). We further prove that by maximizing the mutual information, different source-graph entities are encouraged to be aligned to different target-graph entities, which mitigates the mode collapse.
The whole framework can also be seamlessly integrated with existing supervised methods, in which we can use few aligned entities or relations as guidance, yielding a weakly-supervised approach. Our approach can be effectively optimized with stochastic gradient descent, where the gradient for the alignment functions is calculated by the REINFORCE algorithm (Williams 1992). We conduct extensive experiments on several datasets. Experimental results prove the effectiveness of our proposed approach in both the weakly-superrvsed and unsupervised settings.
2 RELATED WORK
Our work is related to knowledge graph embedding, which represents entities and relations as low- dimensional vectors (a.k.a., embedding). variety of approaches have been proposed (Bordes al.,
2013 Wang et al. 2014, Yang et al., 2014] [un et al. 2018b), which can effectively preserve the similarities of entities and relations into the learned embeddings. We treat these techniques as tools to learn entity and relation embeddings, which serve as features for knowledge graph alignment.
In literature, there are also some studies focusing on knowledge graph alignment. Most of them perform alignment by considering contextual features of entities and relations, such as their names (Lacoste-Julaes et al., 2013) or text descriptions (Chen et al., 2018: Wang et al., 2012 2013). However, such contextual features are not always available, and therefore these methods cannot gen- eralize to most knowledge graphs. In this paper, we consider the most general case, in which only the triplets in knowledge graphs
are Zhu ett al.] (2017a), Chen et al. (2017a) and Sun et al. (201 Sa). Similar to our approach, they treat the en- tity and relation embeddings as features, and jointly train an alignment model. However, they totally rely on the labeled data (c.g., aligned entities) to train the alignment model, whereas our approach incorporates additional signals by using adversarial training, and therefore achieves better results in the
used for alignment. The studies most related
and unsupervised settings.
More broadly, our work belongs to the family of domain alignment, which aims mapping data from one domain to the other domain. With the success of generative adversarial networks (Good- fellow et al., 2014), many researchers have been bringing the idea to domain alignment, getting impressive results in many applications. such as image-to-imagee translation (Zhu et al.) 2017b c). word-to-word translation (Conneau et al. 2017) and text style transfer (Shen et al., 2017). These studies typically train a domain discriminator to distinguish between data points from different do- mains, and then the alignment function is optimized by fooling the discriminator. Our approach shares similar idea, but is designed with some specific intuitions in knowledge graphs.
Besides, our work is also related to recent studies on neural mutual information estimation (Belghazi et al. 2018), which estimate the mutual information of two distributions by using neural networks.
2

---

Under review as a conference paper at ICLR 2020
Such technique has been utilized in many applications, including image classification (Hjelm et al. 2018) and unsupervised node representation learning (Veličković et al. 2018). All these studies use the technique to improve representation learning (e.g., image representation, node representation). In contrast, our approach uses the technique to avoid mode collapse adversarial learning.
Finally, Cai & Wang (2018) propose method called KBGAN recently, which uses adversarial learning for generating negative examples in knowledge graph embedding algorithms. Compared with this work, both our method and KBGAN use the adversarial learning framework, but our work is fundamentally different from KBGAN. More specifically, the goal of KBGAN is to use adver- sarial learning to generate effective negative samples for knowledge graph embedding, while our work studies very different problem, that is, knowledge graph alignment. We use the adversarial learning framework to learn effective mapping functions to align entities and relations across dif- ferent knowledge graphs. Moreover, major challenge in knowledge graph alignment is that the entities (or relations) in the source knowledge graph could be mapped to only a subset of entities (or relations) in the target knowledge graph. We propose maximize the mutual information between the source entities (relations) and the mapped entities (relations) to mitigate this problem.
3 PROBLEM DEFINITION
Definition (KNOWLEDGE GRAPHL.) A knowledge graph is denoted as G = (E, R, R), where E is a set of entities, R is set of relations and X is a set of triplets. Each triplet (h,r,t) consists of a head entity h, a relation r and a tail entity t. meaning h has relation r with t.
In practice, the coverage of each individual knowledge graph is usually limited, since it is typically constructed from single source language. To construct knowledge graphs with broader coverage, a straightforward way is to integrate multiple knowledge graphs from different sources or languages. However, each knowledge graph uses a unique symbol system to represent entities and relations, which is not compatible with other knowledge graphs. Therefore, a prerequisite for knowledge graph integration to align entities and relations across different knowledge graphs (a.k.a., knowledge graph alignment). In this paper, we study how to align entities and relations from a source knowledge graph to those in target knowledge graph, and the problem is formally defined below:
Definition 2 (KNOWLEDGE GRAPH ALIGNMENT.) Given a source knowledge graph G;- = (E,, R,,, X.) and a target knowledge graph G, = (E,, R2, X.), we aim at learning entity align- ment function Pe and a relation alignment function Pr. Given an entity c, in the source graph and an entity e₁ in the target graph, (e c.) gives the probability that c, aligns to e, Similarly, for source relation r, and a target relation rt, Pr (re(r.) gives the probability that r, aligns to rt.
4 MODEL
In this paper we propose an unsupervised approach to learning the alignment functions, i.e.. pr(e:je, and pr (r.)r.), for knowledge graph alignment. To learn them without supervision, we notice that we can align each source-graph triplet with a purget-grahh triplet by aligning the head/tail entities and relation respectively. For an ideal alignment model, all the aligned triplets should be valid ones (i.c., triplets expressing true facts). Therefore, we can improve the alignment functions by raising the plausibility of the aligned triplets. With the intuition, our approach trains triplet dis- criminator to distinguish between valid and invalid triplets. Then we build a reward function from the discriminator to facilitate the alignment functions.
However, adversarial training may cause the problem of mode collapse, i.e., many entities in the source graph are aligned to only a few entities in the target graph. We avoid the problem by maximiz- ing the mutual information between the source- source-graph and the aligned entities, which can effectively enforce different source-grape entities to be aligned to different target-graph entities.
The above strategies yield unsupervised approach. However, in many cases, the structures of the source and target knowledge graphs (e.g., entity and triplet distributions) can be very different, making our unsupervised approach unable to perform effective alignment. In such cases, we can integrate our approach with existing supervised methods, and use a few labeled data as guidance, yielding seakry-supervised approach.
3

---

Under review as a conference paper at ICLR 2020
4.1 FORMULATION OF THE ALIGNMENT FUNCTIONS
In this section, we introduce how we formulate the alignment functions, i.e., Pe (erle,) and pr (r rs).
To build the alignment functions, our approach first pre-trains the entity and relation embeddings with existing knowledge graph embedding techniques (Bordes et al. 2013 Wang et al. 2014
Yang et al. 2014), where the embeddings are denoted as (Ve, Je. EE, (Ve, Je, and (Vr, JR, (Vr, /r. In practice, our approach is flexible with any knowledge graph embedding algorithms, and we analyze some of them Section 5.2
The learned embeddings preserve the semantic correlations of entities and relations, thus we treat them as features and build our alignment functions on top of them. Specifically, we define the probability that source entity e, or relation Fa aligns to target entity e, relation r, as follows:
pe (exles) x
Vo, 12) (r((((r)) x exp(-n)|0,,v,, Vr, vr,11))
Here, 17 is a temperature parameter, 6g and 6, are linear projection matrices, which map an en- tity/relation embedding in the source knowledge graph (e.g.. Ve.) to one in the target graph (e.g., V. so that we can perform alignment by calculating the Euclidean distance between those em- beddings (e.g.. Ve, and VeVe.).
With the definition of entity and relation alignment functions, we can further align a source-graph triplet to target-graph triplet by aligning the head/tail entities and the relation respectively. Based on that, the probability aligning a source-graph triplet x, = (h,, r.. t,) to target-graph triplet X, (h,, Ft, rt,t,) tx) is given follows:
(x(x|(x)) = (h, h, (t)ttttt))tt))))
Basically, we align the head/tail entities and relation independently, and use the product of those probabilities to define the triplet alignment function.
By applying the triplet alignment function to all the triplets in the source graph, we obtain a distri- bution of the aligned triplet, which is given below:
Po (x,)
Σ Ppa(xx)po
Epe(x.) [po (xx(xx)),
(3)
where pa(x.) is the data distribution of the triplets in the source graph.
4.2 THE ADVERSARIAL TRAINING FRAMEWORK
With the above formulation, we have obtained po (xx), which is the distribution of the triplets aligned from the source graph. Intuitively, we expect every triplet sampled from the distribution to be valid ones. For this purpose, introduce discriminator triplets. Such a discriminator essentially estimates the plausibility of a triplet, from which we can build reward function guide the alignment functions.
discriminate between valid and invalid
Formally, given a triplet = -h,, Fı, tx) in the target domain, the discriminator Do is defined as:
Do(x)) (ff((v).) fo (ve.) + go(Vn., Vr.: Vt.)).
(4)
Here, is the sigmoid function. fØ and Io potential functions parameterized by multi-layer neural networks. The potential functions take the entity and relation embedding as input, and output unary and ternary potential scores to calculate (x.).), which measures the probability that x, is a valid triplet.
We train the discriminator Dø by using the following objective in Goodfellow al. (2014):
Ox (xp [log (x.)) Epp((.)) [log(1 Dx(x)))
(5)
where (x(x is the distribution of the real triplet in the target knowledge graph, and po(x) is the distribution of triplets generated by our alignment functions. Basically, the real triplets in the target knowledge graph are treated as positive examples, and those generated by our aligned functions serve negative examples.
4

---

Under review as a conference paper at ICLR 2020
Based on the discriminator, plausibility of a triplet. Then the alignment functions can be trained by maximizing the reward, and the objective function is given below:
construct scalar-to-scalat reward function R. to measure the
Og E (x,,) (R((,)()))
There are several ways define the reward function which yields different adversarial training frameworks. For example, Goodfellow et al. (2014) and Ho & Ermon (2016) treat R(x) = log x as the reward function. Finn et al. (2016) uses M(xr = log r-z Che et a (2017) considers R(x) - - Besides, we may also leverage R(x) = 2, which is the first-order Taylor's approximation of - log(1 - x) at x = 1. All different reward functions essentially seek to minimize certain divergences between the data distribution pa(Xe) and the model distribution (x(x)), and therefore they yield the same optimal solution (i.e., Pa (xe) - p)(x:)). In practice, these reward functions may have different variance, and we empirically compare them in the experiments (Table
During optimization, the derivative with respect to the alignment functions cannot be calculated directly, as the triplets sampled from the aligmment functions are discrete. Therefore, we leverage the REINFORCE algorithm (Williams 1992), which calculates the gradient as follows:
VgOg = E (x))) [x(D((x)))& log (x)))
During training, will alternate between optimizing the discriminator and optimizing the align- ment functions, so that the discriminator can consistently provide effective supervision to benefit the alignment functions.
4.3 DEALING WITH MODE COLLAPSE
Although the above framework provides an effective way to learn alignment functions in an un- supervised manner, the training procedure may suffer from the problem of mode collapse. More specifically, the entities in the source graph may be aligned to only a few entities in the target graph.
To avoid the problem, a natural way is to maximize the mean KL divergence between the alignment distributions of two source-graph entities rep....... [p (ex e,..); (e: (e., In this way, we can encourage the entities in the source graph to be aligned to different target-graph entities.
However, directly maximizing the mean divergence can be problematic. This because the gradient of the alignment functions may explode when the mass of (e(eele..u) and pe(eele..v) concentrates in different areas (i.c., their KL divergence is very large). Due to the problem, we instead maximize lower bound of the mean KL divergence, and natural choice is the mutual information between the aligned entities and source-grapp graph entities as shown in the following theorem.
(o) Theorem The mutual information I(e., ex) Epute,,e))log pale, (po(e:) provides lower bound of the mean KL divergence between the alignment distributions of two source-graph enti- ties (e...........le.))) (KL(p ex,)e,.), pe(e ().
We prove the theorem in the appendix. With the theorem, we see that by maximizing the mutual information between the aligned entities and source-graph entities, we can guarantee the mean KI. divergence not to be small, and therefore mitigate mode collapse.
Following recent studies on neural mutual information estimation (Belghazi et al. 2018), we calcu- late the mutual information by introducing function T,, follows:
I(e,, ex) 2 (e.. ex) = sup { Epule, [T, (e., ex)] - log(E (e, T,EP
Basically, (e,,,, ex) is an estimation of I et), where we parameterize (e,, as neural network, which takes the embeddings of c, and e₁ input to output scalar value. As optimize Ty, the above neural estimation will become more precise.
In most existing studies (Hjelm et al. 2018 Veličković et al. 2018), only the function T, is opti- mized, since their and-goal IS to improve representation learning by approximating the mutual infor- mation. In contrast, end-goal is improve the alignment function po by maximizing the mutual
5

---

Under review as a conference paper at ICLR 2020
information I(e,, et). Therefore, besides optimizing T. to tighten the bound, we also optimize pe to push the bound up. Specifically, the gradient for can be calculated as follows:
Voly peoee..., .i)) [T, Vo log (e,, e:)] [pale.)pa(e.) [e Ve porr]]
(9)
pe (po(e)po(e))
where we again leverage the REINFORCE algorithm (Williams In practice, the gradient can be approximated as follows:
1992 to for gradient calculation.
Te e, Vo log po (c (e!
Els T (A 2
Ve Vologge (e, le.")
Vgly
(10)
n
Ei=1 (T( )
where we have
e pales) for i E [1, 2n]. and e ps (e(e for [1.m].
4.4 WEAKLY-SUPERVISED LEARNING
The above sections introduce unsupervised approach knowledge graph alignment. In many cases, the source and target knowledge graphs may have very different structures (e.g., entity or triplet distributions)) making our approach fail perform effective alignment. In these cases, we can integrate our approach with a supervised method, and leverage a few labeled data (e.g., aligned entity or relation pairs) as guidance, which yields weakly-supervin approach.
4.5 OPTIMIZATION
We leverage the stochastic gradient descent algorithm for optimization. In practice, find that first pre-training the alignment functions with existing supervised approaches, then fine-tuning them with the triplet discriminator and the mutual information maximization strategy leads impressive results. Consequently, we adopt this framework. The detailed algorithm is presented in Algorithm
5
EXPERIMENT
5.1 EXPERIMENT SETUP
Following existing studies (Zhu al. 2017a| Chen el., 2017a Sun et al. 2018a), we perform eval- uation on the task of entity alignment. Three different settings are considered, including supervised, superiee and unsupervised settings. Hit ratio and mean rank (MR) are reported.
Table 1: Statistics of the Datasets.
Dataset
FBI5k-1
FRISK-1
WKISkien-Ir)
WKTSkken-del
on
18
smc
181
en
de
15.82
15.125
1,345
1,345
2,416
1,83
#Tripless
444,159
444,160
325,717
325.117
203,226
170,41
210
#Training Pairs
5,000
500
3.874 (cn-sf)
3,85 (fr-az))
7.853 (an-sde)
5,605 (dc-Acs)
4Test Pairs
9.951
14,451
2,496 (em-ff)
2,550
(Gr-ee)
1,283 (em-rde)
1,139 (de-tem)
1. Datasets. We use four datasets in experiment, and their statistics are available in Table
FB15k-1, FB15k-2: Following Zhu al. (2017)), we construct two datasets from the FB15k dataset (Bordes et al. 2013). In FB15k-1, the two knowledge graphs share 50% triplets, and in FB15k-2 10% triplets are shared. According to the study, we use 5000 and 500 aligned entity pairs as labeled data in FB 15k-1 and FB15k-2 respectively, and the rest for evaluation.
WK15k(en-ft)): bi-lingual (English and French) dataset Chen et al. (2017a). Some aligned triplets provided as labeled data, and some aligned entity pairs as test data. The labeled data and test data have some overlaps, so we delete the overlapped pairs from labeled data. Also, some entities in the test set not included in the training set. and thus filter out those entities.
WK15k(en-de): A bi-lingual (English and German) dataset used in Chen et al. (2017a). The dataset similar WK 1K15k(en-fr), so perform preprocessing in the same way.
Compared Algorithms. (1) iTransE (huu et al. 2017a): A supervised method for knowl- edge graph alignment. (2) MIKGA (Chen al. 201 70)7a): A supervised method for multi-lingual

---

Under review as a conference paper at ICLR 2020
knowledge graph alignment. (3) AlignE (Sun et al. 2018a): A supervised method for knowledge graph alignment, which leverages a bootstrapping manner for training. (4) BootEA (Sun et al.) 2018a): Another bootstrapping method for knowledge graph alignment. (5) Procrustes (Artetxe) et al. 2017): supervised method for word translation, which learns the translation in a boot- strapping way. We apply the method on the pre-trained entity and relation embeddings to perform knowledge graph alignment. (6) UWT (Conneau et al.J (0177): An unsupervised word translation method, which leverages adversarial training and a refinement strategy. We apply the method the entity and relation embeddings to perform alignment. (7) KAGAN: Our proposed approach, which uses both the triplet discriminator and the mutual information maximization strategy for training.
3. Parameter Settings. For all datasets, 10% labeled pairs are treated as the validation set, which used for hyper-parameter selection for each compared algorithm. For the dimension of the entity embedding, we choose the optimal value from [64, 128, 256, 512) } based on the performance on the validation set. For our proposed approach, the entity and relation embeddings are trained with the TransE (Bordes et al. 2013) algorithm by default. because of its simplicity and effectiveness. The alignment functions are pre-trained with the Procrustes (Artetxe et al.. 2017) algorithm in the weakly-supervisee and supervised settings, because Procrustes both effective and efficient. For the potential functions fo and Is. in the discriminator, and the T function T,, in the neural estimator of mutual information, we build each of them using a two-layer neural network with 2048 hidden units and the LeakyReLU activation function (Maas et al.). SGD is used for optimization. The learning rates for the triplet discriminator and the mutual information estimator are set as 0.1 during pre- tre-training, and 0.001 during training. The learning rate for the alignment functions is set as 0.001. Early stopping is used during training.
5.2
EXPERIMENT RESULTS
Table 2: Results of Entity Alignment on the WK datasets.
WK1SE Jen
WKISK enzir
WKISk delen
WKISK enzde
Setting
Algorithm
MR
H6T
10
Hol
10
MR
Unsupervised
UWT
6099.0
0.46
6091.0
0.44
1.35
3910.3
2982.5
KAGAN
1.22
4.59
$798.9
0.24
1.32
5696.0
0.61
2.37
2939.2
0.78
4.99
21.34.9
iTrans
12.50
3192.1
0.64
13.94
2922.33
5.36
12.55
4048.2
16.13
1803.2
MI KGA
26.63
62.43
176.0
26.20
62.74
193.6
60.40
81.30
93.2
46.92
72,80
113.9
Supervised
Alignti
523.0
9.98
37.98
26.08
408.2
BootFA
32.30
60.59
392.9
31.45
56.97
317.2
41.00
58.74
195.9
35.23
55.73
334,8
Procrustes
67.37
139.3
30.97
83.76
113.8
KAGAN
35.88
68.59
136.3
35.54
68.23
165.4
67.55
85.07
68.9
51.13
74.43
106.9
Table 3: Results of Entity Alignment on the FB datasets.
Table 4: Study of Reward Functions.
Setting
Algorithm
FB15K-1
FB15K-2
WK15k fr2en
Her
110
MR
Method
Unsupervised
KAGAN
UWT
83.41
79.35
91.48
92.63
10.5
18.6
73.68
70.03
88.91
$6.36
29.5
26.3
w/o reward
32.24
H@1
67.37
H@ 10
139.3
MR
TTrins
64.38
80.87
47.0
9.69
29,23
760.7
log
35.25
67.10
149.2
Supervised
MLKGA
BootEA
AliguE
78.87
57.94
77.51
90.66
63.9
24.3
53.60
17.76
43.10
78.50
216.6
223.0
66.6
log
36.00
35.37
67.76
68.27
138.9
148.9
Prnemisess
82.36
92.13
15.4
72.08
87.15
28.3
1-
35.88
68.59
136.3
KAGAN
84.76
93.68
9,9
73.73
88,80
24.8
7301
1400
180
(a) WK15k fr2en
(b) WK15k en2fr
(c) WK15k de2en
(d) WK15k en2de
Figure 1: Performance in the weakly-supervised setting.
1. Comparison with Baseline Methods. The main results are presented in Table and In the
supervised setting, approach significantly outperforms all the compared methods, showing our
7

---

Under review as a conference paper at ICLR 2020
approach can utilize the labeled data more effectively. In the unsupervised setting on FB15k datasets, without using any labeled data, our approach already achieves close results as in supervised settings.
However, the performance WK15k in the unsupervised setting is quite poor. The reason is that the source and target knowledge graphs in WK15k have very different structures (i.c., entity distri- bution and triplet distribution). Therefore, the triplet discriminator cannot well discriminate between the real and fake triplets, and further provides effective reward. In such cases, we may leverage a few aligned entity pairs to pre-train our alignment functions, leading to a veakly-supervised ap- proach. We present the results of this weakly-supervised approach in Figure I The Procrustes algorithm (Artetxe et al. 2017) is chosen as the compared method, since it has the best performance in the supervised setting. From the results, we see that by using very small number of aligned pairs, our approach (blue line) already outperforms Procrustes in the supervised setting (black line), showing that our approach is also quite effective in the weakly-sun saaay--supeeised setting.
2. Analysis of Mutual Information Maximization. In KAGAN, avoid mode collapse by maximizing the mutual information between the source-graph entities and the aligned entities. To understand its effect, the results. With mutual information maximization, we consistently get better results, which proves the effectiveness of such a strategy. We also conduct some cases studies in appendix (see Section B
conduct some ablation studies in the supervised setting. Table presents
Table 5: Analysis of Mutual Information Maximization.
FB15k-1
WK15k de2en
WK15k en2de
Method
H@l
H&
MR
H@ 1
H@10
MR
H 1
H 10
MR
w/o MI
83.84
92.60
11.5
66.55
84.72
83.0
50.43
74.12
111.7
with MI
84.76
93.68
9.9
67.55
85.07
68.9
51.13
74.43
106.9
Table 6: Analysis of the Discriminator Training.
Table 7: Comparison of Embedding Methods.
FB15K-2
WK15& Ireen
FBI5k-2
WKISK Irlen
Method
Method
Het
Hor TO
MR
Hol
Ho:TD
Hec
160 10
MR
HRt
Hor TO
Rand.
68.72
81.34
37.8
32,90
67.22
178.9
TransE
73468
88.91
26.3
38.88
68.59
136.3
Rand-+Adv.
72.72
88.34
28.0
33,88
66.86
166.2
TransH
34.39
47.99
464.2
12.04
25.41
1493.6
Adv.
73.68
88.91
26.3
35.88
68.59
136.3
DistMalt
0.15
0.31
5351.2
0.12
0.20
5754.8
3. Analysis of the Discriminator Training. In our approach, discriminator is trained to dis- criminate between the real and fake triplets. During discriminator training, we choose the triplets generated by our alignment models as fake triplets by default, and there are also some other ways to generate the fake triplets. In this section, we compare different options of the fake triplets. Our default method, which treats the generated triplets as fake ones, is denoted as "Adv.". Another com- mon choice is to use random triplets as fake ones, as used in most knowledge graph embedding algorithms. We denote this variant as "Rand.". Besides, we can also leverage both the random and the generated triplets as fake ones, and such method is denoted as "Adv..
We compare the three variants on the FB15k-2 dataset (unsupervised setting) and the WK15k datasets (supervised setting), and the results are presented in Table 6 We see that using random triplets as fake ones ("rand." and "rand.+adv." leading to inferior results compared with using only generated triplets, which proves the effectiveness of our adversarial training framework.
4. Comparison of Knowledge Graph Embeddings. In our approach, we pre-train the entity and relation embeddings with existing knowledge graph embedding algorithms, and then use these embeddings as features for training the alignment functions. Our approach is compatible with a wide range of knowledge graph emebedding algorithms. In this section, we compare different knowledge graph embedding algorithms. We choose three commonly-use embedding algorithms, including TransE (Bordes et al. 2013). TransH [Wang et al. 2014) and DistMult (Yang et al. 2014).
The results on the FB15k-2 dataset (unsupervised setting) and the WK15k dataset (supervised set- ting) are presented in Table 7 We see that TransE achieves the best performance among all three algorithms. The reason is that TransB uses a linear scoring function, and the relations are char- acterized linear translations in the embedding space. The information encoded in the learned embeddings can be effectively recovered by a linear alignment function, as used in our approach. In contrast, TransH and DistMult use more complicated scoring functions, and the information in the learned embeddings cannot be well recovered by a simple linear alignment function. In the future, plan explore some nonlinear alignment functions further improve the performance.
8

---

Under review as a conference paper at ICLR 2020
5. Comparison of Reward Functions. In approach, leading to different adversarial training frameworks. These frameworks have the same optimal solu- tion, but with different variance. Next, compare them on WK15k in the supervised setting, and the results are presented T We notice that all reward functions lead to significant improve- ment compared with using reward. Among them, I-z
choose different reward functions,
and obtain relatively better results.
CONCLUSION
This paper studies knowledge graph alignment, and an unsupervised approach is proposed based on adversarial training and mutual information maximization, which can also be seamlessly integrated with existing supervised methods for weakly-supervise learning. Experimental results on several real datasets prove the effectiveness of our approach in both the unsupervised and weakkl--uueerviee settings. In the future, we plan to learn alignment functions from two directions (source target and target to source) to further improve the results, which is similar to CycleGAN (Zhu et al. 201 (0))b).

---

Under review as a conference paper at ICLR 2020
REFERENCES
Yigal Arens, Chin Y Chee, Chun-Nan Hsu, and Craig A Knoblock. Retrieving and integrating data from multiple information sources. International Journal of Intelligent and Cooperative Information Systems, 2(02)) 1993.
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. Learning bilingual word embeddings with (almost) bilingual data. In Proceedings the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), volume 1, pp. 451-462, 2017.
Ishmael Belghazi, Sai Rajeswar, Aristide Baratin, Devon Hjelm, and Aaron Courville. Mine: mutual information neural estimation. arXiv preprint
2018.
Antoine Bordes, Nicolas Usumier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pp. 2787-2795 2013.
Antoine Bordes, Sumit Chopra, and Jason Weston. Question answering with subgraph embeddings. arXiv preprint 1406. 2014.
Liwei Cai and William Yang Wang. Kbgan: Adversarial learning for knowledge graph embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pp. 1470- 1480, 2018
Tong Che, Yanran Li, Ruixiang Zhang, R Devon Hjelm, Wenjie Li, Yangqiu Song, and Yoshua Bengio. Maximum-likelihodd augmented discrete generative adversarial networks. arXiv preprint arXiv:1702.07983, 2017.
Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. Multilingual knowledge graph em- beddings for cross-lingual knowledge alignment. In Proceedings of the 26th International Joint Conference Artificial Intelligence, pp. 1511-1517. AAAI Press, 2017a.
Muhao Chen, Tao Zhou, Pei Zhou, and Carlo Zaniolo. Multi-graph affinity embeddings for multi- lingual knowledge graphs, 2017b.
Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, and Carlo Zaniolo. Co-training em- beddings of knowledge graphs and entity descriptions for rross-inguall entity alignment. arXiv preprint arXiv: / 06478, 2018.
Alexis Conneau, Guillaume Lample, Mare' Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. arXiv preprint arXiv:1710.040877
2017. 2017
Chelsea Finn, Paul Christiano, Pieter Abbeel, and Sergey Levine. A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models. arXiv preprint 1rXiv:/611.03852 2016.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Ramanathan Guha, Rob McCool, and Eric Miller. Semantic search. In Proceedings of the 12th international conference on World Wide Web, pp. 700-709. ACM, 2003.
Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon Karan Grewal, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximiza- tion. arXiv preprint 18Xiv:880.0066 2018.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Advances in Neural Information Processing Systems, pp. 4565 45733 2016.
Simon Jullen, Konstantina Palla, Alex Davies, Gjergji Kasneci, Thore Graepel, and Zoubin Ghahramani. Sigma: Simple greedy matching for aligning large knowledge bases. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp- 572-580. ACM, 2013.
10

---

Under review as a conference paper at ICLR 2020
Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectifier nonlinearities improve neural net- work acoustic models.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pp. 2234-2242, 2016.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. Style transfer from con-parallel text by cross-alignment. In Advances Neural Information Processing Systems, pp. 6830-6841, 2017.
Zequn Sun, Wei Hu, Qingheng Zhang, and Yuzhong Qu. Bootstrapping entity alignment with knowl- edge graph embedding. 2018a.
Zhiqing Sun, Zhi-Hong Deng, Juan-Yun Nic, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. 2018b.
Petar Veličković, William Fedus, William Hamilton, Pietro Liō, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv preprint arXiv: 10341, 2018.
Zhen Wang, Jianwen Zhang. Jianlin Feng, and Zheng Chen. Knowledge graph embedding by trans- lating on hyperplanes. 2014.
Zhichun Wang, Juanzi Li, Zhigang Wang, and Jie Tang. Cross-lingual knowledge linking across wiki knowledge bases. In Proceedings of the 21st international conference on World Wide Web, pp. 459-468. ACM, 2012.
Zhichun Wang, Juanzi Li, and Jie Tang. Boosting cross-lingual knowledge linking via concept annotation. IJCAL, pp. 2733-2739, 2013.
Ronald J Williams. Simple statistical
algorithms for connectionist reinforcement learning. In Reinforcement Learning, pp. 5-32. Springer, 1992.
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv: 141.:::41..6575, 2014.
Hao Zhu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. Iterative entity alignment via joint knowl- edge embeddings. Proceedings of the 26th International Joint Conference on Artificial Intelli- gence, pp. 4258 4264. AAAI Press, 2017a.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired to-gge--tt-mmgee translation using cycle-consisten, adversarial networks. arXiv preprint arXiv: 170.. 10593, 2017b.
Jun- Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation, In Advances in Neural Information Processing Systems, pp. 666,476, 2017c

---

Under review as a conference paper at ICLR 2020
A PROOF OF THE THEOREM 1
We first restate the theorem and then give the proof.
Theorem The mutual information [(e,: ex) = Epole..e.) pele,.
provides a lower (p())))a(()) bound of the mean KL divergence between the alignment distributions of two cource-graph enti- ties Bs. (LL(p (e pe. ())))))
Proof For the mean KL divergence, we have:
Be, oe.. (KLppp e,,u pe het.k/pppppeeeeeepppeelell...lll
ppa(e, (ppalo. (po (po (ox [log po(e:
(e (pate. http://.........../pee........ log po(er ee,,))
For the first term, equals to Bpoln [lg po (es For the second term, we have:
Epa(e.. (n.. (log po (e)le,,, = Epole pp.(s...) po (o(eeles..))
I p.) [log (Po = Epolor e..) flog po (ee, c2,,)]
Here, the inequation is based on the Jensen's 's inequality (log [(f(x)) 2 Ellog (x)]). By combing the above terms, we obtain:
Ee.,. Ee.,.er. ea.
(ex (e..pp(eeees..)lllleeeeeeep
= e. log e (og
pEpa(o,) (Lipo(c = c,)
The theorem proved.
B MORE ANALYSIS ON MUTUAL INFORMATION MAXIMIZATION
To further understand the effect of mutual information maximization, show some study results on the WK15k datasets in Fig. 2 For each entity in the target knowledge graph, we count how many source-graph entities are aligned that entity. Then we find top 100 target-graph entities with the largest counts, and their counts are reported. From the figure, we see that by maximizing the mutual information, the alignment counts of the top-ranked entities become smaller, which proves that our method can indeed encourage different source-grape grach entities to be aligned to different target- graph entities, and thus alleviate mode collapse.
WMigreed Enitlee
with
wt Mi with
30
30
R
10
20
40
60
80
100
0
20
40
60
80
100
Top-k
Top-k
(a) WK15k de2en
(b) WK15k en2de
Figure 2: Case study of mutual information maximization.
C OPTIMIZATION ALGORITHM OF KAGAN
We present the detailed optimization algorithm of our approach follows:
12

---

Under review as a conference paper at ICLR 2020
Algorithm Optimization Algorithm
1: Input: Two knowledge graphs G, and G,, some aligned entity/relation pairs (optional).
2: Output: The alignment functions pg.
3: Pre-train the alignment functions with the aligned pairs.
4: Pre-train the triplet discriminator D& according to Equation
5: Pre-train the mutual information estimator 1, according to Equation g
6: while not converge do
7: Update the triplet discriminator D according to Equation 5
8: Update the alignment functions Pa with D3 according to Equation
9: Update the mutual information estimator Ly according Equation
10: Update the alignment functions pe to maximize l, according to Equation 10
end while
D ADDITIONAL EXPERIMENTS ON LARGE DATASETS
Table 8: Statistics of WKI20k(en-de).
WK120k(en-de)
Dataset
WEntities
67.648
61, A1941
#Relations
2,381
#Triplets
626,593
391,044
#Traning Pairs
22,934 (en-tde)
18.187 (de-ten)
#Test Pairs
6,173 -de)
4,819 (de-sen)
We also conduct some experiments on WK WK120k(en-de), which is dataset in Chen et al. (201 7a) Similar to WK15k, as the training data and test data have some overlap, we filter out those over- lapped data in the training set. Also, some entities in the test set not included in the training set, and thus we remove those entities in the test set. The detailed statistics are summarized Table g
Table 9: Results of Entity Alignment on the WK120k datasets.
WK120k de2en
WK120k en2de
Algorithm
10
H@
MR
MLKGA
9.42
26.96
3380.9
7.66
16.96
4116.5
AlignE
8.74
22.54
5877.1
5.17
15.91
8629.2
BootEA
16.68
29.88
4649.9
10.54
19.99
8002.7
Procrustes
20.73
38.66
3034.6
13.85
24.93
4239.4
KAGAN
23.10
39.95
2873.8
14.60
25.69
3664.1
The results of the compared methods are presented in Table 9 outperforms all the baseline methods, which shows the effectiveness of KAGAN large datasets.
We see that our proposed approach
13


Reviews:
[{"review_id": "S1lVTZvPKr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors aim at learning alignments between knowledge graphs. For this they use a discriminator that functions as an adversary to the parameterized triple alignment function leading to adversarial training. Furthermore, they regularize the training by maximizing a neural mutual information estimator. This requires training another approximating neural network along the way.\nSeveral experiments seem to indicate that their approach improves over competing methods in the unsupervised and weakly-supervised setting.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HJl5yDjEqS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a formulation to align entities and relation across different knowledge-basis. Each entity (or relation) is associated with a distribution over the entities (or relations) in the other KG through an exponential kernel-density-like-estimate. \n\n\nThe main contribution seems to be the use of GAN to generate good negative triplets instead of the traditional search-for-hard-negatives in constrained problems. This is a novel contribution and I see the value of this line of work.\n\nThe reason why I am tending not to accept this work is because it very similar(/same) to existing work like KBGAN (https://www.aclweb.org/anthology/N18-1133/). Most of the key ideas have already been covered there, and I would like to see a comparison of this work with that before acceptance.\n\n\nnits: \nsome column names in the table are inconsistent - hit vs h, mrr vs mr.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "Byl2uPu0Fr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose an unsupervised knowledge graph alignment framework, KAGAN, using adversarial training. Specifically, they utilize a triple discriminator to discriminate the aligned source triples and target triples, and a reward function to minimize the divergence between source triples and target triples. Also, they propose to leverage the lower bound of the mean KL divergence (mutual information) to resolve the mode collapse problem. The proposed method can be incorporated with a supervised method to be a weakly-supervised approach. Even though there are a family of unsupervised approaches for domain alignment, this paper is the first to solve the knowledge graph alignment problem in an unsupervised/weakly supervised way.\n\nStrength:\n1.\tThe paper addresses a critical knowledge graph alignment problem using GAN based on triplets, not usually entity alignment in literature, but also considers the relation alignment in knowledge graph.\n2.\tThe paper tries to solve the problem in an unsupervised way, and shows the on-par performance with weakly supervised methods in FB15k dataset.\n3.\tThe paper considers mode collapse problem and tries to solve the problem via mutual information rather than mean KL divergence, also gives the theoretical proof.\n4.\tDetailed experimental analysis and ablation studies show the effectiveness of the proposed method on small datasets.\n5.\tThe paper is well-written and easy to follow.\n\n\nI have two concerns as follows:\n1.\tThe authors conduct experiments on multiple small KG datasets such as FB15K and WK15K. But the reviewer finds that the baseline papers authors mentioned also have experiments on larger datasets like WK120K (Chen et al., 2017a), WK60k, DBP-WD or DBP YG (Sun et al., 2018a). It is essential to conduct the experiment on larger datasets to verify the effectiveness of the proposed method.\n2.\tIs it necessary to construct a reward function to update the alignment function using REINFORCE algorithm? For instance, current distribution matching methods can define the discrepancy between the two distributions (such as target distribution and aligned source distribution). It can directly optimize the loss in an end-to-end differentiable way instead of a reinforcement learning way. It can avoid the sampling and provide a more stable optimization process.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Thanks for your detailed feedback to the reviewers, which clarified us a lot in many respects.
This paper potentially discusses an interesting problem, and the concern raised by Review #2 was addressed in the revised paper.
However,  given the  high competition at ICLR2020, this paper is unfortunately below the bar.
We hope that the reviewers' comments are useful for improving the paper for potential future publication.

The 
Decision: Reject