NoiGAN: NOISE AWARE KNOWLEDGE GRAPH EMBEDDING WITH GAN
Authors: Kewei Cheng
Yikai Zhu
Ming Zhang
Yizhou Sun
Venue:  (2020)
Keywords: Knowledge graph embedding
Noise aware

Abstract: Knowledge graph has gained increasing attention in recent years for its successful applications of numerous tasks. Despite the rapid growth of knowledge construction, knowledge graphs still suffer from severe incompletion and inevitably involve various kinds of errors. Several attempts have been made to complete knowledge graph as well as to detect noise. However, none of them considers unifying these two tasks even though they are inter-dependent and can mutually boost the performance of each other. In this paper, we proposed to jointly combine these two tasks with a unified Generative Adversarial Networks (GAN) framework to learn noise-aware knowledge graph embedding. Extensive experiments have demonstrated that our approach is superior to existing state-of-the-art algorithms both in regard to knowledge graph completion and error detection. 

Under review as a conference paper at ICLR 2020
NOIGAN: NOISE AWARE KNOWLEDGE GRAPH EMBEDDING WITH GAN
Anonymous authors Paper under double-blind review
ABSTRACT
Knowledge graph has gained increasing attention in recent years for its success- ful upplications of numerous tasks. Despite the rapid growth of knowledge con- struction, knowledge graphs still suffer from severe incompletion and inevitably involve various kinds of errors. Several attempts have been made to complete knowledge graph as well to detect noise. However, none of them considers unifying these two tasks even though they are inter-dependent and can mutually boost the performance of each other. In this paper, proposed to jointly combine these two tasks with a unified Generative Adversarial Networks (GAN) framework to learn noise-awar awsre knowledge graph embedding. Extensive experiments have demonstrated that our approach is superior to existing state-of-the-art algorithms both in regard to knowledge graph completion and error detection.
INTRODUCTION
Knowledge graph, a well-structured effective representation of knowledge, plays pivotal role in many real-world applications such as web search (Graupmann et al. 2005), question answer- ing(Hao et al., 2017] Yih al. (0055), and personalized recommendation (Zhang al.] 2016). It is constructed by extracting information as the form of triple from unstructured text using infor- mation extraction systems. Each triple rh,r, represents a relation r between a head entity h and a tail entity t. Recent years have witnessed extensive construction of knowledge graph, such as Freebase (Bollacker al.] 2008), DBPedia (Auer al.) 2007). and YAGO (Suchanek et al. (007). However, these knowledge graphs suffer from severe sparsity as we can never collect all the infor- mation. Moreover, due to the huge volumes of web resources, the task to construct knowledge graph usually involves automatic mechanisms to avoid human supervision and thus inevitably introduces many kinds errors, including ambiguous, conflicting and erroneous and redundant information.
To address these shortcomings, various methods for knowledge graph refinement have been pro- posed, whose goals can be arguably classified into two categories: (1) knowledge graph completion, the task to add missing knowledge to the knowledge graph, and (2) error detection, the task to iden- tify incorrect triples in the knowledge graph. Knowledge graph embedding (KGE) currently hold the state-of-the-ar in knowledge graph completion for their promising results (Bordes et al. 2013 Yang et al.] 2014). Nonetheless, they highly rely on high quality training data and thus are lack of robustness to noise (Pujara et al. (011)). Error detection in knowledge graph problem due to the difficulty of obtaining noisy data. Reasoning based methods are the most widely used methods for this task (Paulheim 2017). Without the guidance of noisy data, they detect er- rors by performing reasoning over the knowledge graph to determine the correctness of triple. A rich ontology information is required for such kind of methods and thus impede its application for real-world knowledge graphs.
challenging
Existing works consider knowledge graph embedding and error detection independently whereas these two tasks are inter-dependent and can greatly influence each other. On one hand, error de- tection model is extremely useful to prepare reliable data for knowledge graph embedding. On the other hand, high quality embedding learned by KGE model provides basis for reasoning to iden- tify noisy data. Inspired by the recent advances of generative adversarial deep models (Goodfellow) et al. 20144) in this paper, we proposed to jointly combine these two tasks with unified GAN frame- work, known as NoiGAN, to learn noise-aware knowledge graph embedding. In general, NoiGAN consists of two main components, noise-aware KGE model to learn robuster representation of

---

Under review as a conference paper at ICLR 2020
knowledge and an adversarial learning framework for enror detection. During the training, noise- aware KGE model takes the confidence score learned by GAN as guidance to eliminate the noisy data from the learning process whereas the GAN requires that KGE model continuously provides high quality embedding as well as credible positive examples to model the discriminator and the generator. Cooperation between the two components drives both improve their capability. The main contributions of this paper are summarized as follows:
We propose a unified generative adversarial framework NoiGAN, to learn aware knowledge graph embedding. Under the framework, the KGE model and error detection model could benefit from each other: the error detection model prepares reliable data for KGE model to improve the quality of embedding it learns, while the KGE model provides promising reasoning model for the error detection model to better distinguish noisy triples from the correct one.
Our proposed framework can be easily generalized to various KGE models enhance their ability in dealing with noisy knowledge graph.
We experimentally demonstrate that our new algorithm is superior to existing state-of-the algorithms. The KGE model and GAN can alternately and iteratively boost performance in terms of both knowledge graph completion and noise detection.
2 RELATED WORK
2.1 KNOWLEDGE GRAPH COMPLETION
Embedding based methods currently hold the state-of-the-art in knowledge graph completion for their promising results [Wang et al., 2017b). They aim to capture the similarity of entities by em- bedding entities and relations into continuous low-dimeniional vectors. Existing methods can be roughly divided into two categories: translational distance models and semantic matching models. Translational distance models measure the plausibility a fact as the distance between two entities after translation carried out by the relation. TransE (Bordes et al., 2013). TransH (Wang et al.) 2014) and TransR (Lin et al. 2013) are the representative approaches in this category. Semantic matching models measure plausibility of facts by matching latent semanties of entities and relations embodied in their vector space representations. The typical models include RESCAL (Nickel et al., 2011), DistMult (Yang al.] 2014) and ComplEx (Trouillon et al. 2016). To optimize the KGE model.negative sampling is usually required to minimize the margin based ranking loss. A con- ventional method to construct negative samples is randomly sampling. However, negative samples generated through a random mode are often too easy to be discriminated from positive facts and thus make little contribute towards the training. Some recent works proposed incorporate GAN for better negative sampling improve the quality of embeddings (Wang et al. [2018] 18 Cai & Wang, 2017). Nonetheless, none of the above methods has taken the potential noisy data into consideration. which leads to their sensitivity to unreliable data (Pujara et al. 2017). In this paper, we proposed novel technique to enable current embedding models to cope with noisy data.
2.2 ERROR DETECTION IN KNOWLEDGE GRAPH
Due to lack of noisy data samples, error detection in knowledge graph is a challenging task. Ex- isting methods can be either ontology based or anomaly detection based. Ontology based methods address this problem by exploring additional ontology information. A larger number of ontology reasoners are developed
utilize logic programming to derive uncovering contradictions among observed facts (Luther et al., 2009 Dentler et al.) 2011 Ding et al. 2007). Rich ontology informa- tion required for such kind of methods and thus impede its application to real-world knowledge graphs. Another kind of methods is anomaly detection based methods (Wienand & Paulheim) [014]] Fleischhacker et al. 2014). The main drawback of anomaly detection based methods are that they do not necessarily identify errors, but also natural outliers, which will compromise the objectivity of its results. More recently, a novel confidence -wwee proposed to incorporate triple confidence into KGE model to detect noises while learning knowledge representations simultancously. However, measures the triple confidence merely based on the how well the triple fits the model, which is easily affected by model bias.
aware framework (Xie et al. 2018; 8;; Shan et al. 2018)
2

---

Under review as a conference paper at ICLR 2020
L RGB
Discriminator
D ((0.0.0:))))
r..r.t)
(h's (t))
C.h.r.0)
(K.r.
R
x.
Embedding Model
Generator
G ((02n)))
(1,7,1)
(K.r.)))
Nolse-aware KGE
GAN
Figure Overview of the NoiGAN Framework.
3 PROPOSED METHOD:NOIGAN
In this section, we illustrate our proposed framework, NoiGAN, in details. The goal of our proposed framework to learn noise-aware KG embedding with Generative Adversarial Networks (GAN). As shown in Figure D it consists of three components: (i) Noise aware KGE model, which incorpo- rates a confidence score C(h,r, t) into the KGE model to isolate the impact of noise over embedding vectors. Meanwhile, it provides high quality embedding to model discriminator and generator (Sec- tion 3.1;; ; (ii) Triple generator G((h', (", r, I); Oa), which learns to generate the most likely triples to be the noisy triples by corrupting the correct triple (h, (). It aims at providing reliable noisy data for the discriminator as well as noise aware KGE model (Section 3.2.1]; (iii) Discrim- inator D( ł); (D), which tries to distinguish correct triples from noisy triples, and assigns confidence score for each triple fact to describe the correctness of it. The discriminator is used to determine confidence score C(h, r, in the noise aware KGE model. It also naturally provides guidance to the generator to produce higher quality noisy data (Section (.2.2).
Before further discussing the detail of the algorithm, first give formal definition to knowledge graph. A knowledge graph, denoted by g = {E, R, T) consists of a set of entities E, a set of relations R and a of of observed facts T r(,, h E, R). A fact represented by triple (h, r.,, t), where h, and r denote head entity, tail entity, and relation, respectively.
3.1 NOISE AWARE KNOWLEDGE GRAPH EMBEDDING MODEL
To learn a more robust representation for noisy knowledge graph, we propose noise-aware KGE model to eliminate the impact of noise over embedding vectors by incorporating confidence score into KGE model. It can be casily adapted to any KGE models. For example, we can follow TransE to represent entities and relations. Given fact (h, t). TransE aims to learn corresponding low dimensional vectors h, and in the same space for h, and f, so that the distance between h + and t is minimized. The scoring function of TransE is then defined as fr, () |h+ + r + To optimize the KGE model, et al. 2013) according to (Sun et al. 2019).
loss function similar the negative sampling loss in (Mikolov
[(-log fr(h,t)) (K.T,))T
Σ
E (M (..
log f-(h',t)))
[h(,,,
where y the margin, a is the sigmoid function, represents the observed facts from the knowledge graph and (h', r, r') is the negative sample for triple (h, r). To reduce the effects of randomness, we sample multiple negative triples for each observed fact. We denote the negative triples set of triple (h, as (h, t). Negative triples set is constructed by replacing the head tail entity of the observed triples with an entity sampled randomly from entity set E:
N(h, r, t) ((h', r. r)l) e F. (h,,r, t')]t'
A vast majority of existing embedding models assume that all triple facts hold true to knowledge graph, which is inappropriate. In fact, knowledge graph contains many kinds of errors (e.g., am- biguous, conflicting, erroneous, redundant) due to the automatic construction process. To isolate the
3

---

Under review as a conference paper at ICLR 2020
impact of noise over embedding vectors, following (Shan et al.) 20188. 8) adopted the concept con- fidence to describe whether triple fact is noisy or not. In (Shan et al. 2018), confidence of a triple measured by how well the triple fits to the KGE model. A central issue with such measurement is that it is easily affected by model bias and thus lead to unreliability. Unlike (Shan et al. 2018), we learned the confidence scores using a discriminator, which can be more impartial and promising. We will discuss how to obtain such discriminator in Section 3.2.2] With the introduction of confidence, can eliminate the noisy data from the learning process of KGE model.
LKGE = Σ ((1r::0((--og0007-66010) (b,r,))T
E ( (M)
log (o
[(h,r,
where rh,,r, 1) is the confidence score assigned to each positive triple (h, (). To be specific, C(h, r,t) can be both a binary variable or soft value from the interval [0. 1]. We denote the previous one as the hard version and the later one as the soft version.
3.2 GAN FOR NOISE IDENTIFICATION
After introducing our noise aware KGE model, we now show how to identify the error in knowledge graph to enhance embedding quality. Typically label information about noisy triples is costly and labor intensive to obtain, which is the central challenge to the task of error detection. Without the guidance of noisy data, previous works propose to leverage ontology information to exploit reasoning for inconsistency detection. However, these ontology information can also be difficult to gain. To address this problem, inspired by recent advance of GAN, we propose a novel end-to-end adversarial learning framework to detect noise in knowledge graph. The proposed GAN cooperates with the noise aware KGE model in an interactive way. GAN utilizes the embedding learned by the noise aware KGE model to distinguish true triples and noisy triples. Mcanwhile the noise-aware KGE model requires GAN to learn a confidence score so as to eliminate the impact of noise over embedding vectors.
Our adversarial learning framework consists of two components, a generator and a discriminator. Unlike traditional GAN whose ultimate goal is to train good generator, we aim produce good discriminator to distinguish noisy triples from true triples. To address noisy data deficiency issue, the generator and the discriminator act two players in a minimax game: the generator learns to continually generate "more confusing" triples and thus provides better quality noisy data for the discriminator, whereas the discriminator is trained draw a clear distinction between the true triples and the noisy triples generated by its opponent generator. Formally, it can be formulated as follows:
max D) G
Σ B(u,r.))-(log t(( (KNDET
(t log(1 D((h', t'):0)))
where T represents the true triples set. To get rid of the noise in knowledge graph, construct T with top 10% triples which fit the KGE the best. We discuss the detailed implementation of the generator and discriminator as follows.
3.2.1 TRIPLE GENERATOR
To identify the error in knowledge graph, first we have to achieve reliable examples of noisy data. Since such data is costly and labor intensive to obtain, we instead introduce a generator to produce them. The main goal of the generator to generator high-quality fake triples that can fool discrim- inators, which aims at bringing better classifier to identify error in knowledge graph. In addition, to simultaneously make KGE model aware of the existing noisy data, we also explicitly incorporate the noisy triples generated by the generator negative samples to train the noise aware KGE model.
Note that, the generator takes embedding vectors learned by the noise-aware KGE model as input. Given true triples (h, r),, the generator aims to select the most likely triples to be the noisy triples (h', t') from its negative samples candidate set set.V NM r, t). To achieve this goal, a two-layer fully-connected neural network is introduced to model the probability distribution over the candidate
4

---

Under review as a conference paper at ICLR 2020
negative samples [h, t). In particular, this MLP uses ReLU as the activation function for the first layer and adds Softmax to the second layer. It takes the concatenation of the embedding vectors h', and t' of triple (h', r,, as input and output the probability whether the triple (h', rr, is the most likely noisy triple. As the output of the generator is a discrete triple, training for the generator policy gradient
is little tricky due data indifferentiabilityy issue. A common solution is based reinforcement learning instead (Wang et al. (2177).. In particular, for a generated triple, the reward from discriminator is defined as fD(h', ,, t'), which exactly the probability of the triple (h', t') to be true. Thus, in order to fool the discriminator, the generator is trained to maximize the expected reward follows:
RG Σ h.r.t
B.w.no--acaaaaoooollll Joh'', 6')
(1)
At the very beginning, because of lack of guidance, the generator generates "noise" by random sampling. However, such "noise" is too easy for discriminator to distinguish and result in a low reward. By learning to maximize the reward, the generator continually learns better probability dis- tribution of the candidate negative samples and keep generating "more confusing" triples to improve the capacity of the discriminator. Moreover, this generator can also be used to produce high quality negative samples for the noise aware KGE model.
3.2.2 DISCRIMINATOR
Discriminator aims to distinguish the true triples from the noisy triples. Both positive and negative examples are required train the discriminator. Generator learns to provide reliable negative ex- amples for the discriminator while observed triples in knowledge graph can be regarded as positive examples. However, as knowledge graph noisy, considering all observed triples as positive sam- ples will inevitably introduce noise. We therefore utilize an interesting observation, when noisy data exist, deep learning models tend to memorize these noise in the end, which leads the poor gener- alization performance (Han et al., 2018). Specifically, we take top 10% triples which fit the KGE the best (with lowest f- t)) as positive training examples. Considering discriminator is essentially a binary classifier, the objective function of the discriminator can be formulated minimizing the following cross entropy loss:
CD -- Σ log(fo(h,T,t)) (KMTET
Σ
log(1- fr(h',, (())
(M (r
where fD(h, r, (MLP(h, t)) where MLP is a two-layer neural network with ReLU as acti- vation function and (xx) 1/(1 exp(-z)) is the standard sigmoid function. we use TransE as
our KGE model, the MLP takes the vector h + r (h, r, I) being true. [n, r, r)() will later be used to define C(h, r, 1) for each positive triple (h, r, () in knowledge graph. To be specific, C(h, can be either a binary value or a soft value. When a binary variable, it represents the classification result regard to whether a triple is a true triple. When it takes soft value, it indicates the probability of the triple (h, t) being true. We denote the previous one as a hard version our proposed model and the later one as the soft version.
as input and output the probability of the triple
Algorithm NoiGAN Framework
Require: Observed triples in knowledge bases T
1: Initialize Ch,, tt) as for all triples.
2: Train noise aware KGE model with random negative sampling until convergence.
3: for 1 N do
4: Take top 10% triples which fit the noise aware KGE the best as positive examples.
5: Train GAN with the selected positive examples until convergence.
6: Update C(h, r, I) according to discriminator of GAN.
7:
Train noise aware KGE model with negative samples generated by the generator until con- vergence.
8: end for
5

---

Under review as a conference paper at ICLR 2020
Dataset
#Relations
#Entities
#Train
#Valid
#fest
FB15K
1.345
14,951
483,142
50,000
59,071
WNI8
18
40,943
141,442
5,000
5,000
FB15K-237
237
14,541
272,115
17,535
20,466
WNT8RR
11
40,943
86,835
3,034
3,134
YAGO3-10
37
123,182
1,079,040
5,000
5,000
Table 1: Data Statistics
3.3
JOINT TRAINING
NoiGAN is trained in an iterative way, through which each component of the model can alternately and iteratively boost performance of each other. The training process is summarized in Algorithm I We train each component of NoiGAN alternately by fixing the parameters of the other components. We start by training the noise aware KGE model. We initiate the confidence score C(h, r.() as 1 for all triple facts and pretrain the noise aware KGE model with random negative sampling to gain reliable embedding vector. With these embedding vectors, we start the training of GAN model from the generator. The generator Iearss to understand the probability distribution of the candidate negative samples so as to select the most likely noisy data. In particular, the discriminator gives it reward as guidance. Once the generator finishes training, it can be used to prepare noisy data for the discriminator. In addition, we take top 10% triples facts which fit the noise aware KGE the best as credible triple facts. With noisy data as negative examples and triple facts as positive examples, discriminator learns a classifier to distinguish the positive triples from the negative triples. After we obtain the well trained GAN model. given the confidence score assigned by the discriminator and the negative samples generated by the generator, we retrain the noise aware KGE model and update the embedding vectors. The process repeats until convergence. The embeddings learned by the noise aware KGE model is regarded as our final representation for entities and relations.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Datasets. We evaluate our NoiGAN on five benchmark datasets, including FB15K, WN18 _FB15K- 237, WN18RR and YAGO3-10. These henchmark datasets benefit from human curation that results in highly reliable facts. To simulate the real-world knowledge graphs extracted automatically from unstructured text data, we modified these benchmark datasets to include noisy triples. Since all kinds of noise might be contained while we construct knowledge graphs, our approach to introducing noise to substitute the true head entity or tail entity with any randomly selected entity. Following this approach, we construct five KGs based on each benchmark dataset with noisy triples be different ratio of (e.g., 40%, 70% and 100%) of true triples All noisy datasets share the same entities, relations, validation and test with the original benchmark dataset, with all generated noisy triples fused into the original training set. The statistics of these knowledge graphs are summarized in Table II
Baselines. NoiGAN is compared with the following state-of-the-art algorithm, including (1) KGE models (e.g., TransE (Bordes al., 2013).DistM (Yang et al. (014)) and RotateE (Sun et al. 2019 (2) robust KGE models e.g., attention based method (Nathani et al. 2019). (3) noise aware KGE models (e.g., CKRL (Xie et al., 2018) and (4) KGE models with GAN (e.g.. KBGAN (Cai & Wang 2017)). ). In particular, there are three kinds of triple confidences defined in CKRL (Xie et al.] 2018). In our paper, we take CKRI. with local triple confidence, called CKRL (LT), as baseline.
To fairly compare different methods, the same loss function and negative sampling strategies employed for all models.
Experimental Sctup of NoiGAN. We evaluate two versions of our NoiGAN model mentioned in Section 3.2.2 The soft version is denoted as NoiGAN (soft) while the hard version is denoted NoiGAN (hard). To show that our NoiGAN can be easily generalized to various KGE models, TransE (Bordes et al., 2013) and RotateE (Sun et al. 2019) are implemented as score function for NoiGAN, Adam (Kingma & Ba 2014) is adopted the optimizer. We set the parameters for all

---

Under review as a conference paper at ICLR 2020
Dataset
FBT5K
YAGO3-10
WN18
Metric
AUC
Spe
AUC
Spe
AUC
Spe
CKRL (LT)
.548
.006
.455
.018
.704
.000
10%
NoiGAN-TransE (soft)
.499
.791
.929
.625
.794
.452
NoiGAN-TransE: (hard)
.46
.821
.942
.730
.836
.565
CKRL (LTT
.687
.008
.521
.000
.710
.000
20%
NoiGAN-TransEE (soft)
.933
.733
.914
.619
.756
.336
NoiGAN-TransE (hard)
.954
.823
.934
.711
.799
.524
CKRL (LT)
.725
.018
.577
1.000
.725
.000
40%
NoiGAN-TransE (soft)
.31
.699
.894
.552
.682
.345
NoiGAN-TransE (hard)
.950
.780
.999
.711
.741
.464
Table 2: Evaluation Results Knowledge Graph Error Detection.
methods by a grid search strategy. The range different parameters is set as follows: embedding di- mension & E {250, 500, 1000}, batch size b E {256, 512, 1024), and fixed margin E (9, 12, 24). Afterwards, we compare the best results of different methods. Both the entity embeddings and the relation embeddings
uniformly initialized and regularization is imposed them. As men- tion in Section 3.2 we implement both discriminator and the generator as simple two-layer fully connected neural networks. The size of hidden states for each of the two networks is set 10.
4.2 KNOWLEDGE GRAPH NOISE DETECTION
To verify the capability of NoiGAN in distinguishing noises in knowledge graphs, we evaluate NoiGAN terms of classification performance. To be specific, we classify the training data by determining whether triple is being true. For NoiGAN (hard), inator to classify noise. Whereas, for NoiGAN (soft), the discriminator only assigns a soft score between 0 to 1 to each triple, indicating the probability of triple being true. We thus classify the triples by regarding the triple who have C(h, t) 0.5 as the true triple and the remaining as noise. The experiments are conducted on three benchmark datasets, including FB15K, WN18 and 10 with noisy triples to be different ratio of 10%, 20% and 40% of true triples. Although FBISK and WN18RR suffer from test triple leakage in the training set (Dettmers al., (018)). we report the results over training data, which won't 't be affected by the issue. As we make similar ob- servations on NoiGAN-Transf and NoiGAN-Rotath, we only report the experimental results w.r.t. NoiGAN-TrrasSE to save space.
directly utilize the discrim-
Evaluation Metric. Two classification evaluation metrics are adopted, including (1) AUC; (2) A special measurement defined as the proportion of actual noises that are correctly identified. It can be calculated as TN-TP. where TN represents true negative while FP represents false positive.
TN
We compared against CKRL (LT) it is the only baseline method which is able to detect noise in the training dataset. We can observe that: (1) Our model consistently achieves the best performances over all datasets in all cases, which demonstrates the capability of our models in detecting noises. (2) NoiGAN-Transt (hard) has significant improvements in noise detection compared to NoiGAN-TransE (soft). Considering that NoiGAN-Transs (soft) merely assigns low confidence scores noisy triples while NoiGAN-Transs (hard) completely eliminates these data, this result further validates that our method can classify noisy data accurately. (3) The baselines tend to classify all triples into the same class, which shows their inability in detecting noise.
Results. The results w.r.t NoiGAN-TransE can be found in Table 2
4.3 KNOWLEDGE GRAPH COMPLETION
In addition to its promising results on noise detection, our approach is also superior to existing state- of-the-art algorithms in terms of the quality of learned embedding. We conduct experiments three benchmark datasets, including FB15K- WN18RR and YAGO3-10 10 with noisy triples to be different ratio of 40%, 70% and 100% of true triples.

---

Under review as a conference paper at ICLR 2020
FB15K-237
YAGO3-10
WNISRR
MRR
H@1
H@3
H@ 10
MRR
H@1 H@3
H@@ 10
MRR
H@1 H@3
H@10 10
DistMult
.118
.136
.236
.388
.292
.203
.325
.470
.424
.382
.437
.577
KBGAN
.266
.186
.290
.427
.071
.041
.070
.124
.215
.036
.356
.507
Attention
.436
.360
.464
.589
.443
.363
.489
.583
TransE
.330
.229
.371
.531
.410
.299
.469
.620
.229
.022
.403
.526
CKRL (LT)
.330
.229
.372
.529
.377
.264
.438
.585
.231
.027
.404
.530
0%
NoiGAN-TransE (soft)
.388
.217
.358
.518
.370
.267
.420
.567
.231
.026
.407
.524
NoiGAN-TransE (hard)
.308
.209
.349
.502
.345
.248
.393
.531
.223
.031
.381
.503
RotatE
.326
.228
.363
.521
.380
.269
.437
.594
.472
.429
.488
.557
NoiGAN-RotatE (soft)
.326
.229
.363
.520
.430
.330
.487
.620
.466
.420
.485
.554
NoiGAN-RotatE, (hard)
.325
.228
.363
.518
.413
.312
.471
.602
.465
.420
.483
.551
DistMult
.207
.127
.229
.369
.089
.038
.095
.188
.364
.310
.400
.449
KBGAN
.177
.079
.214
.363
.059
.033
.061
.102
.178
.010
.312
.447
Attention
.326
.249
.352
.480
.230
.035
.357
.586
TransE
.182
.026
.271
.463
.150
.001
.231
.429
.185
.008
.331
.455
CKRL (LT)
.190
.035
.276
.471
.122
.007
.175
.344
.186
.010
.327
.462
40%
NoiGAN-TransE (soft)
.212
.064
.294
.482
.109
.004
.156
.306
.188
.005
.337
.469
NoiGAN-TransE (hard)
.227
.095
.296
.473
.321
.204
.383
.538
.168
.008
.300
.407
RotatE
284
.189
.322
.469
.098
.034
.116
.228
.345
.438
.481
NoiGAN Rotatt (soft)
.294
.197
.331
.488
.220
.120
.266
.420
.403
.339
.447
.506
BoiGAN-Rotatte (hard)
.96
.198
.336
.489
.223
.121
.269
.423
.403
.339
.445
.507
DistMult
.19
.122
.215
.342
.074
.030
.077
.160
.328
.266
.378
.420
KBGAN
.123
.040
.145
.288
.026
.016
.027
.040
.161
.007
.276
.429
Attention
.255
.174
.280
.414
.192
.012
.323
.453
TransE
.167
.019
.244
.440
.085
.005
106
.242
.169
.005
.300
.439
CKRL (L.T)
.171
.021
.253
.449
.090
.119
.262
.169
.299
.441
70%
NoiGANNTTransE (soft)
.189
.037
.274
.464
.124
.003
.183
.358
.166
.002
.289
.442
NoiGAN-TransE (hard)
.199
.059
.275
.457
.210
.091
.268
.443
.152
.004
.267
.392
RotatE
254
.154
.299
.444
.037
.001
.044
.103
.357
.292
.411
.456
NoiGAN- -otattE (soft)
.279
.179
.321
.121
.154
.296
.295
.423
.473
MoiGAN-RotatE (hard)
.282
.184
.321
.478
.128
.040
.160
.310
.368
.298
.424
.474
DistMult
.196
.122
.215
.342
.064
.021
.067
.147
.241
.370
.422
KBGAN
.091
.030
.099
.213
.009
.002
.008
.021
.138
.010
.218
.392
Attention
.192
.112
.221
.339
.131
.003
.214
.335
TransE
.157
.016
.227
.422
.065
.003
.076
.191
.153
.002
.267
.422
CKRL (LT)
.161
.018
.233
.430
.070
.083
.211
.151
.250
.419
100%
NoiGAN-TrannEE (soft)
.173
.020
.258
.454
.100
.001
.145
.286
.147
.000
.247
.415
NoiGAN-TransSi (hard)
.185
.044
.262
.446
.144
.016
.213
.384
.129
.004
.223
.339
Rotath:
2344
.137
.276
.418
.019
.001
.020
.051
.315
.239
.383
.429
NoiGAN-Roo MiGGAN-RtaatE (soft)
.283
.201
.312
.447
.022
.002
.024
.061
.327
.240
.401
.453
NoiGAN-Rotall (hard)
.211
.175
.309
.021
.002
.022
.058
.327
.399
.452
Table 3: Evaluation Results Knowledge Graph Completion
8

---

Under review as a conference paper at ICLR 2020
Distance (TransE)
Distance (NoiGAN)
Score (NoiGAN)
Triple
(McDonald's (business),
Positive
Adiningirestauran/euiiin
19.94
16.60
0.8664
Philippinc cuisine (national cuisine))
(MeDonald's (business),
Negative
sininngrestaurran/euiinnee,
20.51
32.67
0.0171
Joe Walsh (human))
(voice actor (filmakkng occupation),
Positive
ppcopleproffesion/poopc.cccc whth profession, withhttii..poofffsinn
21.58
20.28
0.8409
Michael Dohson (human))
(head teacher (profession),
Negative
pxoopie/rofessio/ppopt...wiii whth .rossppoofssion,
22.60
37.57
0.0058
Michael Dobson (human))
Table 4: Examples of Noisy Triples and Positive Triples in FB15K Dataset.
Evaluation Metric. We mask the head or tail entity of each test triple, and require each method to predict the masked entity. During evaluation, use the filtered setting (Bordes et al. 2013). The Hit@ K (H@K K) and Mean Reciprocal Rank (MRR) are adopted as the evaluation metrics.
Results. Results of reasoning shown in Table 3 In particular, attention based method exceed the memory capacity our machines on YAGO3- 10 dataset, only the results on the FB FB15K-20 -37 and WN18RR datasets reported. Note that TransE, CKRL (LT) and NoiGAN-TransE share the same score function TransE, they are comparable with each other. Similarly, RotatE and NoiGAN- RotatE share the same score function RotatE and thus they are comparable with each other. We make the following observations: (1) NoiGAN consistently outperforms the baseline methods which share the same score function with it on noisy dataset sets. The performance gain is significant especially on datasets with 100% noise. (2) Either TransE or KoiGAN-Rotath will achieve the best performance on almost all noisy datasets. particular, hard version of NoiGAN performs better then soft version in most cases. (3) Even though attention based method claims that they can ensure robust performance, the results show that our NoiGAN significantly outperforms it in terms of robustness. (4) If we do not introduce noise, the performance of NoiGAN models is almost the same as their variants. addition, the improvement introduced by the NoiGAN becomes more significant as the noise rate in KGs rises. This further proves the robustness of the NoiGAN.
4.4 CASE STUDY
To demonstrate the power of discriminator distinguishing noisy triples, we present some triples and their confidence score in Table We conduct experiments over FB15K with 40% noise using the BuiGAN-Transt (hard) model. All hypeparaaeete are set as described in [. To be clear, distance of triple is calculated criminator. As shown in 4 we can see that TransE performs poorly at detecting some "triky" errors, such as logie error (e.g.. (head teacher, preppleprofession/people with_this thiis-srofssson, Michael Dobson)) and grammar eтrоr (e.g. (McDonalds, surprise, NoiGAN (hard) has the ability detect both.
h + t|l, while the scores of the triples are learned by the dis-
Joe Walsh).). To our
5 CONCLUSION
In this paper, propose novel framework NoiGAN, to jointly combine the tasks of knowledge graph completion and error detection for noise aware knowledge graph embedding learning. consists of two main components, noise-aware KGE model for knowledge graph completion and an adversarial learning framework for error detection. Under the framework, the noise-aware KGE alternately and iteratively boost performance of
model and the adversarial learning framework each other. Extensive experiments show the superiority of our proposed NoiGAN both in regard to knowledge graph completion and error detection.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Sören Auer; Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dbpedia: A nucleus for web of open data. In The semantic web, pp. 722-735. Springer, 2007.
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: collab- oratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp. 1247-1250. AcM, 2008.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pp. 2787-2795, 2013.
Liwei Cai and William Yang Wang. Kbgan: Adversarial learning for knowledge graph embeddings. arXiv preprint arXiv: 171100071, 2017.
Kathrin Dentler, Ronald Cornet, Annette Ten Teije, and Nicolette De Keizer. Comparison of rea- soners for large ontologies in the owl 2 el profile. Semantic Web, 2(2):71-87, 2011.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Li Ding, Pranam Kolari, Zhongli Ding, and Sasikanth Avancha. Using ontologies in the semantic web: A survey. In Ontologies, pp. 793113. Springer, 2007.
Daniel Fleischhacker, Heiko Paulheim, Volha Bryl, Johanna Völker, and Christian Bizer. Detecting errors in numerical linked data using cross-checked outlier detection. In International semantic web conference, pp. 357-372. Springer, 2014.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Jens Graupmann, Ralf Schenkel, and Gerhard Weikum. The SphereSearch engine for unified ranked retrieval of heterogeneous XML. and web documents. In Proceedings of the 31st international conference on very large data bases, pp. 529-540. VLDB Endowment, 2005.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in neural information processing systems, pp. 8527-8537, 2018
Yanchao Hao, Yuanzhe Zhang, Kang Liu, Shizhu He, Zhanyi Liu, Hua Wu, and Jun Zhao. An end- to-end model for question answering over knowledge base with cross-attention combining global knowledge, In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 221-231, 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69888 6980, 2014.
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference artificial intelligence, 2015.
Marko Luther, Thorsten Liebig, Sebastian Böhm, and Olaf Noppens. Who the heck is the father of bob? In European Semantic Web Conference, pp. 66 80. Springer, 2009.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed represen- tations words and phrases and their compositionality. In Advances in neural information pro- cessing systems, pp- 11111-1199, 19, 2013.
Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. Learning attention-based em- beddings for relation prediction in knowledge graphs. arXiv preprint arXiv:/906.0/195 2019.
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. three-way model for collective learning multi-relational data. In ICMI., volume 11, pp. 809-816, 2011.
10

---

Under review as a conference paper at ICLR 2020
Heiko Paulheim. Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic web, 8()):489-508 2017.
Jay Pujara, Eriq Augustine, and Lise Getoor. Sparsity and noise: Where knowledge graph em- beddings fall short. Proceedings the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1751-1756, 2017.
Yingchun Shan, Chenyang Bu, Xiaojian Liu, Shengwei Ji, and Lei Li. aowfidence-aware negative sampling method for noisy knowledge graph embedding. In 2018 IEEE International Conference on Big Knowledge (ICBK), pp. 33-40. IEEE, 2018.
Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. YAGO: core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pp. 697-706. ACM, 2007.
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nic, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint UrXiv:1902.101991 2019.
Théo Trouillon. Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Com- plex embeddings for simple link prediction. In International Conference on Machine Learning, pp. 2071-2088 2016.
Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang. Peng Zhang, and Dell Zhang. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pp. 515-524. ACM, 2017a.
Peifeng Wang. Shuangyin Li, and Rong Pan. Incorporating gan for negative sampling in knowledge representation learning. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 21(1)) 2017b.
Zhen Wang. Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by trans- lating on hyperplanes. In Twenty-Eighth AAAI conference on artificial intelligence, 2014.
Dominik Wienand and Heiko Paulheim. Detecting incorrect numerical data in dbpedia. In European Semantic Web Conference, pp. 518. Springer, 2014.
Ruobing Xie, Zhiyuan Liu, Fen Lin, and Leyu Lin. Does william shakespeare really write ham- let? knowledge representation learning with confidence. In Thirty-Second AAAI Conference Artificial Intelligence, 2018.
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv: arXiv: 1412.6575, 2014.
Wentau Yih, Ming Wei Chang, Xiaodong He, and Jianfeng Gao. Semantic parsing via staged query graph generation: Question answering with knowledge base. In IJCLPP, pp. 1321-1331, Beijing, China, July 2015.
Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei- Ma. Collaborative knowl- edge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 353-362. ACM, 2016.


Reviews:
[{"review_id": "ryl29S9x5H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a GAN-oriented framework for training robust-to-noise neural link predictors. My main concern is that CKRL is the only baseline -- ignoring years of prior works in this space (see e.g. [1, 2]).\nFurthermore, [2] shows that two of the three datasets used by the authors suffer from test triple leakage in the training set.\n\nFinally, the considered datasets do not really test for the presence of noise - authors may want to check out e.g. https://arxiv.org/abs/1812.00279 (there are several works in this space, all of which were systematically ignored by this paper).\n\nFinally, authors claim neural link predictors were never used for denoising, but actually [3] use them to learn a prior distribution over triples in a probabilistic DB setting.\n\n\n[1] https://arxiv.org/abs/1806.07297\n[2] https://arxiv.org/abs/1707.01476\n[3] https://ai.google/research/pubs/pub45634", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "S1l96w7HKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to provide a novel noise-aware knowledge graph embedding (NoiGAN) by combining KG completion and noise detection through the GANs framework. More specifically, NoiGAN repeatedly utilizes a GAN model to 1) approximate the confidence score for facts identifying reliable data (discriminator) and 2) generate more challenging negative samples (generator). Then, it uses this confidence score and negative samples to train a more accurate link prediction model. The authors validate the proposed model through several experiments.\n\nThis paper reads well and the results appear sound. I personally find the idea of incorporating confidence score into a link prediction model to achieve a more accurate model very interesting. Furthermore, the provided experiments support their intuition and arguments outperforming considered baselines.\n\nAs for the drawbacks, I find the baselines considered in this work outdated missing many SOTA and related works in link prediction and noise detection [1,2, 3, 4, 5]. Further, I believe this work needs more experimental results and an ablation study capturing different aspects of the presented method. My concerns are as follows:\n\n•\tConsidering the existing reverse relation issue in FB15K and WN18, I suggest conducting the experiments on the FB15K-237 and WN18RR from [6] instead. \n•\tI suggest considering more recent link prediction models as baselines.\n•\tI am wondering if the only difference between NoiGAN and KBGAN [7] is incorporating the confidence score in the link prediction loss?\n•\tConsidering the fact that NoiGAN repeatedly retrains GAN and link prediction model, I suggest providing a comparison of computational complexity.\n•\tI am wondering if NoiGAN can only work with pre-knowledge of noisy triples in KG? If not, why didn’t you report NoiGAN performance with 0% noise in Table 3?\n•\tI find utilizing few examples to evaluate the power of discriminator in distinguishing noisy triples (Table 4) not satisfactory at all. I suggest experimenting with more data and providing the per-relation breakdown performance of the discriminator.\n\nOn overall, although I find the proposed model quite novel and interesting, the paper needs more experimental results to validate the idea.\n \n[1] Pinter, Yuval, and Jacob Eisenstein. \"Predicting Semantic Relations using Global Graph Properties\".\n[2] Nathani, Deepak, et al. \"Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs\". \n[3] Balažević, Ivana, Carl Allen, and Timothy M. Hospedales. \"TuckER: Tensor Factorization for Knowledge Graph Completion\".\n[4] Sun, Zhiqing, et al. \"Rotate: Knowledge graph embedding by relational rotation in complex space\".\n[5] Pezeshkpour, Pouya, Yifan Tian, and Sameer Singh. \"Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications\".\n[6] Dettmers, Tim, et al. \"Convolutional 2d knowledge graph embeddings.\", AAAI-18.\n[7] Liwei Cai and William Yang Wang. “Kbgan: Adversarial learning for knowledge graph embeddings”. \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HklhuRpxiH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presented a jointly learning framework based on GAN for tackling both knowledge graph completion and noise detection simultaneously. Existing works only deal with each of task independently and did not investigate the benefits of coping with both tasks together. The paper is well motivated. In order to achieve them, the paper presented a GAN framework in order to train a noising KG embedding as well the generator and discriminator. The key connections between two parts are through the confidence of a noise triple and generation of the negative sample triples. The whole framework looks quite interesting and promising. The experimental results are provided to validate the effectiveness of the proposed model. \n\nThere are two key concerns about this paper:\n\n1) It is well known that both GAN and RL are hard to train, not to mention combining them together to joint train in order to deal with data indifferenceability issue of discrete triple generation. Are the results easy to reproduce?\n\n2) Choosing 10% triples as positive training examples seems very ad-hoc. Have you studied the sensitivity of the number of percentage of triples as positive training examples on the system performance?\n\n3) I don't know too much about the methods from knowledge graph noise detection so maybe one baseline - CKRL is enough for representing state-of-the-arts. However, for knowledge graph completion task, TransE is most simple baseline and they are rich state-of-the-art methods in this line such as [1]. It is not convincing to show the advantages of the proposed NoiGAN without such comparisons. \n\n[1]  “RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space.” ICLR'19. \n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a noise-aware knowledge graph embedding (NoiGAN) by combining KG completion and noise detection through the GANs framework. The reviewers find that the idea is interesting, but the comparison to SOTA is largely missing. The paper can be improved by addressing the reviewer comments. 
Decision: Reject