Weakly-supervised Knowledge Graph Alignment with Adversarial Learning

This paper studies aligning knowledge graphs from different sources or languages . Most existing methods train supervised methods for the alignment , which usually require a large number of aligned knowledge triplets . However , such a large number of aligned knowledge triplets may not be available or are expensive to obtain in many domains . Therefore , in this paper we propose to study aligning knowledge graphs in fully-unsupervised or weakly-supervised fashion , i.e. , without or with only a few aligned triplets . We propose an unsupervised framework to align the entity and relation embddings of different knowledge graphs with an adversarial learning framework . Moreover , a regularization term which maximizes the mutual information between the embeddings of different knowledge graphs is used to mitigate the problem of mode collapse when learning the alignment functions . Such a framework can be further seamlessly integrated with existing supervised methods by utilizing a limited number of aligned triples as guidance . Experimental results on multiple datasets prove the effectiveness of our proposed approach in both the unsupervised and the weakly-supervised settings . 1 INTRODUCTION . Knowledge graphs represent a collection of knowledge facts and are quite popular in the real world . Each fact is represented as a triplet ( h , r , t ) , meaning that the head entity h has the relation r with the tail entity t. Examples of real-world knowledge graphs include instances which contain knowledge facts from general domain ( e.g. , Freebase 1 , WordNet 2 ) or facts from specific domains such as biomedical ontology ( e.g. , UMLS 3 ) . Knowledge graphs are critical to a variety of applications such as question answering ( Bordes et al. , 2014 ) and semantic search ( Guha et al. , 2003 ) . Research on knowledge graphs is attracting growing interest recently in both academia and industry communities . In practice , each knowledge graph is usually constructed from a single source or language , the coverage of which is limited . To enlarge the coverage and construct more unified knowledge graphs , a natural idea is to integrate multiple knowledge graphs from different sources or languages ( Arens et al. , 1993 ) . However , different knowledge graphs use distinct symbol systems to represent entities and relations , which are not compatible . Therefore , it is critical to align the entities and relations across different knowledge graphs ( a.k.a. , knowledge graph alignment ) before integrating them . Recently , many methods have been proposed to align entities and relations from a source knowledge graph to a target knowledge graph ( Zhu et al. , 2017a ; Chen et al. , 2017a ; b ; Sun et al. , 2018a ) . These methods first represent the entities and relations in low-dimensional spaces and then learn mapping functions to align the entities and relations from the source knowledge graph to the target one . Though these methods are proven quite effective , they rely on a large number of aligned triplets for training supervised alignment models , and such aligned triplets may not be available or can be expensive to obtain . As a result , the performance of these methods will be comprised . Therefore , it would be desirable to design an unsupervised or weakly-supervised approach for knowledge graph alignment , which requires a few or even without aligned triplets . 1 https : //developers.google.com/freebase/ 2 https : //wordnet.princeton.edu/ 3 https : //www.nlm.nih.gov/research/umls/ In this paper , we propose an unsupervised approach to knowledge graph alignment with adversarial training ( Goodfellow et al. , 2014 ) . Our proposed approach first represents the entities and relations in low-dimensional spaces with existing knowledge graph embedding methods ( e.g. , TransE ( Bordes et al. , 2013 ) ) and then learns alignment functions , i.e. , pe ( et|es ) and pr ( rt|rs ) , to map the entities and relations ( es and rs ) from the source knowledge graph to those ( et and rt ) in the target graph . Intuitively , an ideal alignment function is able to map all triples in the source graph to valid ones in the target graph . Therefore , we train a triplet discriminator to distinguish between the real triplets in the target graph and those aligned ones from the source graph . Such a discriminator measures the plausibility of a triplet in the target graph and provides a reward function for optimizing the alignment functions , which are optimized to fool the discriminator . The above process naturally forms an adversarial training procedure . By alternatively optimizing the alignment functions and the discriminator , the whole process can constantly enhance the alignment functions . Despite the effectiveness of adversarial learning in many scenarios , one big problem it may suffer from is the mode collapse ( Salimans et al. , 2016 ) . Specifically , in our case , it means that many entities in the source knowledge graph are aligned to only a few entities in the target knowledge graph . We propose to mitigate this problem by maximizing the mutual information between the entities in the source graph and those aligned entities , which can be effectively and effectively optimized with some recent techniques on mutual information neural estimation ( Belghazi et al. , 2018 ) . We further prove that by maximizing the mutual information , different source-graph entities are encouraged to be aligned to different target-graph entities , which mitigates the mode collapse . The whole framework can also be seamlessly integrated with existing supervised methods , in which we can use a few aligned entities or relations as guidance , yielding a weakly-supervised approach . Our approach can be effectively optimized with stochastic gradient descent , where the gradient for the alignment functions is calculated by the REINFORCE algorithm ( Williams , 1992 ) . We conduct extensive experiments on several datasets . Experimental results prove the effectiveness of our proposed approach in both the weakly-supervised and unsupervised settings . 2 RELATED WORK . Our work is related to knowledge graph embedding , which represents entities and relations as lowdimensional vectors ( a.k.a. , embedding ) . A variety of approaches have been proposed ( Bordes et al. , 2013 ; Wang et al. , 2014 ; Yang et al. , 2014 ; Sun et al. , 2018b ) , which can effectively preserve the similarities of entities and relations into the learned embeddings . We treat these techniques as tools to learn entity and relation embeddings , which serve as features for knowledge graph alignment . In literature , there are also some studies focusing on knowledge graph alignment . Most of them perform alignment by considering contextual features of entities and relations , such as their names ( Lacoste-Julien et al. , 2013 ) or text descriptions ( Chen et al. , 2018 ; Wang et al. , 2012 ; 2013 ) . However , such contextual features are not always available , and therefore these methods can not generalize to most knowledge graphs . In this paper , we consider the most general case , in which only the triplets in knowledge graphs are used for alignment . The studies most related to ours are Zhu et al . ( 2017a ) , Chen et al . ( 2017a ) and Sun et al . ( 2018a ) . Similar to our approach , they treat the entity and relation embeddings as features , and jointly train an alignment model . However , they totally rely on the labeled data ( e.g. , aligned entities ) to train the alignment model , whereas our approach incorporates additional signals by using adversarial training , and therefore achieves better results in the weakly-supervised and unsupervised settings . More broadly , our work belongs to the family of domain alignment , which aims at mapping data from one domain to the other domain . With the success of generative adversarial networks ( Goodfellow et al. , 2014 ) , many researchers have been bringing the idea to domain alignment , getting impressive results in many applications , such as image-to-image translation ( Zhu et al. , 2017b ; c ) , word-to-word translation ( Conneau et al. , 2017 ) and text style transfer ( Shen et al. , 2017 ) . These studies typically train a domain discriminator to distinguish between data points from different domains , and then the alignment function is optimized by fooling the discriminator . Our approach shares similar idea , but is designed with some specific intuitions in knowledge graphs . Besides , our work is also related to recent studies on neural mutual information estimation ( Belghazi et al. , 2018 ) , which estimate the mutual information of two distributions by using neural networks . Such a technique has been utilized in many applications , including image classification ( Hjelm et al. , 2018 ) and unsupervised node representation learning ( Veličković et al. , 2018 ) . All these studies use the technique to improve representation learning ( e.g. , image representation , node representation ) . In contrast , our approach uses the technique to avoid mode collapse in adversarial learning . Finally , Cai & Wang ( 2018 ) propose a method called KBGAN recently , which uses adversarial learning for generating negative examples in knowledge graph embedding algorithms . Compared with this work , both our method and KBGAN use the adversarial learning framework , but our work is fundamentally different from KBGAN . More specifically , the goal of KBGAN is to use adversarial learning to generate effective negative samples for knowledge graph embedding , while our work studies a very different problem , that is , knowledge graph alignment . We use the adversarial learning framework to learn effective mapping functions to align entities and relations across different knowledge graphs . Moreover , a major challenge in knowledge graph alignment is that the entities ( or relations ) in the source knowledge graph could be mapped to only a subset of entities ( or relations ) in the target knowledge graph . We propose to maximize the mutual information between the source entities ( relations ) and the mapped entities ( relations ) to mitigate this problem . 3 PROBLEM DEFINITION . Definition 1 ( KNOWLEDGE GRAPH . ) A knowledge graph is denoted as G = ( E , R , X ) , where E is a set of entities , R is a set of relations and X is a set of triplets . Each triplet x = ( h , r , t ) consists of a head entity h , a relation r and a tail entity t , meaning h has relation r with t. In practice , the coverage of each individual knowledge graph is usually limited , since it is typically constructed from a single source or language . To construct knowledge graphs with broader coverage , a straightforward way is to integrate multiple knowledge graphs from different sources or languages . However , each knowledge graph uses a unique symbol system to represent entities and relations , which is not compatible with other knowledge graphs . Therefore , a prerequisite for knowledge graph integration is to align entities and relations across different knowledge graphs ( a.k.a. , knowledge graph alignment ) . In this paper , we study how to align entities and relations from a source knowledge graph to those in a target knowledge graph , and the problem is formally defined below : Definition 2 ( KNOWLEDGE GRAPH ALIGNMENT . ) Given a source knowledge graph Gs = ( Es , Rs , Xs ) and a target knowledge graph Gt = ( Et , Rt , Xt ) , we aim at learning an entity alignment function pe and a relation alignment function pr . Given an entity es in the source graph and an entity et in the target graph , pe ( et|es ) gives the probability that es aligns to et . Similarly , for a source relation rs and a target relation rt , pr ( rt|rs ) gives the probability that rs aligns to rt .

Summary: The authors propose KAGAN, a novel method for knowledge graph (KG) alignments using GANs. In contrast to most other methods, KAGAN does not rely on a supervised setting where a set of already aligned triples is used as seed. In addition, the authors propose modifications such that their method can also integrate information about aligned triples.