Knowledge Association with Hyperbolic Knowledge Graph Embeddings (2020)

Abstract: Capturing associations for knowledge graphs (KGs) through entity alignment, entity type inference and other related tasks benefits NLP applications with comprehensive knowledge representations. Recent related methods built on Euclidean embeddings are challenged by the hierarchical structures and different scales of KGs. They also depend on high embedding dimensions to realize enough expressiveness. Differently, we explore with low-dimensional hyperbolic embeddings for knowledge association. We propose a hyperbolic relational graph neural network for KG embedding and capture knowledge associations with a hyperbolic transformation. Extensive experiments on entity alignment and type inference demonstrate the effectiveness and efficiency of our method.

## introduction knowledge graphs (kgs) have emerged as the driving force of many nlp applications, e.g., kbqa @xcite , dialogue generation @xcite and narrative prediction @xcite . different kgs are usually extracted from separate data sources or contributed by people with different expertise. therefore, it is natural for these kgs to constitute complementary knowledge of the world that can be expressed in different languages, structures and levels of specificity @xcite . associating multiple kgs via entity alignment @xcite or type inference @xcite particularly provides downstream applications with more comprehensive knowledge representations. entity alignment and type inference seek to find two kinds of knowledge associations, i.e., sameas and instanceof, respectively. an example showing such associations is given in figure 1 . specifically, entity alignment is to find equivalent entities from different entity-level kgs, such as united states in dbpedia and united states of america in wikidata. type inference, on the other hand, associates a specific entity with a concept describing its type information, such as united states and country. the main difference lies in whether such knowledge associations express the same level of specificity or not. challenged by the diverse schemata, relational structures and granularities of knowledge representations in different kgs @xcite , traditional symbolic methods usually fall short of supporting heterogeneous knowledge association @xcite @xcite . recently, increasing efforts have been put into exploring embeddingbased methods @xcite @xcite . such methods capture the associations of entities or concepts in a vector space, which can help overcome the symbolic and schematic heterogeneity @xcite . embedding-based knowledge association methods still face challenges in the following aspects. (i) hierarchical structures. a kg usually consists of many local hierarchical structures @xcite . besides, a kg also usually comes with an ontology to manage the relations (e.g., subclassof ) of concepts @xcite , which typically forms hierarchical structures as illustrated in figure 1 . it is particularly difficult to preserve such hierarchical structures in a linear embedding space @xcite . (ii) high parameter complexity. to enhance the expressiveness of kg embeddings, many methods require high embedding dimensions, which inevitably cause excessive memory consumption and intractable parameter complexity. for example, for the entity alignment method gcn-align @xcite , the embedding dimension is selected to be as large as 1, 000. reducing the dimensions can effectively decrease memory cost and training time. (iii) different scales. the kgs that we manipulate may differ in scales. for example, while the english dbpedia contains 4, 233, 000 entities, its ontology only contains less than a thousand concepts. capturing the associations between entities and concepts has to deal with drastically different scales of structures and search spaces, while most existing methods do not consider such difference. to tackle these challenges, we propose a novel hyperbolic knowledge association method, namely hyperka, inspired by the recent success of hyperbolic representation learning @xcite @xcite . unlike the euclidean circle circumference that grows linearly w.r.t. the radius, the hyperbolic space grows exponentially with the radius. this property makes the hyperbolic geometry particularly suitable for embedding the hierarchical structures that drastically span their sizes along with their levels. it is also capable of achieving superior expressiveness at a low dimension. to leverage such merit, hyperka employs a hyperbolic relational graph neural network (gnn) for kg embedding and captures multi-granular knowledge associations with a hyperbolic transformation between embedding spaces. for each kg, hyperka first incorporates hyperbolic translational embeddings at the input layer of the gnn. then, several hyperbolic graph convolution layers are stacked over the inputs to aggregate neighborhood information and obtain the final embeddings of entities or concepts. on top of the kg embeddings, a hyperbolic transformation is jointly trained to capture the associations. we conduct extensive experiments on entity alignment and type inference. hyperka outperforms sota methods on both tasks at a moderate dimension (e.g., 50 or 75). even with a small dimension (e.g., 10), our method still shows competitive performance. ## hyperbolic geometry the hyperbolic space is one of the three kinds of isotropic spaces. for simplicity, we follow @xcite and let @xmath0. we hereby introduce some basic operations of hyperbolic geometry, which we use extensively. hyperbolic distance. the distance between vectors u and v in the poincaré ball is given by: when points move from the origin towards the ball boundary, their distance would increase exponentially, offering a much larger volume of space for embedding learning. vector translation. the vector translation in the poincaré ball is defined by the möbius addition: transformation. a transformation is the backbone of both gnns @xcite and transformation-based associations @xcite . the work in @xcite defines the matrix-vector multiplication between poincaré balls using the exponential and logarithmic maps. the hyperbolic vectors are first projected into the tangent space at 0 using the logarithmic map (log 0 : d n,1 → t 0,n d n,1 ), then multiplied the transformation matrix like in the euclidean space, and finally projected back on the manifold with the exponential map (exp 0 : t 0,n d n,1 → d n,1 ). specifically, the two projections on vector @xmath1,1 are defined as follows: through such inverse projections, theoretically, we can apply any euclidean counterpart operations on hyperbolic vectors. the transformation that maps a vector @xmath2,c into d m,c can be done using the möbius version of matrix-vector multiplication: in this section, we introduce the technical details of hyperka -the hyperbolic gnn-based representation learning method for knowledge association. different from existing relational gnns like r-gcn (schlichtkrull et al., 2018), avr-gcn @xcite and compgcn @xcite ) that perform a relation-specific transformation on relational neighbors before aggregation, our method models relations as translations between entity vectors at the input layer, and performs neighborhood aggregation on top of them to derive the final entity embeddings. this allows our method to benefit from both relation translation and neighborhood aggregation without increasing computation complexity. ## experiments we evaluate the proposed method hyperka on two tasks of knowledge association, i.e. entity alignment (section 5.1) and entity type inference (section 5.2). the source code is publicly available foot_0 . ## conclusion and future work we propose a method to capture knowledge associations with a new hyperbolic gnn-based repre-sentation learning model. the proposed hyperka method extends translational and gnn-based techniques to hyperbolic spaces, and captures associations by a hyperbolic transformation. our method outperforms sota baselines using lower embedding dimensions on both entity alignment and type inference. for future work, we plan to incorporate hyperbolic rnns @xcite to encode auxiliary information for zero-shot entity and concept representations. another meaningful direction is to use hyperka to infer the associations between snapshots in temporally dynamic kgs @xcite . we also seek to investigate the use of hyperka for cross-domain representations of biological and medical knowledge @xcite .