A Localized Geometric Method to Match Knowledge in Low-dimensional Hyperbolic Space (2022)

Abstract: Matching equivalent entities across Knowledge graphs is a pivotal step for knowledge fusion. Previous approaches usually study the problem in Euclidean space. However, recent works have shown that hyperbolic space has a higher capacity than Euclidean space and hyperbolic embedding can represent the hierarchical structure in a knowledge graph. In this paper, we propose a localized geometric method to find equivalent entities in hyperbolic space. Specifically, we use a hyperbolic neural network to encode the lingual information of entities and the structure of both knowledge graphs into a low-dimensional hyperbolic space. To address the asymmetry of structure on different KGs and the localized nature of relations, we learn an instance-specific geometric mapping function based on rotation to match entity pairs. A contrastive loss function is used to train the model. The experiment verifies the power of low-dimensional hyperbolic space for entity matching and shows that our method outperforms the state of the art by a large margin.

## introduction knowledge graph (kg) is knowledge base that uses a graph-structured topology to integrate entities, relations, and metadata. real-world kgs such as dbpedia, wikidata, and yago benefit a variety of downstream applications such as question answering @xcite , and fact checking @xcite . in general, a kg is constructed from one single knowledge base or built in one single language. thus it is impractical to reach full coverage of the domain @xcite . to increase the completeness of the knowledge base, a conventional approach is fusion of multiple kgs. one pivotal step for fusion is to align equivalent entities across different kgs. conventional entity alignment approaches mainly compare symbolic features of entities @xcite or reason the co-relations by ontology matching @xcite . with the prosperity of node embedding @xcite , recent works favor learning entity embeddings for alignment and compare entities using embedding distance metrics. existing embedding methods for entity alignment can be classified into three types: attributebased @xcite , relation-based @xcite and graph-based @xcite . however, these embedding-based works study the problem in euclidean space, where the embeddings are euclidean vectors. recent research has proven that euclidean space does not provide the most powerful geometrical representations for complex data that exhibit a highly non-euclidean latent anatomy @xcite . to tackle this challenge, a variety of remarkable embedding methods have been developed to represent the data in hyperbolic space. the distinctive feature of hyperbolic spaces enables us to embed hierarchical data while preserving the latent hierarchical structure @xcite . in this paper, we propose to solve the entity alignment problem in hyperbolic space. since entity alignment is a downstream task for embedding, how to use the hyperbolic embedding to match the entities is a challenge. furthermore, all operations in neural networks such as vector addition, matrixvector multiplication, and vector inner product are defined in euclidean space. therefore, existing neural network models are not applicable anymore for hyperbolic embeddings. to address these challenges, we use a hyperbolic version of neural networks. specifically, we utilize a hyperbolic graph neural network model to learn the low-dimensional hyperbolic embeddings for entities on two kgs respectively. two mapping functions are used to implement the initialization, attention-based aggregation, and reduction of dimension in the model. the pre-trained semantic embeddings are projected into the hyperbolic space as the entity features. we consider the output of the hyperbolic neural network model as the final low-dimensional hyperbolic embeddings for entities. then the next task is to map embeddings between two kgs. existing works use a unified global map function to match entities. however, the asymmetry of structure on different kgs makes it difficult to learn a unified relationship for all pairs. the reason behind the varied relations is the heterogeneous nature of data sources for kgs. for example, there are 30,291 inner-relations between 15,000 entities in dbpedia kg. however, there are only 26,638 inner-relations between these entities in yago kg. as a result, the structure of one dbpedia sub-graph may be the same as its counterpart in yago kg, where the structure of another sub-graph may be different from its counterpart. to address the localized relations between entities, we propose an instance-based geometric mapping function where local parameters are learned from its embeddings for each entity. in hyperbolic space, more generic/ambiguous nodes (e.g., root in a kg) tend to be placed closer to the origin while moving more specific objects (nodes at low levels) towards the boundary. figure 1 shows the embeddings of two kgs in a 2-dimensional poincaré disk. we can see that the nodes at low levels will be closer to the original point while nodes at higher levels are placed towards the boundary. this motivates us to design a novel geometric mapping function based on rotation to match two entities across kgs. ideally, after the rotation, the entity will overlap with the equivalent entity in the hyperbolic space. instead of learning a unified rotation function for all entities, we use instance-based rotation functions. as shown in figure 1 , after the rotation of embedding for e 1 through θ 1 , e 1 will overlap with its corresponding entity e ′ 1 . however, the angle between embeddings of e 2 and e ′ 2 is θ 2 , which is totally different from θ 1 . to train the model, we minimize the hyperbolic distance after mapping between a pair of aligned entities and push negative samples away from the target one. for each entity, we find the aligned entity by searching for the nearest neighbor in terms of hyperbolic distance. our novelty over existing works can be summarized as: ## related work the majority of existing entity matching methods rely on kg embeddings @xcite . according to the kg embeddings approaches, these models can be roughly categorized into three groups: relation-based, attributes-based, and graphbased models. relation-based models mainly employ the translational methods @xcite to learn the embedding based on relationship triples. iptranse @xcite is an entity alignment model based on translation. it encodes both entities and relations into a unified low-dimensional semantic space. mtranse @xcite encodes entities and relations of each kg in a separated embedding space. bootea @xcite leverages the bootstrapping idea to iteratively label likely alignment. rsns @xcite feeds the relational paths into recurrent neural networks to learn embeddings. to increase the robustness, mul-tike @xcite unifies multiple views of entities and embeds entities with several combination strategies. attributes-based models consider the correlations among attributes of entities. for example, jape @xcite assumes that similar entities should have similar correlated attributes. attre @xcite exploits large numbers of attribute triples and models the various types of attribute triples to generate attribute embeddings. then the embedding shifts two kgs into the same space by computing the similarity. with the prosperity of graph neural networks @xcite @xcite , many works propose to utilize graph convolutional networks to model the structure of kg. gcnalign @xcite trains gcns to embed entities of each kg into a unified vector space. rdgcn @xcite further incorporates relation information in kg and captures neighboring structures via dual-graph convolutional network. alinet @xcite aims to mitigate the non-isomorphism of neighborhood structures in an end-to-end manner and controls the aggregation of both direct and distant neighborhood information using a gating mechanism. rrea @xcite abstracts existing entity alignment methods into a unified framework and derives two key criteria for an ideal transformation operation. rnm @xcite is a relationaware neighborhood matching model. it utilizes neighborhood matching to enhance the entity alignment and uses an iterative framework to leverage the positive samples and the relation alignment in a semi-supervised manner. dual-amn @xcite uses an encoder to model both intra-graph and cross-graph information. easy @xcite removes the labor-intensive pre-processing by fully discovering the name information provided by the entities themselves and jointly fuses the features captured by the names of entities and the structural information of the graph. activeea @xcite introduces active learning to reduce the cost of labeling and annotation. temporal kg is also studied to match time-aware entities @xcite . hmea @xcite utilizes visual information to learn image embeddings. it combines the structure and visual representations in the hyperbolic space to predict alignment results. hyperka @xcite also aligns entities across kgs in hyperbolic space. however, hyperka directly aggregates neighborhood information in hyperbolic space and fails to leverage the power of hyperbolic neural networks (e.g., dimensionality reduction and attention mechanism). furthermore, it uses a unified linear transformation function where we use a localized geometric method; thus hyperka ignores the non-linear distance ratio with respect to the radius, the isometry of hyperbolic geometry and the locality of mapping. lastly, it randomly associates each entity with a vector. instead, we associate the entity with a pretrained semantic embedding. besides entity align-ment, tensor completion @xcite is another method to increase the completeness of the knowledge base. ## methodology we first associate each entity e with a vector as the feature. specifically, we follow rnm @xcite to initialize the entity vector with the pre-trained semantic embedding x e , which can represent the lingual information of entity names. however, existing pre-trained word embedding is learned from euclidean neural networks or in euclidean space. to address this problem, we map euclidean features into the hyperboloid manifold by: to utilize the structure of the kgs for entity alignments, we introduce the aggregation operation in hyperbolic space. on each knowlege graph, we aggregate neighbor's vectors with that of the center entity by: then we aggregate the neighbor's vector in the tangent space and map them back to hyperbolic space: here we use conc(•, •) to denote the concatenation operation, and the softmax function is used to normalize the weights. the attention mechanism enhances the important neighbors. to further reduce the dimension of the hyperbolic vectors, we feed z (k+1) i into a hyperbolic linear layer: existing works either utilize a unified linear transformation function for all entities or directly enforce two embeddings close to each other. however, these works ignore the asymmetry of structure on different kgs and the localized nature of relations. intuitively, the relations between two entities may vary from one pair to another. for example, the equivalent entity of "new orleans" is "la nouvelle-orléans" in the french wikipedia kg. however, the corresponding entity "times square" in french is still "times square". these two relations are totally different. the first relation is translation but the second pair of entities are identical to each other. as another example, there are 30,291 inner-relations between 15,000 entities in dbpedia kg. however, there are only 26,638 inner-relations between these entities in yago kg. as a result, the structure of one dbpedia sub-graph may be same as its counterpart in yago kg, where the structure of another sub-graph may be totally different from its counterpart. the asymmetry of relations on different kgs makes it difficult to learn a unified relationship for all pairs. in this paper, we propose to use a localized mapping function. specifically, for each entity, we learn the parameter in the mapping function from its hyperbolic embedding. since kgs often exhibit hierarchies, the root node or the node at a low level on the kg tends to be located near the original point in the hyperbolic space generally. considering this property, we design a parameterized geometric mapping function where the parameters are computed from the hyperbolic embedding. let h 1 and h 2 be the embeddings of entities after k iterations on g 1 and g 2 , respectively. suppose we have a pair of equivalent entities across g 1 and g 2 : (e i , e j ) where e @xmath0. our instance-specific mapping function is a rotation: to train the model, we propose to minimize the hyperbolic distance between f (h 1 i ) and h 2 j for each pair of @xmath1, e j )|e 1 ∈ e 1 , e 2 ∈ e 2 }. at the same time, we propose to push negative samples away from an entity. in order to achieve this, we design a loss function formulated as: ## conclusion we proposed gea-h, a geometric entity alignment method in hyperbolic space. gea-h learns lowdimensional hyperbolic embeddings for entities in kgs with an attention-based hyperbolic graph neural network. we design a geometric function based on rotation for mapping entities and learn the localized parameters of mapping function for each entity to address the locality of mapping.