Time-aware Graph Neural Network for Entity Alignment between Temporal Knowledge Graphs (2021)

Abstract: Entity alignment aims to identify equivalent entity pairs between different knowledge graphs (KGs). Recently, the availability of temporal KGs (TKGs) that contain time information created the need for reasoning over time in such TKGs. Existing embedding-based entity alignment approaches disregard time information that commonly exists in many large-scale KGs, leaving much room for improvement. In this paper, we focus on the task of aligning entity pairs between TKGs and propose a novel Time-aware Entity Alignment approach based on Graph Neural Networks (TEA-GNN). We embed entities, relations and timestamps of different KGs into a vector space and use GNNs to learn entity representations. To incorporate both relation and time information into the GNN structure of our model, we use a self-attention mechanism which assigns different weights to different nodes with orthogonal transformation matrices computed from embeddings of the relevant relations and timestamps in a neighborhood. Experimental results on multiple real-world TKG datasets show that our method significantly outperforms the state-of-the-art methods due to the inclusion of time information.

## introduction knowledge graphs (kgs) provide a means for structured knowledge representation through connected nodes via edges. the nodes represent entities and the edges connecting these nodes denote relations. a kg stores facts as triples of the form (e s , r, e o ), where e s is the subject entity, e o is the object entity, and r is the relation between entities. many large-scale kgs including yago @xcite and dbpedia (lehmann et al., 2015) * is the corresponding author have been established and are widely used in nlp applications, e.g., question answering and language modeling @xcite . since most kgs are developed independently and many of them are supplementary in contents, one of core challenges of kgs is to align equivalent entity pairs between different kgs. to address this issue, embedding-based approaches are leveraged to model entities and relations across multiple kgs and measure the similarities between entities @xcite . it has been proven that the utility of multi-relation information is helpful for an effective entity alignment approach @xcite . in addition to relation information, many kgs including yago3 @xcite , wikidata @xcite and icews @xcite also contain time information between entities, i.e., some edges between entities have two properties, relation and time as shown in figure 1 . facts in such temporal kgs (tkgs) can be represented as quadruples shaped like (e s , r, e o , τ ) where τ denotes the timestamp. noteworthily, timestamps in most tkgs are presented in arabic numerals and have similar formats. thus, timestamps representing the same dates across multiple tkgs can be easily aligned by manually uniforming their formats. however, the existing embedding-based entity alignment approaches disregard time informa-tion in tkgs, leaving much room for improvement. taking the case in figure 1 as an example, given two entities, george h. w. bush and george walker bush, existing in two tkgs respectively, time-agnostic embedding-based approaches are likely to ignore time information and wrongly recognize these two entities as the same person in the real world due to the homogeneity of their neighborhood information. to address this issue, an intuitive solution is to incorporate time information into entity alignment models. inspired by the recent successful applications of gnn models in entity alignment, in this paper, we propose a novel time-aware entity alignment approach based on graph neural networks (tea-gnn) for entity alignment between tkgs. different from some temporal gnn models which discretize temporal graphs into multiple snapshots, we treat timestamps as properties of links between entities. we first map all entities, relations and timestamps in tkgs into an embedding space. to incorporate relation and time information into the gnn structure, we utilize a time-aware attention mechanism which assigns different importance weights to different nodes within a neighborhood according to orthogonal transformation matrices computed with the embedddings of the corresponding relations and timestamps. to further integrate time information into the final entity representations, we concatenate output features of entities with the summation of their neighboring time embeddings to get multi-view entity representations. ## problem formulation formally, a tkg is represented as @xmath0, r, t , q) where e, r and t are the sets of entities, relations and timestamps, respectively. ## the proposed approach to exploit both relation and time information for entity alignment, we first create a reverse link for each link so that each pair of reverse links between entities can represent relation directions and handle the begin and end of the relation. an orthogonal transformation-based time-aware attention mechanism is employed in each gcn layer to assign different weights to entities according to relation and time information between them. finally, entity alignments are predicted by applying a distance function to multi-view representations of entities. where τ 0 ∈ t * is the first time step in the time set denoting the unknown time information. a fact without known time information can be denoted as (e s , r, e o , [τ 0 , τ 0 ]) to deal with heterogeneous temporal knowledge bases where a significant amount of relations might be non-temporal. ## conclusion the main contributions of this paper are threefold: for future work, we will try to integrate other types of information, e.g,. attribute information, into our model and extend our model for other learning tasks of temporal graphs.