A uto ETER : Automated Entity Type Representation for Knowledge Graph Embedding (2020)

Abstract: Recent advances in Knowledge Graph Embedding (KGE) allow for representing entities and relations in continuous vector spaces. Some traditional KGE models leveraging additional type information can improve the representation of entities which however totally rely on the explicit types or neglect the diverse type representations specific to various relations. Besides, none of the existing methods is capable of inferring all the relation patterns of symmetry, inversion and composition as well as the complex properties of 1-N, N-1 and N-N relations, simultaneously. To explore the type information for any KG, we develop a novel KGE framework with Automated Entity TypE Representation (AutoETER), which learns the latent type embedding of each entity by regarding each relation as a translation operation between the types of two entities with a relation-aware projection mechanism. Particularly, our designed automated type representation learning mechanism is a pluggable module which can be easily incorporated with any KGE model. Besides, our approach could model and infer all the relation patterns and complex relations. Experiments on four datasets demonstrate the superior performance of our model compared to state-of-the-art baselines on link prediction tasks, and the visualization of type clustering provides clearly the explanation of type embeddings and verifies the effectiveness of our model.

## introduction in recent years, knowledge graph (kg) has been viewed as a powerful technique for recognition systems and prevalent in many fields such as ecommerce, intelligent healthcare, and public security. knowledge graphs collect and store a great deal of commonsense or domain knowledge in factual triples composed of entity pairs with their relations. the existing large scale kgs such as freebase @xcite , wordnet @xcite , yago @xcite have shown their validity in various applications, including question answering @xcite , dialogue generation @xcite and recommender systems @xcite . however, the existing kgs are inevitably incomplete whether they are constructed manually or automatically, limiting the effectiveness when exploited for downstream applications. some existing kg inference approaches such as inductive logic programming algorithm @xcite , markov logic networks based method @xcite and reinforcement learning-based approach @xcite try to predict entities or relations in kgs but face the limited performance and suffer from the low efficiency. compared to the above approaches, knowledge graph embedding models could learn the latent representations of the entities and relations and show the best performance on the kg completion task. however, most of the kg embedding models such as transe @xcite and its variants transh @xcite , transr @xcite ) learn kg embeddings relying on single triples, which simply exploit the structure information implied in kgs. entity types define categories of entities that are valid to enhance the representation of entities. in many type-embodied models such as tkrl @xcite and transt @xcite , the explicit types are necessary while some kgs (i.e., wordnet) lack them, which limits the versatility of these approaches. joie @xcite jointly encodes both the ontology and instance views of kgs. nevertheless, ontologies' concepts always represent the general categories of entities but cannot reflect the specific types, primarily associated with different relations. jain @xcite learned the type embeddings by defining the compatibility between an entity type and a relation. still, it ignores the semantics implied in a whole triple consisting of a relation jointly with its linked two entity types. moreover, all the previous typebased approaches neglect the diversity of entity type representations specific to various relations. as figure 1 shows, contrary to the previous researches considering entity types, the triples in the entity level could be extended to triples in the type level. each entity has multiple types, and diverse types should be focused on for different specific relations. additionally, some models embed the entities and relations into the complex vector space instead of the frequently-used real space to improve the capability of representation learning, including com-plex @xcite and rotate @xcite . nevertheless, none of the existing embedding models could model and infer all the relation patterns and the complex 1-n, n-1 and n-n relations, simultaneously. to conduct the kg inference from the perspectives of both entity-specific triples and type-specific triples on any kg, whether the explicit types exist, we propose autoeter to automatically learn the diverse type representations of each entity when focusing on the various associated relations. intuitively, the high-dimensional entity embeddings imply the individual features to distinguish the diverse entities. in contrast, the low-dimensional type embeddings capture the general features to discover the similarity of entities according to their categories. inspired by the translational-based principle in transe, we expect that given a head entity and its associated relation, the tail entity's type representation can be obtained by type head +@xmath0. particularly, the latent type embeddings of two head or two tail entities focused on the same relation should be close to each other since they imply the same type. furthermore, the embeddings of the entity-specific triples and the type-specific triples are capable of modeling and inferring symmetry, inversion, composition, and complex 1-n, n-1, n-n relations. the contributions of this work are summarized as follows: ## autoeter: kge with automated entity type representation to cope with the above limitations, we describe the proposed model autoeter, which aims to automatically learn a variant of type representations semantically compatible with various relations and infer all the relation patterns and complex relations. as figure 2 shows, we first embed the entities and relations into complex space via the entity-specific triple encoder with a hyper-plane projection strategy ( §3.1). additionally, the type-specific triple encoder is developed to learn type embeddings incorporated with a relation-aware projection mechanism ( §3.2). meanwhile, the type embeddings are constrained by their similarity derived from the associated relations ( §3.3). afterward, we propose the overall optimization objective with both entity-specific triple and type-specific triple representations and the similarity constraint of the type embeddings ( §3.4). ## experiment results in this section, we evaluate our model autoeter for kg completion on four real-world benchmark datasets. additionally, we visualize the clustering results of type embeddings for demonstrating the effectiveness of representing types automatically. ## conclusion and future work in this paper, we propose an autoeter framework to learn type representations for enriching kg embedding automatically. we introduce two classes of encoders to learn the entity-specific triple and typespecific triple embeddings, which could model and infer all the relation patterns of symmetry, inversion and composition as well as the complex 1-n, n-1 and n-n relations. we also constrain the type embeddings by the type similarity. our experiments on four real-world datasets for link prediction illustrate the superiority of our model and the visualization of the type embeddings clustering verifies the availability of representing types automatically. in future work, we intend to extend our approach to obtain the better type representations incorporating the supervision of ontologies.