A Good Neighbor, A Found Treasure: Mining Treasured Neighbors for Knowledge Graph Entity Typing (2022)

Abstract: The task of knowledge graph entity typing (KGET) aims to infer the missing types for entities in knowledge graphs. Some pioneering work has proved that neighbor information is very important for the task. However, existing methods only leverage the one-hop neighbor information of the central entity, ignoring the multi-hop neighbor information that can provide valuable clues for inference. Besides, we also observe that there are co-occurrence relations between types, which is very helpful to alleviate false-negative problem. In this paper, we propose a novel method called Mining Treasured Neighbors (MiNer) to make use of these two characteristics. Firstly, we devise a Neighbor Information Aggregation module to aggregate the neighbor information. Then, we propose an Entity Type Inference module to mitigate the adverse impact of the irrelevant neighbor information. Finally, a Type Co-occurrence Regularization module is designed to prevent the model from overfitting the false negative examples caused by missing types. Experimental results on two widely used datasets indicate that our approach significantly outperforms previous state-of-the-art methods.

## introduction knowledge graphs (kgs) store huge amounts of structured data in the form of triples, i.e., (head entity, relation, tail entity). each entity in kgs is labeled with one or more types. as shown in figure 1 , the entity einstein not only belongs to scientist type, but also physicist type. the entity type information is very important and can benefit many natural language processing (nlp) applications, such as entity linking @xcite , relation extraction @xcite , knowledge graph embedding @xcite ) and text generation @xcite . unfortunately, kgs usually suffer from entity type incompleteness problems. more specifically, one entity may have multiple types, while the annotated entity type information is usually incomplete. take figure 1 as an example, the entity newton should be labeled with scientist, mathematician, physicist and englander types. however, only the scientist type is annotated in the kg. according to the statistics on the fb15k @xcite , 10% of entities have the /music/artist type, but missing the /people/person type, which indicates the type incompleteness problem is not negligible. therefore, we focus on knowledge graph entity typing (kget), which aims to infer the missing types from existing types for entities in kgs. great efforts have been devoted to tackling the kget task, which can be mainly divided into embedding-based methods @xcite and graph neural network-based methods @xcite . embedding-based methods focus on learning lowdimensional vector representations of entities, rela-tions and entity types, then predict missing types based on a scoring function. although embeddingbased methods are simple and intuitive, they ignore the rich neighbor information of entities. by contrast, graph neural network-based methods effectively leverage the neighbor information by modeling it as the graph-structured data to infer the missing types, which has shown to be the most effective for the kget task @xcite . despite these successful efforts, existing methods ignore the multi-hop neighbor information, and type cooccurrence information, which is very important for the kget task. multi-hop neighbor information can provide more valuable clues for inference. for example in figure 1 , based on the one-hop neighbor information alone, it is difficult to predict the entity newton has mathematician type. fortunately, multi-hop neighbor information can provide more conclusive clues. for instance, there are facts that newton and leibniz (two-hop neighbor) both invent calculus, and leibniz has mathematician type. combining the two, the model can easily conclude newton has mathematician type. nevertheless, aggregating multi-hop neighbors may bring irrelevant information. as shown in figure 1 , only the triple (newton, born in, england) plays a decisive role in inferring the englander type for newton, while others contribute less. therefore, the first challenging problem is how to mine treasured neighbor information for inference. type co-occurrence information can facilitate alleviating the false-negative problem. some entity types should have been stored in kgs, but they are not marked. most existing methods simply treat them as negative samples. therefore, they face a serious false-negative problem that affects model performance. we observe a rich amount of type co-occurrence information in kgs, which can be used to address the false-negative problem. for instance, both scientist type and physicist type often go hand in hand. however, scientist type and actor type rarely belong to the same entity. if the model makes use of the prior knowledge, it can alleviate the memorization of false labels and accurately predict the missing types. therefore, the second challenging problem is how to leverage the type co-occurrence information. in this paper, we propose a novel method termed as mining treasured neighbors (miner) to address aforementioned problems. the proposed method consists of three modules: neighbor information aggregation module, entity type inference module and type co-occurrence regularization module. first, the neighbor information aggregation module aims to aggregate more neighbor information, including one-hop neighbor information and multi-hop neighbor information. this module can generalize to any number of hops. to mitigate the adverse impact of the irrelevant neighbor information, the entity type inference module mines the valuable neighbor information for central entities via two ways: type-specific local inference and type-agnostic global inference. in addition, we leverage the type co-occurrence information to alleviate the memorization of the false labels. we propose type co-occurrence regularization module to correct false negative examples caused by missing types. experimental results on two widely used datasets indicate that our approach significantly outperforms previous state-of-the-art methods. our contributions are summarized as follows: ## related work knowledge graph entity typing (kget) is an essential sub-task of knowledge graph completion (kgc) that has been researched for decades. existing approaches for the task can be mainly divided into embedding-based methods and graph neural network-based methods. embedding-based methods. the entities with known types can be treated as as special triples with a unique relation "has type", e.g., (newton, has type, physicist). in this way, the kget can be formulated as a link prediction task. existing knowledge graph embedding (kge) methods @xcite @xcite @xcite can be used to infer the missing types of newton by completing the triple (newton, has type, ?). although using kge methods can address the kget task to some degree, it lacks the diversity of relation types. therefore, @xcite propose the ete model to tackle this problem. ete first learns entity embeddings and relation embeddings with a kge model on kgs that do not have entity types, then trains the embedding of each entity to be closer to the embedding of its type. for better expressing and reasoning capability, @xcite propose the connecte model, which considers both local entity typing information and global triple knowledge in kgs. connecte first uses transe @xcite to obtain the entity embeddings, then infers the missing types according to two inference mechanisms. one is e2t mechanism that focuses on mapping entities from entity space to entity types space. another is trt mechanism, which is based on the assumption that the relation can remain unchanged when replacing the entities in the triple with their types. ## task definition formally, we consider a kg g containing the triples in the form of (e, r, ẽ) and the entity type information in the form of (e, t), where e, ẽ ∈ e, @xmath0, @xmath1, and e, r, t are the entity set, relation set and entity type set, respectively. one entity may have multiple types, while the annotated entity type is usually incomplete. the neighborhoods of entities can provide more valuable and richer information for inferring missing types. following the setting of @xcite , we also regard the existing types of each entity as its one-hop neighbors. therefore, this paper aims to infer the missing types by considering the neighbor information (i.e., onehop neighbor information and multi-hop neighbor information) of entities. ## method our approach is shown in figure 2 , which consists of three primary components: (1) neighbor information aggregation, which aggregates the information from one-hop neighbors and multi-hop neighbors; (2) entity type inference, which mines valuable neighbor information for inferring missing types; and (3) type co-occurrence regularization, which prevents the model from overfitting false-negative samples by using type co-occurrence information. we will detail these three modules as follows. ## conclusion in this paper, we propose a novel method called miner to mine treasured neighbors. first, miner aggregates one-hop neighbor and multi-hop neighbor information. then, miner predicts the entity types by type-specific local inference and typeagnostic global inference. finally, we use type co-occurrence regularization to prevent our model from overfitting the false-negative samples. experi-mental results on two widely used datasets indicate that our approach significantly outperforms previous state-of-the-art methods.