The Integration of Semantic and Structural Knowledge in Knowledge Graph Entity Typing (2024)

Abstract: The Knowledge Graph Entity Typing (KGET) task aims to predict missing type annotations for entities in knowledge graphs. Recent works only utilize the in the local neighborhood of entities, disregarding in the textual representations of entities, relations, and types that are also crucial for type inference. Additionally, we observe that the interaction between semantic and structural knowledge can be utilized to address the false-negative problem. In this paper, we propose a novel S emantic and S tructure-aware KG E ntity T yping (SSET) framework, which is composed of three modules. First, the Semantic Knowledge Encoding module encodes factual knowledge in the KG with a Masked Entity Typing task. Then, the Structural Knowledge Aggregation module aggregates knowledge from the multi-hop neighborhood of entities to infer missing types. Finally, the Unsupervised Type Re-ranking module utilizes the inference results from the two models above to generate type predictions that are robust to false-negative samples. Extensive experiments show that SSET significantly outperforms existing state-of-the-art methods.

## introduction a knowledge graph (kg) is a graph-structured knowledge base that consists of triples in the form of (head entity, relation, tail entity). each entity in the kg is annotated with one or multiple types. for example, entity "albert einstein" belongs to the "/scientist/physicist" type. types (or concepts) in kgs provide a high-level summary of their instance entities, which is crucial in many natural language processing (nlp) tasks such as entity linking @xcite , relation extraction @xcite , and question answering @xcite . currently, most kgs (e.g., freebase @xcite and yago @xcite ) are manually constructed by domain experts, which the type annotations are usually incomplete. the statistics on the fb15ket dataset show that 10% of the entities with the type "/music/artist" do not have type "/people/person" @xcite . in light of this, we focus on the knowledge graph entity typing (kget) task, which aims to infer missing type annotations for entities in a kg. recent studies in the kget task can be categorized into three groups: (1) embedding-based methods @xcite , (2) gnn-based methods @xcite @xcite , and (3) transformer-based methods @xcite . however, each group of solutions has its drawbacks. specifically, embedding-based methods ignore the rich neighbor information of entities. gnn-based methods fall short of capturing the interactions between non-directly connected neighbors of the target entity @xcite . the only transformerbased method tet suffers from its high computational complexity. more importantly, all existing methods ignore the textual information in the kg and suffer from serious false-negative problem @xcite , which ultimately limits the type inference performance. we argue that the textual representations of entities, relations, and types provide important semantic knowledge for type inference. for example, in figure 1 , it is hard to infer that "albert einstein" belongs to the type "/scientist/physicist" based solely upon the structural knowledge since triple (albert einstein, award, nobel prize in physics) is missing from the kg. fortunately, the description of the neighbor entity "theory of relativity" reveals that the theory is in the domain of "physics", which can help us to conclude that "albert einstein" has "/scientist/physicist" type. moreover, we observe that semantic and structural knowledge can complement each other to alleviate the false-negative problem. some valid type annotations are plausible but missing from the kg. simply considering these missing type annotations as negative samples will result in significant performance degradation. we find that the semantic knowledge obtained from the textual information provides an informative prior for the alleviation of the false-negative problem. if both semantic and structural knowledge support that an entity should belong to a particular type, we can assign a higher relevance score to that type. such an agreement allows us to mitigate the impact of false-negative samples with high confidence. in summary, the main objective of this paper is to capture and integrate semantic and structural knowledge from the local subgraph of each entity in the kg and apply both types of knowledge to the entity typing task. based on the aforementioned observations, we propose a novel semantic and structure-aware kg entity typing (sset) framework, which consists of three modules. the semantic knowledge encoding module encodes factual knowledge in the kg with a masked entity typing task. the structural knowledge aggregation module leverages knowledge from 1-hop neighbors, multi-hop neighbors, and known types of entities to predict missing types. the unsupervised type re-ranking module utilizes the preliminary inference results obtained from the two modules mentioned above to generate type predictions that are robust to false-negative samples. our contributions are summarized as follows: embedding-based methods. ete @xcite is the first model specialized for the kget task. it employs kg embedding methods @xcite to embed entities and relations, and learns type embeddings by minimizing the distance between embeddings of entities and corresponding types. connecte @xcite jointly embeds entities and types into different spaces and proposes two type inference mechanisms, namely "e2t" and "trt". despite the simplicity and intuitiveness, these methods ignore the rich neighbor information of entities, which seriously limits their performance @xcite . ## problem specification we consider a kg g to consist of triples in the form of (e s , r, e o ) and entity type assertions in the form of (e, t), where e s , e o , @xmath0, @xmath1, @xmath2. the notations e s and e o denote the subject and the object of a triple. e, r, t are the set of entities, relations, and types, respectively. in kgs, an entity can have multiple types, but some type annotations are missing from the kg. here, we use t (e) = {t|(e, t) ∈ g} to denote the set of known types of entity e. we denote the set of triples related to entity e by n (e) = {(e, r, ẽ) ∪ (ẽ, r, e) ∈ g}. in most kg datasets such as fb15k @xcite and yago43k @xcite , entities are provided with labels and descriptions, while relations and types are represented by their textual identifiers. we assume that these textual information are meaningful and contain valuable semantic knowledge for the kget task. in this paper, we aim to reveal missing type annotations based on the local graph structure of entities and corresponding textual information. ## method figure 2 provides an overview of the architecture of our proposed sset framework, which consists of three modules. first, the 1) semantic knowledge encoding module (sem) infuses the knowledge from triples and entity-type assertions into the pretrained language model (plm). we expect the finetuned language model can be utilized to encode textual representations of entities, relations, and types, and to conduct preliminary type inference. then the 2) structural knowledge aggregation module (ska) aggregates knowledge from the local subgraph of each target entity and utilizes 1-hop neighbors, multi-hop neighbors, and known types of the entity to infer its missings types. finally, the 3) unsupervised type re-ranking module (utr) exploits the agreement between semantic and structural knowledge to alleviate the false-negative problem and generate the final type rankings. ## conclusion in this paper, we propose a novel model sset for the kget task. sset focuses on modeling the integration and interaction between semantic and structural knowledge from various perspectives. the sem module utilizes structural triples and entity-type assertions as the corpus to fine-tune the plm. the ska module fuses pre-trained textual embeddings into learnable structural embeddings to address the "cold start" issue and leverages the preliminary inference results from the sem module to mitigate the false-negative problem. finally, the utr module enables us to combine inference scores from different modules with minimal computational overhead. the experiment results show that the proposed sset model significantly and consistently outperforms all baseline methods.