Multi-level Shared Knowledge Guided Learning for Knowledge Graph Completion (2024)

Abstract: In the task of Knowledge Graph Completion (KGC), the existing datasets and their inherent subtasks carry a wealth of shared knowledge that can be utilized to enhance the representation of knowledge triplets and overall performance. However, no current studies specifically address the shared knowledge within KGC. To bridge this gap, we introduce a multi-level Shared Knowledge Guided learning method (SKG) that operates at both the dataset and task levels. On the dataset level, SKG-KGC broadens the original dataset by identifying shared features within entity sets via text summarization. On the task level, for the three typical KGC subtasks—head entity prediction, relation prediction, and tail entity prediction—we present an innovative multi-task learning architecture with dynamically adjusted loss weights. This approach allows the model to focus on more challenging and underperforming tasks, effectively mitigating the imbalance of knowledge sharing among subtasks. Experimental results demonstrate that SKG-KGC outperforms existing text-based methods significantly on three well-known datasets, with the most notable improvement on WN18RR (MRR: 66.6%→ 72.2%, Hit@1: 58.7%→67.0%).

## introduction knowledge graphs (kgs) are directed multirelation graphs, with entities as nodes and relations as edges, denoted as a set of triples (h, r, t). their distinctive advantage lies in efficiently representing and managing extensive knowledge, offering high-quality structured information for diverse downstream tasks, including question answering @xcite , information re- * xiaodong wang is the corresponding author. existing kgc methods are mainly divided into structure-based methods and text-based methods. structure-based methods @xcite @xcite typically map entities and relations into low-dimensional vectors and calculate the probability of valid triples by various scoring functions. text-based methods @xcite @xcite adopt pre-trained language models to semantically encode textual descriptions of entities. they can encode unseen entities in training time, while making reasoning less efficient. recent advancements, such as the bi-encoder structure proposed in studies like @xcite , aim to reduce the training cost of language model encoders. this shift has led to text-based methods beginning to surpass structure-based methods in terms of performance. while these methods exhibit a strong capability to complete knowledge graphs, challenges persist in the effective sharing of knowledge among datasets and subtasks. specifically, we find that the same (h, r) or (r, t) often appear in different triples (h, r, t). according to our analysis in figure 1a , 42.1% of triples can find other triples sharing the same (h, r) with themselves, and 66.9% of triples can find those sharing the same (r, t) with themselves in the wn18rr training set. for instance, (kirsten dunst, film actor, @xcite the greater the proportion of triples sharing the same (r, t) or (h, r), the more accessible knowledge we acquire. (b) the average number of connected entities in many-to-one and one-to-many relations indicates the imbalanced distribution between head entities and tail entities, respectively. spider-man), (willem dafoe, film actor, spider-man), (james franco, film actor, spider-man) all have the same relations and tail entities. this suggests the potential existence of shared knowledge, such as ''american film actor,'' among various head entities. leveraging this dataset-level shared knowledge is essential to enhance the learning ability of triples and assist the model in correctly identifying answers from lexically similar candidates. notably, we observe considerable performance variations across various kgc subtasks, even when applied to the same dataset. for kg-bert @xcite on the wn18rr dataset, the hit@10 prediction results differ notably for head entities (54%) and tail entities (60.7%). this discrepancy arises from certain relations, such as gender and city, linking more head (tail) entities and fewer tail (head) entities. as shown in figure 1b , the issue of imbalanced distribution of head entities and tail entities is prevalent in knowledge graphs, yet it receives limited attention in research. existing multi-task learning methods treat head entity and tail entity prediction equally, ignoring the intricacies of more complex tasks. therefore, making the model focus on more challenging tasks while learning the shared knowledge across multiple subtasks becomes an urgent concern. this task-level shared knowledge can enhance the model's learning of entity and relation embeddings. in this paper, we introduce a multi-level shared knowledge guided learning method (skg) for knowledge graph completion. to capture datasetlevel shared knowledge within specific entity sets, we jointly train original triples, triples with identical head entities and relations, and triples with identical relations and tail entities. for task-level knowledge sharing, we incorporate relation prediction in multi-task learning to assist entity prediction task, enabling the model to acquire more relation-aware entity information. in each iteration, our loss weight allocation scheme assigns higher loss weights to tasks that are more challenging and underperforming, effectively addressing the imbalanced distribution of head and tail entities. in summary, our contributions include: ## method in this section, we introduce a multi-level shared knowledge guided learning method (skg) for knowledge graph completion. we elaborate the entire architecture of the proposed model in section 3.1. in sections 3.2 and 3.4, we illustrate how our method captures shared knowledge at both dataset and task levels for kgc. these insights are seamlessly integrated at the bi-encoder architecture, as explained in section 3.3. the following sections provide a detailed overview of the training and inference processes of our model. ## conclusion in this paper, we introduce a multi-level shared knowledge guided method for efficient knowledge graph completion. our approach effectively addresses the challenges of inadequate knowledge learning and imbalanced subtasks in multitask learning. through extensive experiments on benchmark datasets, we demonstrate that skg-kgc consistently outperforms competitive base-line models, particularly excelling on wn18rr with its extensive entity descriptions. these findings provide new insights for multi-task learning and other tasks related to knowledge graphs. in future research, we aim to explore the integration of text-based methods with graph embeddings to extract the semantic and structural information in knowledge graphs.