COSIGN : Contextual Facts Guided Generation for Knowledge Graph Completion (2024)

Abstract: Knowledge graph completion (KGC) aims to infer missing facts based on existing facts within a KG. Recently, research on generative models (GMs) has addressed the limitations of embedding methods in terms of generality and scalability. However, GM-based methods are sensitive to contextual facts on KG, so the contextual facts of poor quality can cause GMs to generate erroneous results. To improve the performance of GM-based methods for various KGC tasks, we propose a COntextual FactS GuIded GeneratioN (COSIGN) model. First, to enhance the inference ability of the generative model, we designed a contextual facts collector to achieve human-like retrieval behavior. Second, a contextual facts organizer is proposed to learn the organized capabilities of LLMs through knowledge distillation. Finally, the organized contextual facts as the input of the inference generator to generate missing facts. Experimental results demonstrate that COSIGN outperforms state-of-the-art baseline techniques in terms of performance.

## introduction a knowledge graph (kg) represents a network of real-world entities and illustrates the relationship between them @xcite . knowledge graph completion (kgc) is a task aimed at inferring missing facts based on existing facts within kgs, including static kgc (skgc) @xcite , temporal kgc (tkgc) @xcite and few-shot kgc (fkgc) @xcite . tkgc involves time facts with timestamps, while fkgc predicts facts with limited or zero trained samples for relationships. previous approaches primarily relied on graph embedding, which involves embedding entities and relationships into high-dimensional vectors to represent their associations @xcite . the training and inference models of skgc @xcite @xcite depend on various transitional relationships on graph paths, while tkgc @xcite @xcite and @xcite @xcite methods further integrate with special components or learning paradigms to handle additional time information or training requirements. however, these differences in methods result in significant maintenance costs and an inability to adapt to emerging knowledge queries, ingestion, and presentation. recently, generative models (gms) including large language models (llms) have demonstrated advanced performance in handling various natural language processing tasks. despite having heterogeneous inputs and outputs, generative models transform these tasks into a "text-to-text" format, taking the text as input and producing another text as output @xcite @xcite . moreover, gms embed a significant amount of real-world knowledge from pre-training @xcite @xcite , which holds potential benefits for kgc tasks. however, gm-based methods are sensitive to contextual facts on kg @xcite . as illustrated in figure 1 , when the reasoning process lacks contextual facts, the flexibility of the model will be invoked, resulting in significant differences in the generated results. therefore, it is necessary to gather effective contextual facts to guide the model's generation. however, scattered contextual facts can make it difficult for the generative model to understand thereby generating erroneous results. therefore, organizing scattered contextual facts into structured and logical contextual facts is an important challenge of gm-based methods for various kgc tasks. to address the important challenge, we propose a contextual facts guided generation (cosign) model to infer missing facts on kgs. first, to enhance the inference ability of the generative model, we designed a contextual facts collector. given a triple (s, r, o, m), the collector collects all the paths that the head entity s can infer to the tail entity o in the subgraph as the contextual facts. second, a contextual facts organizer is proposed, which replaces inefficient manual organization works with automated information organization, to learn the organized capabilities of llms through knowledge distillation @xcite . finally, the organized contextual facts as the input of the inference generator to generate missing facts. additionally, we add prefix constraints to our model's inference process to ensure the rationality of the generated results. the contributions of our work can be summarized as follows: to the best of our knowledge, our work is the first to explore how to distillate the llm to achieve the kgc. ## preliminaries knowledge graph completion. let e, r, t , and f denote a finite set of entities, relations, timestamps, and facts, respectively. a fact list @xmath0, r, o, m) 1 , ..., (s, r, o, m) n } where s, @xmath1, @xmath2. for instance, in the context of skgc, m denotes null because it does not consider the time. however, in tkgc, it signifies a timestamp and @xmath3. knowledge graph completion (kgc) predicts the missing entities for the queries (s, r, ?, m) or ?, r -1 , o, m , where r -1 ∈ r -1 is the reverse edge of r. generative knowledge graph completion. the generative models have transformed the representation of kgc @xcite . the typical generative models of an encoder and a decoder, which can be viewed as p (y|x) = |y| @xmath4, x), where x is the input sequence and y is the generated output sequence. in generative kgc, x represents the query @xmath5, r, ?, m) (or ?, r -1 , o, m ), and the predicted ground-truth object (or subject) as y. it is noteworthy that both x and y are flattened textual representations of knowledge @xcite . ## proposed method figure 2 shows there are three key technical components in cosign: the contextual facts collector, the contextual facts organizer, and the inference generator. please note that we take the temporal kgc task as an example when introducing the method, but our method also applies to the static kgc task and few-shot kgc. ## experiment we combined pytorch @xcite with the huggingface @xcite transformers library to implement cosign, and evaluated its performance on an amd epyc 7t83 cpu (64 cores) and two rtx-4090 gpus (24gb each). ## conclusion in this paper, we propose a generative model, called cosign, for various kgc tasks. different from previous methods, we do not directly use contextual facts to guide the model to infer missing facts. in contrast, we use the organizational ability of the llm to transform the collected scattered contextual facts into coherent contextual facts, and use coherent contextual facts to guide the model to infer missing facts. specifically, we designed a contextual facts collector to achieve human-like retrieval behavior. then, a contextual facts organizer is proposed to learn the organized capabilities of llms through knowledge distillation. finally, the organized contextual facts as the input of the inference generator to generate missing facts. experimental results on five datasets for three types of kgc tasks highlight its significant performance.