Knowledge Graph-Enhanced Large Language Models via Path Selection (2024)

Abstract: Large Language Models (LLMs) have shown unprecedented performance in various real-world applications. However, they are known to generate factually inaccurate outputs, a.k.a. the hallucination problem. In recent years, incorporating external knowledge extracted from Knowledge Graphs (KGs) has become a promising strategy to improve the factual accuracy of LLM-generated outputs. Nevertheless, most existing explorations rely on LLMs themselves to perform KG knowledge extraction, which is highly inflexible as LLMs can only provide binary judgment on whether a certain knowledge (e.g., a knowledge path in KG) should be used. In addition, LLMs tend to pick only knowledge with direct semantic relationship with the input text, while potentially useful knowledge with indirect semantics can be ignored. In this work, we propose a principled framework KELP with three stages to handle the above problems. Specifically, KELP is able to achieve finer granularity of flexible knowledge extraction by generating scores for knowledge paths with input texts via latent semantic matching. Meanwhile, knowledge paths with indirect semantic relationships with the input text can also be considered via trained encoding between the selected paths in KG and the input text. Experiments on real-world datasets validate the effectiveness of KELP.

## introduction recently, large language models (llms) such as @xcite and llama @xcite have shown exceptional performance, such as unprecedented reasoning capabilities @xcite , across various nlp tasks @xcite @xcite . however, in scenarios where certain new knowledge beyond the scope of training corpus is required, current llms are usually criticized for generating factually inaccurate outputs @xcite @xcite . as a consequence, it becomes imperative to develop effective and efficient techniques for incorporating new knowledge into pretrained llms. to facilitate the incorporation of new knowledge in llms, extracting external knowledge from knowledge graphs (kgs) (known as @xcite ) has become a promising way to improve the factual accuracy of llm outputs @xcite @xcite . here, the structure of kgs plays a critical role, since the relationship between entities can effectively contribute to novel knowledge required by various tasks, such as multi-hop reasoning @xcite . in general, there are two mainstreams to achieve kg-enhanced llms. the first mainstream incorporates new knowledge for llms during their training phase, which is usually achieved by designing new training objectives or tasks @xcite . nevertheless, these techniques typically require significant computational resources. on the contrary, the second mainstream incorporates new knowledge for llm during the inference phase, where this new knowledge is incorporated by prompt engineering, that is, designing new prompts to include the triplets (head, relation, tail) derived from kgs @xcite . usually, prompt engineering stands out as the most computationally efficient approach to incorporate new knowledge for llms, as new information can be directly introduced together with the text input without an additional training process. nevertheless, integrating new knowledge for llms with prompt engineering bears two significant disadvantages despite its effectiveness and efficiency. first, most existing methods solely rely on llms to identify relevant triplets @xcite . however, llms can only provide binary figure 1 : an example of the phenomenon that semantically unrelated contexts in the input prompts can possibly contain important knowledge to correct/improve the generation of large language models. in this example, there exist potential relationships between "japan" and "akita inu" that are challenging to directly identify and capture. to properly handle the problems discussed above, we introduce a novel approach for kg-enhanced llms, i.e., kelp (knowledge graph-enhanced large language models via path selection), aiming to flexibly capture potentially impactful knowledge as in-context facts to improve the factual accuracy of the llm outputs given input texts. in particular, kelp consists of three key components: (i) knowledge path extraction, (ii) sample encoding, and (iii) fine-grained path selection. specifically, we first extract knowledge paths from kg based on the entities identified in the input texts as the candidate knowledge. subse-quently, we train a path-text encoder to encode the indirect connections between input texts and the knowledge paths extracted from kg, with similarity defined on the latent semantic space, such that whether an instance of knowledge (represented by a path in the kg) is potentially useful for a certain input text (i.e., is the potentially impactful knowledge) can be quantitatively measured. based on the latent similarity score, two coverage rules are introduced to further refine the selected paths with high flexibility. through these meticulously designed steps, kelp strives to flexibly capture potentially impactful knowledge with fine granularity (based on quantitative scores) to refine llm outputs. the contribution of this paper can be concretely summarized into three folds as follows: ## problem formulation in this section, we introduce the task of enhancing llm performance with kgs. the kg is defined as @xmath0, r, t ), where e and r are sets of entities and relations, respectively, and @xmath1, r, t)|h, @xmath2, @xmath3, each contains a head entity h, a tail entity t, and a relation r. in addition, we have a pretrained large language model (llm) denoted as lm . given a question q as the given task, if we denote all the entities contained in q as a set e q , the goal is to utilize the background kg g as input prompts to support the generation of lm based on question q. ## methodology in this section, we introduce the details of our proposed framework kelp, which is presented in figure 2 . kelp is structured into three phases: (i) knowledge path extraction, (ii) sample encoding, and (iii) fine-grained path selection. during knowledge path extraction, we extract a set of knowledge paths for each entity in the entity set e q of q from the background kg g. for sample encoding, we employ a sentence encoder m trained on the latent semantic space that can encode the input question q and paths in the extracted knowledge path sets to obtain their distance (i.e., the possibility that the paths can influence the output of llms), thus ensuring capturing the potentially impactful knowledge in the paths. in the final fine-grained path selection phase, we propose two coverage rules to guarantee that the selection of knowledge paths is sufficiently flexible, thereby ensuring the acquisition of the most diverse and representative paths for the inference of llms regarding the input question q. additionally, we also design an alternative strategy called relation-only ranking to generalize kelp to cases where the sizes of the knowledge path sets become substantially massive. ## experiments in this section, we introduce the extensive experiments conducted on two different tasks to demonstrate the effectiveness of the proposed method kelp. our experimental setup closely follows the experimental settings outlined in kg-gpt @xcite , ensuring consistency and comparability with existing research in the field. ## conclusion in this paper, we present kelp, an innovative method to enhance large language models (llms) with knowledge graphs (kg), aiming to flexibly capture potentially impactful knowledge that may lack direct semantic relevancy to the input texts. specifically, we establish a training dataset with real examples of path-text pairs that demonstrate the correction of llm outputs by including external knowledge as contexts. subsequently, we train a path-text encoder to measure whether an instance of knowledge (represented as a given path in kg) contains potentially impactful knowledge for a specified input text. based on the similarity score, two coverage rules are introduced to further refine the selected knowledge paths with high flexibility. through experimental validation on two datasets, kelp has proven its superiority over other stateof-the-art baselines on kg-enhanced llms.