Can Knowledge Graphs Reduce Hallucinations in LLM s? : A Survey (2024)

Abstract: The contemporary LLMs are prone to producing hallucinations, stemming mainly from the knowledge gaps within the models. To address this critical limitation, researchers employ diverse strategies to augment the LLMs by incorporating external knowledge, aiming to reduce hallucinations and enhance reasoning accuracy. Among these strategies, leveraging knowledge graphs as a source of external information has demonstrated promising results. In this survey, we comprehensively review these knowledge-graph-based augmentation techniques in LLMs, focusing on their efficacy in mitigating hallucinations. We systematically categorize these methods into three overarching groups, offering methodological comparisons and performance evaluations. Lastly, this survey explores the current trends and challenges associated with these techniques and outlines potential avenues for future research in this emerging field.

## introduction large language models (llms) seek to emulate human intelligence through statistical training on extensive datasets @xcite . llms operate on input text to predict the subsequent token or word in the sequence while identifying patterns and connections between words and phrases, aiming to comprehend and generate human-like text. due to their stochastic decoding processes, i.e., sampling the next token in the sequence, these models exhibit probabilistic behavior, potentially yielding varied outputs or predictions for the same input across different instances. additionally, if the training data includes misinformation, biases, or inaccuracies, these flaws may be mirrored or amplified in the content produced by these models. llms also face challenges in accurately interpreting phrases or terms when the context is vague and resides in a knowledge gap region of the model, leading to outputs that may sound plausible but are often irrelevant or incorrect @xcite . this phenomenon, often termed "hallucinations," undermines the reliability of these models @xcite . addressing the issue of hallucinations in these models is challenging due to their inherent probabilistic nature. to effectively tackle this issue, there have been continuous research efforts in making knowledge updates and model tuning @xcite @xcite . however, adding random information does not improve the model's interpretation and reasoning capabilities. instead, providing more granular and contextually relevant, precise external knowledge can significantly aid the model in recalling essential information @xcite . one emerging research trend is enhancing llms through integrating knowledge representation tools such as knowledge graphs (kgs) @xcite . zheng et al. @xcite demonstrate that augmenting these models with comprehensive external knowledge from kgs can boost their performance and facilitate a more robust reasoning process. the strategies for enhancing llms with kgs can be grouped into three main categories, each uniquely contributing to the refinement of the model as shown in figure 1 : enhancing the inference process, improving the learning mechanism, and establishing robust methods for validating the model's decisions. in this survey, we critically review kg augmentation methods used in specific stages to reduce hallucinations in llms and improve their performance and reliability. in section 3, we classify these methods into three overarching categories: related works: there are several related surveys which discuss llm augmentation using external knowledge @xcite @xcite @xcite @xcite . however, to our knowledge, this is the first survey to exclusively focus on critically reviewing llm augmentation methods utilizing structured knowledge from knowledge graphs. specifically, our emphasis is on addressing hallucinations in llms through kg integration. ## preliminaries we now introduce the preliminaries and definitions that will be used throughout the survey. ## knowledge graph-enhanced llms the llms primarily have three points of failure: a failure to comprehend the question due to lack of context, insufficient knowledge to respond accurately, or an inability to recall specific facts. improving the cognitive capabilities of these models involves refining their inference-making process, optimizing learning mechanisms, and establishing a mechanism to validate results. this survey comprehensively reviews existing methodologies aimed at mitigating hallucinations and enhancing the reasoning capabilities of llms through the augmentation of kgs using these three techniques. we classify them as knowledge-aware inference, knowledge-aware learning, and knowledge-aware validation. figure 2 details key works from each of these categories. ## discussion, challenges and future in this section, we examine the effectiveness of kgenhanced llm techniques in reducing hallucinations and enhancing performance and reliability in llms. we also identify key challenges associated with each method and propose potential research avenues in this evolving field. ## conclusion in this survey, we systematically investigate the integration of kgs into llms to mitigate hallucinations and improve reasoning accuracy. we emphasize the benefits of using kgs to enhance llm performance across various phases at inference, model training, and output verification stages. while substantial progress has been made, we emphasize the need for continuous innovation and propose future directions to facilitate the development of more advanced kg-augmented llms.