Refining Noisy Knowledge Graph with Large Language Models (2025)

Abstract: Knowledge graphs (KGs) represent structured real-world information composed by triplets of head entity, relation, and tail entity. These graphs can be constructed automatically from text or manually curated. However, regardless of the construction method, KGs often suffer from misinformation, incompleteness, and noise, which hinder their reliability and utility. This study addresses the challenge of noisy KGs, where incorrect or misaligned entities and relations degrade graph quality. Leveraging recent advancements in large language models (LLMs) with strong capabilities across diverse tasks, we explore their potential to detect and refine noise in KGs. Specifically, we propose a novel method, LLM_sim, to enhance the detection and refinement of noisy triples. Our results confirm the effectiveness of this approach in elevating KG quality in noisy environments. Additionally, we apply our proposed method to Knowledge Graph Completion (KGC), a downstream KG task that aims to predict missing links and improve graph completeness. Traditional KGC methods assume that KGs are noise-free, which is unrealistic in practical scenarios. Our experiments analyze the impact of varying noise levels on KGC performance, revealing that LLMs can mitigate noise by identifying and refining incorrect entries, thus enhancing KG quality.

## introduction knowledge graphs (kgs) provide a structured framework for representing interconnected data, widely used in research fields such as natural language processing and recommendation systems. however, automated kg construction often introduces noise, leading to inaccurate or misaligned triples that degrade the quality and reliability of downstream tasks like knowledge graph completion (kgc) @xcite . addressing noise in kgs is crucial for maintaining kgc performance, as this task relies on accurate triples to infer missing links and enhance kg completeness. large language models (llms), which demonstrate impressive capabilities across a variety of tasks like question answering @xcite , summarization @xcite , and translation @xcite , offer a promising solution for kg noise detection. by encoding extensive factual and contextual knowledge, llms can evaluate the coherence of entity-relationship pairs based on learned semantic patterns @xcite . leveraging lms for noise detection presents a potential advancement over traditional noise detection methods, which typically depend on kg embedding models or rule-based techniques that may not effectively handle nuanced or context-specific noise. in this study, we propose a novel approach, llm_sim, which uses a llm, llama3 foot_0 , to detect and refine erroneous triples in noisy kgs. our llm_sim, generates candidate triples for detected noise and refines them using contextual similarity, matching them to existing kg triples. our experiments show that llm_sim is particularly effective under high noise conditions, underscoring the value of llms for kg refinement. the contributions of this paper are as follows: the remainder of this paper is organized as follows. section 2 reviews related work on noise detection and kgc methods. section 3 explains our proposed method llm_sim for detecting and refining noisy kgs. section 4 describes the experimental setup, including datasets and model configurations. section 5 presents experimental results, highlighting the effectiveness of our approach. finally, section 6 concludes the paper and outlines future research directions. ## related work in noise detection research, various approaches have been proposed to demonstrate the effectiveness of their models in detecting noise. these methods can be broadly categorized into three types: traditional kg embedding models, noise detection based on pre-trained language models(plms), and unsupervised rule-based noise detection models. knowledge graph embedding (kge) models assess the validity of triples by estimating confidence scores for embedded representations, based on the principle that correct triples approximate the vector equation h + r ≈ t, where h and t represent the head and tail entities, and r represents the relationship. notable models in this category include transe @xcite , rotate @xcite , distmult @xcite , and complex @xcite . however, noise in training data can degrade embedding quality, as kg embeddings rely on clean data for optimal performance. while kge models can determine a triple's validity, their performance remains limited due to their sensitivity to noisy data. plms, such as gpt-2 xl 4 , approach noise detection by assessing the semantic relationship between text and entities within a triple. they evaluate correctness by measuring the model's confidence or probability score for a given triple. however, these models often struggle with domain-specific or temporal knowledge, as they depend on the contexts present in their training data. additionally, these models primarily rely on associative reasoning rather than causal inference, making it challenging to detect implicit noise in complex knowledge reasoning scenarios. consequently, such models often have limited capacity for inferring non-explicit 4 https://huggingface.co/openai-community/ gpt2-xl relationships and may not effectively detect noise in superficially similar triples. unsupervised rule-based noise detection models use predefined rules or constraints to detect anomalies or noise in triples. for example, @xcite introduced a rule-based triple confidence framework for noise detection in kge, assigning confidence scores to improve noise filtering and enhance the robustness and accuracy of embedding models. probabilistic models have also been applied to noisy data @xcite , using statistical methods to quantify uncertainty and model noise, facilitating robust error correction and data refinement, thereby enhancing the quality, usability, and reliability of kgs. however, these models often struggle to intuitively grasp the semantic information of triples and lack a solid foundation of real facts and logical coherence. although they are not dependent on labeled data, rule-based approaches lose efficacy in dynamic kgs or frequently updated datasets, as fixed rules may become outdated, leading to inefficiency and reduced scalability. ## methodology to effectively detect noise in kgs, which refers to erroneous triples, we propose a novel framework to detect and refine noisy triples in large-scale kgs using llms. our approach consists of two key components: noise detection and noise refinement. figure 1 illustrates the overall process of noise detection and refinement in kgs. the llm detection model first identifies noise within the kg, which is detected as noise data, filtered kg in the figure, which is then passed to the llm refinement for correction. the refined kg along with detected correct triplets, filtered kg, forms a new dataset, renewed kg, which is subsequently utilized for kgc tasks. the refinement process begins with the llm generating five candidate triples for a given noisy triple. the candidates are divided into head-relation and relation-tail pairs, which are then matched against the noisy kg. the most suitable candidate is selected based on similarity calculations and used as the final refinement triple. ## experiment this section describes our experimental setup, including the dataset description, noise construction methods, and evaluation metrics. ## results and analysis in this section, we present the results of our experiments and conduct a thorough analysis to gain insight into the outcomes. ## conclusion the results confirmed that our proposed llm_sim method significantly enhanced kg reliability, benefiting downstream tasks such as kgc. these findings underscored the broader potential of llms for kg-specific tasks by detecting and refining noise in dynamic, evolving kgs. the demonstrated robustness of llms in high-noise settings highlighted their applicability to real-world scenarios where kgs are frequently updated. moving forward, we plan to refine our methods to enhance noise detection and refinement capabilities, aiming for improved robustness and adaptability across diverse datasets.