Verify-in-the-Graph: Entity Disambiguation Enhancement for Complex Claim Verification with Interactive Graph Representation (2025)

Abstract: Claim verification is a long-standing and challenging task that demands not only high accuracy but also explainability and thoroughness of the verification process. This task becomes an emerging research issue in the era of large language models (LLMs) since real-world claims are often complex, featuring intricate semantic structures or obfuscated entities. Traditional approaches typically address this by decomposing claims into sub-claims and querying a knowledge base to resolve hidden or ambiguous entities. However, the absence of effective disambiguation strategies for these entities can compromise the entire verification process. To address these challenges, we propose Verify-in-the-Graph (VeGraph), a novel framework leveraging the reasoning and comprehension abilities of LLM agents. VeGraph operates in three phases: (1) Graph Representation - an input claim is decomposed into structured triplets, forming a graph-based representation that integrates both structured and unstructured information; (2) Entity Disambiguation -VeGraph iteratively interacts with the knowledge base to resolve ambiguous entities within the graph for deeper sub-claim verification; and (3) Verification - remaining triplets are verified to complete the fact-checking process. Experiments using Meta-Llama-3-70B (instruct version) show that VeGraph achieves competitive performance compared to baselines across benchmarks (HoVer and FEVEROUS), effectively addressing claim verification challenges. Our source code and data are available for further exploitation.

## introduction in the era of rapidly advancing large language models (llms), the widespread dissemination of misinformation, combined with the increasing presence of ai-generated content, has made it significantly harder for individuals to assess the reliability of information. consequently, claim verification, leveraging advanced natural language processing (nlp) techniques to automatically determine the veracity of claims, has emerged as a critical research topic @xcite . traditional approaches typically begin by decomposing a given claim (e.g., at the sentence or passage level) into sub-claims, often using methods such as chain-of-thought (cot) prompting @xcite . subsequently, each sub-claim is evaluated by prompting an llm, incorporating knowledge sources (e.g., information retrieval systems) to determine the truthfulness of the overall claim @xcite , as shown in figure 1 @xcite . multi-step reasoning in llms is the process of addressing complex tasks by breaking them into sequential inference steps, where each step builds on the previous one, enabling the model to integrate intermediate results and draw conclusions. recently, more advanced methods have enhanced claim verification task by incorporating multi-step reasoning to resolve ambiguous entities before verifying sub-claims @xcite @xcite , as illustrated in figure 1(b ). these improvements have made such methods more promising for explainable and interpretable claim verification systems. however, despite the advancements achieved by multi-step reasoning mechanisms, several critical challenges persist: i) ambiguous entity interactions: ambiguities in entity relationships remain a significant hurdle for fact verification systems @xcite . this challenge is amplified in multi-step reasoning, where entity disambiguation must span the entire verification process. unlike previous approaches that employ external tools for resolving ambiguities in individual subclaims, effective resolution here requires seamless integration throughout the reasoning pipeline; ii) limitations of llm-based multi-step reasoning agents: many existing approaches rely on static, single-plan veracity prediction @xcite . if a failure occurs at any intermediate step, the entire reasoning process may collapse, thereby underutilizing the adaptive potential of llm-based agents to recover and refine reasoning paths dynamically. in response to these challenges, this study introduces an agent-based framework, named verify-inthe-graph (vegraph), for automatic fact verification. our approach, illustrated in figure 1 (c), consists of three interconnected stages: an llm agent first constructs a graph-based representation by decomposing the input claim into sub-claim triplets. the agent then interacts with a knowledge base to resolve ambiguous entities in triplets, iteratively updating the graph state. finally, the agent verifies triplets, completing the process. overall, the primary contributions of this work are as follows: ## related work claim verification is a long-standing and challenging task that seeks to determine the veracity of a claim by retrieving relevant documents, selecting the most salient evidence, and making a veracity prediction. in the era of large language models (llms), llm-based claim verification has evolved to generate subclaims from input claims using the chain-of-thought (cot) approach, and to retrieve evidence by augmenting the llm with external knowledge sources for verification @xcite . @xcite improves this process by leveraging in-context learning along with the cot method, decomposing the original claim into program-like functions to guide the verification steps. similarly, folk (wang and shu, 2023) translates the claim into first-order-logic (fol) clauses, where each predicate corresponds to a subclaim that requires verification. folk then performs fol-guided reasoning over a set of knowledge-grounded question-answer pairs to predict veracity and generate explanations, justifying its decision-making process. furthermore, pacar @xcite leverages llm agent concept, which incorporates self-reflection technique and global planning to enhance performance. despite the advancement of these methods, which exploit llm reasoning capabilities to interact with external knowledge bases, they are limited to a single interaction with the knowledge base for an ambiguous entity. if the knowledge base fails to identify the requested entity in the query, the entire verification process may collapse. in light of these limitations, our proposed method similarly leverages llm reasoning in conjunction with external knowledge retrieval systems. however, we extend this by incorporating agent-based llm, enabling iterative interactions with the knowledge base to resolve ambiguous entities and execute multi-step reasoning for more robust and in-depth claim verification. ## methodology the main objective of this study is to predict the veracity of a complex input claim c through automated reasoning using an interpretable llm agent, incorporating both structured and unstructured information through graph representation. figure 2 shows the architecture of our proposed framework. specifically, vegraph consists of three stages: (i) the agent represents the claim c with graph triplets, each corresponding to a sub-claim; (ii) the agent interacts with an external knowledge base to resolve ambiguous entities; and (iii) once all ambiguities are addressed, the agent verifies sub-claims corresponding to the remaining triplets. the veracity of the input claim is determined by the veracity of all graph triplets, if all the graph triplets are verified with the information in the knowledge base then the claim c is supported, if one of the triplets cannot be verified then the claim c is ref uted. during processing through stages, the logging module records the activities of the agent for explainability. ## conclusion this study presents vegraph, a novel claim verification framework using the concept of interactive graph representation incorporating llm agent technology to identify ambiguous entities in terms of multi-step reasoning of veracity predictions. specifically, the input claim first is decomposed into a set of triplets. these triplets are then identified with ambiguous entities and verified of fact interactively using the proposed agent llm pipeline. the experiment on two well-known benchmark claim verification datasets indicates promising results of vegraph for claim verification tasks, especially in the case of complex claims.