I nte G round: On the Evaluation of Verification and Retrieval Planning in Integrative Grounding (2025)

Abstract: Grounding large language models (LLMs) in external knowledge sources is a promising method for faithful prediction. While existing grounding approaches work well for simple queries, many real-world information needs require synthesizing multiple pieces of evidence. We introduce “integrative grounding” – the challenge of retrieving and verifying multiple inter-dependent pieces of evidence to support a hypothesis query. To systematically study this problem, we repurpose data from four domains for evaluating integrative grounding capabilities. Our investigation reveals two critical findings: First, in groundedness verification, while LLMs are robust to redundant evidence, they tend to rationalize using internal knowledge when information is incomplete. Second, in examining retrieval planning strategies, we find that undirected planning can degrade performance through noise introduction, while premise abduction emerges as a promising approach due to its logical constraints. Additionally, LLMs’ zero-shot self-reflection capabilities consistently improve grounding quality. These insights provide valuable direction for developing more effective integrative grounding systems.

## introduction large language models (llms) are notorious for their tendency to hallucinate -generating content that appears plausible but is factually incorrect or unsupported @xcite . to alleviate this issue, grounding llms to external knowledge sources has emerged as a promising approach. by anchoring model outputs to verifiable information @xcite @xcite , grounding has enabled llms with more faithful decision-making and responsible generation. in typical grounding setups, systems retrieve relevant documents in response to a query. while this approach has shown success for simple queries where a single piece of evidence suffices, many real-world information needs are inherently complex and require synthesizing multiple pieces of evidence to form a complete answer (figure 1 ). for instance, answering questions about comparative analysis, multi-step reasoning @xcite , or claims requiring evidence from different sources @xcite @xcite often necessitates the integration of multiple pieces of information. we term this problem "integrative grounding": given a hypothesis query, a grounding system needs to retrieve multiple interdependent pieces of evidence to support it. despite its significance, the integrative grounding problem lacks systematic evaluation in current research. existing work on retrieval-augmented generation (rag) primarily focuses on end-to-end reasoning performance @xcite , reasoning with pre-retrieved evidence @xcite , or post-generation evaluation in specific domains @xcite . while these approaches implicitly address aspects of integrative grounding, they offer no comprehensive analysis of the grounding problem itself. similarly, research in automated theorem proving @xcite explores complex inference chains but operates in restricted domains without robust evaluation of the broader integrative grounding challenges. while related fields like natural language inference (nli) and automated proof generation have their own evaluation paradigms, they typically operate under the assumption that sufficient evidence is provided. a critical gap remains in systematically evaluating grounding under the sub-optimal evidence conditions that are common in open-world settings. our work addresses this gap by introducing integround, a novel evaluation framework designed specifically for integrative grounding. unlike standard benchmarks, integround systematically tests a model's behavior across four distinct evidence scenarios: complete, redundant, incomplete, and uninformative. this allows us to rigorously analyze critical failure modes like rationalization-where models invent facts to fill evidence gaps-that are often missed by traditional evaluations. we argue that robust integrative grounding is a crucial prerequisite for building truly reliable and faithful generative systems. in this work, we focus on two critical aspects for evaluating integrative grounding. first, we investigate whether models can effectively verify groundedness by determining if multiple evidence pieces collectively support a query hypothesis (rq1). second, recognizing the challenges posed by reasoning dependencies among evidence, we examine effective planning strategies for llms to reformulate search queries and guide the retrieval process (rq2). to address these questions, we first construct an evaluation dataset from four domains to evaluate a system's ability to integrate multiple pieces of evidence. for rq1, we conduct experiments on groundedness verification. our experiments reveal that although llms are robust to redundant or distracting evidence, they exhibit a strong tendency to compensate for incomplete information by drawing on internal knowledge instead of strictly adhering to the retrieved content. for rq2, we present a systematic investigation of different planning strategies for retrieval, including decomposition-based, query expansion-based, and premise abduction-based approaches. our findings reveal that planning does not universally improve retrieval performance. in fact, undirected planning can degrade performance by introducing noise, while decomposition-based planning shows limited improvement due to its conservative nature. notably, we identify premise abduction as a particularly promising approach that shows consistent improvements and generalizes well to other datasets. this is likely due to its strong logical constraints, which encourage a directed expansion of the search space. furthermore, we demonstrate that leveraging zero-shot self-reflection consistently enhances grounding quality across all planning strategies, highlighting the value of iterative refinement. ## preliminaries proposition. a proposition is a statement that has a truth value. for instance "the sky is green." is a proposition since it can be verified as true or false, while "look up at the sky" is not a proposition. we use p to denote general propositions. for convenience, we use ϕ to denote a proposition that serves as hypothesis. in this paper, the queries are all propositions. ## constructing evaluation data our evaluation is based on data repurposed from two tasks where integrative grounding is required: multiple premise entailment @xcite @xcite and multi-hop question answering (qa) @xcite . ## groundedness verification in this section, we aim to investigate whether models are capable of doing groundedness verification (rq1). ## retrieval planning for integrative grounding in the sections above, we assume models are given a fixed set of queries (hypotheses) for retrieving evidence. recent advancements of proof systems @xcite @xcite and llm agents @xcite provide another perspective on this setting, where planning is integrated to proactively intervene retrieval processes. the objective of planning is to increase the success rate of grounding, i.e., biasing the search space so that it is more likely for an informative set σ to be found. the following part comprises experiments and discussions for three research questions: can planning and verification help retrieval and grounding? moreover, how to optimize these components to maximize grounding performance? ## related work natural language proof generation. prior work has developed proof writing algorithms that generate proof trees based on models' internal knowledge @xcite . while nellie @xcite incorporates retrieved facts for hypothesis decomposition, these approaches typically operate in restricted domains. more recent work has also focused on enhancing this process using principles from informal logic to improve decompositional inference @xcite . fact verification with llms. recent work on fact-checking llm-generated content @xcite @xcite primarily focuses on single-premise verification. other studies have investigated how well llms ground their outputs in provided sources, confirming that even state-of-the-art models struggle with faithfully adhering to evidence @xcite , especially when it contains conflicting information @xcite . while related to our verification component, our work focuses on a comprehensive evaluation of integrative retrieval and planning. rag and multi-hop qa agents. recent llm agents for multi-hop question answering @xcite employ iterative retrieval strategies but focus primarily on reasoning rather than addressing integrative grounding challenges. methods like trace @xcite construct reasoning chains from already-retrieved evidence, whereas our approach emphasizes dynamically planning what to retrieve next based on evidence interdependencies. similarly, while studies like @xcite focus on postgeneration evaluation and others @xcite enhance citation quality, we address verification during the retrieval process to evaluate whether multiple documents collectively support a hypothesis. ## conclusion in this work, we introduce "integrative grounding" as a critical challenge for llms and provide a systematic evaluation framework, integround, to assess it. our investigation yields several key insights with direct implications for building more reliable systems. we demonstrate that while llms are robust to redundant evidence, they exhibit a strong tendency to "rationalize" with internal knowledge when faced with incomplete information, posing a significant risk to faithfulness. in retrieval planning, we find that intuitive strategies like undirected query expansion can degrade performance by introducing noise, whereas logically constrained methods show significant promise. notably, premise abduction prove effective by expanding the search space in a directed manner, and zero-shot selfreflection consistently improve performance across all planning methods by enabling iterative refinement. our findings offer direct guidance for building more robust grounding systems, with a detailed discussion of practical applications for rag pipelines provided in appendix a.2.5.