Inter-Passage Verification for Multi-evidence Multi-answer QA (2025)

Abstract: Multi-answer question answering (QA), where questions can have many valid answers, presents a significant challenge for existing retrieval-augmented generation-based QA systems, as these systems struggle to retrieve and then synthesize a large number of evidence passages. To tackle these challenges, we propose a new multi-answer QA framework – Retrieval-augmented Independent Reading with Inter-passage Verification (RI²VER). Our framework retrieves a large set of passages and processes each passage individually to generate an initial high-recall but noisy answer set. Then we propose a new inter-passage verification pipeline that validates every candidate answer through (1) Verification Question Generation, (2) Gathering Additional Evidence, and (3) Verification with inter-passage synthesis. Evaluations on the QAMPARI and RoMQA datasets demonstrate that our framework significantly outperforms existing baselines across various model sizes, achieving an average F1 score improvement of 11.17%. Further analysis validates that our inter-passage verification pipeline enables our framework to be particularly beneficial for questions requiring multi-evidence synthesis.

## introduction question answering (qa) represents a longstanding challenge in natural language processing @xcite @xcite , which in recent years has become an important knowledge and factuality benchmark task of large language models (llms) @xcite . while most such qa benchmark datasets operate under the assumption that each question has a single answer, this assumption often fails to reflect real-world scenarios where multiple valid answers exist (min et al., 2020; amouyal 1 our code is available at https://github.com/baleche n/river : ... gdpr, is a european union regulation on information privacy ... : ... gdpr requires that ai developers obtain consent from individuals before using their personal data ... ## background: multi-answer qa we first formally define the multi-answer qa task. given a question q and a text corpus of numerous passages @xmath0, a qa system predicts a set of answers @xmath1, a 2 , . . . ) as close as the ground truth answer set a * = (a * 1 , a * 2 , . . . ), where each answer a i is a short-form answer (e.g., an entity) to q. each ground truth answer can have one or more supporting passages in c, with the latter case being multi-evidence multi-answer qa. the predicted answer set a is evaluated by set precision, recall, and f1 with respect to a * . specifically, for each question, if a predicted answer a i exactly matches one of the ground truth answers a * j or one of a * j 's aliases, we count it as a true positive answer, or otherwise a false positive answer. if there exists an a * i such that neither itself nor its aliases are present in a, we count it as a false negative error. thus, a qa system should balance between finding all correct answers (high recall) and ensuring the predicted answers are accurate (high precision) to achieve the optimal f1 score. ## method in this section, we introduce our multi-answer qa framework, ri 2 ver, that introduces the interpassage verification (ipv) stage to the vanilla rag paradigm for multi-answer qa. at a high level, ri 2 ver first finds a large pool of passages and predicts a set of answers grounded on each passage. this retrieve-then-independently-read process prioritizes the recall of correct answers at a cost of precision, leading to many false answers in the predicted answer set. then we filter each answer for each predicted answer , stage 1: retrieval retriever retrieved document pool stage 2: reading who were employed by the curtis institute of music and the juilliard school? reader prompt ... (michael) tree served on the faculty of curtis institute of music, the julliard school, ... ... ivan galamian and renée longy were colleagues at juilliard, before galamian joined curtis in ... curtis high school, operated by the new york city department of education, is ... stage 3: inter-passage verification (ipv) few-shot examples is [answer] a person's name? was [answer] employed at curtis institute of music? was [answer] employed at the juilliard school? categorical verification factual verification llm 1. verification question generation 2. gathering additional evidence retrieved document pool 3. inter-document verification for each predicted answer , david effron llm ✅ ✅ ❌ filtered prompt 2 prompt 1 michael tree ivan galamian renée longy independent reading figure 2: illustration of ri 2 ver. in stage 1, we search broadly for a large pool of passages to cover sufficient evidence. in stage 2, a reader model processes each top-k retrieved passage independently to generate a set of answer candidates. in stage 3, we employ the ipv pipeline to verify each answer against a series of verification questions generated based on the original question, along with an additional retrieval step to gather evidence. as illustrated in figure 2 , ri 2 ver is composed of three stages. first, in the retrieval stage, we broadly search for a large pool of passages. second, in the reading stage, we employ an llm reader with the independent reading paradigm to elicit all potential correct answers ( §3.1). last, in the third stage, we design the ipv pipeline to effectively verify each answer candidate through three steps: ## error analysis to better understand the performance of our framework ri 2 ver, we conducted an error analysis to answer the following two questions: what causes the low precision on both datasets? we randomly sample 100 false positive answers from the final predictions of 8b with ipv and manually label them into five categories shown in in ## conclusion in this paper, we propose a new multi-answer qa framework, ri 2 ver, in which we retrieve broadly for relevant passages, extract all potential answers from each passage independently, and then refine the prediction set with the ipv pipeline that gathers inter-passage evidence to verify each answer candidate with an llm. our framework consistently outperforms various baselines across different model sizes and benchmarks. through further analysis, we demonstrate that the ipv pipeline is beneficial in answering questions that require synthesizing multiple evidence passages. it also enables the reader model to benefit from the high recall from independently reading more passages without suffering from the increase in irrelevant information. we believe this work offers valuable insights and paves the way for developing more robust and reliable multi-answer qa systems.