Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answering (2025)

Abstract: The retrieval augmented generation (RAG) framework addresses an ambiguity in user queries in QA systems by retrieving passages that cover all plausible interpretations and generating comprehensive responses based on the passages. However, our preliminary studies reveal that a single retrieval process often suffers from low-quality results, as the retrieved passages frequently fail to capture all plausible interpretations. Although the iterative RAG approach has been proposed to address this problem, it comes at the cost of significantly reduced efficiency. To address these issues, we propose the diversify-verify-adapt (DIVA) framework. DIVA first diversifies the retrieved passages to encompass diverse interpretations. Subsequently, DIVA verifies the quality of the passages and adapts the most suitable approach tailored to their quality. This approach improves the QA systems’ accuracy and robustness by handling low quality retrieval issue in ambiguous questions, while enhancing efficiency.

## introduction open-domain question answering (qa) systems aim to provide factual responses across diverse topics. however, ambiguity in user queries is common, with over 50% of google search queries falling into this category @xcite . ambiguous questions challenge qa systems to determine user intent, making it essential for them to deliver answers covering all possible interpretations. addressing ambiguous questions is crucial in real-world applications, yet remains underexplored compared to unambiguous questions @xcite . this work aims to fill this gap by tackling the complexities of ambiguous qa. retrieval-augmented generation (rag) framework has made significant progress in open-domain notably, diva achieves better performance to the iterative rag @xcite , while significantly more efficient (that is, 2x faster and 1.8x cheaper). the size of the circle indicates the cost per query ($). closed-book llm indicates the traditional few-shot prompting method used in brown (2020) qa tasks @xcite @xcite and also proven to be an effective solution for addressing ambiguous questions @xcite @xcite . specifically, these approaches first retrieve passages on the given question and prompt the llm to extract plausible interpretations and answers relying on the passages (c.f. fig 2(a) ). despite the success of the rag framework on the ambiguous qa task, we should rethink: is a single retrieval process sufficient to retrieve passages encompassing all plausible interpretations? to answer this question, we conduct preliminary experiments (c.f. sec 2) about the quality of the retrieved passages used in the rag framework. we observe that the passages obtained from the single retrieval process often pose a low quality issue with respect to addressing ambiguous questions. in other words, the retrieved passages often partially or completely failed to cover all plausible interpretations, leading to significant performance degradation in terms of factual accuracy. to address this issue, the iterative rag approach, toc @xcite , has been introduced (c.f. fig 2(b) ) to further explore other interpretations that can not be covered by the single retrieval process. specifically, to further explore missing interpretations, the interpretations extracted in the previous iteration are utilized as queries to retrieve new passages, additional interpretations are then extracted. this exploration process is repeated in multiple times, leading to encompassing more diverse interpretations and corresponding answers. however, we argue that this effectiveness comes with a significant increase in computational overheads due to the iterative passage retrieval and llm reasoning. in our experiments, this method requires an average of 5.5 exploration steps per query. as shown in figure 1 , iterative rag (i.e., toc) significantly outperforms the vanilla rag approach in terms of factual accuracy but at the cost of greatly reduced efficiency, with notable increases in both inference time and api call costs. to this end, we introduce an efficient and robust rag framework for ambiguous qa, referred to as diversify-verify-adapt (diva). diva comprises two key components efficiently addressing the low quality retrieval issue: 1) retrieval diversification (rd) and 2) adaptive generation (ag). the key idea of rd is to infer pseudointerpretations of a question, using them to retrieve a set of passages that broadly cover these interpretations, thus enhancing retrieval quality without any iterative interpretation exploration process. to further enhance the robustness of this framework, we propose an adaptive generation (ag) method. the key idea of ag is to carefully verify the overall quality of the passages retrieved from rd before indiscriminately incorporating them. more specifically, we define a new criterion of quality levels tailored to ambiguous questions: {useful, partialuseful, useless}. subsequently, ag adapts the most suitable approach between relying on the retrieved passages and llm's internal knowledge, each of which is tailored to the specific quality level of the passages. experiments demonstrate that the proposed rd method efficiently diversifies the retrieval process to obtain passages covering diverse interpretations, thereby enhancing both qa and retrieval accuracy. additionally, the proposed ag method successfully discriminate low quality passages, leading to the improvement of the qa performance. consequently, diva outperforms existing baselines on the asqa @xcite and @xcite across various llm backbones in a few-shot setup, achieving superior accuracy and efficiency. the key contributions of this work are as follows: ## preliminary experiments we investigate the quality of retrieved passages and their impact on the performance of the rag framework (as in fig 2 (a)) in ambiguous qa task. ## proposed method: diva based on the findings, we propose an efficient and robust rag framework for ambiguous qa, diversify-verify-adapt (diva). this framework comprises two key components: retrieval diversification (sec 3.2) and adaptive generation (sec 3.3). the retrieval diversification method aims to efficiently diversify the retrieved passages to encompass diverse interpretations. subsequently, the adaptive generation method aims to verify the quality of the passages and adapt the most suitable approach tailored to their quality. fig 2(c ) and algorithm 1 show the overview and inference algorithm of diva, respectively. ## conclusion in this study, we examined the shortcomings of the current rag-based method in dealing with ambiguous questions, specifically its low-quality retrieval and inefficiency. our proposed framework, diva , effectively diversifies the retrieved passages to capture various interpretations, verifies their quality, and adapts the most appropriate approach based on that quality. this strategy improves qa performance while minimizing inefficiency.