F a V e: Factored and Verified Search Rationale for Long-form Answer (2025)

Abstract: Targeting long-form question-answering, chain-of-query (CoQ) has been studied, integrating chain-of-thought (CoT) with retrieval-augmented generation. CoQ answers the complex question step-by-step, through simpler subquestions (SQs) from which relevant knowledge is retrieved. By doing so, CoQ aims to improve the answer comprehensiveness and verifiability, at the expense of latency. Our first contribution is showing that the chaining often incurs harmful effects on both objectives, and SQs left unverified often fail to answer the given question. Second, we propose a better alternative to CoQ, union-of-query which adopts a factored approach to break the harmful chain. Finally, we propose to verify SQs before answers, by fine-tuning the SQ generator using verified SQs and introducing a selector verifying SQs in test time. Employing vicuna-13b, our approach, denoted by FaVe (short for Factored and Verified search), even outperforms ChatGPT baselines while maintaining efficiency.

## introduction in long-form question-answering (lfqa) @xcite , a model is tasked to generate a longform response to answer a complex question. in this work, we target two task objectives for lfqa: answer comprehensiveness and answer verifiability @xcite , where we aim to generate factual claims that are not only comprehensive, meaning they cover all relevant information related to the given question, but also verifiable, ensuring that the claims can be confirmed through external sources @xcite . an existing solution is retrieval-augmented generation (rag) @xcite , utilizing retrieved documents from external corpus as rationales for the generated answer. this enables the user to verify the truthfulness of first, we propose union-of-query (uoq) adopting a factored approach, to break two harmful chains (as shown in figure 1 (b) with scissors) -(c1) the chain between answers and queries and (c2) the chain between different answers -while preserving the chain between sqs. by breaking c1, uoq, before answers, sequentially decodes varying numbers of sqs, i.e., a search session. then, by breaking c2, uoq answers different sqs in parallel, reducing latency. figure 2 shows that uoq (green curve) via the chain break consistently improves the comprehensiveness and maintains comparable latency to that of vanilla rag. second, we propose session verification, leveraging feedback presented in figure 1(b) with green arrows, to verify if an sq is (v q ) relevant to the given question, (v d ) answerable by the retrieved document, and finally (v a ) addressing relevant claims in the final answer. specifically, the verification process comprises generating multiple search session candidates and, to identify the best session, verifying sqs in each session candidate based on the received feedbacks. building upon this process, we introduce answer-aware candidate generator, which is trained using verified sessions via feed1 the comprehensiveness is measured by gold claim recall, used in alce benchmark @xcite on eli5 dataset @xcite . with these two components (i.e., uoq and session verification), we propose factored and verified search session, or fave. we evaluate models using two datasets: eli5 @xcite and strategyqa @xcite . compared to coq, results on eli5 show that fave improves answer comprehensiveness and verifiability by 35.4% and 21.8%, respectively, while reducing latency by a factor of two via parallel answer generation. when evaluated on multi-hop reasoning using strat-egyqa, fave also demonstrates improved coverage in relevant facts by 35.6%, achieving 2.9%pt increases in the final accuracy on yes/no questions. our contributions are threefold: ## related work and motivation though large language models (llms) often produce factually incorrect answers, grounding knowledge or latent rationales of answers are unknown to users, constraining reliability of llm-generated responses. to enhance reliability, rationales can be obtained 1) internally or 2) externally. a notable exemplar of the former is chain-ofthought prompting @xcite , where llm how to measure battery life? ## approach in the following sections, we first describe uoq, the factored generation process targeting a single search session ( §3.1), and then present the session verification process, sampling and verifying multiple search session candidates ( §3.2). the algorithm for our proposed method is presented in algorithm 1. for all inferences, such as session generation and answer generation, we employ vicuna-13b with prompting, unless specified otherwise. detailed prompts are shown in appendix a.2. ## conclusion in this work, targeting the long-form questionanswering task, we aim to improve both the effectiveness (in terms of comprehensiveness and verifiability) and the efficiency (in terms of latency). we propose fave, consisting of union-of-query, as a better alternative to chain-of-query, and session verification. evaluated on eli5 and strategyqa, fave outperforms baselines with lower latency.