Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023)

Abstract: Retrieval-augmented generation has raise extensive attention as it is promising to address the limitations of large language models including outdated knowledge and hallucinations. However, retrievers struggle to capture relevance, especially for queries with complex information needs. Recent work has proposed to improve relevance modeling by having large language models actively involved in retrieval, i.e., to guide retrieval with generation. In this paper, we show that strong performance can be achieved by a method we call Iter-RetGen, which synergizes retrieval and generation in an iterative manner: a model’s response to a task input shows what might be needed to finish the task, and thus can serve as an informative context for retrieving more relevant knowledge which in turn helps generate a better response in another iteration. Compared with recent work which interleaves retrieval with generation when completing a single output, Iter-RetGen processes all retrieved knowledge as a whole and largely preserves the flexibility in generation without structural constraints. We evaluate Iter-RetGen on multi-hop question answering, fact verification, and commonsense reasoning, and show that it can flexibly leverage parametric knowledge and non-parametric knowledge, and is superior to or competitive with state-of-the-art retrieval-augmented baselines while causing fewer overheads of retrieval and generation. We can further improve performance via generation-augmented retrieval adaptation.

## introduction generative large language models (llms) have powered numerous applications, with wellperceived utility. despite being powerful, llms lack knowledge that is under-represented in their training data, and are prone to hallucinations, especially in open-domain settings @xcite . retrieval-augmented llms, therefore, have raised widespread attention as llm outputs can be potentially grounded on external knowledge. previous retrieval-augmented lms @xcite typically adopted one-time retrieval, i.e., to retrieve knowledge using only the task input (e.g., a user question for open-domain question answering). one-time retrieval should suffice to fulfill the information needs if they are clearly stated in the original input, which is applicable to factoid question answering @xcite and single-hop fact verification @xcite , but not to tasks with complex information needs, e.g., multi-hop reasoning @xcite and long-form question answering @xcite . to fulfill complex information needs, recent work proposes to gather required knowledge multiple times throughout the generation process, using partial generation @xcite ) or forward-looking sentence(s) @xcite as search queries. however, such structured workflows of interleaving retrieval with generation have the following limitations: (1) as intermediate generation is conditioned on knowledge retrieved before, with no awareness of knowledge retrieved afterwards, they fail to process all retrieved knowledge as a whole during the generation process; (2) they require multi-round retrieval to gather a comprehensive set of knowledge, and may frequently change the prompts by updating newly retrieved knowledge, thus increasing the overheads of both retrieval and generation. in this paper, we find it simple but effective to enhance retrieval-augmented llms through iterative retrieval-generation synergy (iter-retgen, fig 1 ). iter-retgen iterates retrieval-augmented generation and generation-augmented retrieval: retrieval-augmented generation outputs a response to a task input based on all retrieved knowledge (initially using the task input as the query). this output shows what might be needed to fulfill the task, and thus can serve as an informative context to retrieve more relevant knowledge, i.e., generationaugmented retrieval. the newly retrieved knowledge can benefit another iteration of retrievalaugmented generation. we can also leverage model generations to adapt retrieval, by distilling knowledge from a re-ranker with access to model generations to a dense retriever with access to task inputs only, which may be beneficial in scenarios where user inputs can be easily collected, but relevant knowledge or desirable outputs are not annotated. we evaluate our method on three tasks, including multi-hop question answering, fact verification, and commonsense reasoning. our method prompts an llm to produce a chain of reasoning steps followed by the final answer under a few-shot setting. for in-context demonstrations, we focus on problem-solving and follow @xcite to annotate chains of thoughts, without explicitly considering how generation-augmented retrieval might be affected, which makes it conceptually simple and easy to implement. our method achieves up to 8.6% absolute gains over previous state-of-theart retrieval-augmented methods on four out of six datasets while being competitive on the remaining two. according to our experiments, generation generally benefits from more iterations, with two iterations giving the most performance gains. one may customize the performance-cost tradeoffs by choosing an appropriate number of iterations. we can further improve performance and also reduce iterations via the aforementioned generation-augmented retrieval adaptation. we summarize our findings as follows: ## related work in recent months, there has been a surge in llmpowered applications, such as chatgpt, bing chat, and copilot @xcite . while showing an unprecedented level of performance, llms are subject to the following limitations: (1) due to a high demand for compute and data, it remains an open research question to continually update llms both efficiently and effectively @xcite ; retrieval augmentation is a mainstream direction to connect llms to the external world. previous retrieval-augmented lms @xcite typically receive retrieved knowledge in a passive way: knowledge is retrieved based on the task inputs without lms' intervention. as it is difficult for a retriever to capture relevance, especially in the zero-shot setting, recent work shows a shift towards having llms actively involved in retrieval to improve relevance modeling, e.g., to provide a specific context for retrieval with model generations (e.g., generated search queries @xcite @xcite , partial generation @xcite , or forward-looking sentences @xcite ). @xcite proposed a dsp programming framework that supports various retrieval-augmented methods. recent work interleaves retrieval with generation when completing a single output. such a structured workflow may reduce the flexibility in generation @xcite . iter-retgen avoids interrupting generation with retrieval, but iterates retrieval and generation, i.e., to leverage the complete generation from the previous iteration to retrieve more question: retrieval: retrieval-augmented generation: what is the height of the player who won the 2015 afl rising star award? title: 2015 afl rising star context: the nab afl rising star award is given annually to a stand out young player in the australian football league. the award was won by jesse hogan of melbourne the 2015 afl rising star award was won by jesse hogan of melbourne. jesse hogan is a professional australian rules footballer. he is 198 cm tall. so the answer is 198 cm title: jesse hogan context: jesse hogan … playing for the melbourne football club. a key forward, hogan is 1.95 m tall … made his afl debut in the 2015 season and won the ron evans medal as the afl rising star the 2015 afl rising star award was won by jesse hogan of melbourne. jesse hogan is 1.95 m tall. so the answer is 1.95 m iteration 1 iteration 2 generation-augmented retrieval: retrieval-augmented generation: question: what is the height of the player who won the 2015 afl rising star award? relevant information which helps improve generation in the next iteration. iter-retgen also has the advantage of processing all retrieved knowledge as a whole during the generation process, and is conceptually simpler and easier-to-implement, while being empirically strong in multi-hop question answering, fact verification, and commonsense reasoning. a closely related work called gar @xcite augments queries with generated background information. hyde @xcite also shares a similar spirit, but focuses on zero-shot information retrieval, and proposes to first prompt an llm to produce "hypothetical" paragraphs that cover the information needed to answer a given question, and then use the generated paragraphs to retrieve the real ones. repocoder @xcite focuses on repository-level code completion, and proposes a 2-iteration retrieval-generation paradigm where the second iteration leverages the intermediate code completion for retrieval. by contrast, we propose to synergize retrieval and generation with iter-retgen on various natural language tasks, and explore how we can further adapt retrieval with model generations. ## case study ## conclusion we demonstrate the effectiveness of iter-retgen in answering questions with complex information needs. despite simple, iter-retgen outperforms retrieval-augmented methods that have a more complex workflow, which we believe could serve as a strong baseline for future research on retrieval-augmented generation. we also show that generation-augmented retrieval adaptation can further improve the performance of iter-retgen while also reducing overheads.