A + B : A General Generator-Reader Framework for Optimizing LLM s to Unleash Synergy Potential (2024)

Abstract: Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models (LLMs). Targeting its bottleneck of retriever performance, “generate-then-read” pipeline is proposed to replace the retrieval stage with generation from the LLM itself. Although promising, this research direction is underexplored and still cannot work in the scenario when source knowledge is given. In this paper, we formalize a general “A + B” framework with varying combinations of foundation models and types for systematic investigation. We explore the efficacy of the base and chat versions of LLMs and found their different functionalities suitable for generator A and reader B, respectively. Their combinations consistently outperform single models, especially in complex scenarios. Furthermore, we extend the application of the “A + B” framework to scenarios involving source documents through continuous learning, enabling the direct integration of external knowledge into LLMs. This approach not only facilitates effective acquisition of new knowledge but also addresses the challenges of safety and helpfulness post-adaptation. The paper underscores the versatility of the “A + B” framework, demonstrating its potential to enhance the practical application of LLMs across various domains.

## introduction large language models have achieved remarkable success in natural language processing @xcite . nonetheless, in real-world scenarios, llms sometimes lack of domain-specific or latest information @xcite . to supplement necessary external knowledge, retrieval-augmented generation (rag) has been proposed and attracted much attention @xcite . the basic idea is to employ a two-step "retrieve-then-generate" process. first, rag models leverage a retriever with the goal of discovering relevant documents from the given source, and subsequently generate answers by feeding the retrieval results together with the question into llms. although effective, rag apparently suffers from the bottleneck of retriever performance. in contrast to the retrievethen-read pipeline, @xcite proposed generate rather than retrieve, a.k.a, generate-then-read framework. it aims to generate relevant documents by the llm itself, taking advantage of the memorization ability of llms to simplify and optimize the pipeline. however, this research direction is still under-exploration. in this paper, we first formalize a general "a + b" framework and conduct systematical investigation to unleash the full power in various scenarios, where a and b denote generator and reader, respectively. clearly, they have distinct functionalities. the generator a is tasked with producing context relevant to the input queries, which requires a high degree of factual accuracy, while the reader b aims at interpreting the generated context to furnish appropriate responses, necessitating cognitive reasoning and alignment with human preferences. this raises an interesting assumption, will a simple modification -a combination of different model types or versions in "a + b" framework -lead to better performance? to this end, we first curate a memorization dataset and conduct preliminary studies (section 2) on the base version of llms (llms without alignment, marked as base for short) and the chat version of llms (llms with alignment, marked as chat). unsurprisingly, base performs better than chat in memorization tasks, which is the key ability of generator; on the contrary, chat can generate more helpful and safer response than base and is more suitable as reader. we then delve deeply into the "a + b" framework with various foundation mod-els varying in types and scales, considering both task performance and human preference alignment in knowledge-intensive tasks. note that no source documents are provided and all knowledge are assumed seen during pre-training, largely following "generate-then-read" @xcite . we found that different combinations perform quite differently, but common patterns show that base/chat are indeed good generator/reader, and "a + b" framework consistently outperforms the singular model especially in complex scenarios. deep dive into this general framework not only helps us better understand the effects of pre-training and posttraining (i.e., alignment) of llms, but also offers practical insights in optimizing the performance and safety in real-world applications. actually, besides rag, many other models can also fall into this general "a + b" framework, such as chain-of-thought @xcite and self-refine @xcite . our investigation method can be easily adapted. furthermore, we consider the scenario where source documents are present. this case goes beyond the "generate-then-read" regime and is traditionally solved by vanilla rag. we thus apply the "a + b" framework through continuous learning to integrate source knowledge into llms, demonstrating the potential of our framework in this scenario. although there are some works such as domainspecific adaptations @xcite in this direction, a significant gap remains in research on the effective usage of this integrated knowledge, as well as how to guarantee the safety and helpfulness after adaptation or continuous learning. our "a + b" framework with suitable model types can naturally solve this issue. further experiments and analysis demonstrate the effectiveness of our framework. our main contributions can be summarized as follows: ## preliminary experiments two critical aspects must be considered in knowledge-intensive tasks: accurate knowledge memorization and the generation of high-quality responses. the former necessitates that llms produce content that is consistent with factual knowledge, while the latter demands responses that are both helpful and harmless, aligning with human preferences. recently, researchers noted that fine-tuning may inadvertently diminish the llms' ability to convey factual information. specifically, llms subjected to sft have demonstrated marked performance declines on benchmarks assessing factual knowledge and reasoning capabilities compared to their baseline models @xcite . this observation raises a pivotal question: are unaligned and aligned foot_0 models better suited to distinct roles within knowledge-intensive tasks, for example, as generators and readers, respectively? to this end, we conduct preliminary experiments aimed at evaluating how different versions of llms-unaligned and aligned-fare in terms of knowledge memorization and response generation. ## a + b framework building on the posit from preliminary experiments that the base model and chat model are better suited to different roles in knowledge-intensive tasks, we demonstrate a more nuanced approach to questionanswering. rather than relying on a single model to directly answer queries, we conceptualize the framework as "a + b" (generator-reader) architecture with distinct models. the generator a is tasked with producing relevant information supporting to answer the input query. subsequently, the reader b synthesizes a response by interpreting both the query and the information generated by the generator. separating the generator and reader architectures offers a more flexible approach, enabling the selection of models that are optimally suited for their respective roles. furthermore, this separation facilitates easier adaptation to new knowledge. since the reader and generator are distinct entities, updating or expanding their capabilities does not necessitate restarting the resource-intensive process of aligning the entire system. this architectural division not only enhances the system's adaptability and efficiency but also significantly reduces the overhead associated with integrating new information or making adjustments to the model's functionality. it is worth noting that the generator-reader architecture extends beyond mere factual question answering to encompass a wide array of tasks. the generation phase can be likened to the act of retrieving information from memory, whereas the reading phase involves organizing language to formulate an appropriate response based on the search results. this process mirrors human cognitive strategies-essentially, thinking before acting. furthermore, prior research, such as cot @xcite and rag, employs a similar generatorreader framework. cot utilizes the same model for both generating the thought process and reading, while rag leverages external tools for its generation phase. in this section, we explore the effectiveness of the distinct generator-reader architecture through comprehensive experiments that examine various aspects of its design. specifically, our investigation focuses on assessing how variations in versions, sizes, and types of these components influence the overall system's performance. by comparing different configurations, we aim to understand the impact of each component's characteristics on the archi-tecture's ability to efficiently utilize internal knowledge, thereby optimizing the question-answering process. ## external knowledge scenario in this section, we aim to extend and evaluate the "a + b" framework in scenarios that more closely resemble real-world applications. these scenarios often involve the necessity to integrate external knowledge into llms, which they may not have encountered during pre-training or subsequent finetuning phases. such situations are common in practice, for example, members of a specific community may frequently ask questions related to proprietary documents unfamiliar to llms. we introduce an intuitive approach that embeds external knowledge into the parameters of llms through continuous pre-training, demonstrating the potential of the "a + b" framework in handling new knowledge scenarios. ## conclusion this research introduces the "a + b" framework as a novel approach to enhance llms in knowledge-intensive tasks. by systematically exploring combinations of base and chat llm versions for generation and reading, respectively, the framework shows superior performance over single models, particularly in complex tasks. the extension of the "a + b" framework to include continuous learning for scenarios with source documents enables efficient integration of external knowledge, improving inference efficiency, and addressing safety and helpfulness challenges. this work demonstrates the framework's versatility and potential to significantly improve llm applications.