Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation (2024)

Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating additional information from retrieval. However, studies have shown that LLMs still face challenges in effectively using the retrieved information, even ignore it or be misled by it. The key reason is that the training of LLMs does not clearly make LLMs learn how to utilize input retrieved texts with varied quality. In this paper, we propose a novel perspective that considers the role of LLMs in RAG as “Information Refiner”, which means that regardless of correctness, completeness, or usefulness of retrieved texts, LLMs can consistently integrate knowledge within the retrieved texts and model parameters to generate the texts that are more concise, accurate, and complete than the retrieved texts. To this end, we propose an information refinement training method named INFO-RAG that optimizes LLMs for RAG in an unsupervised manner. INFO-RAG is low-cost and general across various tasks. Extensive experiments on zero-shot prediction of 11 datasets in diverse tasks including Question Answering, Slot-Filling, Language Modeling, Dialogue, and Code Generation show that INFO-RAG improves the performance of LLaMA2 by an average of 9.39% relative points. INFO-RAG also shows advantages in in-context learning and robustness of RAG.

## introduction retrieval-augmented generation (rag) is a popular framework in modern nlp systems that equips neural with retrieved information for text generation like open-domain question answering, dialogue @xcite etc. recently, rag has been applied to large language models (llms) to provide additional knowledge * work done during the tencent rhino-bird research elite program at wechat. ## our info-rag this section introduces our info-rag, an unsupervised training method to enable llms to perform information refinement in rag. firstly, we summarize the retrieved texts in rag into three scenarios and define the positive information gain for each scenario. secondly, we construct sample pairs in which the output has information gain compared to the input for these three scenarios and design three training tasks. thirdly, we train llms under our designed tasks on the unsupervised samples. unsupervised training makes info-rag low-cost and general for rag in various tasks. ## conclusion this paper proposes a novel perspective to reassess the role of llms in rag that considers llms as "information refiner". this means that regardless of the correctness, completeness, or usefulness of the retrieved texts, llms can consistently integrate knowledge within model parameters and the retrieved texts to generate texts that are more concise, accurate, and complete. to achieve it, we propose an information refinement training method named info-rag in an unsupervised manner, which is low-cost and general across various tasks. extensive experiments across 11 datasets of 7 tasks in zero-shot setting show that info-rag improves the performance of llms for rag. info-rag also shows advantages in icl and robustness of rag and can be combined with the sota rag framework to further improve its performance.