A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation (2025)

Abstract: Retrieval augmented generation (RAG) is a popular framework for question answering that is powered by two large language models (LLMs): an embedding model that retrieves context documents from a database that are relevant to a given question, and a generator model that uses the retrieved context to generate an answer to the question. Both the embedding and generator models can be fine-tuned to increase performance of a RAG pipeline on a new task, but multiple fine-tuning strategies exist with different costs and benefits. In this paper, we evaluate and compare several RAG fine-tuning strategies, including independent, joint, and two-phase fine-tuning. In our experiments, we observe that all of these strategies achieve about equal improvement in EM and F1 generation quality metrics, although they have significantly different computational costs. We conclude the optimal fine-tuning strategy to use depends on whether the training dataset includes context labels and whether a grid search over the learning rates for the embedding and generator models is required.

## introduction retrieval augmented generation (rag) is a popular framework for nlp tasks like question answering. rag is powered by two llms: an embedding model that retrieves context documents from a database that are relevant to a given question, and a generator model that uses the retrieved context documents to generate an answer to the question. both the embedding model and generator model can be fine-tuned to improve the end-to-end performance of a rag pipeline. given a dataset of (question, context) pairs, the embedding model can be fine-tuned to retrieve more relevant context documents for a given question. this requires a training dataset with context labels, i.e., where each question is paired with one or more relevant context documents from the database. given a dataset of (question, context, answer) triplets, where the context is either provided as part of the training dataset as context labels or retrieved from the database using a baseline embedding model, the generator model can be fine-tuned to increase the likelihood of generating the correct answer given the question and relevant context documents. although the embedding and generator models can be fine-tuned independently, fine-tuning both models jointly with an end-to-end fine-tuning method such as rag-token or rag-sequence @xcite ) may yield equal or better endto-end performance without the need for context labels. additionally, we consider a two-phase finetuning strategy that uses rag-token to first finetune the generator model while holding the embedding model frozen, then fine-tunes the embedding model while holding the generator model frozen. the choice of learning rate used for fine-tuning may significantly affect the end-to-end performance of the rag pipeline, and the optimal choice of learning rate for the embedding and generator models may be different. we use a grid search to find a suitable choice of learning rates. in this paper, we compare independent, joint, and two-phase fine-tuning and find they all achieve similar end-to-end performance when using a suitable choice of learning rates. based on our experimental results, we make the following conclusions: ## experiments here we evaluate and compare the performance of the rag fine-tuning strategies described in the previous section for four rag pipelines, each consisting of either an mpnet @xcite or minilm @xcite embedding model and either a llama-3-8b-instruct (ai@meta, 2024) or mistral-7b-instruct-v0.1 @xcite generator model. we fine-tune and evaluate on two datasets: hotpotqa @xcite and popqa @xcite . our retrieval system uses the embedding model to retrieve the top @xmath0. we use the same chunking of wikipedia as @xcite , which contains 29.9m chunks. we construct a vector database from the corpus using a faiss index @xcite . each experiment was conducted on a node with 8 nvidia a10 gpus. to minimize the computational expense of our experiments, in each experiment we fine-tune for only 1 epoch (for the two-phase strategy, each model is fine-tuned for 1 epoch) @xcite . in all experiments, we use a linear learning rate schedule. to find nearoptimal choices of learning rates, we perform a grid search over values between 10 -8 and 10 -4 , with grid values separated roughly by factors of 3: specifically, 10 -8 , 3 × 10 -8 , 10 -7 , 3 × 10 -7 , 10 -6 , 3 × 10 -6 , 10 -5 , 3 × 10 -5 , and 10 -4 . ## conclusion in this paper, we compared various strategies for fine-tuning the embedding and generator models of a rag pipeline. from our experiments with four different rag pipelines on hotpotqa and popqa, we observed that independent, joint, and two-phase fine-tuning are all about equally effective for fine-tuning a rag pipeline. while independent fine-tuning is computationally less expensive, joint fine-tuning and two-phase fine-tuning have the benefit of not requiring context labels to perform fine-tuning. in addition, two-phase fine-tuning allows for a more efficient hyperparameter search for the embedding and generator model learning rates compared to joint fine-tuning.