GPT vs RETRO : Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning (2024)

Abstract: Parameter-Efficient Fine-Tuning (PEFT) and Retrieval-Augmented Generation (RAG) have become popular methods for adapting large language models while minimizing compute requirements. In this paper, we apply PEFT methods (P-tuning, Adapters, and LoRA) to a modified Retrieval-Enhanced Transformer (RETRO) and a baseline GPT model across several sizes, ranging from 823 million to 48 billion parameters. We show that RETRO models outperform GPT models in zero-shot settings due to their unique pre-training process but GPT models have higher performance potential with PEFT. Additionally, our study indicates that 8B parameter models strike an optimal balance between cost and performance and P-tuning lags behind other PEFT techniques. We further provide a comparative analysis of between applying PEFT to Instruction-tuned RETRO model and base RETRO model. This work presents the first comprehensive comparison of various PEFT methods integrated with RAG, applied to both GPT and RETRO models, highlighting their relative performance.

## introduction pre-trained large language models have made a demonstrable impact across applications in academia and industry. many use cases, however, require llms adapted to specific tasks and unique information but lack the resources for extensive retraining. to address this, parameter-efficient fine-tuning (peft) @xcite and retrieval-augmented generation (rag) @xcite have become popular methods due to their effectiveness and efficiency, inspiring new lines of research. peft has been proven to be a comparable substitute to supervised fine-tuning (sft) by achieving competitive performance at a fraction of the number of updated parameters @xcite . in * equal contribution. ## related work previous works like @xcite have compared multiple peft methods but lacked comparison for retrieval-based tasks and retrieval aug-mented language models. in this section we focus on recent work that combine finetuning with retrieval. a comprehensive survey @xcite synthetized multiple comparative studies on peft and rag, underscoring the potential benefits of combining these approaches as a promising direction for future investigation. there are multiple works that provide methods to combine rag with fine-tuning to improve accuracy @xcite . multiple studies have explored the comparison between fine-tuning and retrieval. @xcite and @xcite reported improved accuracy using rag over fine-tuning gpt models, while also noting suboptimal results when combining the two methods. @xcite demonstrated improved outcomes by integrating both approaches for specific agriculture and geography tasks. additionally, @xcite compared the efficacy of these methods, including full and qlora finetuning @xcite , in low-frequency entity question-answering tasks. these studies collectively suggest the need for comprehensive investigation into multiple peft techniques combined with rag and maintain retrieval pretrained llms with peft to be unexplored, thereby motivating our research. ## conclusion this study explores parameter-efficient fine-tuning (peft) methods applied to retrieval-augmented generation (rag) models, comparing gpt and retro architectures. retro generally outperforms gpt in zero-shot settings due to their pre-training process that integrates external retrieval, enhancing contextual understanding. however, gpt models show a higher performance potential with peft, indicating more room for improvement during fine-tuning. both retro and gpt models perform optimally around the 8b parameter mark, balancing cost and performance. while p-tuning is effective in larger models, it lags behind other methods in smaller models, particularly for retro. applying peft to instructiontuned retro yields limited improvement compared to base retro, suggesting a saturation point in leveraging pre-training and fine-tuning benefits. our comprehensive analysis offers valuable insights for optimizing large language models with peft and rag to the community.