Preserving Pre-trained Representation Space: On Effectiveness of Prefix-tuning for Large Multi-modal Models (2024)

Abstract: Recently, we have observed that Large Multi-modal Models (LMMs) are revolutionizing the way machines interact with the world, unlocking new possibilities across various multi-modal applications. To adapt LMMs for downstream tasks, parameter-efficient fine-tuning (PEFT) which only trains additional prefix tokens or modules, has gained popularity. Nevertheless, there has been little analysis of how PEFT works in LMMs. In this paper, we delve into the strengths and weaknesses of each tuning strategy, shifting the focus from the efficiency typically associated with these approaches. We first discover that model parameter tuning methods such as LoRA and Adapters, distort the feature representation space learned during pre-training, limiting the full utilization of pre-trained knowledge. We also demonstrate that prefix-tuning excels at preserving the representation space, despite of its lower performance on downstream tasks. These findings suggest a simple two-step PEFT strategy called Prefix-Tuned PEFT (PT-PEFT) , which successively performs prefix-tuning and then other PEFT (i.e., Adapter, LoRA), combines the benefits of both. Experimental results show that PT-PEFT not only improves performance in image captioning and visual question answering compared to vanilla PEFT methods but also helps preserve the representation space of the four pre-trained models.

## introduction understanding the visual scene and expressing it with a natural language are two distinct tasks yet the human brain can comprehensively handle both without difficulty. large multi-modal models (lmms) mimic such capability by training a deep neural network (dnn) such that it learns semantically meaningful connections between vision and language from a large number of imagetext pairs @xcite @xcite . recently, lmms have been widely used due to their broad range of applications, including chatbot, robot control, and video generation @xcite @xcite . in the pre-training, lmms are trained to predict the masked words or next words from the imagetext pair @xcite @xcite . in the second step called fine-tuning, the pre-trained lmms are tailored to the specific downstream task. it has been shown that fine-tuning provides superior performance in various downstream tasks such as image captioning (ic), visual question answering (vqa), and image-text retrieval @xcite @xcite . however, fine-tuned models often suffer from the loss of generalization capability obtained from the pre-training @xcite . since the taskspecific dataset is far smaller than the pre-training unlabeled dataset, the pre-trained model can be easily overfitted to the small-sized downstream task dataset, leading to degraded performance @xcite . various approaches have been suggested over the years to address the problem. in prompt-based approaches, manually designed prompts or trainable continuous embedding vectors are integrated into the input data to adapt the model for downstream tasks @xcite @xcite . in knowledge distillation-based fine-tuning approaches, the model minimizes the distance between the distribution of the pre-trained and finetuned models @xcite @xcite . the common wisdom behind these approaches is to minimize the modification of the pre-trained model parameters while maintaining performance on downstream tasks. one drawback of the full model fine-tuning is the huge computational burden caused by the model parameters update. in an effort to reduce the huge training cost, various parameter-efficient finetuning (peft) techniques have been proposed @xcite @xcite . in these approaches, only a small set of additional modules (e.g., prefix, adapter, lora) is trained instead of relying on full fine-tuning. these approaches are especially beneficial for training the large pre-trained model like gpt @xcite , t5 @xcite , and llama @xcite . training efficiency is a well-known advantage of prefix-tuning. unlike other peft methods, prefixtuning does not modify the model's parameters, leaving the representation space unchanged. to investigate the changes in the representation space, we analyze the feature representation matrices using singular value decomposition (svd). notably, we observe that the representation space of a finetuned model (in ic and vqa) utilizes only a limited set of effective basis vectors (60% of those in the pre-trained model) to express the output. clearly, this limits the model's ability to fully enjoy the benefits obtained from pre-training (see figure 4 ). in contrast, we discover that all the basis vectors are utilized in the prefix-tuned model, implying that the prefix-tuning effectively preserves the inherited representation space from the pretraining. while the prefix-tuning is effective in preserving pre-trained knowledge, the efficacy of this approach is somewhat questionable since the reported evaluation results are not conclusive. some studies claim that the prefix-tuning performs comparable to the model parameter-tuning (e.g., full finetuning, lora, adapter), while others argue that the prefix-tuning struggles in the training of relatively small-sized language models @xcite . an aim of this paper is to propose a simple yet effective tuning strategy to combine the merits of two seemingly distinct approaches. the proposed method, henceforth referred to as prefix-tuned peft (pt-peft), performs the prefix-tuning and the model parameter-tuning sequentially. the key feature of pt-peft is to preserve the pre-trained feature space through the prefix-tuning and then refine the model parameters using the peft method. intuitively, this approach resembles a language model learning a new task using prompt sentences such as "i will provide example sentences describing the given pictures in the news article style. so, please generate the caption for the given images with such style." by providing a context suitable for the new task, the model's adaptability is enhanced, allowing for faster convergence and minimal changes to the weights of the pre-trained model. ## prefix-tuned parameter-efficient fine tuning (pt-peft) prefix implementation prefix embedding vectors are first processed through the prefix encoder, following standard practices in prefix-tuning @xcite (see appendix for details). the processed prefixes are then concatenated with text and/or image tokens to form the input to the lmms. ## experiments 4.1 setup model to demonstrate the generalization capability of our method, we evaluate various pretrained lmms with different architectures and sizes. specifically, we conduct experiments on vinvl-base/large @xcite , ofa-base @xcite , blip @xcite , an blip-2 @xcite models. ## conclusion in this paper, we discovered that fine-tuning methods including lora, adapter, and full fine-tuning could cause the loss of learned knowledge from the pre-training stage. we quantified this loss in representation space using a novel rank-based analysis and identified that prefix-tuning does not cause this critical loss. based on these findings, we proposed a two-step strategy, pt-peft, which first performs prefix-tuning and then applies other finetuning methods. our experiments showed that pt-peft not only preserves the representation space preservation but also improves downstream task performance.