Fast Randomized Low-Rank Adaptation of Pre-trained Language Models with PAC Regularization (2024)

Abstract: Low-rank adaptation (LoRA) achieves parameter efficient fine-tuning for large language models (LLMs) by decomposing the model weight update into a pair of low-rank projection matrices. Yet, the memory overhead restricts it to scale up when the model size increases. We propose Randomized LoRA (RLoRA) which adopts Randomized Walsh-Hadamard Transform to achieve significant reduction in the size of trainable parameters compared to LoRA. At the same time, it allows a PAC-Bayes regularizer to be efficiently incorporated to improve generalization. We evaluate the effectiveness of RLoRA on LLMs RoBERTa, GPT-2 and LLaMA-7B using GLUE, E2E and math reasoning benchmarks. With a much lower memory requirement, RLoRA can give similar performance as the SOTA low-rank adaptation methods for these three tasks and significantly better performance under few-shot settings.

## introduction pre-trained language models (plms), typically consisting of millions or billions of parameters, have achieved the state-of-the-art performance on nlp tasks @xcite . while plms can be fine-tuned for specific downstream tasks, their increasing scale makes the fine-tuning and model storage formidable. parameter-efficient fine-tuning (peft) aims to fine-tune a pre-trained model efficiently by adapting only a small set of parameters. various approaches for peft have been explored. i) the adapter-based approach @xcite @xcite introduces trainable adapter modules to achieve fine-tuning with the plm frozen. this approach however falls short of the inference latency introduced. ii) the masking-based approach updates only some selected subset of the plm's parameters @xcite @xcite figure 1 : performance of peft methods on fine-tuning roberta-base using 200 training data points from the glue datasets. pa: parallel adapter @xcite ; sa: sequential adapter @xcite ; said: intrinsic dimension adaptation @xcite ; lora: low-rank adaptation @xcite , rlora: randomized lora (proposed). @xcite . as it requires access to the plm for updating, its application is restricted. iii) prefix-tuning @xcite and prompt-tuning @xcite prepend additional tokens to the input embeddings and only train these soft prompts where the contextual information is wrapped to the inputs (or hidden states) without modifying the plm. this approach however converges much slower than the adapter-based one @xcite . low-rank adaptation (lora) @xcite ) hypothesizes that the plm's weight update for the fine-tuning possesses a low "intrinsic rank" dominating the desirable optimization trajectory @xcite , and proposes to decompose the update of the weight (e.g., transformer's self-attention weight metrics) into two low-rank projection matrices. this reparameteration trick has been shown particularly effective in reducing the number of trainable parameters compared to other peft approaches. yet, the memory overhead is still considerably large when the size of the model increases. to this end, inspired by the work on random feature projection for kernel machines @xcite ,we propose randomized low-rank adaptation (rlora) which adopts the random matrix approximation to construct the up-projection and down-projection matrices in lora. in particular, we adopt the randomized walsh-hadamard transform (wht) and a learnable scaling vector to reparameterize the low-rank projections to reduce both the time and space requirements. using random matrices to reduce the memory requirement has also been explored in some recent work like lora-fa @xcite , vera @xcite and nola @xcite . to contrast, rlora adopts the randomized wht which is orthogonal and recursively defined, which can further reduce the resources needed for the gradient computation w.r.t. the hidden states. also, the proposed rlora adopts pac-bayes regularization to enhance generalization performance, which is particularly important for adapting plms where only a small training set is available. some related ideas have been explored in @xcite , which empirically showed that the pac-bayesian theories provide a tighter bound and understanding for the role of model size and generalization ability. @xcite ; @xcite used the pac-bayesian framework to estimate the information in neural networks. the adoption of the randomized wht in rlora allows the regularization to be efficiently incorporated. to the best of our knowledge, work on incorporating the pac-bayes generalization bound for regularizing peft is still lacking in the literature. our contribution can be summarized as: our code will be made available for sharing via github. ## background in this section, we provide background and related work on low-rank adaptation, random matrix methods, and pac-bayes generalization bound. ## methodology in this section, we present the details about how the proposed rlora. in particular, section 3.1 presents the randomized wht and how it is integrated into rlora to gain parameter-efficiency and reduction in computational cost during training. in section 3.2, we show theoretically how the posterior distribution of the plm's weight update based on the reparameterization is associated with a lower-dimensional "intrinsic subspace" characterized by the randomized wht. we derive the pac-bayes generalization bound in section 3.3 and explain how the reparameterization by randomized wht allows the bound to be efficiently incorporated into rlora to regularize the fine-tuning. fig. 2 shows an overall illustration of rlora. ## experiment we evaluate the performance of rlora using roberta-base/roberta-large @xcite and gpt-2-medium @xcite . our experiments encompass a diverse range of tasks, including natural language understanding (nlu) and natural language generation (nlg). specifically, we evaluate the models on the glue benchmark @xcite for robertabase/roberta-large. for evaluation on gpt-2 medium, to facilitate direct comparison, we adopt the experimental setup outlined in the original work of lora @xcite . additional details regarding the datasets used can be found in appendix c. all the experiments were conducted using the nvidia tesla v100. ## limitations and future work in this paper, we proposed randomized walsh-hadamard low-rank adaptation to reduce the finetuning resource consumption and scale the pac-bayesian regularization to larger model. the current estimation of the pac-bayes generation bound requires additional gradient computation on the resampled data. better ways to integrate the estimation of the information into the training process is worth pursuing. in addition, how to select a proper rank for low-rank adaptation is another research issue where methods like adalora @xcite and increlora @xcite propose different parameter importance scores for the rank selection. this issue is not yet considered in this paper. ## conclusion we propose the randomized low-rank adaptation rlora which integrates lora with the 3 we train the roberta-base on rte datasets for 1 epoch to get this result. randomized walsh-hadamard transform to gain parameter-efficiency for fine-tuning and pac-bayes regularization to further boost generalization capability. via comprehensive experiments, we demonstrates that rlora can achieve comparable performance as lora where the number of trainable parameters required is only 0.02% of the original model (and 1/35 of lora). also, under few-shot settings, the pac-bayes regularization makes rlora outperforms lora and other sota low-rank adaptation methods. for future work, more effective and efficient ways for estimating the information-based regularization is worth pursuing. in addition, how to selecting the optimal rank is another open research issue.