RA - L o RA : Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models (2024)

Abstract: Deploying large language models (LLMs) with their extensive parameters and high memory demands challenges computational efficiency, particularly in fine-tuning for specific applications with limited resources. Techniques like Low-Rank Adaptation (LoRA) help by training a smaller, modifiable extension of the base model to reduce memory usage. However, combining quantization with LoRA, especially in low-bit scenarios, can lead to performance losses due to quantization errors. Our innovative Rank-Adaptive LoRA (RA-LoRA) addresses this by dynamically adjusting the adapter’s rank using rank-subspace analysis, optimizing performance with fewer parameters. We tested RA-LoRA on state-of-the-art LLMs for 2-bit efficient fine-tuning, showing it can improve model accuracy with minimal trainable parameters, marking a leap forward in quantization-aware fine-tuning methods and highlighting the significance of rank dynamics in optimizing quantized LLMs.

## introduction large language models (llms) such as @xcite demonstrate impressive abilities in tasks like translation and summarization. however, their large size presents deployment challenges due to significant memory demands @xcite . a technique known as weight quantization compresses model parameters, reducing memory needs and boosting computational efficiency. although post-training quantization (ptq) can be conveniently employed for compression of large pre-trained models, it often compromises inference accuracy, particularly in low-bit environments @xcite @xcite @xcite . the field of natural language processing is increasingly adopting the strategy of fine-tuning * corresponding author large-scale, pre-trained llms for specific downstream applications @xcite . while effective in enhancing model performance, fine-tuning traditionally updates the entire pre-trained model, a method becoming unsuitable with the growing size of llms. hence, the push for parameter-efficient fine-tuning methods is clear, offering the advantage of training a smaller, adaptable extension to the base model and substantially lowering the memory footprint for adaptation. techniques like low-rank adaptation (lora) @xcite , which efficiently reparameterizes weight matrices, lead this initiative by providing a feasible technique to fine-tuning of llms in various applications. combining quantization with low-rank adaptation techniques, such as qlora @xcite , lessen the memory footprint for fine-tuning llms by melding parameter-efficient fine-tuning (peft) with model compression, especially ptq. yet, this approach encounters notable hurdles with aggressive low-bit quantization scenarios like 2bits, where quantization errors frequently cause significant accuracy drops. despite efforts by methods like loftq and lq-lora to overcome these challenges through strategic initialization, issues persist at lower bit levels, emphasizing the need for more sophisticated solutions that preserve both memory efficiency and model accuracy during the fine-tuning of quantized llms. our investigation reveals that quantization error in llms inherently opposes low-rank adaptation. this resistance is especially pronounced under aggressive low-bit quantization, attributed to the high-rank subspace of the quantization error. importantly, we discovered that vectors with small singular values play a pivotal role in the finetuning process for correcting quantization errors, hindering effective, parameter-efficient error compensation via low-rank adaptation. moreover, our study indicates that the characteristics of quantiza-tion errors in rank subspace vary depending on their location within the model, specifically in weights of feedforward networks and output projections, and differ across layer numbers. based on this understanding, we introduce a novel, parameter-efficient, quantization-aware finetuning method known as rank-adaptive lora (ra-lora). this method dynamically adjusts the adapter's rank based on rank-subspace analysis to effectively counter quantization errors with fewer parameters during fine-tuning. our evaluation of ra-lora on advanced llms like deberta-v3 and llama-2 for 2-bit parameter-efficient finetuning demonstrates its capacity to significantly enhance model accuracy with a lower parameter count, highlighting ra-lora's superiority over existing quantization-aware fine-tuning approaches. ## challenges despite previous efforts in quantization-aware lora @xcite @xcite to bridge the accuracy gap, 2-bit quantization still results in significant accuracy degradation. to gain fundamental insights into this challenge, we change various factors of quantizationaware lora, such as bit-precision (2∼4 bits), adapter ranks (0∼4096), and the number of alternating steps (1∼5), and explore model weight discrepancy and model accuracy degradation. we first found that the quantization error significantly increases at lower bit-precisions (2-bit), and cannot be adequately compensated by the lora adapters, even with previously proposed initialization methods. fig. 1 (a) investigates the discrepancy between the quantized and the original pre-trained weights in terms of the frobenius norm across the layers. notably, the adapter initialization decreases the discrepancy over the steps (from a 1 b 1 to a 5 b 5 ), but the reduction is marginal compared to the discrepancy between 2-bit and 3-bit quantization errors. additionally, fig. 1(b ) examines the effects of raising the adapter rank and reducing the quantization group size, utilizing loftq initialization. it highlights a notable discrepancy in all 2-bit quantization scenarios when the rank size is set to 64, a common default rank size for many quantization-aware lora methods @xcite @xcite . the observed significant quantization errors in 2-bit quantization directly lead to substantial accuracy degradation during fine-tuning. this is evidenced by the notable gap in perplexity when compared to results from 3-bit and 4-bit fine-tuning, particularly at lower adapter ranks, as demonstrated in fig. 1(c ). despite adapter initialization (2-bit loftq), higher adapter ranks only modestly narrow this gap toward full-precision model performance. this indicates that 2-bit quantization errors are not inherently low-rank, and attempting to fit these high-rank errors within a low-rank subspace leads to considerable accuracy losses. to recap, without adapter initialization, such as in qlora, even with increased ranks, the accuracy gap remains significant. loftq's method of initializing quantization errors offers some mitigation, but the accuracy gap fails to reduce substantially when adapter ranks are low, underscoring the need for addressing the high-rank nature of quantization errors to improve accuracy in 2-bit quantized fine-tuning. ## analysis in this section, we explore the effects of adjusting adapter ranks and analyze the learning behaviors of low-rank adapter weights through decomposition into singular values and vectors. this analysis aims to understand how high-rank quantization errors can be compensated within a constrained subspace. these findings motivate us to devise a novel rank adaptation technique that reduces the number of parameters while enhancing fine-tuning accuracy. ## experiments we evaluate the effectiveness of our ra-lora approach through comparative analysis of taskspecific fine-tuning performance on both natural language understanding(nlu) and natural language generation(nlg) tasks. for nlu, we utilize the glue @xcite dataset, focusing on specific tasks (cola, rte, mrpc, stsb) that exhibit significant accuracy drops in 2-bit quantization-aware lora @xcite . for nlg, we measure perplexity (ppl) using the language modeling benchmark of wikitext @xcite , accuracy with language generation on gsm8k @xcite , and zero-shot reasoning accuracy using the commonsense qa tasks @xcite @xcite . the models employed are deberta-v3 @xcite and llama-2 @xcite . baseline. the performance of ra-lora is evaluated against state-of-the-art quantization-aware lora methods. this includes qlora @xcite and loftq @xcite , which finetunes fp16 adapters on 2-bit quantized pre-trained weights and qa-lora @xcite , whcih merges adapter weights with quantized weights while ensuring that the entire weight configuration remains within the quantized space, enhancing efficiency during inference. trainable parameter ratio. for rigorous comparison, we include the trainable parameter ratio alongside performance metrics. this approach allows us to directly compare the efficiency and effectiveness of our rank adaptation method against static rank approaches. in evaluating rank allocation efficiency, we present the percentage of learnable adapter parameters to the total model parameters. note that our baselines are applying fixed rank 32 in the encoder-only model and 64 in the decoderonly model. (further detailed training settings are provided in appendix a.1.) ## conclusion this paper presents rank adaptive low-rank adaptation (ra-lora) as an innovative solution to the high parameter and memory challenges of deploying large language models (llms). ra-lora adjusts adapter ranks through rank-subspace analysis, significantly mitigating quantization errors in low-bit scenarios and outperforming traditional techniques. our findings reveal that ra-lora boosts accuracy in 2-bit fine-tuning of cutting-edge llms while reducing the number of necessary trainable parameters, marking a substantial leap in quantization-aware fine-tuning by emphasizing the critical role of rank subspace dynamics.