SQFT : Low-cost Model Adaptation in Low-precision Sparse Foundation Models (2024)

Abstract: Large pre-trained models (LPMs), such as large language models, have become ubiquitous and are employed in many applications. These models are often adapted to a desired domain or downstream task through a fine-tuning stage. This paper proposes SQFT, an end-to-end solution for low-precision sparse parameter-efficient fine-tuning of LPMs, allowing for effective model manipulation in resource-constrained environments. Additionally, an innovative strategy enables the merging of sparse weights with low-rank adapters without losing sparsity and accuracy, overcoming the limitations of previous approaches. SQFT also addresses the challenge of having quantized weights and adapters with different numerical precisions, enabling merging in the desired numerical format without sacrificing accuracy. Multiple adaptation scenarios, models, and comprehensive sparsity levels demonstrate the effectiveness of SQFT.

## introduction despite several limitations, such as hallucinations and a significant computational footprint, large pre-trained, foundation, or frontier models have become integral to numerous applications, including language understanding and code generation. these models are trained with extensive corpora on thousands of graphics processing units (gpus), resulting in outstanding zero-shot performance across various tasks and datasets. however, it is frequently the case that they must be adapted to improve their performance on new tasks or data. low-rank adapters (lora) @xcite ) have demonstrated their effectiveness in model adaptation. however, when lora is combined * co-first authors. sparsification lora frozen trainable unable to merge (loss of sparsity) quantization lora frozen unable to merge (different numerical precision) sparsification sparse full tuning expensive trainable trainable base model adapter sparse base model quantized base model figure 1 : limitations of existing approaches for finetuning sparse and quantized models. full fine-tuning is expensive. low-rank adapters (lora) for parameterefficient fine-tuning (peft) on sparse or quantized models cannot easily merge with the compressed weights due to loss of previously induced sparsity or different numerical precision. with model compression techniques, e.g., sparsity or quantization, several challenges prevent merging these adapters into a single compressed and fine-tuned model, as illustrated in figure 1 . these challenges stem from two primary reasons: i) merging dense adapters causes the loss of sparsity in the base model, and ii) adapter merging cannot be achieved due to different numerical precisions. this paper introduces sqft, an end-to-end compression and model adaptation solution for large pre-trained models (lpms) that alleviates the limitations above. sqft is designed to sparsify, quantize, and fine-tune large models and can instantiate efficient pipelines that streamline compression techniques. within the sqft framework, we propose sparse parameter-efficient fine-tuning (sparsepeft), a strategy to address the adapter merging problem for sparse and quantized models, resulting in more effective high-performing models. furthermore, sqft also benefits from weight-sharing techniques applied to traditional parameter-efficient fine-tuning (peft) techniques and incorporates insights from state-of-the-art compression techniques. throughout this paper, we discuss the following contributions: this paper is organized as follows: section 2 describes the stages in the proposed end-to-end solution, sqft. section 3 discusses sqft's evaluation, and we finalize with some concluding remarks in section 4. due to page limits, we include a related work section and additional results in the appendix. ## methodology sqft fine-tunes large pre-trained models (lpms) in an efficient multi-stage approach that includes (1) sparsification, with an optional reduction in the numerical precision, i.e., quantization, (2) fine-tuning with neural low-rank adapter search (nls), (3) sparse parameter-efficient fine-tuning (sparsepeft) with optional (4) quantizationawareness. figure 2 illustrates the alternative model compression and adaptation pipelines that were explored. in the following sections, we discuss the details of each stage and the benefits of accelerating inference and model serving. ## experimental results we evaluate sqft on several state-of-the-art large pre-trained models and datasets. next, we discuss the setup for our experiments. ## conclusion large pre-trained models often require fine-tuning to downstream target tasks and compression to utilize them in resource-constrained environments. this paper presents sqft, a low-cost fine-tuning solution for low precision and sparse foundation models. sqft solves challenges when merging sparse (and quantized) base models and dense