A lpha T uning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models (2022)

Abstract: There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better inference efficiency through model compression has not been thoroughly explored yet.Model compression could provide the benefits of reducing memory footprints, enabling low-precision computations, and ultimately achieving cost-effective inference.To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task.Specifically, AlphaTuning works by employing binary-coding quantization, which factorizes the full-precision parameters into binary parameters and a separate set of scaling factors.During the adaptation phase, the binary values are frozen for all tasks, while the scaling factors are fine-tuned for the downstream task.We demonstrate that AlphaTuning, when applied to GPT-2 and OPT, performs competitively with full fine-tuning on a variety of downstream tasks while achieving >10x compression ratio under 4-bit quantization and >1,000x reduction in the number of trainable parameters.

## introduction self-supervised learning facilitates the increased number of parameters to construct pre-trained language models (plms) (e.g., @xcite ; @xcite ). we expect the continuation of model scaling of the plms, especially for the transformers @xcite , because their general capability follows the power-law in parameter size, exhibiting "the high-level predictability and appearance of useful capabilities" @xcite . therefore, the transformer-based plms have been studied with great enthusiasm for various applications including natural language processing @xcite @xcite @xcite @xcite , automatic speech recognition @xcite , and computer vision @xcite . despite the impressive zero or few-shot learning performance of plms, additional adaptation steps (e.g., fine-tuning on a target task) are required to further enhance performance on downstream tasks. since each downstream task needs to load/store independent adaptation outcomes, if we aim to deploy multiple instances of distinct tasks, adapting plms with limited trainable parameters is crucial for the efficient deployment @xcite . thus, various parameter-efficient adaptation techniques, such as adapter modules @xcite , low-rank adaptation @xcite , prefix-tuning @xcite , prompt tuning @xcite , and p-tuning @xcite , are proposed. although trainable parameters can be significantly reduced by parameter-efficient adaptation schemes, we notice that the memory footprints for inference are not reduced compared to those of plms foot_0 . to enable efficient deployments of multiple downstream tasks, we incorporate model compression and parameter-efficient adaptation. we argue that previous model compression techniques were not practical solutions in terms of parameter-efficiency for adaptations. for example, quantization-aware training (qat) @xcite can perform full finetuning coupled with model compression; however, each task needs dedicated memory storage as much as that of a compressed plm. our key observation to achieve a compression-aware parameter-efficient adaptation is that, once a plm is quantized, only a small amount of quantization-related parameters is needed to be fine-tuned for each target task. as a result, both the overall memory footprints and the number of trainable parameters for adaptation can be substantially reduced. figure 1 illustratively compares two different approaches enabling both model compression and parameter-efficient adaptation. fine-tuned and quantized lms can be achieved through a;c;d or a;b;d as shown in figure 1 . in the case of a;c;d, we may have a large number of trainable parameters, and/or ptq may degrade performance on downstream tasks. to address such issues, we investigate a;b;d scheme, called "alphatuning" in this work. specifically, we factorize the parameters of large plms into binary values and scaling factors. then, alphatuning conducts the adaptation by training only the scaling factors that occupy a small portion in the quantization format, while freezing the other binary values. note that, to conduct a;b, we consider post-training quantization (ptq) @xcite @xcite because the qat demands significant computational overhead for training from a scratch with the whole dataset. in this paper, our contributions are as follows: parameter-efficient adaptation of lms adapting language models efficiently for a task and domain-specific data has been at the center of the community's interests since the emergence of largescale language models. one promising approach is in-context learning (icl) @xcite , in which the language model learns and predicts from the given prompt patterns. as the technique elicits reasonable few-shot performances from the large-scale language models without parametertuning, a plethora of works @xcite @xcite have investigated the underlying mechanism and proposed various methods to further exploit this approach. another class of techniques is to adopt external or partially internal parameters such as continuous prompt embeddings to enable parameter-efficient lm adaptation, which is based on the intuition that specific prompt prefixes may better elicit certain lm behaviors. earlier works explored the discrete prompt token space @xcite , but later work showed that optimizing on the continuous word embedding space yielded better results @xcite @xcite , even performing on par with full fine-tuning @xcite . another similar line of works explored introducing new parameters within the transformer blocks or partially training existing parameters @xcite @xcite . finally, some works have suggested unifying all existing approaches related to parameter-efficient fine-tuning @xcite . ## quantization for alphatuning enterprise-scale lms, such as 175b gpt-3, face challenges in the prohibitive cost of massive deployment mainly resulting from their huge parameter size. to facilitate cost-effective lms by alleviating memory requirements without noticeable performance degradation, we can consider compression techniques, such as quantization @xcite , pruning @xcite , and low-rank approximation (n. @xcite . memory reduction by model compression is also useful to reduce latency because memory-bound operations dominate the overall performance of lms with a small batch size @xcite . in addition, model compression can save the number of gpus for inference because gpus present highly limited memory capacity @xcite . in this work, we choose quantization as a practical compression technique because of its high compression ratio, simple representation format, and the capability to accelerate memorybound workloads @xcite ). let us discuss our quantization strategy for lms (see more details in appendix c). we choose nonuniform quantization since uniform quantization demands aggressive activation quantization (to exploit integer arithmetic units) which is challenged by highly non-linear operations (such as softmax and layer normalization) of the transformers @xcite . even though uniform quantization can mitigate performance degradation by frequent activation quantization/dequantization procedures @xcite or additional high- bcq format given a full-precision weight vector @xmath0, bcq format approximates w to be w ≈ q @xmath1, α ∈ r is a scaling factor to be shared by g weights, and @xmath2, +1} g is a binary vector. note that g represents a group size or the number of weights sharing a common scaling factor. thus, g is a hyper-parameter for quantization. when @xmath3, α and b can be analytically determined to minimize the mean squared error (mse). if @xmath4, however, α and b need to be obtained by heuristic methods such as greedy approximation @xcite and iterative fine-tuning method @xcite . for a weight matrix @xmath5, row-wise quantization (i.e., @xmath6: lower g results in slightly increased weight size after quantization. where α @xmath7, b @xmath8, +1} hout×h in , and diag(•) denotes the function of a vector that outputs a zero-matrix except for the vector elements in its diagonal. a linear operation of @xmath9, then can be approximated as follows: figure 2 describes the row-wise bcq examples based on greedy approximation @xcite when q varies. note that increasing q and/or decreasing g can reduce the mse after quantization at the cost of a lower compression ratio. transformer quantization ## experimental results to extensively demonstrate the influence of al-phatuning, we apply detailed adaptation techniques and hyper-parameter selections that we explored by using gpt-2 models on webnlg (in the previous section) to additional downstream tasks and opt models @xcite . ## conclusion in this paper, we proposed alphatuning as the first successful compression-aware parameter-efficient adaptation method for large-scale lms. through a few representative generative lms (such as gpt-2), we demonstrated that once linear layers are quantized by bcq format, training only scaling factors can obtain reasonably high scores. we also empirically proved that quantizing an already adapted lms would degrade scores significantly. incorporating various model compression techniques and parameter-efficient adaptation methods would be an interesting research topic in the future.