BitDelta: Your Fine-Tune May Only Be Worth One Bit

Authors: james  liu
guangxuan  xiao
kai  li
jason d lee
song  han
tri  dao
tianle  cai
  mit
princeton  university
together  ai
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 01:47:06

Abstract: Large Language Models (LLMs) are typically trained in two phases: pre-training on large internet-scale datasets, and fine-tuning for downstream tasks. Given the higher computational demand of pre-training, it is intuitive to assume that fine-tuning adds less new information to the model, and is thus more compressible. We explore this assumption by decomposing the weights of fine-tuned models into their pre-trained components and an additional delta. We introduce a simple method, BitDelta, which successfully quantizes this delta down to 1 bit without compromising performance. This interesting finding not only highlights the potential redundancy of information added during fine-tuning, but also has significant implications for the multi-tenant serving and multi-tenant storage of fine-tuned models. By enabling the use of a single high-precision base model accompanied by multiple 1-bit deltas, BitDelta dramatically reduces GPU memory requirements by more than 10x, thus reducing per-user generation latency by more than 10x in multi-tenant settings. We validate BitDelta through experiments across Llama-2, Mistral and MPT model families, and on models up to 70B parameters, showcasing minimal performance degradation in all tested settings.

Main paper: 
 introduction: 
 After large-scale pretraining, foundation models are typically fine-tuned for specific downstream tasks [16, 43, 44] . This pretrain-finetune paradigm has revolutionized machine learning; LLMs have not only proven effective for critical tasks such as instruction following and alignment [39] , but are also performant on a wide array of niche yet highly impactful applications [61, 42] . Through fine-tuning, LLMs are adeptly equipped to align with distinct user preferences or specialized task requirements, showcasing an unprecedented level of adaptability. Thus, the prospect of serving millions of uniquely fine-tuned models, each tailored to individual tasks and user needs, presents a promising vision for the future of machine learning.Realizing this vision is challenging due to two key reasons: 1) Expensive Storage. Each new fine-tuned model is large, even if we have relatively few base models, making them expensive to store and challenging to manage on disk. 2) Expensive Serving. Distinct fine-tuned models each demand significant GPU memory, making it difficult and expensive to concurrently serve such models without noticeable downtime. To tackle these issues, we decompose the fine-tuned model weights into the weights of the base pre-trained model and a delta induced by the fine-tuning process. By compressing this delta while maintaining model performance, we aim to sidestep the prohibitive costs associated with storage and GPU memory demands.From the delta decomposition point of view, parameter-efficient fine-tuning (PEFT) methods like LoRA [25, 24, 46, 15, 9] effectively enforce a highly structured and compressed form of delta during fine-tuning, a powerful insight for model serving of PEFT-based fine-tunes. Sheng et al. [49] and Chen et al. [7] explore multi-tenant serving of LoRA-based fine-tunes.Figure 1 : Overview of BitDelta. BitDelta applies 1-bit quantization to the weight delta between fine-tuned and base models. For each weight matrix, we quantize its delta as its sign bits and a trainable high-precision scale factor. The scale factor is initialized to achieve the best approximation error in L 2 norm and further refined with a few distillation steps. BitDelta shows minimal degradation in model performance and reduces memory consumption in multi-tenancy serving by representing multiple fine-tuned models with a single high-precision base model and multiple 1-bit deltas.Nevertheless, recent work has shown that PEFT methods may not yet match the model quality of full parameter fine-tuning, especially on high resource tasks [6] , and are fairly sensitive to hyperparameter choice and prompting methods [38] . Biderman et al. [2] show that LoRA's reduced expressivity, although providing desirable regularization, leads to significantly worse performance compared to full fine-tuning in math and programming tasks. As a result, we notice that among the 2307 LLMs (as of time of writing) on the Open LLM Leaderboard [1] with a valid README file, only < 20% indicate that they exclusively use LoRA. Most models are full parameter fine-tunes, model merges [64, 28, 59 ] of full parameter fine-tunes, or model merges of LoRA based fine-tunes (which are effectively high-rank). It is also attractive to approximate general deltas with low-rank matrices post-training (in particular, post-fine-tuning). However, experimental results show that this is challenging (Table 1 ), as deltas from full parameter fine-tunes tend to be fairly high-rank (Figure 2 ). We instead draw from the insight that motivates PEFT methods in general: Given the higher computational demand of pre-training, it is intuitive to assume that fine-tuning adds less new information to the model, and is thus much more compressible. In fact, we find that we can efficiently quantize the delta to merely 1 bit with almost no performance drop. We propose BitDelta, an efficient post-training quantization (PTQ) solution that acts on the weight delta between a finetuned model and its underlying base model. BitDelta consists of two stages: 1) We quantize the delta between a fine-tuned model's weight matrix and base model's weight matrix into a scaling factor multiplied by a binary matrix. Specifically, we Table 1 : Comparison between BitDelta and a SVD based method, with Llama 2-7B and Llama 2-7B Chat as the base and fine-tuned models. BitDelta is performant across the board, whereas the SVD-based method fails to sufficiently capture the fine-tuned information. BitDelta creates opportunities to efficiently serve multiple fine-tuned models with shared servers: By only storing a single full-precision base model, and (dynamically) loading and performing batched inference over multiple 1-bit deltas, we can efficiently represent multiple fine-tuned models. Compared to naively using full precision fine-tuned models, deltas compressed by BitDelta are more than 10× smaller, and can therefore be loaded faster. This addresses the storage challenge. Moreover, since LLM inference is memory-bound [32, 5, 3] , the latency of each decoding step is proportional to the GPU memory consumption of the model weights. With an efficient CUDA kernel implementation, we can translate this memory reduction into a latency reduction, similar to other quantization methods [19, 33] . Using the W IN T 1 A F P 16 kernel from BitBLAS [58] , we improve the multi-tenant serving latency of full-parameter fine-tuned models by more than 10×.Finally, we study a few extensions of BitDelta, where we quantize the base model and where we iteratively apply BitDelta. Experimental results show that our method is quite general and can be applied to various use cases.2 Related Work 
full model compression: 
 Quantization. Quantization techniques are widely used to reduce memory consumption and improve LLMs' generation latency. Xiao et al. [60] implement a technique that rescales between activations and parameters, effectively mitigating outlier activations to facilitate smoother quantization. Dettmers et al. [14] develop an approach that decomposes matrix multiplications into 8-bit computations, with an additional 16-bit process for handling outliers. Exploring further, Frantar et al. [19] introduce a method that iteratively rounds weight columns to 3-4 bits of precision. Similarly, Lin et al. [33] propose an activation-aware quantization scheme that selectively preserves crucial weights while compressing the majority to 3-4 bits. Kim et al. [29] devise a sparse, low-precision pattern focusing on a small yet significant set of weights. Chee et al. [4] utilize incoherence processing to quantize model weights to as low as 2 bits with minimal impact on performance.Pruning. Pruning also aims to reduce the memory consumption of neural networks. It accomplishes this by pushing certain parameter values to zero, inducing sparsity in the model [31, 21, 22, 67] . However, these methods may fail to take advantage of modern hardware like GPUs unless using certain structured sparsity patterns like 2:4 (50%) sparsity [36] . Frantar and Alistarh [18] demonstrate a pruning method on LLMs that successfully utilizes the 2:4 sparsity pattern and achieves a 50% sparsity ratio. It is challenging to obtain higher sparsity while being hardware-friendly.Early work on post-training delta compression. Most related to our work, a few studies explore the idea of post-training delta compression by adopting existing compression techniques like GPTQ, unstructured pruning [22] , or even classic lossless compression algorithms. Isik et al. [26] focus on reducing the delta size to save storage. Yu et al. [64] utilize pruning to improve model merging applications. Yadav et al. [62] reduces the size of PEFT modules to save storage. Ryu et al. [47] combines quantization with a low-rank approximation to reduce the delta size. The concurrent and independent work by Yao and Klimovic [63] also explores using delta compression to improve multitenant serving, but focuses more on reducing the model loading time from disk to GPU. Compared to existing work, we offer a much simpler and faster method, BitDelta, achieving a compression ratio of more than 10× while also being friendly to modern accelerators.3 BitDelta 
method: 
 BitDelta consists of two stages: 1) We quantize each weight matrix into a scalar multiplied by a binary matrix † . 2) We further calibrate the scalar factors using model distillation. We describe each stage in this section:1-bit quantization. Let W base , W fine ∈ R n×m be weight matrices from the base model and finetuned model respectively. We define the weight delta as ∆ = W fine -W base , representing the modification in weights post-fine-tuning. For efficient representation of this weight delta, we aim to obtain a binarized estimator by encoding its sign bits, denoted as ∆:EQUATIONwhereEQUATIONand α is a high-precision scaling factor for the entire matrix. To minimize the quantization error of ∆ in L 2 norm:EQUATIONwe initialize α as follows:EQUATIONSurprisingly, we find that the above quantization approach already does quite well and retains most of the fine-tuned models' performance.Scale distillation. The scaling factor α intuitively plays a more significant role in the low-bit regime. Additionally, per-matrix L 2 weight error is not a perfect measure of degradation in overall model quality. We further optimize these scales by performing model distillation to align the output logits of the quantized model to that of the original fine-tuned model. More concretely, we freeze the model weights and optimize for the following objective:EQUATION(5) † In our experiments, we only quantize the linear layers in the Transformer blocks as they contribute the majority of the parameters and computation. where X is a calibration dataset, and Z(•) are the logits of the respective models. Scale distillation is fairly robust to choice X, as 1) the process is extremely parameter efficient, and 2) the crucial aspect of the process is to logit match with the fine-tuned model, regardless of the actual text content.For our experiments, we distill on the C4 dataset [45] , consisting of generic internet data, using 800 samples of length 128. We use the same subset of C4 over all models to control for seed-based variations. We use the Adam optimizer [30] with lr = 10 -4 , β = (0.9, 0.999), ϵ = 10 -8 . 1x80 GB A100 GPU is used to distill 7B and 13B models, and 6x80GB A100 GPUs are used to distill 70B models (2x for finetune, 4x for binarized). Scale distillation is fast; we can compress 70B models in roughly 10 minutes.Compared to full parameter and parameter efficient fine-tuning methods, BitDelta is extremely cheap. While fine-tuning methods require training thousands to millions of parameters, BitDelta only necessitates training a single parameter per weight matrix. Moreover, BitDelta operates efficiently with input sequences of length 128, unlike fine-tuning methods that demand longer sequences to saturate the context window (4k, 8k, etc.). Crucially, BitDelta requires only 200 training steps (assuming a batch size of 4), which is significantly less compared to the 10000-1000000 steps at higher batch sizes needed by fine-tuning methods. Thus, in terms of methodology cost, we liken BitDelta more to post-training quantization (PTQ) schemes like GPTQ [19] and AWQ [33] , rather than full parameter or parameter efficient fine-tuning, while being faster than most PTQ schemes.The ability to compress the delta to merely 1-bit opens up multiple opportunities for improving efficiency, enabling more effective model storage [26] -where a single base model can be maintained alongside multiple compressed deltas -and facilitating model hot-swapping [7, 49] . With hotswapping, the base model remains in GPU memory, and compressed deltas are dynamically loaded in accordance to incoming requests. In both cases, the compression ratio can be directly translated into reductions in storage needs and loading times.Moreover, BitDelta enables the possibility of a multi-tenant serving system like Punica [7] or S-LoRA [49] but for general fine-tuned models instead of just LoRA models. Concretely, we consider the scenario where multiple models fine-tuned from the same base model are served with the same server. This setting greatly exploits the GPU resource and saves each fine-tuned model's inference cost when their traffic is low or unbalanced. With BitDelta, we can keep one high-precision base model with multiple compressed deltas in the GPU memory. Compared to directly serving multiple fine-tuned models, this approach greatly saves memory consumption.Since LLM inference follows the memory-bound computation pattern where the generation latency is proportional to the GPU memory used by the model weights, the lower memory consumption also suggests the opportunity to improve the serving latency. For example, Punica and S-LoRA exploit LoRA's structure and memory saving by computing the activation product between the shared base weight, and low-rank fine-tuned delta weights separately. Similarly, we decompose the forward pass of each linear layer as follows:X ′ i = W fine,i X i ≈ W base X i + ∆i X i Kernel (6)where X i and X ′ i represent input and output features to the i-th fine-tuned model, and the base model weight and the 1-bit delta are computed separately. For a batch of requests, W base X i can be computed with the classic batched GEMM kernel. We utilize the BitBLAS [58] W IN T 1 A F P 16 kernel that allows us to calculate ∆i X in a batched setting while keeping the 1-bit deltas quantized until they are transferred to the GPU cache. This kernel fuses the dequantization operation with the GEMM calculation, reducing the data moving overhead by a large factor.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: The paper does not include theoretical results. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
setup: 
 Baselines. Our primary baselines are the original fine-tuned models without compression. We also compare with 8-bit RTN, 4-bit GPTQ [19] , and 2-bit QuIP# [54] on evaluations where we run BitDelta on quantized base models.Models and datasets. We benchmark fine-tuned models based on the Llama-2 [53] , Mistral [27] , and MPT [51] model families: Vicuna, Xwin-LM, Solar-70B, Zephyr, OpenChat 3.5, Dolphin 2.2.1, and OpenOrca [10, 52, 56, 55, 57, 23, 37] . We evaluate on eight tasks: MT-Bench, 25-shot ARC Challenge, 5-shot BBH, 10-shot HellaSwag, zero-shot TruthfulQA, zero-shot LAMBADA, zero-shot Table 4 : Comparison of model responses from Zephyr-7B-β for Question 9 in MT-Bench, a concise advertisement task. BitDelta-Initial is unable to follow the instructions, producing an advertisement that is overly formal and makes no attempt to adhere to the word limit. With the addition of scale distillation, BitDelta successfully produces a concise, catchy advertisement slightly over the word limit. *Prompt slightly modified for clarity.Please create a slogan about bioenergy that satisfies the following: 1. Make the tone sound casual 2. Embed an advertisement for a company called "FlexPower" 3. Fewer than 10 words.BitDelta "FlexPower's groundbreaking bioenergy solution: Produce fuel without compromising food production!" (10 words) This headline is catchy, casual, and highlights the unique selling point of the... GPT-4 Score: 8 BitDelta-Initial "FlexPower's groundbreaking technology unlocks the key to harnessing the power of renewable bioenergy while minimizing ethical concerns" GPT-4 Score: 4Winogrande, and 5-shot GSM8K [66, 12, 50, 65, 34, 40, 48, 13] . We use FastChat [66] to evaluate on MT-Bench, and use lm-evaluation-harness [20] to evaluate on the other tasks. We denote our methodology before scale distillation is applied as BitDelta-Initial.We primarily focus on high-margin metrics where fine-tuning is significantly impactful and aggregate the other metrics. See Tables 7 to 10 in the Appendix for full results. BitDelta performs quite well on the aggregated metrics, even outperforming the baseline in many cases. However, it's important to contextualize these results with regard to the base model itself, which is also performant on these metrics. It's difficult to attribute performance to our methodology or to the underlying base model in such cases. Because of this, we highlight TruthfulQA, GSM8K, and MT-Bench, which base models tend to struggle on, to show that BitDelta accurately preserves fine-tune information. SVD comparison. We compare BitDelta to a low rank approx. of the weight delta on Vicuna-7B v1.5. For the low rank approx., we decompose ∆ = U ΣV and approximate ∆ = AB where A = U Σ, B = ΣV . During distillation, we treat all entries of the low rank matrices as trainable parameters. We compare against two settings: r = 16 (most commonly used) and r = 128 (memory equivalence with Bit-Delta). We find that the low rank approx. fails to fully capture the fine tune information, and underperforms across the board (Table 1 ). In particular, the low rank approx. heavily underperforms on MT-Bench [10] , a difficult multi-turn instruction following dataset fairly indicative of real world performance. Interestingly, distillation is not as effective for the low rank approx. compared to BitDelta.BitDelta is performant across various model families, across a wide range of model sizes, and across many fine-tuning techniques. We benchmark on Llama-2, Mistral, and MPT, families, and on models ranging from 7B to 70B parameters. Shown in Table 2 , we find that BitDelta is very general and can recover all types of finetune information, including SFT-based methods [43] on Mistral-7B v0.1 Instruct, RLHF-based methods [11] on Llama 2 Chat, and context extension methods (RoPE scaling) [8, 41] on Vicuna-7B v1.5 16k.We note that GSM8K for BitDelta-Initial on Mistral-7B v0.1 Instruct and Zephyr-7B-β is abnormally high; we attribute this to how performant the base model Mistral-7B v0.1 is on this task in comparison. Scale distillation is effective, raising TruthfulQA and GSM8K scores to within 1-2 points of the baseline fine-tune, and generally raising MT-Bench scores to within 0.1-0.2 points. Case Study. We present a sample response from Zephyr-7B-β in Table 4 , highlighting the efficacy of scale distillation. BitDelta-Initial does not have a casual tone, and makes no attempt to adhere to the word limit. With the introduction of scale distillation, BitDelta exhibits greater instruction following capabilities, producing a catchy response that slightly exceeds the word limit.Quantized base models. Because 8-bit RTN, GPTQ, and QuIP# work with 16-bit activations, we can keep the fine-tune weights W fine and scaling factors α in high precision in the compression process, only quantizing the base weights W base . As shown in Table 6 , we find that BitDelta is still performant when applied to quantized base models. Ablation over fidelity of ∆.By successively applying BitDelta, treating the compressed model from the previous iteration as our base model, we can vary the granularity over the delta, associating it with multiple 1-bit masks. One advantage of doing this is the ability to assign arbitrary scale factors to each 1-bit mask. In contrast, when increasing the bit size, scale factors are implicitly fixed with respect to each other. Figure 3 shows how the TruthfulQA of Llama 2-7B plus an increasingly granular delta approaches that of Vicuna-7B v1.5. Full results are in Table 9 .For simplicity, we consider the setting where each model receives one distinct request simultaneously. It would be insightful to develop more sophisticated serving systems, which we leave to future work. Following the decomposition in Eq. ( 6), the W IN T 1 A F P 16 kernel is used to compute the batched matrix multiplication between B binary matrices (N × M ) and B high-precision activations (L × N ) where N, M are intermediate dimensions and L is the sequence length. We focus on decoding latency which dominates runtime, as opposed to prefill latency. Tokens are generated one by one when decoding, meaning L is always 1. For all latency experiments we use a single A100 80GB with power limit set to 500W.Kernel latency. We benchmark the decoding latency of our kernel, a batched linear operation over multiple 1-bit deltas, corresponding to the delta component of Eq. (6) . We compare this to the S-LoRA kernel, a batched linear operation over multiple low-rank deltas, and also compare this to the base weight backbone shared over all deltas. We set r = 128 for S-LoRA, to maintain memory equivalence with BitDelta at N = M = 4096.We profile the latency of the backbone (W base X) and deltas (∆X) separately. Although X's memory footprint scales with batch size, it is negligible compared to W base , which remains constant. For typical low to medium batch settings, which is typical for B × N ≪ N × M . In such settings, the overall memory footprint of the backbone is effectively independent of batch size, as shown in Figure 4 (left). This is in contrast with that of the deltas, which scales with the batch size, as each BitDelta underperforms slightly compared to S-LoRA in large-batch settings as the LoRA kernel is highly optimized for GPU. We emphasize that closing or even surpassing the gap is tractable. For example, Ma et al. [35] point out that W IN T 1 A F P 16 requires no multiplication operations and that new hardware can be co-designed with this in mind to drastically reduce energy/latency costs. End-to-end latency. We benchmark the end-to-end decoding latency on Llama 2-7B variants with an input length of 128 (we find the decoding latency is less sensitive to the input length), ablated across batch size. For BitDelta and S-LoRA, the forward pass consists of the addition of two components: a single backbone pass (batch independent) and a delta pass (scales with batch size).We compare BitDelta and S-LoRA with a naive method that computes each W i X i separately in the forward pass. This naive approach scales poorly with batch size as it effectively maintains a separate backbone (W i ) for each client in the batch. Given the substantial memory footprint of the backbone, this leads to significant memory usage as batch size increases. In contrast, BitDelta and S-LoRA share a single backbone across all clients in the batch, with only the 16× smaller deltas scaling with batch size. This allows for more efficient memory utilization and better performance at larger batch sizes.We find that BitDelta and S-LoRA introduce overhead when the batch size is low. However, BitDelta and S-LoRA scale better and successfully translate the saved GPU memory to improved decoding latency, starting at B = 2. This is exacerbated at larger batch sizes, where the naive approach succumbs to out-of-memory issues and BitDelta and S-LoRA are still performant. In the B ≥ 16 regime, used in modern serving solutions, BitDelta has a >10× lower per-user decoding latency than the naive method.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: The paper fully discusses the BitDelta methodology on an algorithmic level, and how to replicate the experiments in Section 4. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
conclusion: 
 We propose BitDelta, a simple but effective approach to efficiently quantifyings the weight delta arising from the fine-tuning of LLMs down to 1 bit. BitDelta encodes the sign bits of the weight delta and a per-weight matrix scaling factor, which is calibrated further through distillation. This allows for representing multiple full-parameter fine-tuned models with one base model and multiple 1-bit deltas, enhancing applications in multi-tenancy serving by reducing GPU memory requirements and improving generation latency. BitDelta is fast and accurate, showcasing minimal performance degradation, and opens new avenues for efficient model deployment and resource utilization in machine learning. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The paper provides the source code to reproduce the main results.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: The paper provides training details in Section 3.1.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: The paper does not report error bars / statistical significant tests as of now.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We have cited the creators/original owners of assets the paper uses/references. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: The source code associated with the paper is well documented. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: The reviews are mostly favourable, although varying in their level of support for the paper. A couple of reviewers misunderstood initially the main idea and I urge the authors to pay special attention to this aspect when revising the paper. Having said that, after reading through the paper myself I did not see any parts where the paper would be impossible to understand so I do not think any misunderstandings like those could be a base for rejection. To make it more evident, both reviewers dXr8 and dvPC revised their scores after the rebuttal (although dXr8 did so silently), suggesting there are no fundamental flaws in that regard in the paper.

Only reviewer b1KL kept their score and mentioned the paper should make it clearer that its focus is on serving, and that evaluation needs to be redesigned - however, it is not very clear to me what the reviewer here had in mind. With regard to the positioning of the paper, as mentioned above, although there might be some room for improvement I do not think it warrants rejection. With regard to evaluation, the biggest shortcoming seems to be related to the fact that the proposed method only makes sense in a multi-tenancy setting, but strictly speaking no such case has been evaluated. While this might see as a serious shortcoming, given its relevance for the central point of the paper, it is important to note that the current results are sufficient to infer the expected performance in a multi-tenancy setting. More specifically, the criticism is related to the memory consumption (dXr8, b1KL) and latency (my own comment).

With respect to the memory, while Table 5 provides the core information needed to understand and validate benefits of the method, the fact the authors do not provide any end-to-end memory, especially while varying the number of fine-tuned models, is a notable omission and should be fixed.

With respect to latency, I find the current comparison to be very confusing. After carefully reading the entire section multiple times, I do not think there is any serious mistake there, but it is unnecessarily difficult to understand. For example: "batch size" referring to the number of models is very confusing, referring to only the int1-fp16 part of the method as BitDelta in Figure 4, but encompassing the entire W_baseX + deltaX in Figure 5 does not help, etc. (in general, it took me much longer than reasonably needed to understand why BitDelta vs. Backbone in Figure 4 exhibits the exact opposite trend to BitDelta vs. Naive in Figure 5). I would urge the authors to rewrite this section thoroughly.

Having said that, both of the aspects above seem to be matters pertaining more to the presentation of the current results rather than omission of important experiments needed to support claims made by the paper. Given that and the fact that, after the rebuttal, overall motivation or contributions do not seem to be questioned by the reviewers, who largely lean towards acceptance, I also recommend acceptance. Having said that, I do trust the authors to revise their paper accordingly to the feedback from the review process, as it seems there is many minor places that should be improved before publication.

Reviews: review: paper_summary: paper_summary: The paper proposes to quantize the weight delta of a fine-tuned LLM to 1-bit and observes that the model quality only drops a little.
During the binarization step, it requires calibrating the scaling factor with a few hundreds of distillation steps. This is less than a full fine tuning. Evaluation show that the proposed method produces higher model quality than other post-training quantization techniques such as GPTQ and QuIP.
strength_weakness: strength_weakness: While requiring low storage size, the proposed method introduces an extra binary-float matmul during inference. Although it is indeed a special kernel and can have much lower inference time than the float matmul, the overhead will become more significant when the base model is low-precision.
questions: questions: 1. Line 133 mentioned the scale distillation is robust to the choice of calibration dataset. Does it mean that it can also use synthetic data? It will be an improvement if the paper presents such an ablation.

2. Line 226: can the quantized delta be merged with the quantized base model? For example, by synchronizing the scaling factors.
limitations: limitations: The paper does not have negative societal impact as far as the reviewer can tell.
paper_summary: paper_summary: This paper introduces Bitdelta, a method that enables quantization with just 1 bit. The main idea is to compress the weight delta into a scalar and a binary matrix. The experimental results demonstrate that Bitdelta achieves better performance compared to other techniques.
strength_weakness: strength_weakness: - A significant contribution of Bitdelta lies in its ability to reduce memory consumption through 1-bit quantization. However, the paper lacks detailed evidence to support this claim. It is necessary to compare memory consumption for each compression technique, but currently, only Bitdelta is shown.
- It would be better for the paper to mention and explain the figures within the proper location of text. For example, Figure 1 and Figure 4 are included but not mentioned in the text. It is hard to understand without sufficient explanation.
- Table 1 lacks information about the base models for Bitdelta and SVD. It should clearly specify whether these are based on Llama-7B or Llama-7B Chat.
questions: questions: - I am curious about the clear differences between Bitdelta and other compression technologies. For example, can Bitdelta, which adopts the Post-Training Quantization (PTQ) method, be used in combination with other PTQ techniques? Also, what happens if you combine Bitdelta with compression techniques like pruning? It would be beneficial to discuss the relationships between various compression technologies
- (lines 165-167) The authors said that generation latency is proportional to the GPU memory used. How can this claim be proven? It would be helpful to mention references or provide supporting data
- I think BitNet has a similar purpose with the design of 1-bit quantization. Comparing Bitdelta to BitNet is required
limitations: limitations: I think the paper will be strengthened with the measurements on GPU (or memory) consumption are added
paper_summary: paper_summary: Aiming at storage and serving overhead caused by multiple finetuned LLMs for various downstream tasks, this paper proposes a memory-friendly model compression method namely BitDelta which binaries the delta of each weight matrix and uses self-distillation to learn optimal scaling factors.
strength_weakness: strength_weakness: The paper lacks innovation and has several points that do not hold up under scrutiny. 

For the first contribution, the paper proposes decomposing multiple fine-tuned models into a shared pretrained model and their respective deltas, and then binarizing the deltas. The specific decomposition method is not described in the paper. Based on experience, this can be understood as using LoRA to fine-tune LLMs and binarizing the learned low-rank matrices. This is not novel, as there are already existing low-bit model fine-tuning methods, such as Q-LoRA and QA-LoRA, which are more memory-friendly and do not require retaining an additional fp16 model.

Regarding the second contribution, which involves using self-distillation to learn scaling factors, the paper's description is insufficient. For example, the initialization method of these factors is not well-explained. Furthermore, if the entire fp model's output is used to supervise the quantized model, the computational resource consumption is high. The paper could explore layer-wise optimization mechanisms to address this issue.
questions: questions: 1) The author should describe the advantages of BitDelta compared to PEFT methods, such as QLoRA, QA-LoRA, and LoftQ. Because they don't require saving the fp16 pretrained model, which results in more efficient storage utilization. 
2) More detailed description should be given, such as GPTQ+Δ in Table 6. Additionally, the performance of FP16+Δ being superior to 4-bit GPTQ and 2-bit Quip does not entirely prove that directly quantizing the base model is impractical, as INT8-RTN still shows better performance. If we consider 8-bit GPTQ, or other sota PTQ methods such as Omniquant, AWQ, it might also perform better while reducing memory usage by half. Therefore, it is necessary to provide additional arguments from other perspectives to explain why directly quantizing the base model is not preferable. 
Extensive additional experiments are not necessary, but it is important to clearly explain the aforementioned issues.
limitations: limitations: The authors have addressed the limitations.
paper_summary: paper_summary: This paper introduces BitDelta which quantizes the aggregated weight updates (the authors call it “delta”) to 1-bit after full fine-tuning. The paper claims that the approach has two applications: 1. First, it shows that the delta is highly redundant and 2. It is useful in the multi-client-single-server applications where the high precision model will be saved on the server and each client only store it’s own 1-bit delta. They show that in this scenario, the generation takes up to 10x memory reduction (and similar in latency improvement).
strength_weakness: strength_weakness: 1, The author claim that BitDelta shows the potential redundancy of information added during fine-tuning. However, this is not a new finding and almost all PEFT approaches (for example LoRA) are based on this fact. 

2. It seems that the paper completely missed the full fine-tuning costs and just measured the memory/latency of the serving step. However, I would suggest to have “apple to apple” comparisons and compare the fine-tuning+serving cost (including memory and runtime) of “full-fine tuning+1bit optimization+serving” against “fine-tuning+serving” in PEFT schemes (like LoRA). 

3. In Table 6, the paper shows higher accuracy for FP+delta compared to GPTQ. Again, I would rather like to see memory Vs. accuracy tradeoff in such comparisons as a function of number of clients. The fact is FP+delta does not have the same memory as GPTQ (please current me if I am wrong).

4. It would be nice to have a performance model for the latency of the decoding as a function of client numbers. This is also missed and needs to be include as this is the main claim of the paper.
questions: questions: Please check the "weaknesses" section. I would be happy to discuss and change my score.
limitations: limitations: I think the authors should define and present the limitations of the method more clearly.

Comments: title: Thank the authors for response
Followup
Reply
comment: Thank the authors for the response. I have read all replies and comments from other reviewers.
We thank the reviewer for the response. We will make sure to properly position the paper in the final manuscript. Though, we are wondering if the reviewer could clarify how they think our evaluation approach should be re-defined. Our baselines (For accuracy, fine-tuned models without BitDelta applied. For latency, serving fine-tuned models separately.) are fairly reasonable in this context. Are there specific aspects that the reviewer thinks are misaligned?

Given that we have additionally addressed the other concerns (fine-tuning costs, latency results, etc.), we would greatly appreciate it if the reviewer could reconsider their score.
Thanks for your answers. The authors claim that they do not present a fine-tuning scheme, but this rather a "serving" approach. I think they should highlight this as the main message of the paper and re-defining the evaluation approach. I would stick to my current score.
- Thank you for your efforts in addressing the weaknesses and questions. Based on the authors’ responses, I understand where I was confused, and I have updated my review decision (borderline accept -> weak accept).
- Also, although the authors logically explained the memory consumption aspects, the actual memory consumption of the technique is affected by various factors; therefore, providing the actual numbers would be beneficial.