Paper: EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR-2024)

Reviewer comment (Novelty):
Novelty concern: I think the paper presents a combination of existing works (scale-aware optimization from LSQ, common distillation technique, and LoRA). Can the authors clarify the difference in contribution from the combination of existing works? If there is a difference, how does the performance differ compared to the combination?

Author response:
**Q1: Clarify the difference in contribution from the combination of existing works (scale-aware optimization from LSQ, common distillation technique, and LoRA).** 
In this paper, we propose a data-free, quantization-aware and parameter-efficient fine-tuning framework for low-bit diffusion models, which seamlessly integrates three tightly coupled components, QALoRA, scale-aware optimization and data-free distillation. 
Overall, our work has been well recognized by other reviewers, such as  "the idea of QALoRA is novel" (Reviewer 5Wot), "this paper introduces low rank adapter and distillation loss ...is the first to achieve very good performance..." (Reviewers ec6g) and "the proposed QALoRA is data-free and efficient." (Reviewer BLfX). 

To elaborate, **our scale-aware optimization differs from the ``STEP SIZE GRADIENT SCALE" used in LSQ in both formula and function.** From a formula standpoint, we adjust the gradient of QALoRA weight using the averaged quantization scale, whereas LSQ adjusts the gradient of quantization scales using the number of weights and precision. The goal of our technique is to improve the learning effectiveness of QALoRA, a critical issue in its optimization. As referred to Table 3 in the paper, our scale-aware optimization significantly improves EfficientDM's performance, leading to a 10.95 decrease in sFID.

Common distillation methods require access to the training dataset, which may be challenging due to massive size or copyright concerns. The recent work LLM-QAT [ii] proposes a distillation method that utilizes generations produced by full-precision (FP) models. However, it requires generating massive amounts of data using the teacher model beforehand and training the entire quantized model, incurring significant training costs. **Our distillation scheme stands out by not requiring the generation and storage of any data.** Instead, we input the same random Gaussian noise to both FP and quantized models and minimize the mean squared error between their denoising results. The distillation process is conducted in a data-free manner, which is efficient and easy to implement.

**The proposed QALoRA is also distinct from the existing work QLoRA [iii], which applies LoRA to quantized models.** As referred to ``Quantization-aware low-rank adapter" part in the paper, QLoRA incurs limitations when both weights and activations are quantized. The full-precision LoRA weights can not be merged with quantized model weights, leading to increased model size and massive floating-point calculations during inference. In contrast, the proposed QALoRA is a new variant of LoRA that can be seamlessly integrated with quantized model weights. After fine-tuning, there is no additional storage needed and the calculations between quantized weights and activations can be efficiently implemented for faster inference.

Reviewer comment (Reproducibility):
How to get $\bf{x}_t$ in equation (7) is not described properly. Do the authors sample $\bf{x}_T \sim \mathcal{N}$, and run several solver steps using the FP model to get $\bf{x}_t$, or otherwise? If it is the case, the equation (7) is not written properly.

Author response:
**Q1: How to get $\mathbf{x}_t$ in Eq. (7)? The Eq. (7) is not written properly.** Please refer to Q2 in general response. The input data $\mathbf{x}_t$ is derived from denoising random Gaussian noise $\mathbf{x}_T \sim \mathcal{N}(0,1)$ with the FP model through $T-t$ iterations. We have revised the explanation of Eq. (7) in the revised paper.

Reviewer comment (Experiments):
The proposed Temporal LSQ (TLSQ) method uses a different activation quantization scale for different time steps. Can the authors show the learned scales and analyze how the scale factors of certain layers change w.r.t. the time steps on different datasets?

Author response:
**Q2: Can the authors show the learned scales and analyze how the scale factors of certain layers change on different datasets?** As referred to Figure B to C in the supplementary material of revised paper, we visualize the learned quantization scales and the variance of the activations collected from W4A4 LDM-4 model over ImageNet $256\times256$ and LSUN-Bedrooms. Layers exhibiting significant shifts in activation distribution show considerable variation in the learned scale across different steps, and vice versa.

Reviewer comment (Experiments):
The paper mentioned that "we interpolate the learned temporal quantization scales to deal with the gap of sampling steps between fine-tuning and inference". I found steps=100 experiments on CIFAR-10 and LSUN in Table 1 and Appendix A, I wonder if the authors experimented with using a different schedule with fewer steps? Does this scale-deciding technique work well in the fewer-step regime?

Author response:
**Q4: Does scale-deciding technique work well in the fewer-step regime?** As referred to Table 2 in the paper, the performance on ImageNet $256\times256$ is evaluated with 20 steps, which demonstrates the strong performance of EfficientDM in the fewer-step regime. Moreover, we conduct experiments on LSUN dataset with 20 steps, as referred to Table B in the supplementary material of revised PDF. The proposed EfficientDM consistently outperforms previous quantized diffusion models in the fewer-step regime, which demonstrates the effectiveness of the interpolated quantization scales.

**Performance of unconditional image generation on LSUN-Bedrooms with 20 steps. `N/A' denotes failed image generation.**
| Method | Bitwidth (W/A) | Model Size (MB) | IS$\uparrow$ | FID$\downarrow$ |
| :---: | :---: | :---: | :---: | :---: |
| FP     | 32/32       | 1045.4      | 2.21        | 9.74          |
| Q-Diffusion | 8/8        | 261.67      | 2.13        | 10.10         |
| Ours   | 8/8        | 261.69      | **2.20**    | **9.75**      |
| Q-Diffusion | 6/6        | 196.33      | **3.78**     | 149.39        |
| Ours   | 6/6        | 196.36      | 2.27        | **10.39**     |
| Q-Diffusion | 4/4        | 130.99      | N/A         | N/A           |
| Ours   | 4/4        | 131.02      | **2.24**    | **16.35**     |

Reviewer comment (Reproducibility):
Is QALoRA applied for all the weights, including the convolutions and the attention layers?

Author response:
**Q5: Is QALoRA applied for all the weights, including the convolutions and the attention layers?** As referred to ``Quantization settings" in Section 5.1 in the paper, the input embedding and output layers are fixed to 8-bit, whereas other convolutional and fully-connected layers are applied with QALoRA modules.

Reviewer comment (Writing):
For the good of future efficient diffusion, can the authors discuss more relevant limitations and raise questions worth future studying? The current discussion is not specific.

Author response:
**Q6: Discuss limitations and raise questions worth future studying.** Thanks for your valuable comment. We have updated the ``Limitations and future work" part in the revised paper. Specifically, while EfficientDM can attain QAT-level performance using PTQ-level data and time efficiency, it employs a gradient descent algorithm for optimizing QALoRA parameters. Compared to PTQ-based methods, this places higher demands on GPU memory, particularly with diffusion models featuring massive parameters. To address this, memory-efficient optimization methods can be integrated. Moreover, efficient diffusion models for tasks such as video or 3D generation are still under explored.

Reviewer comment (Experiments):
It would be beneficial to evaluate EfficientDM over recent text-to-image diffusion models, such as Stable Diffusion.

Author response:
**Q1: Evaluate EfficientDM over recent text-to-image diffusion models.**
As referred to Figures K-M in the supplementary material, we include visualization results to evaluate EfficientDM over text-to-image Stable Diffusion models.

Reviewer comment (Novelty):
Recent work TDQ [1] also introduces a quantization method that adjusts the quantization scale at various denoising steps. The differences should be discussed.

Author response:
**Q2: The differences from TDQ should be discussed.** Please refer to Q1 in the general response. TDQ incorporates additional MLP modules for each quantized layer to approximate quantization scales. This approach entails training these extra MLP modules, leading to a significant surge in training expenses. Conversely, TLSQ introduces a limited number of adjustable quantization scales per layer, making TLSQ more computationally efficient than TDQ. Empirically, TLSQ achieves a 20.9% faster training iteration speed compared to TDQ over DDIM model on CIFAR-10 dataset.

Reviewer comment (Theory):
Formulating the gradient of LoRA weights can help elucidate the reasons for ineffective learning of QALoRA.

Author response:
**Q3: Formulating the gradient of LoRA weights.**  Referring to Eq.~(6) in the paper and considering the impact of STE [i], the gradient of LoRA weights $\mathbf{BA}$ can be expressed as:
\begin{gather}
    \frac{\partial \mathbf{Y}}{\partial (\mathbf{BA})}= \frac{\partial \mathbf{Y}}{\partial \hat{\mathbf{W}}} \frac{\partial \hat{\mathbf{W}}}{\partial (\mathbf{BA})} = \hat{\mathbf{X}}^\top
\end{gather}
The key insight is that the gradient magnitudes of LoRA weights remain unaffected by the quantization scale, which is exactly the step size of the quantization step-like function. Consequently, in layers with larger quantization scales, the minor updates to LoRA weights will be diminished by the round operation, which causes the ineffective learning of QALoRA.

[i] Bengio, Yoshua, Nicholas Léonard, and Aaron Courville. "Estimating or propagating gradients through stochastic neurons for conditional computation." arXiv, 2013.

Reviewer comment (Novelty):
This paper proposes **TLSQ**, which is quite similar to TDQ in [1]. TDQ is applicable to diffusion models with both continuous time and arbitrary discrete time steps. The paper should clarify the number of time steps used in TLSQ and discuss the settings.

Author response:
**Q3: The paper should clarify the number of time steps used in TLSQ and discuss the settings.**
In practice, we set the number of time steps as 100 during fine-tuning and interpolate the learned temporal quantization scales when sampling with fewer time steps. We include the settings in Section 5.1 in the revised paper.

Reviewer comment (Reproducibility):
Table 3 shows the ablation study results. Does the **QALoRA** use LSQ algorithm to fine-tune the low rank adapter parameters and quantization step-size parameters?

Author response:
**Q4: Does the QALoRA use LSQ algorithm to fine-tune the low rank adapter parameters and quantization step-size parameters?**
Yes, the weight quantization step-size parameters are optimized during the fine-tuning process.

Reviewer comment (Experiments):
Are there any results on using LSQ method on quantized diffusion models on dataset other than Cifar10?

Author response:
**Q5: Additional experimental results of using LSQ [ii] method on quantized diffusion models.**
As referred to Table A in the supplementary material, we conduct experiments using LSQ method on LSUN dataset. Despite consuming $15.8\times$ less time, our approach surpasses LSQ at W8A8 bit-width, demonstrating superior FID performance.  At W6A6 bit-width, our approach achieves an FID that is only 0.07 higher than LSQ, striking a better balance between performance and efficiency.

Reviewer comment (Reproducibility):
In **Data-free fine-tuning for diffusion models** part, is $\mathbf{x}_t$ in Eq(7) sampled from Gaussion noise with an FP model?

Author response:
**Q6: Is $\mathbf{x}_t$ in Eq. (7) sampled from Gaussian noise with an FP model?** Yes. Please refer to Q2 in the general response. The input data $\mathbf{x}_t$ is derived from denoising random Gaussian noise $\mathbf{x}_T \sim \mathcal{N}(0,1)$ with the FP model through $T-t$ iterations. We have revised the explanation of Eq. (7) in the revised paper.

Reviewer comment (Reproducibility):
In **Variation of activation distribution across steps** part, it proposes to assign a separate step size parameter for activation quantization in each denoising time step, and the results shown in Table 2 are obtained from 20-step sampling. Is the total time steps fixed to 20 for the fine-tuning. Is the data-free fine-tuning in Eq(7) fixed for 20 steps?

Author response:
**Q7: Is the data-free fine-tuning in Eq. (7) fixed for 20 steps?** 
As referred to Q2, we set the denoising step for fine-tuning as 100 in practice and interpolate the learned scales for sampling with fewer steps.

Reviewer comment (Reproducibility):
In Sec3.2 Eq(3), the quantization scheme has three parameters, $l, u, s$, are they all trainable? If so, is it optimized with LSQ [2] or LSQ+ [3] algorithm?

Author response:
**Q8: In Eq. (3), the quantization scheme has three parameters, are they all trainable?**
No. Only quantization scale $s$ is trainable and optimized with LSQ algorithm, while $l$ and $u$ are determined by the target bit-width. We have revised it in the revised paper.