Title: AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Network Quantization

Diffusion Models
Abstract: We present in this paper a novel post-training quantization (PTQ) method, dubbed AccuQuant, for diffusion models. We show analytically and empirically that quantization errors for diffusion models are accumulated over denoising steps in a sampling process. To alleviate the error accumulation problem, AccuQuant minimizes the discrepancies between outputs of a full-precision diffusion model and its quantized version within a couple of denoising steps. That is, it simulates multiple denoising steps of a diffusion sampling process explicitly for quantization, accounting the accumulated errors over multiple denoising steps, which is in contrast to previous approaches to imitating a training process of diffusion models, namely, minimizing the discrepancies independently for each step. We also present an efficient implementation technique for AccuQuant, together with a novel objective, which reduces a memory complexity significantly from $\mathcal{O}(n)$ to $\mathcal{O}(1)$, where $n$ is the number of denoising steps. We demonstrate the efficacy and efficiency of AccuQuant across various tasks and diffusion models on standard benchmarks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method to quantize diffusion models for image generation post training that is more robust than state-of-the-art techniques, providing better image quality at the same quantization bit width. The main idea is to optimize the quantization process by jointly considering multiple timesteps of the diffusion process in order to prevent accumulation of quantization errors. This is also implemented efficiently via a gradient approximation approach that avoids backpropagation though the selected group of timesteps in sampling chain.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- the paper is overall well written and clearly motivated\n- the method seems novel and it is is scientifically sound. The idea to jointly consider multiple timesteps when optimizing the quantization process indeed allows to attenuate the error accumulation process of models optimizing individual timesteps\n- experimental results are extensive enough to properly characterize the performance of the method compared to the state-of-the-art under various quantization bit widths and to assess the design decisions such as the gradient approximation\n\nWeaknesses:\n- some minor presentation issues. For example: the FID2FP32 metric is mentioned in the introduction before being defined and may confuse readers. Also, it needs some explanation why we need this metric and it is not enough to just measure the regular FID\n- results are good but they are not always the best. It seems that for the experiments where numbers are reported, TAC [43] is a strong baseline. The authors should better discuss this method.\n- The ablation on the group size suggest that beyond a certain size the proposed method will result in (significantly) worse performance than working independently on each timestep. It is not very clear how one would choose the optimal group size.\n- It would be interesting to check the performance of the method on recent models using very few timesteps, so that a single group could be formed for optimization\n\nQUESTIONS:\nThe fact that the approximated gradients work better than the \"correct\" counterpart is quite surprising. The authors attempt an explanation suggesting that that the removed term is quite noisy but I am not fully convinced by this explanation. The method without approximation should minimize Eq. (11) having the full precision result as ground truth and the current denoiser, so how is it possible that the correct objective is noisier than the simplified objective? Is it possible this might be due to small batch size for the experiment without approximation?\n\nLIMITATIONS:\nSome limitations are acknowledged in the Appendix. The authors should integrate them in the main text. Societal impact of more efficient generative models is not discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes AccuQuant, a novel post-training quantization method for diffusion models that reduces accumulated quantization errors by simulating multiple denoising steps during quantization. It also introduces a memory-efficient objective and demonstrates strong performance across various tasks and models.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: The paper presents a strong motivation by identifying and analyzing the unique challenge of error accumulation in quantized diffusion models. The presented AccuQuant is a thoughtful approach by simulating multiple denoising steps during quantization, aligning closely with the actual inference behavior of diffusion models. This is a meaningful departure from prior per-step quantization strategies. The thorough evaluation on standard benchmarks strengthens showcases real-world applicability.\nWeakness:\n1: In the paper, the method is tested across various diffusion models and benchmarks, it’s unclear if tasks beyond generation (e.g., image restoration, inpainting, or editing) were considered, if these tasks can’t be evaluated, it is better to give reasonable explanations.\n2: The work has a strong assumption that minimizing output discrepancies over a few denoising steps is sufficient to mitigate long-range error accumulation. But some error behaviors might emerge only over longer sampling chains, the authors should explain this issue more. \n3. Although the paper introduces an efficient implementation (O(1) memory), there may still be computational overhead in simulating multiple denoising steps during quantization, especially for large-scale models. A more detailed analysis of runtime and energy trade-offs during quantization would help others understand its deployment feasibility.\n\nQUESTIONS:\n1:How does AccuQuant determine the optimal number of denoising steps to simulate during quantization to balance accuracy and efficiency?\n\n2: How good does AccuQuant generalize to tasks beyond image generation, such as image restoration ( a small scale experiment  will do)?\n\n3: What are the trade-offs regarding computational cost between AccuQuant and previous step-wise quantization methods during the quantization process?\n\nLIMITATIONS:\nThe major limitation of this work is its adaptation to other domains such as inpainting, restoration etc.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes that PTQ for diffusion models should not be based solely on single-step denoising alignment, but rather on multi-step joint alignment to better account for accumulated errors. To address the high computational cost of aligning multiple steps, the authors introduce an efficient gradient approximation method based on empirical observations. Extensive experiments demonstrate that the proposed approach achieves state-of-the-art PTQ performance for diffusion models.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The paper presents a clear and well-motivated rationale for aligning multiple denoising steps to mitigate cumulative errors, and accordingly introduces a novel and reasonable optimization objective.\n2. The authors thoroughly consider the computational complexity involved in aligning cumulative errors and effectively address it through mathematical reformulation and gradient approximation techniques.\n3. Extensive experiments demonstrate that the proposed method consistently outperforms existing PTQ approaches for diffusion models across multiple evaluation dimensions.\n\n### Weaknesses\n\n1. Increasing the group size consistently leads to suboptimal results, which is rather puzzling. This may suggest that the alignment objectives defined in Equations (10) and (12) are potentially inaccurate or not well-justified.\n2. The manuscript does not report the overall cost of the proposed method, including actual runtime and memory efficiency, which are crucial for evaluating its practical applicability.\n\nQUESTIONS:\n1. Could enforcing multi-step alignment lead to suboptimal results? For example, if we keep the same calibration process but only include the loss between the final-step outputs of the quantized and full-precision models, could that potentially lead to better optimization? This concern may be reflected in Table 4, where increasing the group size actually leads to performance degradation.\n2. What is the actual computational overhead of the proposed multi-step joint optimization? Given the complexity, would a lightweight QAT approach achieve better results with similar resource costs?\n3. Why does removing gradient approximation lead to worse results? The results in Table 5 are counterintuitive. This may suggest that the optimization objective in Equation (12) is inherently flawed, as also questioned in Point 1 above.\n\nLIMITATIONS:\nNo further discussion is needed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nPrior work on post-training quantization (PTQ) has attempted to compress diffusion models by calibrating quantization parameters one step at a time, but ignores the fact that quantization errors accumulate across the sequence. This paper introduces AccuQuant, a novel PTQ method that explicitly addresses error accumulation. A key technical innovation is an efficient gradient approximation technique that reduces memory usage. Empirically, it achieves stronger fidelity to the full-precision model than prior quantization methods on a variety of benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Novel multi-step calibration: Unlike previous PTQ approaches that minimize error per step in isolation, AccuQuant explicitly calibrates over groups of steps. This is a significant innovation: by “simulating” the sampling trajectory, the method directly accounts for the accumulated error from previous steps. This better aligns the quantized model’s outputs with the full-precision model over time, addressing a core limitation of existing methods.\n2. Memory-efficient implementation: The paper presents a clever gradient approximation that avoids storing all intermediate activations when backpropagating through multiple steps. This reduces memory complexity from $O(n)$ to $O(1)$, making multi-step calibration practical for large models. \n\nWeaknesses:\n1. Reproducibility (no released code): The authors state they do not release code due to copyright restrictions. Especially, some experimental results show their performance closer to benchmark.\n2. Variable gains at higher bit-widths: In some settings, the improvements over baselines are modest. For instance, on CIFAR-10 at 6/6 bits the gap between AccuQuant (FID=5.79) and the best prior (TFMQ-DM, FID=7.84) is clear, but at 4/8 bits the advantage is smaller (FID 4.75 vs TAC’s 4.89). This suggests that when models are already near full precision, the benefit may be incremental. It would be helpful to understand when multi-step calibration provides the biggest payoff (e.g. extremely low bits vs. moderate bits).\n\nQUESTIONS:\n1. Accumulation Quantization Error has been used in CNN architectures or Transformer architectures. Compared to literature works, how is current method different?\n\n2. The paper primarily compares with quantization methods specifically designed for diffusion models. However, it would strengthen the evaluation to include more recent diffusion-focused approaches such as PTQ4DM and Vidit-Q.\n\nAdditionally, since diffusion models often incorporate Transformer-based components, it would be valuable to compare against advanced Transformer quantization methods such as SmoothQuant and RepQ-ViT, to better assess general applicability and robustness\n\n[1] Repq-vit: Scale reparameterization for post-training quantization of vision transformers\n\n[2] Vidit-q: Efficient and accurate quantization of diffusion transformers for image and video generation.\n\n[3] Post-training quantization on diffusion models. \n\n[4] Smoothquant: Accurate and efficient post-training quantization for large language models. \n\n\n3. Understand the authors have conducted experiments on group size. However, will the optimal group size changes when dataset or model changes?\n\n4. The authors have conducted experiments analyzing the gradient approximation. Would it be possible to also compare the loss computed using full-precision models, rather than focusing solely on the quantized models?\n\nLIMITATIONS:\nGradient approximation effects: The proposed gradient approximation makes the method feasible, but it is inherently an approximation. The paper does not deeply analyze whether this approximation affects the optimality of calibration. It is possible that the loss surface changes and the learned quantization parameters are suboptimal compared to a full-gradient approach (if that were practical). Quantifying any trade-off would strengthen the work. Not sure how the gradient approximation affect the overall loss.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-17"}