Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models
Authors: Tianao Zhang
Zhiteng Li
Xianglong Yan
Haotong Qin
Yong Guo
Yulun Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Difussion-LLM
Quantization
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: This paper proposes Quant-dLLM, a 2-bit weight-only post-training quantization framework for diffusion large language models that achieves state-of-the-art accuracy.

Abstract: Diffusion large language models (dLLMs), which offer bidirectional context and flexible masked-denoising generation, are emerging as a compelling alternative to autoregressive (AR) LLMs. However, like AR LLMs, their model sizes continue to grow, motivating weight compression for deployment. Although post-training quantization (PTQ) is effective for AR LLMs, directly transferring it to dLLMs at 2-bit leads to unsatisfactory performance. To tackle these challenges, we propose Quant-dLLM, an ultra-low-bit PTQ framework tailored to dLLMs. Since masked-denoising activations in dLLMs differ from the fully visible signals assumed by standard PTQ methods, we introduce Masked Calibration Simulation (MCS) to align calibration with the timestep-dependent masking, which yields more reliable calibrations. Moreover, we propose a Data-aware Any-order Quantizer (DAQ) that learns ultra-low-bit weight representations via an optimization algorithm. It performs iterative approximation guided by our simulated calibration data. In addition, under a strict 2-bit budget, we introduce Adaptive Blockwise Mixed Precision (ABMP), a sensitivity-based precision allocation scheme that adaptively assigns bit width across channel groups. When restricted to 2-bit precision, Quant-dLLM consistently achieves higher accuracy than state-of-the-art (SOTA) AR-transfer PTQ methods on dLLMs. We will release the code and models soon.

Published as a conference paper at ICLR 2026
QUANT-DLLM: POST-TRAINING EXTREME LOw-BIT QUANTIZATION FOR DIFFUSION LARGE LANGUAGE MODELS
Tianao Zhang Zhiteng Li Xianglong Yan', Haotong Qin', Yong Guo', Yulun Zhang Shanghai Jiao Tong University, PZhiyuan College, SJTU, 'ETH Zürich, *Huawei
ABSTRACT
Diffusion large language models (dLLMs), which offer bidirectional context and flexible masked-denoising generation, are emerging compelling alternative autoregressive (AR) L.Ms. However, like AR L.Mss, their model sizes continue grow, motivating weight compression for deployment. Although post-training quantization (PTQ) is effective for AR LLMs, directly transferring it to dLLMs at 2-bit leads unsatisfactory performance. To tackle these challenges, we propose Quant-dLLM, an ultra-low-bit PTQ framework tailored to dLLMs. Since masked- denoising activations in dLLMs differ from the fully visible signals assumed by standard PTQ methods, we introduce Masked Calibration Simulation (MCS) to align calibration with the Limestep-dependee masking, which yields more reliable calibrations. Moreover, we propose Data-aware Any-order Quantizer (DAQ) that learns ultra-low-bit weight representations via an optimization algorithm. It performs iterative approximation guided by our simulated calibration data. In addition, under a strict 2-bit budget, we introduce Adaptive Blockwise Mixed Precision (ABMP), censitivity-basee precision allocation scheme that adaptively assigns bit width across channel groups. When restricted to 2-bit precision, Quant- dLLM consistently achieves higher accuracy than mate-of-thee-nt (SOTA) AR- transfer PTQ methods on dLLMs. The code and models will be available at https://githuse com/ 2TA2785/Quantt
INTRODUCTION
Transformer-bant large language models (LLMs) (Vaswani, 2017) have achieved strong results across wide range of natural language processing and text generation tasks. Recent advances are led by autoregressive systems such as LLaMA (Touvron et al., 2023) and Qwen (Yang al., 2025), driven by scaling model size billions of parameters. Recently, diffusion-basee large language models (dLLMs) have emerged as promising alternative (Nie et al., 2025; Ye et al., 2025). They generate text by iteratively denoising masked sequences with bidirectional context and a üimeste--dependen visibility schedule, which supports parallel token recovery and fine-grained control of structure. Efficient dLLM deployment remains difficult multi-step denoising and large parameter counts increase memory. Thus, training-free weight compression is essential.
Model Performance Comparison
GPTO
GPTAQ
60
ours
Avg. Accuracy (%)
50-
40-
30
20
10-
cream
Model
Figure 1: dLLMs' performance on general tasks. Our Quant-dLLM yields the best accuracy at equal memory cost.
In recent years, large body of work explores compression for autoregressive large language models, including weight quantization (Frantar et al., 2023: Lin et al., 2024; Ashkboos et al., 2024). low rank factorization (Zhang et al., 2024; Yuan et al., 2023), network pruning (Sun et al., 2024; Frantar & Alistarh, 2023), and knowledge distillation (Zhong et al., 2024: Gu al., 2024). Among these
"Equal contribution "orresponning author; Yulun Zhang, yulun 100 gmail.com

---

Published as a conference paper at ICLR 2026
approaches, post-training quantization (PTQ) is especially attractive because it avoids finetuning and scales to billion--pranae parameter models. On autoregressive models, PTQ has advanced with curvature- aware error control and with layout or rotation techniques that reduce outliers, and it delivers strong accuracy at low-bit precision with simple deployment. Recently, Lin et al. (2025) systematically transfers classic low-bit PTQ methods from autoregressive L.MS to diffusion language models and reports that weight-only 4-bit performs well, whereas 3-bit shows noticeable degradation on math and code tasks. Building on these observations, we extend weight-only PTQ 2-bit budget and find that performance drops sharply, which highlights the mismatch with diffusion-style inference.
This work focuses on 2-bit weight-only PTQ for dLLMs. Inspired by DB-LLM (Chen et al., 2024), we encode each weight matrix as combination of binarized matrices with row- row-column scaling. This keeps bitwise- frrindll execution and increases expressiveness under the same 2-bit budget. In dLLMMs, we observe that it has lower reconstruction error than fixed 2-bit codes and more stable fittigg under masked, dimestep-dependent activations. The parameterization also reveals structured sparsity that reduces memory usage, making it natural foundation for our diffusion-aligned PTQ.
In this work, we develop a training-free, weight-only PTQ pipeline tailored to dLLMs under a 2-bit budget. Our analysis finds two main sources of error at 2 bits. (i)Timestep and mask schedules shift the activation distribution away from fully visible calibration. (i)QQuatizzttion errors accumulate across denoising steps and become larger at later stages. To address these issues, first propose a diffusion-aligned calibration simulation that takes the inference visibility schedule: calibration batches drawn with explicit coverage over timesteps and masking ratios. Building on these calibrated statistics, we develop a data-aware any-order quantizer that reconstructs each weight matrix as combination of binarized matrices with row-column scaling. It is optimized with a Data-aware Objective Reformulation and Row-column Successive Re-scaling. The order K can be increased while preserving binary-friendee execution ($3.3). Finally, under a strict 2-bit weight budget, we introduce adaptive blockwise mixed precision that assigns different orders to blocks by importance, granting higher effective precision to a small set of sensitive regions while keeping the majority 2-bit. This stabilizes late denoising steps without changing activation ($3.4)
Our key contributions can be summarized as follows:
Masked Calibration Simulation (MCS), We construct timestep-aware, partially visible calibration inputs that mirror the diffusion denoising process, reducing the distribution mismatch between calibration and inference.
Data-aware Any-order Quantizer (DAQ). We introduce row-column scaled multi- binary parameterization with Data-aware Objective Reformulation (DOR) and Row-column Successive Re-scaling (RSR), and extend it to an any-order composition.
Adaptive Blockwise Mixed Precision (ABMP). We propose an importance-guidee per- block order assignment that adheres to strict 2- bit average budget, concentrating represen- tational capacity on the most critical model components.
Quant-dLLM. We present Quant-dLLM, which seamlessly integrates MCS, DAQ, and ABMP into standard PTQ pipelines, achieving state-of-the-art (SOTA) 2-bit weight-only accuracy on recent dLLMs. We will release the code soon.
2 RELATED WORKS
2.1 DIFFUSION LANGUAGE MODELS
Diffusion models have seen remarkable success in generating continuous data such as images, video, and audio by learning reverse forward noise process. However, extending this framework to language introduces a unique set of challenges due to the inherently discrete and structured nature of textual data. Early work addressed this by modeling the reverse dynamics of discrete diffusion process with an absorbing state (He et al., 2022; Austin et al., 2021a). More recently. Masked Diffusion Models (MDMs) (Lou et al., 2023; Ou et al., 2024; Shi et al., 2024) have gained increasing attention by adopting simpler forward process that gradually replaces input tokens with designated [MASK] token. This year, efforts have successfully scaled these models to the billion-parameter regime. Notable examples include LLaDA (Nie et al., 2025), which uses a bidirectional transformer as a denoiser to achieve performance comparable to autoregressive LLMs, and Dream (Ye et al., 2025), which is initialized from pre-trained autoregressive model and demonstrates competitive generation capabilities. These advancements confirm the viability of diffusion approaches as an alternative to autoregressive models for large-scale language modeling.
2

---

Published as a conference paper at ICLR 2026
(a) Masked Calibration Simulation
(b) Data-aware Any-order Quantizer
Callention
Coltration
Cap
W
Henak
Inttialization
800
Caliration
Iegartence Matris 7
Impectance
Wx
Bick-riie Bi-nidtte
Objective Refomulation
Jaigen Rode
Zait gusttiatian
JAD( W)
K- cedert
1-B oedkt
Importanee
Rallbaate
Calbration Bata
Strdannd
Row-column
Successivee
Rescaliar
Rovaseding
Re-scaling
Mar Inqusteet
(c) Adaptive Blockwise Mixed Precision
Figure 2: Overview of our Quant-dI.LM. Masked Calibration Simulation: Aligns calibration with diffusion by simulating masked, timestep-aware inputs. Adaptive Blockwise Mixed Precision: Assigns binary orders by importance under a 2-bit average. Data-aware Any-order Quantizer: Builds multi Muntii RC forms with data-aware optimization.
Despite these encouraging results, the deployment of dLLMss remains constrained by the computa- tional demands of their billions of parameters. While some recent works propose caching strategies accelerate dLLMM inference (Wu et al., 2025; Liu et al., 2025; Ma et al., 2025), research takes different, complementary approach. We explore the potential PTQ techniques to dLLMs, aiming to reduce quantized model memory footprint while preserving generation quality.
2.2 WEIGHT-ONLY QUANTIZATION
Weight-only quantization is primary strategy for compressing LLMs, typically categorized into quantization-awar training (QAT) and post-training quantization (PTQ). While QAT integrates quantization into the training process to achieve low-bit representations, it often demands significant computational resources and extensive training time. In contrast, PTQ applies quantization directly to a pre-trained model's 's weights, making it a faster and more resource-efficient alternative.
Recent advancements in PTQ for autoregressive LLMs include methods like GPTQ (Frantar et al., 2023) and GPTAQ that leverage Hessian- guided error compensation, well as approaches such as Slim-LLM (Huang et al., 2024b), which employs mixed- precision schemes to enhance the perception of important weight information. Other techniques like AWQ Lin et al. (2024) use activation-aware scaling to achieve strong performance at low bitwidths. For extreme compression, binarization methods like BiL.LM (Huang et al., 2024a) and ARB-LIMM (Li et al., 2025b) push the boundaries to I-bit, demonstrating competitive results. Sub-1-bit methods (Dong et al., 2025; Yan et al., 2025) push compression further by lowering average bitwidths while maintaining strong accuracy. A recent study by Lin et al. (2025) explored transferring these PTQ methods to dL.LMs. While effective at 3-4 bits, found that performance degraded severely at 2-bit precision, highlighting notable challenge in ultra-low-bit quantization for dLLMs. To address this gap, we develop tailored 2-bit weight-only PTQ framework, achieving significant improvement over the SOTA method for autoregressive LLMs.
(eessdan-guied
3 METHOD
Overview. Figure 2 summarizes our pipeline. Section 3.1 reviews masked diffusion for language models and binarization. Section 3.2 introduces Masked Calibration Simulation (MCS), aligning calibration with timestep-dependent masking. Section 3.3 presents the Data-aware Any-order Quan- tizer (DAQ), which fits RC-scaled binary matrices via data-aware closed-form successive ee-scaling. Section 3.4 introduces Adaptive Blockwise Mixed Precision (ABMP), reallocating precision by importance, upgrading key blocks to 3-bir bit and reducing others to 1-bit under strict 2-bit budget.
3.1 PRELIMINARY
Masked diffusion for language models. We consider a diffusion decoder that operates exclusively on masked token positions. For a token sequence y E {cz, ... eg , an absorbing token m (i.e.. [MASK]) )), and a monotonic visibility schedule or E [0, 1], the forward noising process is defined as: q(ve y) = Cat(a, y + (1 a;) m).
(1)
The decoding (denoising) process proceeds from t=1 + 0, where visible tokens are copied and masked tokens are predicted via na (3x) (ooltmax(ff()))))
3

---

Published as a conference paper at ICLR 2026
Given a weight matrix W R"x", we row-centered matrix W W - with J 1 Σz=1 W and approximate it by a binary carrier B E (-1,+1) modulated by separable row-column scales: W (a, a a, E R", ag E m The scales and carrier are obtained by
Binarization.
min Kr,.
W W- (a, ag B
This is typically solved via closed-form alternating updates for ar,, a2 and sign rounding for B (see $3.3). The single-scale case recovers the classic solution sign (W), (a,);
1 m Σ, Way
Algorithm 1 MCS: Masked Calibration Simulation
func MCS(X, T, a(t))
6:
a ((l)) for to T. do
Input: X' -fll sequences; T - total timesteps;
7:
y prefix ratio; att) - visibility schedule
8:
f if E P clse Bernoulli(a)
Output: Masked calibration set Dealib
9:
(x(t) t x, rf = 1 clse [MASK]
1: Dealib f- 00
10: end for
2: for all x E X do
11: Dealib < Dealib U (({tt)
3:
L <- length of x
12: end for
4: p - (1..... 17- LJ}
13: end for
5: for = to 7 do
14: return Dealib
3.2 MASKED CALIBRATION SIMULATION (MCS)
Conventional calibration techniques assume autoregressive activations and static token visibility. However, such assumptions fail to capture the distributional shift introduced by timestep-dependent masking in dLLMs, leading to inaccurate quantization statistics and performance degradation. To address this, we propose Masked Calibration Simulation (MCS), a method that aligns the calibration process with the model's native denoising mechanism, as detailed in Algorithm
MCS simulates activation distributions by constructing ameeteep-wwars batches that jointly stratify across time and visibility ratio. This simulates the temporal uncertainty and partial token observations encountered during inference. Given fully visible sequence x E V we first fix deterministic visible prefix: P {1..... ly - L]}. J. For each timestep t (t), we first compute the corresponding visibility ratio (t)., which determines the expected proportion of visible tokens. Then, we construct a binary mask r {0, 1 such that tokens in the always-visible set P are assigned r; 1, while the remaining positions are independently sampled from Bernoulli distribution with probability att), ie., T2 = 24 ~ Bernoulli (t) ) for i P. Finally, we set the masked input as (t) = x, if and (1) [MASK] otherwise.
Applying this over a grid of T timesteps yields a calibration dataset whose activation statistics closely match masked denoising. We then aggregate second-order statistics over this set. In $3.3, we further convert these into an elementwise importance mask to weight positions adaptively. For implementation, we discretize (0, 1] into T uniform timesteps, a default 7 0.25, and fix random seeds to ensure full experimental reproducibility.
3.3 DATA-AWARE ANY-ORDER QUANTIZER (DAQ)
To effectively parameterize weights under the masked activation statistics from MCS, we introduce the Data-aware Any-order Quantizer (DAQ). Instead of mapping weights to small set fixed quantization levels, DAQ approximates each weight matrix as combination of several binary matrices, each modulated by its own separable row-column scaling factors (a,, xx. This formulation, expressed superposition of binarized components, preserves the computational efficiency of binary operations while significantly expanding the model's representational capacity:
W
Σ k=1
(ath)
(K)T or
BR
B (-1,-11+
The integer K denotes the order of the representation, which directly corresponds to the bit-width assigned to weight block ($3.4).
We begin by analyzing the first-order case (K = 1), where a single row-column scale pair modulates one binary matrix: W = (a, a @ B. While minimizing weight reconstruction error is a common proxy, more direct objective is to minimize the error the layer's output, thereby better aligning
4

---

Published as a conference paper at ICLR 2026
the quantization process with the model's behavior during inference. To this end, incorporate the simulated calibration data X from MCS and define the data-aware quantization error as: L2 = |[X- - |.. However, directly computing this objective is computationally prohibitive due to the large number of matrix multiplications involving the calibration data X. To create tractable objective, we follow prior work (Li et al., 2025b) and reformulate the enror. By pre-computing the uncentered second moment of the simulated activations:
SMCS = E Σ Xx(() x ~
Xx(1) X,(1),,
I
LET b
the objective can be expressed as an efficient quadratic form: L2 TTr(WWWW W) SMaS (W- (W-W)T) While this reformulation is efficient, directly optimizing the quadratic objective remains challenging. Therefore, we develop the following strategies to create tractable yet effective approximation.
Algorithm 2 DAQ: Data-aware any-order Quantizer
fune DAQ(W, Z. K, T, A) Input W. Z Raxm K, T 1 Output
func binary re init(X) 1 [ 1m
I: ar + m
2: a2 t
X diag(a,) 1n
K W Σ
3: B f- sign(X)
a [a
T
(())
4: return Ct,, Cicy B
fune update_a, (X,,a,,A)
1: A- = build.impottnnc.mmaa. ()
I: uu (A² @ X B) a
2: W == 0nxm
v -A² (diago.) ae +eln
3:
for k = 1,2,
K do
3: return u 2
4:
R + W-
W
5:
ar
(k)
ac
(k)
(k)
binary.rc.nitf(f
func update a. (X,,,arA)))
I: u (A² X B) ar
6: S(K)
(k)
(k)
Or
2: v (A3) (diag a,) + +
7: W + W + (K) (4)
3: return u Ø v
8:
end for
fune update B W, WW (a 1h=11 {a 1
(N) 1K
9:
for -=
T do
1k=1
for k = 1,2.... .......... K do
1: for i 1, ,,..., do
WiA Egst (S()) B())
2: for j =
do
s s [(a ) (4) K
12:
(kk) t W W
3:
k=1
13:
t update R (k) (8), az
A)
4:
b+ search W 8)
for k 1, 1..... K do
14:
(K) ad
R
B A)
6:
(B( + by
end for 8: end for
((k] (k) (a( )
16:
end for
end for
17:
[B(h) ===1 + update B
K
W, (a 1k=1,
10: return (B(*)
fune iwwoiiancceeaaaaaaeeaeeo
A)
(a!
K ]k=
I: μ < mean (Z). o a std(
18:
W - DK == ((A)
④ (N)
2: M < T( > 30)
19:
end for
3: return 1,xm + (A 1) M
20: return W
Data-aware Objective Reformulation (DOR). Our analysis reveals that quantization error is not uniformly distributed but concentrates on small, critical subset of weights. This insight allows us to distill the complex, data-dependent information from SMes into computationally efficient proxy. We construct importance matrix Z gaxm with the same shape as W. Let = diag (SMcS - 7 Im 1 E Rm and D = diag(d). We normalize W column- wise by D and square elementwise: Z = (WD !) o (WD-1) We then standardize Z along a chosen block on the calibration and detect outliers with the 30 rule:
7 (Z 11) 2 a II -= [=[(Z|33)
(5)
The 30 threshold isolates the small heavy-tailed portion of high- imphiimpoeeeeee weights that drives most of the error, while keeping the selected fraction compact and stable across layers. Finally, we form the importance mask: A 1nxm + (A - 1) II, which means elements beyond 30 receive weight >> and the others receive weight 1. Therefore, distill the effect of SMcs into A and optimize
5

---

Published as a conference paper at ICLR 2026
the data- Frobenius proxy:
Σ (ar,, | @ (W- W)ll TT((A (W- (A) (w (W-W))))
(6)
When A = 1 axm. this objective reduces to the standard Frobenius reconstruction emoo.
Row-column Successive Re-scaling (RSR). We minimize the data-aware Frobenius proxy using iterative method RSR. The optimization objective is given by:
(a,, 0.) A @ ( aa, (B)))
(7)
The RSR algorithm operates by alternately optimizing set of scaling factors while holding the other fixed. By differentiating LA with respect to each scales and setting the gradient zero, we obtain closed-form update rules that guarantee monotonic decrease in the objective:
ar (A W a.
Ø (AA"(diag(e) a.) + 1n)
(8)
a2 = [A W WOB) ar
@ [(A (diag( or) a,)) + =mmm
(9)
Following the updates to the scaling factors, the binary matrix B is refined through a per-entry minimal-error search:
Bi = arg min bE(+1}
Wiy - (ar,)ac.))) bl.
(10)
Here, @ and @ denote element-wise product and division, respectively. diag(-) (-) forms diagonal matrix, 1, Im are all-ones vectors, and > is a small constant to stabilize division.
K-order extension. The K-order representation is constructed through greedy, iterative process that extends the single-order framework. We begin by finding the optimal first-order approximation W()., Then, we compute the residual (()) w (((1) and apply the same RSR procedure to find the optimal approximation for this residual. This process is repeated K times, with each new binary component fitting the residual from the previous stage. This formulation naturally extends to any
order K. progressively optimizing the approximation. The full procedure is detailed in Algorithm 2.
Algorithm 3 ABMP: Adaptive Blockwise Mixed Precision
1: Given block partition G and importance Z
2: Compute 8g [i.))en Zy for all g
3: Set k + [0.05 G and sort blocks by 89
4: Assign by 3 to top-k; by to bottom-k; others b =2 ( [3 Eg by = 2)
5: Apply DAQ quantization to each block using its assigned order K = bg.
3.4 ADAPTIVE BLOCKWISE MIXED PRECISION (ABMP)
The DAQ representation from $3.3 provides a flexible multi-binary parameterization. However, uniform application order inherently suboptimal. Such strategy fails to differentiate between model components of varying importance, leading to misallocation of the bit budget, which means highly salient regions suffer from performance degradation due to insufficient precision, while less critical regions consume more bits than necessary.
To address this, we introduce Adaptive Blockwise Mixed Precision (ABMP), a strategy that in- telligently redistributes the quantization budget at block level. The core principle of ABMP is to strategically reallocate precision under a strict per-layer budget, ensuring the model's average bit-width remains exactly 2-bit:
bg 2, by (1,2,3)
(11)
DEG
This approach concentrates representational capacity where it is most impactful, enhancing model fidelity without increasing the overall memory footprint. The allocation process is guided by the importance of each block. To quantify this importance, we leverage the elementwise importance matrix Z (from $3.3) to compute an aggregate importance score = E(())e Zu for each block g E G. We then rank all blocks within a layer based on these scores. To enforce the 2-bit average budget, the number of blocks assigned higher precision (3-bit) is precisely matched by equal number blocks assigned a lower precision (I (I-bit). Specifically, we allocate 3-bit precision to the top-k most important blocks and 1-bit precision to the bottom-k.

---

Published as a conference paper at ICLR 2026
Based experimental analysis. we set the default reallocation ratio 5% per layer (i.e., = [0.05 [.) As detailed in our ablation studies, this ratio provides a robust balance between performance and stability, with diminishing returns observed beyond 10%. The complete procedure is outlined in Algorithm 3. By dynamically adapting the bit-width at a structured, block-wise level, ABMP effectively preserves critical information in dL.LMs, which is particularly crucial for stabilizing performance during the later, more sensitive stages of the denoising process.
3.5
QUANT-DLLM PIPELINE
As illustrated in Figure 2, our proposed Quant-dLLM framework is composed of three key compo- nents: Masked Calibration Simulation (MCS), Data-aware Any-order Quantizer (DAQ), and Adaptive Blockwise Mixed Precision (ABMP). The end-to-end pipeline operates in a layer-wise manner, requiring only a pre-trained dLLM and a small set of calibration data.
The workflow begins by first processing the calibration data with MCS ($3.2) to generate timestep- aware dataset whose activation statistics align with the duffusion-denoisiee framework proceeds layer by layer. For each target layer, we first compute an importance matrix Z from the simulated calibration data. This matrix then passed to ABMP ($3.4), which determines the optimal per-block bit-width assignment {by} } under a strict 2-bit average budget. Finally, DAQ ($3.3) quantizes the layer's 's weight matrix according to the bit-widths allocated by ABMP. This process is repeated for all targeted layers until the entire model is quantized. Since both quantization and bit allocation are performed layer-wise, our framework maintains a low memory overhead and avoids any gradient- based updates, making a highly efficient PTQ solution.
process. Subsequently, the
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
All experiments are conducted on single NVIDIA A800-80GB GPU using the PyTorch and Huggingface frameworks. As post-training quantization (PTQ) framework, Quant-dLLM does not require any training or gradient backpropagation. We use 128 calibration samples from the C4 dataset, with each sequence having a length of 4096 tokens, and the group size is fixed at 128 for all experiments, For evaluation, we use recent dLLMs, including LLaDA-8B-Base (Nie et al., 2025), LLaDA-8B-Instrue (Nie et al., 2025), LLaDA-1.5 (Zhu et al., 2025), Dream- and Dream-7B-Ba IB-Instruct (Ye et al., 2025). We evaluate Quant-dLLM on three task categories:
Ye et al. (2025),
General-knowledge QA: MMLU (Hendrycks et al., 2020), ARC-E/C (Clark et al., 2018), HellaSwag (Zellers et al., 2019), WinoGrande (Sakaguchi et al., 2020), PIQA (Bisk et al., 2020), and BBH (Suzgun et al., 2022).
Mathematical and scientific reasoning: GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), and GPQA (Rein et al., 2024).
Code generation: HumanEval (Chen et al., 2021) and MBPP (Austin et al., 2021b).
General tasks are evaluated in n-shot setting with 128 Monte Carlo samples, while code tasks report the Pass@) metric. We compare our method against three strong PTQ baselines tailored for low-bit regimes: GPTQ (Frantar et al., 2023), traditional block-wise weight-only quantization method; GPTAQ (Li et al., 2025a), variant of GPTQ that incorporates activation into its compensation; and Slim-LLM, (Huang et al., 2024b), a mixed- mixed-precision weight-only SOTA PTQ method, Baselines use the same calibration data and sequence length as ours.
4.2 MAIN RESULTS
Table 1 reports the weight-only quantization results for GPTQ, GPTAQ. Slim-LLM, and our Quant- dLLM on five different dLLM models across seven general knowledge tasks. Our method consistently delivers the highest average accuracy among 2-bit baselines on all five models, surpassing GPTQ, GPTAQ, and Slim-ILLM by clear margin. Averaged over all models, our method improves the mean score from 36.5 (GPTQ), 35.6 (GPTAQ), and 40.9 (Slim-LLM) to 51.3.
On a per-model basis, our approach consistently improves the average score over Slim-LLM. For LLaDA-SB- -B--se, the average score rises from 42.39 to 54.06, improvement of over 27%. Our method also preserves a large fraction the prec-iiin performance on the LLaDA series under the same extreme b-bit constraints. For example, 87.72% of the full- pell-preoision score. Beyond the average, our method provides consistent per-task improvements over Slim-LL.M and achieves the best score on all seven general benchmarks for all
-LaDA-8B-Baatd our average score reaches
7

---

Published as a conference paper at ICLR 2026
Table Results of GPTQ, GPTAQ. Slim-LLM, and our Quant-dLLM with 2-bit weight quantization among 7 tasks on LLaDA-Base. LLaDA-Instrun LLaDA-1.5, Dream-Base and Dream- The numbers parentheses represent the number used for evaluation. Best results are marked in bold.
Melhod
MMLU(5)
Wino.(5)
PIQA())
ARC-C(O)
ARC-E(0)
Hella.())
BBH()
Avg
FP
65.76
74.59
74.70
44.11
74.20
54.27
42,60
61.46
LLaDA-SB-Base
GPTQ
GPTAQ
34.75
34.73
57.85
59.04
56,42
57.45
22.27
22.44
37.04
38.17
35.13
33.77
5.34
4.06
35.87
35.34
Slim-I.IM
47.98
60.85
62.46
26.11
53.41
39.25
6.64
42.39
Quant-dLLM
56.87
68.19
69.75
36.26
68.69
46.45
32.18
54.06
НP
63,91
72.30
74.43
53.24
79.34
53,95
47,89
63.58
LLDDA-SB-Instruct
GPTQ
42.15
58.88
61.21
27.22
51.18
37.46
3.93
40.29
GPTAQ
44,94
60.22
61.04
29.01
52.44
37.18
4.55
41.34
Slim-LLM
50.35
60.69
65.72
34.73
65.57
41.34
27.48
49.41
Quant-dLLM
54.07
65.67
70.73
43,86
71,97
47.39
35.02
55.53
64.21
54.38
74.54
54.27
79,80
54.38
56.27
62.55
GPTQ
6.91
39.27
LLaDA-1.5
GPTAQ
47.26
35.89
58.81
28.41
51.39
35.89
5.38
37.58
Slim-LLM
33.70
Quant-dLLM
54.32
47.82
71.16
47.18
73.06
47.82
38.09
54.21
69.43
73.56
74.59
55.38
82.58
54,42
50.12
65.73
Dream-7B-Bin
GPTQ
23.84
50.51
53.86
19.88
28.70
29.69
3.08
29.94
GPTAQ
23.90
52.49
55.88
20.82
30.43
30.98
4,00
31.21
Slim-LLM
25.54
51.62
54.24
20.05
31.14
30.83
4.18
31.09
Quant-dLLLM
40.22
59.19
63.71
29.35
54,29
40.87
25.59
44.75
72.93
75.14
57,25
54.43
57.92
67.31
Dream-7B-Inste -BI-ntruut
GPTO
23.96
50.59
54,90
18.52
30.77
30.11
5.32
30.60
GPTAQ
24.32
49.72
56.42
20.73
31.82
30.97
5.11
31.30
Slim-LLM
25,29
51.93
56.09
20.56
36.99
31.55
7.61
32.86
Quant-dLLM
43.14
57.93
66.59
33.62
62.96
41.36
30.35
47.99
Mathematics Science Performance
Code Performance
Avg. Accuracy
Ilada
Model Figure 3: Average accuracy of mathematical & scientific reasoning, and code generation datasets on
Model
LLaDA series and Dream series.
five models tested. These results demonstrate that under strict 2-bit weight quantization, our approach delivers robust and transferable accuracy improvements across different dI.LM families.
In Figure 3, we further illustrate our method's 's superiority mathematics, science, and code tasks. Previous methods show significant performance collapse, while our Quant-dLLM the only method that effectively retains accuracy, providing a robust solution for wide range of applications. For example, on the LLaDA-IIntrrct model, our method achieves an average accuracy of over 30% for mathematics and science tasks, while all baselines remain below 12%. For code generation, method achieves over 15% accuracy on LLaDA -nnstruct and 1 while basclines hover near 0%. This remarkable resilience highlights that our Quant-dI.LM pipeline achieves significant performance increase compared to prior methods.
4.3 ABLATION STUDY
Unless otherwise specified, all ablation studies are conducted on LLaDA-8B-BaL Base and Dream-7B- Base, reporting the 5-shot accuracy on MMLU under a strict 2-bit average budget, where group size and block size are both 128. We vary factor at time while keeping all other settings fixed.
Effectiveness MCS. To verify the effectiveness of our Masked Calibration Simulation (MCS), we compare the performance of Quant-dLLM with and without MCS. As shown in Table 2a, on the LLaDA-8B-BB B-Baas model, the MMLU accuracy improved from 52.10% to 56.87% when using MCS. On the Dream-7B Base model, the MMLU accuracy improves from 37.81% % 40.22%. These results demonstrate that aligning the calibration data with the denoising process the diffusion model, MCS effectively reduces distribution mismatch, thereby significantly improving quantization performance.
8

---

Published as a conference paper at ICLR 2026
Table Ablation studies on LLaDA-8B-Bast and Dream-7B-Base. We report MMLU in 5 shots. (a) Effectiveness of MCS
(b) Effectiveness of ABMP
Model
MCS MMLU(S) 1
Model
ABMP
0%
5%
10%
15%
x
52.10
X
54.32
-.-DD--B-Base
56.87
/
56.87
55.87
53.01
37.81
x
32.75
Dream-7BB 7B-Base
Dream-7B- 7B-Base
40.22
/
34.50
40.22
31.11
(c) Ablation Study for DAQ
(d) Ablation Study for Calibration Set Size.
Model
DAQ Component MMLU(S)
Model
Calib. Data Size MMLU(5)
baseline
39.26
64
54.49
LLaDA-8B-Bal
RSR w/o DOR
48.32
LLaDA-8B-Ball
128
56.87
RSR w/ DOR
56.87
256
55.59
baseline RSR w/o DOR
27.84
38.24
Dream- Breum--B-Baane 7B-Base
34.73
Dream-7B: Bream-BB-Base
40.22
RSR w/ DOR
40.22
256
39.64
Effectiveness of ABMP.
We investigate the impact of enabling ABMP and varying its bit allocation ratio. As shown in Table 2b, simply enabling ABMP provides significant improvement over the 5% reallocation ratio achieves the best MMLU
uniform 2-bit baseline. On LLaDA-8B-Bast accuracy of 56.87% surpassing the 54.32% baseline. On Dream-781 Beeam-B--Basee a 10% ratio performs best, increasing accuracy from 32. 75% to 40.22%. This confirms that strategically reallocating precision to the most critical regions is highly effective strategy for boosting overall performance.
Ablation Study for DAQ. We conduct a breakdown study to evaluate the contributions of the core components within our DAQ, as shown in Table 2c. The baseline represents a simple quantizer without our proposed enhancements. By introducing RSR without the data-aware objective, performance substantially increases, with MMLU accuracy rising from 39 9926% to 48.32% on LLaDA-80 -----BBBB Base. Incorporating the full DAQ with the data-aware objective (RSR w/ DOR) provides another major boost to 56.87%. A similar trend is observed Dream-7B- Be-aee. This clearly indicates that both the iterative re-scaling and the data-aware objective are crucial for DAQ's state-of-the-ar performance.
Ablation Study for Calibration Set Size. We evaluate the impact of the calibration size Quant-dI.LM's performance. As shown in Table 2d, using 128 samples, each with sequence length of 4,096 yields the best performance. On LLaDA-8B-Base. increasing from 64 128 samples improves the MMLU accuracy from 54.49% to 56.87%. A further increase to 256 samples leads to slight drop 55.59%. Once moderate threshold is met, using more samples doesn't 't significantly improve performance. This shows Quant-dLLM is practical for aesource-limitee environments.
4.4 MODEL SIZE ANALYSIS
Efficient model size reduction is critical for deploying dLLMs on resource-constrained hardware. We evaluate LLaDA-8B- 3B-Base under different quantization schemes and observe that Quant-dLLM achieves the smallest model size while maintaining accuracy. This advantage comes from our row-column scaling factor design, which imposes less mem- ory overhead than the conventional zero-point and scaling factor used in typical low-bit guantization. The detailed comparison is summarized in Table
Table 3: Model size of LLaDA-8B-
Base under different methods.
Model
Method
Bit
Memory (GB)
FP16
16
16.09
LLADA-88
GPTQ
2
3.70
Slim-LLLM
Quant-dLLM
2
3.69
CONCLUSION
In this paper, we propose Quant-dL.LM. novel 2-bit PTQ framework that effectively addresses the unique challenges of quantizing dLLMs.. By introducing Masked Calibration Simulation to handle timestep-dependent inputs, and developing Data-aware Any-order Quantizer and an importance- guided Adaptive Blockwise Mixed Precision strategy, our method significantly mitigates quantization errors. Extensive experiments demonstrate that Quant-dLLM consistently outperforms SOTA 2-bit PTQ methods, establishing a new SOTA 2-bit weight-only quantization for dLLMs and enabling efficient deployment resource-constrained environments.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by the National Natural Science Foundation of China (62501386, 625B (025) and also sponsored by CCF-Tencent Rhino-Bird Open Research Fund.
ETHICS STATEMENT
The research conducted in the paper conforms, in every respect, with the ICLR Code of Ethics.
REPRODUCIBILITY STATEMENT
We have provided implementation details in Section 4. We will also release all the code and models.
LLM USAGE STATEMENT
Large Language Models (LLMs) were used solely for polishing writing. They did not contribute to the research content scientific findings of this work.
REFERENCES
Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlierffree 4-bit inference in rotated llms. In NeurlPS, 2024.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models discrete state-spaces. In NeurlPS, 2021a.
Jacob Austin, Augustus Odena, Maxwell Nye. Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2 108 07732, 2021b.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, al. Piqa: Reasoning about physical commonsense in natural language. In AAAI, 2020.
Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, al. Db-llm: Accurate bual-binarization for efficient Ilms. arXiv preprint arXiv: 1960, 2024.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.033: 2021.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint wrXivv1803.05455 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10.14661 2021.
Peijie Dong, Lujun Li, Yuedong Zhong, Dayou Du, Ruibo Fan, Yuhan Chen, Zhenheng Tang, Qiang Wang, Wei Xue, Yike Guo, et al. Stbllm: Breaking the 1-bit barrier with structured binary llms. In ICLR, 2025.
Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models be accurately pruned in one-shot. In /CML, 2023.
Elias Frantar, Salch Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. In ICLR, 2023.
10

---

Published as a conference paper at ICLR 2026
Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. Knowledge distillation of large language models. In ICLR, 2024.
Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. Diffusion- bert: Improving generative masked language models with diffusion models. arXiv preprint arXiv:2211.15029 2022.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In ICLR, 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv 2021.
preprint
Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. Billm: Pushing the limit of post-training quantization for 1lms. In ICML, 2024a.
Wei Huang, Haotong Qin, Yangdong Liu, Yawei Li, Qinshuo Liu, Xianglong Liu, Luca Benini, Michele Magno, Shiming Zhang, and Xiaojuan Qi. Slim-llm: Salience-driven mixed-precision quantization for large language models. arXiv preprint arXiv:2405. 20.2224455.99..... 2024b.
Yuhang Li, Ruokai Yin, Donghyun Lee, Shiting Xiao, and Priyadarshini Panda. Gptaq: Efficient finetuning-free quantization for asymmetric calibration. In ICML, 2025a.
Zhiteng Li. Xianglong Yan, Tianao Zhang, Haotong Qin, Dong Xie, Jiang Tian, Linghe Kong, Yulun Zhang, Xiaokang Yang, al. Arb-llm: Alterattng refined binarizations for large language models. In ICLR, 2025b.
Haokun Lin, Haobo Xu, Yichen Wu, Ziyu Guo, Renrui Zhang, Zhichao Lu, Ying Wei, Qingfu Zhang, and Zhenan Sun. Quantization meets dlims: A systematic study of post-training quantization for diffusion llms. arXiv preprint arXiv:2508. 14896, 2025.
Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device Ilm compression and acceleration. MLSys, 2024.
Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint
25l::2250006295. 2025.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. arXiv preprint arXiv:2310. 16834, 2023.
Xinyin Ma. Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache; The cache for diffusion language models. arXiv preprint arXiv:2505. 15781, 2025.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji- Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint 099992250..0999. 2025.
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions clean data. arXiv preprint arXiv:2406.0377
2024.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. adversarial winograd schema challenge at scale. In AAAI, 2020.
Winogrande: An
Jiaxin Shi, Kehang Han, Zhe Wang, Amaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data. In NeurIPS, 2024.

---

Published as a conference paper at ICLR 2026
Mingjie Sun, Zhuang Liu, Anna Bair, and Zico Kolter. A simple and effective pruning approach for large language models. In JCLR, 2024.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261. 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint M.Xiv:2302.13971,, 2023.
A Vaswani. Attention is all you need. In NeurlPS, 2017.
Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion Ilm by enabling kv cache and parallel decoding. arXiv preprint arXiv:2505.2266 2025.
Xianglong Yan, Tianao Zhang, Zhiteng Li, and Yulun Zhang. Progressive binarization with semi- structured pruning for llms. arXiv preprint arXiv:2502.01705. 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.092: 2025.
Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong, Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508. 15487, 2025.
Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, and Guangyu Sun. Asvd: Activation- aware singular value decomposition for compressing large language models. arXiv preprint arXiv:2312.05821, 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? ACL, 2019.
Mingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, and Bohan Zhuang. Loraprune: Pruning meets low-rank parameter--effcient fine-tuning. In ACL, 2024.
Qihuang Zhong, Liang Ding, Shen, Juhua Liu, Bo Du, and Dacheng Tao. Revisiting knowledge distillation for autoregressive language models. In ACL, 2024.
Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. Llada 1.5: Variunce-reduced preference optimization for large language diffusion models. arXiv preprint arXiv:2505.19222, 2025.
12


Reviews:
[{"review_id": "W1D11ZUJUx", "reviewer": "Reviewer_dsVf", "summary": "The paper correctly highlights the timestep-dependent masking in dLLMs and the accumulated denoising-step error, both of which invalidate classic AR-LLM PTQ assumptions (page 1–2). This motivation is strong and well supported by experimental trends.", "questions": "Can the authors comment on sacalability? i.e. scaling to larger models (≥30B) and comment on the generality claim.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The DAQ method builds on RC-scaled binarization and extends it to multi-order binary compositions (Algorithm 2, page 5).\nStrong points:\nData-aware objective reformulation using second-moment statistics (Eq. 4).\nImportance mask using 3\\sigma outlier detection (page 5) is computationally cheap.\nClosed-form alternating updates (Eqs. 8–9) are well derived.\nMulti-order fitting of residuals is natural and preserves binary-friendly execution.\nThe ablations (Table 2c) show DAQ contributes the largest accuracy jump.\n\nAcross 5 dLLMs and 7 tasks, the improvement over GPTQ/GPTAQ/Slim-LLM is consistent and large (Table 1, page 7).\n\nPerformance remains robust even on math and code tasks, where low-bit AR PTQ collapses (Figure 3, page 8).", "weaknesses": "The 3σ rule (page 5) is heuristic. Can the authors give a intuition or analysis on:\n-stability of \\DELTA across calibration seeds,\n- sensitivity of performance to \\sigma threshold or \\lambda,\n- whether importance correlates with diffusion-step Jacobians or outlier distributions\n\nGroup size = 128 is used universally. No analysis on: effect of block size, potential misalignment between block boundaries and meaningful weight structure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7ANpWWfYGh", "reviewer": "Reviewer_skef", "summary": "The paper proposes 2-bit weight-only quantization methods tailored for masked diffusion language models (MDLMs). To address the limitations of GPTQ on MDLMs, it introduces three techniques: MCS, which corrects statistics during masked denoising; DAQ, which matches (WX) instead of (W) by using data/activations (X) during quantization; and ABMP, which assigns 3-bit to top-K sensitive blocks and 1-bit to bottom-K (best around 5–10%). The improvement on standard benchmarks is significant.", "questions": "* I wonder if SVDQuant [1] (with native low-bit CUDA kernels, activation quantization) could be combined with the proposed method?\n* Does the distribution mismatch in MDLMs arise from the increasing number of “MASK” tokens over timesteps? If so, could this be partially resolved by [2]?\n\n[1] Li, et al. “SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models.” ICLR 2025. [arXiv:2411.05007](https://arxiv.org/pdf/2411.05007).\n\n[2] Deschenaux, et al. “Partition Generative Modeling: Masked Modeling Without Masks.” [arXiv:2505.18883](https://arxiv.org/pdf/2505.18883).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The method effectively prevents GPTQ from breaking down at 2-bit extreme quantization. Accuracy remains high enough to maintain, to some extent, performance on complex tasks such as math and coding, where 2-bit GPTQ mostly fails. Empirical memory saving exceeds 4× for an 8B MDLM.", "weaknesses": "* Comparisons are limited: no low-rank adapters, QAT, hybrid weight+activation quantization.\n* The writing needs overall improvement and simplification.\n* The naming of sub-methods should be improved: there are too many acronyms, and several are grammatically awkward (missing nouns, missing hyphens, missing capitalization after hyphens, “order-**K**”, Quantiz**ation**, etc.). The lowercase “d” in dLLM is also confusing. It is recommended to consult an LLM.\n* The extra “=0” at the end of each function should be removed when compiling the algorithm environment.\n* The full-precision model is missing in the math and coding bar plots.\n* Although significantly better than GPTQ, the performance drop is still notable, leaving room for improvement.\n* Empirical latency/throughput is not reported.\n* K is hand-tuned but not heuristic.\n* Activation statistics are not reported, so the effect of the correction is invisible.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eTrDUvQTMK", "reviewer": "Reviewer_xcW1", "summary": "The paper proposes Quant-dLLM, a 2-bit, weight-only PTQ pipeline designed explicitly for diffusion LLMs (dLLMs). It has three parts: first,  a masked calibration simulation (MCS)  that creates a synthetic dataset for the calibration of the quantizer appropriate for dLLM, seconds, a data-aware any-order quantizer (DAQ) inspired by DB-LLM that represents weights as a sum of row–column–scaled binary matrices, and finally, a mixed-precision approach inspired by Slim-LLM that allocates orders $K \\in \\{1,2,3\\}$ per block under a strict 2-bit average budget. Across LLaDA and Dream series models, Quant-dLLM outperforms strong 2-bit weight-only PTQ baselines (GPTQ, GPTAQ, Slim-LLM) on general, math/science, and code tasks.", "questions": "* Section 4.4: any discussion or latency/throughput? Are there any costs associated with the quantisation method that would affect deployment, e.g. additional kernels, etc?`\n\n* Ablation studies: I would like to see some ablation studies (even in the appendix) on the effect of the quantization block size or the effect of $\\lambda$\n\n* Line 320: regarding the bit relocation strategy. Is this inspired by another mixed-precision work (in which case, please cite). Have you tried less-heuristic, more systematic alternatives for reallocation?\n\n* Line 184:  “Leading to inaccurate quantization statistics …” Do you have measurements (e.g., per-timestep activation histograms, KL/TV distances) that quantify the mismatch between AR-style calibration and masked-denoising activations?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is a well-motivated PTQ paper that clearly identifies the idiosyncrasies of diffusion LLMs (dLLMs) and proposes a data-aware combination of PTQ and mixed-precision quantization to aggressively compress models under a strict 2-bit average precision budget. The approach is conceptually elegant and technically well-executed, addressing a real gap between autoregressive and diffusion-style inference.\n\nThe results show consistent improvements across all benchmarks compared to existing low-bit PTQ baselines (GPTQ, GPTAQ, Slim-LLM), including on challenging math and code tasks. The paper is carefully presented, with clear motivation, ablation studies, and sound empirical analysis. Overall, this is a strong, mature piece of work that meaningfully advances post-training quantization for a new class of models.", "weaknesses": "* Limited analysis of runtime/latency and kernel costs. Quant-dLLM reports a favourable model size (Table 3) but lacks end-to-end latency/throughput on full models and a cost breakdown for DAQ/ABMP kernels. Reporting wall-clock metrics on common hardware and the proportion of time in quant ops would help assess deployability\n\n* Sensitivity to hyperparameters is only partially explored:  λ (importance mask weight), 3σ thresholding, group/block sizes, or order K choices beyond the \\{1,2,3\\} scheme.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xTI1wgCk03", "reviewer": "Reviewer_ayMd", "summary": "This paper proposes an ultra low bit PTQ method Quant-dLLM, which aims at addressing the specific characteristics in DLLM quantization including calibration, sensitivity distribution and low-bit quantization optimization. Extensive comparison and ablation results demonstrate its effectiveness.", "questions": "1. The paper focuses on the 2-bit quantization but lacks specific 2-bit quantization baselines, such as QuIP[1] and PBLLM[2], which would provide a more comprehensive comparison.\n2. In addition to accuracy, the authors should also provide memory usage and runtime statistics for the quantization process. Efficiency is an important factor that determines the broader impact of the proposed method\n\n---\n\n> [1] Chee J, Cai Y, Kuleshov V, et al. Quip: 2-bit quantization of large language models with guarantees[J]. Advances in Neural Information Processing Systems, 2023, 36: 4396-4429.\n\n>[2] Shang Y, Yuan Z, Wu Q, et al. Pb-llm: Partially binarized large language models[J]. arXiv preprint arXiv:2310.00034, 2023.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "By considering the unique characteristics of DLLMs, the authors identify the shortcomings of existing LLM quantization methods in this context and conduct targeted explorations to overcome them. As one of the pioneering studies in this direction, this work offers meaningful insights and guidance for subsequent research.", "weaknesses": "1. The paper requires improvements in writing, as it lacks a logical flow and several parts are inadequately explained, causing confusion for the readers.\n2. The paper introduces block-wise bitwidth allocation but lacks a clear definition of what constitutes a *block*. As far as I understand, it should refer to the same concept as *group* in group-wise quantization, rather than a transformer block. This should be made consistent throughout the paper.\n3. The importance matrix is unstructured and is related to the scaling factors and bitwidth allocation. If this information is required during inference, such as identifying the corresponding weights for the scaling factor when GEMM, the additional memory overhead introduced by the importance matrix will be non-negligible. For instance, an importance matrix could increase the average bitwidth of the weights by 1bit. The authors should provide a more detailed explanation of this aspect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sxEcjKxTG5", "reviewer": "Reviewer_hDfx", "summary": "This paper introduces Quant-dLLM, a post-training quantization (PTQ) framework tailored for diffusion-based large language models (dLLMs) under an extreme 2-bit weight-only budget. The authors propose three key components: 1) Masked Calibration Simulation (MCS) to align calibration with timestep-dependent masking; 2)Data-aware Any-order Quantizer (DAQ) to reconstruct weights using multi-binary matrices with row-column scaling; 3)Adaptive Blockwise Mixed Precision (ABMP) to allocate varying precision across blocks under a strict 2-bit average.\nExtensive experiments on recent dLLMs (LLaDA, Dream) show that Quant-dLLM significantly outperforms strong baselines (GPTQ, GPTAQ, Slim-LLM) across general QA, math, and code tasks.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, with detailed algorithms and clear exposition. The authors commit to releasing code and models.\n2. Quant-dLLM shows improvements of challenging tasks like math and code. Ablation studies verify the effectiveness of each module.\n3. MCS directly tackles the distributional mismatch caused by masked denoising in dLLMs, a unique challenge not present in autoregressive LLMs.", "weaknesses": "1. Missing Baselines: The paper focuses on 2-bit quantization and lacks classical comparison methods [1] [2]. \n2. Incremental Novelty: The core ideas, multi-binary approximation and mixed-precision allocation, are extensions of prior work (e.g., Slim-LLM). The authors should clearly articulate the technical distinctions and why these ideas are nontrivial to adapt to dLLMs.\n\n[1] QuIP: 2-Bit Quantization of Large Language Models With Guarantees\n\n[2] DB-LLM: Accurate Dual-Binarization for Efficient LLMs", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)