Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Authors: Zheng Gong
Ying Sun
Venue: ICML 2025 poster (2025)
Keywords: Post-Training Quantization
Discrete Graph Diffusion Models
Primary area: deep_learning->graph_neural_networks

Abstract: Discrete Graph Diffusion Models (DGDMs) mark a pivotal advancement in graph generation, effectively preserving sparsity and structural integrity, thereby enhancing the learning of graph data distributions for diverse generative applications. Despite their potential, DGDMs are computationally intensive due to the numerous low-parameter yet high-computation operations, thereby increasing the need of inference acceleration. A promising solution to mitigate this issue is model quantization. However, existing quantization techniques for Image Diffusion Models (IDMs) face limitations in DGDMs due to differing diffusion processes, while Large Language Model (LLM) quantization focuses on reducing memory access latency of loading large parameters, unlike DGDMs, where inference bottlenecks are computations due to smaller model sizes. To fill this gap, we introduce Bit-DGDM, a post-training quantization framework for DGDMs which incorporates two novel ideas: (i) sparse-dense activation quantization sparsely modeling the activation outliers through adaptively selected, data-free thresholds in full-precision and quantizing the remaining to low-bit, and (ii) ill-conditioned low-rank decomposition decomposing the weights into low-rank component enable faster inference and an $\alpha$-sparsity matrix that models outliers. Extensive experiments demonstrate that Bit-DGDM not only reducing the memory usage from the FP32 baseline by up to $2.8\times$ and achieve up to $2.5\times$ speedup, but also achieve comparable performance to ultra-low precision of up to 4-bit.

Outlier-Aware Post- Tost-Training Quantization for Discrete Graph Diffusion Models
Zheng Gong'
Ying Sun
Abstract
G
GT
G(G
(a)
Discrete Graph Diffusion Models (DGDMs) mark pivotal advancement in graph generation, effec- tively preserving sparsity and structural integrity. thereby enhancing the learning of graph data dis- tributions for diverse generative applications. De- spite their potential, DGDMs are computationally intensive due to the numerous low-parameter yet high-computation operations, thereby increasing the need of inference acceleration. A promis- ing solution to mitigate this issue is model quan- tization. However, existing quantization tech- niques for Image Diffusion Models (IDMs) face limitations in DGDMs due to differing diffusion processes, while Large Language Model (LLM) quantization focuses reducing memory ac- cess latency of loading large parameters, unlike DGDMs, where inference bottlenecks are compu- tations due to smaller model sizes. To fill this gap, we introduce Bit-DGDM, a post-training quantization framework for DGDMs which in- corporates two novel ideas: sparse-dense ac- fivation quantization sparsely modeling the acti- vation outliers through adaptively selected, data- free thresholds in full-precision and quantizing the remaining to low-bit, and (ii) ill-conditioned low-rank decomposition decomposing the weights into low-rank component enable faster inference and an a-sparsity matrix that models outliers. Ex- tensive experiments demonstrate that Bit-DGDM not only reducing the memory usage from the FP32 baseline by up to 2.8x and achieve up to 2.5x speedup, but also achieve comparable per- formance to ultra-low precision of up to 4-bit.
Pa(G"~11G
sampling noise
recursive denoising
Graph Diffision (Digress
4.3
Image Diffasion (SD1.4'22)
Language Model (Gemma2-202)) 2022
2.9
(b)
2.0
2.2
2.1
1.2
1.3
0.8
0.07
Parameters (B)
Computation (TMACS)
Inference Time (s)
Figure (a) illustration of DGDM the generation of graph. (b) A comparison of the representative models widely used in DGDMs, IDMs. and LLMs, including Digress (Vignac et al., 2023), SD1.4 (Rombach et al., 2022) and Gemma2-2b (Team et al., 2024), terms of parameter sizes, computation time, and inference time. The LLM evaluated with 512 context tokens and 256 output tokens, while DGDM and IDM are assessed over 50 diffusion steps.
1. Introduction
Graph generation plays a pivotal role in graph learning (Li et al., 2024a; Liu et al., 2023c; Gong al., 2023), with applications spanning drug design (Wu et al., 2018), code completion (Brockschmidt et al., 2018), and social network analysis (Newman, 2006). Recently, diffusion models (Rom- bach et al., 2022; Podell et al., 2024) have emerged cutting-edge and effective approach for generative tasks, with significant attention being directed toward graph diffu- sion models (Jo et al., 2022; Vignac et al., 2023), A notable development is the emergence of Discrete Graph Diffu- sion Models (DGDMs) (Yi et al., 2023; Liu al., 2024), which employ discretized diffusion processes, illustrated in Fig. ((a). These approaches exhibit particular capability in preserving sparsity and structural integrity of the gener- ated graphs. However, DGDMs face computational chal- lenges due to their reliance on a recursive denoising process, thereby necessitating dedicated acceleration techniques.
'Attificial Intelligence Thrust, Hong Kong University of Sci- ence and Technology (Guangzhou), Guangzhou, China. Corre- spondence to: Ying Sun Eyings@hhuustg..cdu..r gz..d..cn>
Proceedings of the nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Quantization (Jacob et al., 2018; Wang et al., 2020) rises as crucial direction for model acceleration, wherein floating- point numbers are converted into integers, thereby enhanc- ing computational efficiency from hardware perspective. However, existing quantization methods primarily focus on IDMs and LLMs. For IDMs, quantization approaches are centered on calibrating errors between the Gaussian de- noising processes of full-precision and quantized models, such as time step-aware calibration (Li et al., 2023) and Gaussian noise correction (He et al., 2024b). Moreover, given huge model sizes, LLMSs inference latency is domi- nated by weight loading, prompting LLM-oriented quanti- zation methods to emphasize memory access optimization through vector quantization (Egiazarian et al., 2024) and non-uniform quantization (Kim et al., 2024). However, ex- isting quantization methods are inadequate to address the unique challenges of DGDMs:
sight that the inference efficiency of DGDMs is primarily constrained by computation and the observation that out- liers are prominently present in both weights and activa- tions, Bit-DGDM introduces a novel approach combining sparse-dense activation quantization with ill-conditioned low-rank weight decomposition to effectively mitigates the impact of outliers. Moreover, we implement equidistributed & adaptive sparse-dense kernels tailored for the decomposed matrix multiplication. These techniques achieve high effi- ciency at precisions as low as 4 bits, minimizing memory usage and speeding up inference without sacrificing model performance or the quality of generated graphs. To the best of our knowledge, we are among the first to propose a quantization framework for DGDMs.
The detailed contributions of our work are as follows:
Sparse-Dense Activation Quantization: Recognizing that activations in DGDMs contain significant outliers, our sparse-dense activation quantization method utilizes a sparse matrix modeling outliers and dense matrix modeling remaining values based on adaptively selected, data-free thresholds (Sec. 4.2).
(1) The inference efficiency of DGDMs is bounded by computations instead of memory access. The functioning of DGDMs relies on substantial low-parameter yet high- computation operations (e.g., FiLM network (Perez et al., 2018)), which render DGDMs computationally intensive and result in longer runtime compared to basic Image Diffu- sion Models (IDMs) and Large Language Models (LLMs), as depicted in Fig. 1(b). However, due to the inherent dif- ferences between the discrete diffusion process in DGDMs and the Gaussian denoising process in IDMs, most quantiza- tion methods designed for IDMs face significant limitations when applied to DGDMs. Moreover, since DGDMs have substantially smaller parameter sizes compared L.LMs (0.07B versus 2B, as shown in Fig. 1(b)), the quantization methods designed to reduce the latency of LLM weight loading may be inefficient for the quantization of DGDMs.
II-Conditioned Low-Rank Weight Decomposition: To address the outliers in weights and enhance computational efficiency, we decompose the weights into low-rank com- ponents and an a-sparsity matrix that models outliers, ensuring guaranteed recovery (Sec. 4.3).
Equidistributed & Adaptive Sparse-Dense Kernels: To optimize inference speed and precision, we quantize the dense part of the product of decomposed weights and activations into low-bit representations and design equidistributed, adaptive sparse-dense kernels for effi- cient sparse-dense matrix multiplications (Sec. 4.4).
(2) Considerable outliers in DGDMs are observed in both weights and activations. The outliers, referring to entries with magnitudes significantly larger than others, can substantially impact quantization precision by amplifying magnitudes and compressing the distinctions of remaining values. In text and image tasks, where outliers primarily oc- cur in activations rather than weights, existing quantization methods like smoothing (Li et al., 2024b; Xiao et al., 2023) attempt to transfer these outliers from activations to weights. However, in graph generation tasks, outliers are significantly present in both activations and weights (Sec. 4.1), render- ing both aspects highly susceptible to outlier effects. This dual vulnerability diminishes the effectiveness of smoothing methods in mitigating outliers (Sec. 5.2). Consequently, se- lecting an appropriate quantization approach to mitigate the influence of outliers in both activations and weights is cru- cial for maintaining the performance of quantized DGDMs.
We conduct extensively experiments across multiple DGDMs and diverse datasets to demonstrate that Bit-DGDM can effectively quantize DGDMs and ob- tain the superior performance compared with other SOTA quantization baselines. Notably, Bit-DGGDM achieves 2.5x inference speed while reducing the memory foot- print by up to 2.8x (Sec. 5.2 and Sec. 5.3).
Evaluation:
2. Related Work
The related work of this paper falls in two categories: Graph Generation Models and Model Quantization. Detailed re- cent advancements of Quantization for LLMs and Outlier- Aware Quantization can be found in Appendix A.
2.1. Graph Generative Models
With the widespread prevalence of graph data (Sun et al., 2024; Ji et al., 2025; Gong & Sun, 2024b;a; Huang et al., 2022; Shi et al., 2024; Han et al., 2024), graph generation has found extensive applications, including inverse protein folding (Yi et al., 2023), molecule generation (Wu et al.,
In this paper, we propose post-training quantization frame- work for DGDMs, called Bit-DGDM. Grounded in the in-
2

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
2018), and program synthesis (Brockschmidt al., 2018). Early on, there was considerable interest in graph generation using RNNs (You et al., 2018), VAEs (Simonovsky & Ko- modakis, 2018), GANs (De Cao & Kipf, 2018) or normaliz- ing flows (Zang & Wang, 2020). Inspired by the tremendous success of diffusion models in the image domain, Niu et al. (2020) relied on Gaussian noise and generated adjacency ma- trices by thresholding continuous values to represent edges, while Jo et al. (2022) extended this approach incorporate both node and edge attributes. Given the discrete nature of graph structures, Haefeli et al. (2022) were the first to de- sign a discrete diffiusion model specifically for unattributed graphs, demonstrating that discrete diffusion can enhance graph generation. Furthermore, Vignac et al. (2023) intro- duced the first Markovian noise model, Digress, tailored for attributed graphs, and Yi et al. (2023) extended Digress to inverse protein folding task where the node features are represented as 3D coordinates. In our study, we focus on the quantization of discrete graph diffusion models, as they in- herently capture discrete structures and demonstrate greater efficacy in graph generation.
[ and [&], respectively. For graph g (x, E) with [V nodes, X E | attributes, and each row x; E g denote the one-hot en- coding of node attributes. E E R ||||||I represents the one-hot encoding ei E RI of each edge, treating the absence of edge as a particular edge type. Here, we take the 2D graph generation problem as an example, while the input for the other application of graph generation will be discussed in detail in Appendix C.
is the matrix of node categorical
3.1. Discrete Graph Diffusion Models
Discrete graph diffusion models consist of two main com- ponent: noise process and recursive denoising process.
Noise process. The noise process q progressively cor- rupts graph 9 and creates a sequence of increasingly noisy graphs (G',..,,9*),, where G' = =(x( E²) It has a Markovian structure, where g(g (" = (( (g) IT (g In line with Vignac et al. (2023), we add noises by sampling each node and edge type from a categorical distribution as
(0")g" (X'- E'
(1)
2.2. Model Quantization
where the transition matrix N = a'I 8 I. m' and N E = a'I + 3 m'm m'x and m'a denote the true cat- egorical distributions of node and edge attributes as the prior distributions, respectively. Since (1. )² m'x
Model quantization aims to save memory costs and speed up computations by reducing the precision of weights and activations. It can be generally divided into two cate- gories. Quantization-avure training (QAT) (Jacob et al., 2018; Zhuang et al., 2018) integrates quantization during the training process, enabling high performance lower precision. However, it necessitates significant time and computational resources. In contrast, post-training quanti- zation (PTQ) (Wei et al., 2022a; Lin et al., 2022) does not require fine-tuning and only requires a small amount of unla- beled data for calibration, offering a faster and less resource- intensive alternative. Due to the huge computation costs of diffusion models, recent quantization approaches (He et al., 2024b;a; Li et al., 2024b) for diffusion models focused PTQ. These approaches have pushed the boundaries of PTQ to 4-bit quantization by leveraging new rounding strate- gies (Nagel et al., 2020), layer-wise calibration (Wang et al., 2020), second-order statistics (Li et al., 2021), and mixed- precision schemes (He et al., 2024b). However, these meth- ods primarily focus on continuous diffusion models, partic- ularly in the image domain. To the best of our knowledge, there is lack of methods addressing PTQ for DGDMs.
we have the closed-form N x N
X ....N's = a''I+
3 1.m'xx. where a' = II a and 3 = =1 - a², sat-
isfying Vi, limm N Nx11 11, = m'x- Thus, it then follows
xX' ( = Cat (X XN X and
X'N X'N X X
q( xXX'XXXXX) = Cat(X
) (2)
where Cat denotes categorical distribution. The edge transi- tion distribution is defined similarly and omitted for brevity.
Recursive denoising process. The denoising process pe (G = p(G") II Po (G parameterized by 0 aims to recover the distribution p(g") (for brevity, go := 9).
The reversed transition po () is a product of categor- ical distributions over nodes and edges, i.e., Po( and pa Pa (g') Aligned with the no-paraameterizationn used in continuous diffusion models (Karras et al., 2022; Ho et al., 2020), po (x('-1 is modeled as:
(x ( A
Σ X
x X X0) pa X
(3)
3. Preliminary
where pe X g') is a neural network predicting posterior probability of x given a noisy graph G*. The architec- tures of neural networks will be illustrated extensively in Appendix C. The reversed process for edges is similar to the nodes and thus omitted. After training, the new graph sam- ples can be generated by sampling GT from prior distribu- tions m'x and m'g and subsequently sampling (G(" resulting in a generation trajectory (G".GT" .......!)!
In this section, we first revisit discrete graph diffuuion mod- els and then introduce the quantization process. Follow- ing Vignac et al. (2023), we focus on graphs with categorical node and edge attributes, particularly well-suited for diverse structured data like chemical compounds (Wu et al., 2018) and proteins (Ingraham et al., 2019). Let X' and & denote the categorical space of nodes and edges with cardinalities
3

---

Outlier-Aware Post- Post-Training Quantization for Discrete Graph Diffusion Models
Remark 3.1 (DGDMs versus IDMs). In DGDMs, both the noise process and the recursive denoising process model the node attributes and graph structures through discrete sampling from categorical distributions. In contrast, IDMs rely on Gaussian noise and denoising processes, where each step is represented as continuous values. This difference renders many IDM-oriented quantization methods, such as Gaussian noise correction (He et al., 2024b; Li et al., 2023), difficult to apply to DGDMs.
Activation (X) and Weight (W)
Kurtosis (c) xf and
in lnyeer (Layer Index:3)
actoss different layers
x1 hesviertals
0.1%outiie
outhess
flattrr shope
Quilar
Court
Bamore 0.2% Outler
9.003
0.006
oul%ess
X
0.5% outhiers
Layer Index
Outliz
Hemmaee 1% Outiie
Camt
235
3.2. Quantization
10
200
Quantization is effective approach reduce model size and accelerate computations. Given a tensor X, the quanti- zation process is defined as:
1.75
-.005
0.00
0.05
Layer lncks:
Figure 2. The left panels illustrate the numerical distributions of activations and weights for the layer with index 3. On the right, the top and bottom panels show how removing the largest and smallest 0.1% and 0.5% of outliers significantly reduces the kurtosis of the numerical distributions. A lower kurtosis corresponds to a flatter distribution, which facilitates quantization.
x
Xg- clamp(
+ +2,00,2 20 -1). 1):
(4)
8x
where X is the low-bit representations of X, 8x xx)i is the scaling factor, 2x
min(X)
28-1
"x
is the zero point, and 2 - denotes the maximum quan- tized value with respect to the bit width b. The dequantized tensor is formulated as:
and weights in one layer of DGDMs, along with the top and bottom 0.1 % and 0.5% quantiles marked as outliers. It is ev- ident that these outliers substantially increase the magnitude. In the right of Fig. 2, kurtosis (R) is employed as a statistical measure, defined as K ((2:)=21 = ((EL ():-)))"))
X Q (Q(X)) 8x (Xe- -xx)
(5)
n (2))(=)))) A higher kurtosis value indicates heavier tails, while lower kurtosis corresponds
Due to the high computational costs of diffusion model train- ing, we focus on post-training quantizing the parameters of well-trained discrete graph diffusion models, i.e., 0 in Eqn. (3), to reduce memory requirements and enhance the efficiency of graph generation, e.s. moooccll synthesis (Wu et al., 2018) and protein folding (Ingraham al., 2019).
smoother distribution (Westfall, 2014). Removing a small number of the most significant outliers is shown to significantly reduce kurtosis, thereby improving quantization precision.
4.2. Sparse-Dense Activation Quantization
4. Bit-DGDM: Outlier-Aware PTQ for DGDM
The analysis of DGDMs in Sec. 4.1 indicates significant presence of outliers in the activations. Additionally, we observe that the volume of activations is substantially larger than that of the weights (e.g., 10t versus 10x for a single layer, as shown in Fig. 2). highlighting the computational intensity of DGDMs. Therefore, it is imperative to propose efficient quantization method for activations that not only ensures quantization precision but also mitigates the impact of outliers. With this in mind, we introduce method to filter out outliers and quantize the activations. We first perform a simple yet effective decomposition of activation X R into a sparse matrix Sx containing the outliers and a remaining dense matrix Dx:
In this section, we first discuss the impact of outliers in the quantization of DGDMs in Sec. 4.1. Then we introduce the our post-training quantization framework Bit-DGDM comprising sparse-dense activation quantization and ill- conditioned low-rank weight decomposition, detailed in Sec. 4.2 and Sec. 4.3, respectively. At last, we introduce our kernel implementation for efficiently inference on the quantized DGDMs Sec. 4.4. The overall architecture and algorithm of our framework are shown in Fig. 3 and Alg.
4.1. Outlier Analysis of DGDMs
The quantization process based on Eqn. (4) and (5) reveals that the presence of outliers significantly amplifies the scale factor by increasing the maximum value and decreasing the minimum value. Since adjacent ranges of the scale factor are quantized into neighboring integers, larger scale factor im- plies that a wider range of floating- point numbers is mapped to the same integer, leading to greater quantization errors and ultimately degrading quantization precision. In the left panels of Fig. 2, we visualize the distributions of activations
x Dx + Sx.
(6)
Dx can be effectively quantized into low-bit representa- tion Dx due to its significant reduced value ranges. The outliers contained in sparse Sx are filtered from X via Sx = X[x > Tmax|> < Tmin]. where the thresholds Tmux//min are selected based on the percentile of the value distribution.
Data-free Adaptive Threshold Selection. The selection of thresholds Tmax/min typically requires a substantial num-
4

---

Outlier-Aware Post- Wost-Training Quantization for Discrete Graph Diffusion Models
Difficult to quantize
Easy to quantize
Sparse- Dense Quantization
1.0
(Sec. 4.2)
1.0
Quantization and Efficient K (Sec. 4.4)
X
0.
0.
0.9
Bacch
Bateh
Botch
Soput Chamea
Ippur Churnel
Ippot Chansel
Sparse Sx
Dense Dx
XW = (Sx + Dx) Dw)
Il-conditioned Decomposition
Difticult to quantize
Easy compute
Dy + SxLR + XSW
0.75
(Sec. 4.3)
0.7
16-bit bit
u.50 [W]
Sparse-
W
0.25
+
rank=32
low-bit
multiplication
Ourpor Charme
16-bit
Inpur Channel
a- Sparsity Sw
Low-Rank Dw LR"
Figure 3.
Our quantization framework Bit-DGDM consists of three phases. (i) Decomposing the activations into sparse matrix modeling
outliers and a dense quantized matrix modeling remaining values (Sec. 4.2). (ii) Utilizing the ill-conditioned low-rank decomposition method for decomposing weights into a-sparsity matrix and low-rank components (Sec. 4.3). (iii) Quantizing the dense parts low-hit representations and implementing efficient kernel for sparse-dense matrix multiplication (Sec. 4.4).
ber of activation values to ensure accurate percentile estima- tion. A straightforward approach involves inferring activa- tion values using a training dataset. However, this necessi- tates access to large raw dataset. Acquiring such datasets is often challenging due to their size, privacy concerns, and copyright restrictions. Moreover, avoiding the use of raw datasets enhances the applicability of post-training quanti- zation. With this in mind, we propose a data-free method for adaptive threshold selection. Our approach begins by sampling multiple initial graphs } {X"", E²) from uniform distributions of node and edge types:
sion reduction robustness for the outlier-existnng matrix, we propose ill-conditioned low-rank weight decomposition method. The overall algorithm for weight decomposition is illustrated in Alg. 2. We assume that the weight W E Rm is superposition of Dw and Si::
W = Di + SW:
(8)
where Sty is sparse matrix modeling the outliers of weight W, and Div is a rank-r matrix modeling the remaining component. Herein, denote the ideal or target values, while the following variables without denote the variables to be optimized. We define the sparsity of Sw following o-sparsity constraint (Yi al., 2016; Netrapalli et al., 2014);
X ~ U((11 (X)).. E (((()...,,1))))) (7)
These graphs are processed through the full-precision DGDM to obtain intermediate activation values in each steps. Thresholds from all steps within the same layer, using predefined per- centiles, thereby facilitating effective quantization without relying on raw datasets.
Definition 4.1 (a-sparsity). The sparse matrix S is a- sparsity if at most a-fraction of nonzero entries per row and column for a E [0, 1), which we denote as:
then selected based on the activations
Sa : (S R X Vi. |S S an, ||S S am).
4.3. il-conditioned Bon-Rank Weight Decomposition
Our analysis of outliers in activations and weights indicates that, unlike in LLMs where outliers predominantly occur in activations, DGDMs exhibit significant outliers in model weights as well. This characteristic may render quantization methods via smoothing inefficient (Xiao et al., 2023), as reducing the magnitude of activations comes at the cost of amplifying the magnitude of weights, thereby exacerbat- ing the impact of outliers on weight quantization. More- over, quantization approaches that mitigate outlier effects through SVD decomposition (Li et al., 2024b) may also be compromised, as outliers can substantially influence the magnitude of singular values and the direction of singular vectors. Inspired by robust principal component analysis (RPCA) (Candès al., 201 1) for enhancing the dimen-
avoid the low-rank constraint on Dw we write Dw as product of Dw LR T with L E R.xr and R E Rarr We utilize the low-rank matrix LR and the sparse matrix Sw to reconstruct the weight matrix W through the following optimization objective:
min LeRmxr RReexx Sw.S.
[R T + Sw W
(9)
where || || denotes Frobenius Norm. To enable gradi- ent descent on efficiently optimizing L and R, we denote the loss function in k-th iteration L, == L(L,, R₂) = LLR. + Sw. - WIß- To enforce the a-sparsity con- straint of Sw in Def. 4.1, we employs a sparsification oper-
5

---

Outlier-Aware Post- fost-Training Quantization for Discrete Graph Diffusion Models
ator To (.) for any matrix M E Rmxx:
Assumption 4.3 (a-Sparsity of Outliers). Let Sw E Rmxn be a-sparsity matrix representing the outliers. Specif- ically, we impose the condition S ((L/(ur)/2R)) to ensure guaranteed recovery.
if [MM 2 [M (an) 1.
[M]j
(M)(M)
and [ 2 [M] (
0,
otherwise,
Remark 4.4. Our assumption aligns with the most com- mon RPCA models, which address ill-conditioned low-rank compositions under a sparsity assumption. Notably, this assumption based on the observation that the number of outliers in the weights is inherently limited, whereas their magnitude has a more significant impact on quantization.
(10)
where F and 11199) denote the entry with a-th largest
magnitude in the i-th row and j-th column, respectively. In
the (k + 1) optimization iteration, the sparse matrix Sw is updated as:
SWA++ = (W R
(11)
Based on the rank-sparsity uncertainty principle, a ma- trix cannot simultancously exhibit incoherence and spar- sity (Chandrasekaran et al., 2011). The two assumptions outlined above ensure the uniqueness of our weight decom- position solution. We now present our main theorem:
After updating Sw the gradients w.r.p La and R& can be exactly computed as:
VI, (L + SW - W) R
(12)
VRL = (L,RR, + SWA++1 W) Lg.
Theorem 4.5 (Guaranteed Recovery). Let Diy be a rank- matrix with j-incoherence, and let Siy be aparsityy matrix with a S O(1 ((11/1)3/2kk) The iterative optimization at step k satisfies
However, the vanilla gradient descent optimization suffers from the ill-conditioned matrix, and thus we incorporate the scaled terms (R. R&)) and (L L&) -1 to overcome the impacts of outliers following (Tong et al., 2021). We update the low-rank components:
ILaRR - DilF S 0.03(1 - dist(Lk, RR: L' R*) < 0.02(1 o,
o, 0,
L++11 = L Th++1 VT,, Lx
(R., RR)
-1
(13)
R = R 7k+1 VR, Lk (L Lx)
-1
where the step size 17 [0.1.2/3].
where 1/4+11 denote the step size at the (k + 1) iteration.
Proof. The proof is detailed in Appendix B.
4.4. Quantization and Kernel Implementation
Initialization. We introduce an initialization of low-rank components Ly and Rg as well as the sparse matrix S'. for reducing the effects of outliers in the weight decompo- sition. We first assign the sparse matrix Sw. Ta (W) to remove the obvious outliers with largest magnitude. Then we initialize Lo- U Ro Vo Eg where Uo Vo the best rank-r approximation of W Sw..-
To efficiently handle matrix multiplication between activa- tions and weights in the presence of outliers, two conditions must be satisfied: (1) outliers should be computed with high precision, and (2) the quantized matrices should be computed using the same bit-widths. With these consider- ations, we proceed as follows: (i) We first decompose the activations and weights using Eqn. (6) and (8). (ii) We then perform quantization on the dense components Dx and Dw to the same bit-width, ensuring the removal of outliers. For the multiplication involving activation outliers S x, we decompose Dw into a low-rank approximation LR" to maintain precision and efficiency. (iii) Finally, we compute the product of the sparse weight outliers Sw with the raw activation X. This process is formulated as:
Theoretical Guarantee. We present the recovery guar- antee of our proposed ill-conditioned low-rank weight de- composition method here. We start with the p-incoherence assumption of Di following RPCA works (Candès et al., 2011; Chen, 2015):
Assumption 4.2 (p-incoherencce). Div E mmx is a rank-r matrix with p-incoherence, i.e.,
XW = (Dx + Sx) + Sw)
(14)
U. U2112,00 S
jur V
pr and IV.12,000 S V n
Dx Dw + SSD + DxSw + SSSW
m
for 1 S μ S. max(m, n), where U.E.V V is the compact SVD of Div, and |M]| max; C M ).
[6X]
DxDW + LRLR DD + S
(6x))
D Dx + SxLR T + +XSw
Herein, U, E Rmxrr and V, E Raxr are composed of r left and right singular vectors, respectively. Σ. E x is a diagonal matrix consisting of r singular values of D organized in a -on-incceasin order, i.e., 1n ) 2 ... 2
low-hit hit
Sparse Dense Multiplication
Equidistributed and Adaptive Sparse-Dense Kernel. To optimize the computational efficiency of step (iii) in Eqn. (14), the sparse-dense matrix multiplication plays a
a, (Diw) > And we define k == 01 (D())// (Diw).
6

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Table Molecule generation performance of frel-precision Digress (Vignac et al., 2023) (i.e., haseline) and quantized Digress with different quantization methods on QM9 and MOSES datasets. "WxAy" denotes that the weights (W) and activations (A) are represented æ-bit and y-bit precision, respectively. "Mem." " refers to the peak memory usage during the model inference.
Molecule Generation
QM9
MOSES
Valid (%)
Unique (%)
Speedup (1)
Mem. (GB 4
Novel (%)
Valid (%)
Unique (%)
Speedup (f)
Mem. (GB ))
Method
Precision
FP32
99.0
96.2
Ix
6.4
93.8
82.5
99.8
1x
8.7
basclinc
BF16
98.8
96.0
1.3x
3.4
93.6
82.5
99.7
1.3x
4.6
RTN
W4A4
57.6
88.6
2.7x
1.6
84.2
63.8
87.9
2,8x
2.3
GPTQ
W4A4
72.4
86.1
2.7x
1.6
85.1
72.0
92.6
2.8x
2.3
SVDQuant
W4A4
95.6
92.7
2.2x
2.0
91.6
80.6
96.5
2.2x
3.0
DuQuant
W4A4
96.5
94.6
2.1x
2.1
92.8
81.1
98.0
2.1x
3.2
SqueezeLLM
W4A16
95.9
94.8
1.1x
2.3
92.5
81.3
98.0
1.1x
3.6
Bit-DGDM
W4A4
98.2
95.5
2.5x
2.3
93.1
81.9
98.9
2.5x
3.8
critical role. The term XSw, where Sw adheres to a- Sparsity, implies that each row and column contains at most an a-fraction of nonzero entries. In other words, the nonzero elements are evenly distributed across each row and column. To address this, we design a equidistributed sparse-dense kernel that assigns each thread a column, ensuring uniform workload distribution across threads. For the term S x LR given that the occurrence of outliers in the activations is ran- dom, we have developed an adaptive sparse-dense kernel based on (Flegar& & Quintana-Ori 2017). This approach assigns an equal number of nonzero entries to each thread. Although this necessitates additional synchronization, as each column of L is processed by multiple threads, en- sures a more balanced workload in terms of time. Since the computations for XSW and x LR are independent, the equidistributed and adaptive sparse-dense kernels are launched in a single call. The detailed descriptions of these two kernels are demonstrated in Appendix E.
and MOSES (Polykovskiy et al., 2020), well as non- molecular benchmarks (Martinkus et al., 2022) such as SBM and planar graphs. Moreover, we investigate protein inverse folding task, an essential application of graph generation, where node relationships are provided by 3D) coordinates and assess the model's ability recover the correct amino acid sequence given protein's 3D structure. We utilize the SOTA GRADE-IF (Yi et al., 2023) as the DGDM, based on Equivariant Graph Neural Network (Satorras et al., 2021).
Baselines. As there is a lack of PTQ baselines for DGDMs, we compare Bit-DGDM against various PTQ methods for LLMs and IDMs including RTN (round-to-nearest] GPTQ (Frantar et al., 2023), SqueezeLLM (Kim al., 2024), SVDQuant (Li et al., 2024b), DuQuant (Lin et al., 2024). Their descriptions are provided in Appendix F.I.
Metries. The metrics for evaluating the effectiveness, of molecular graph generation include Validity, Uniqueness, and Novelty, while the key metrics for assessing 3D pro- tein inverse folding are Perplexity and Recovery (detailed descriptions can be found in Appendix F.2). The efficiency of model inference after quantization evaluated using Speedup and Peak Memory Usage (Mem.).
5. Experiments
In this section, conduct empirical experiments to demon- strate the efficacy of our Bit-DGDM We include quantiza- tion cost analysis and other results in Appendix F.
Quantization details. For Bid-DGDM, we first determine the thresholds, Tmax and Tiins for identifying activation out- liers by selecting the top 0.1% highest and lowest values, respectively. To obtain the activation values, randomly generate 32 samples using the full-precision DGDMs. For low-rank weight decomposition, we set the rank r to 32, the o-sparsity parameter a to 1%, and the step size to 0.
5.1. Experimental Setup
Models and Datasets. We conduct a comprehensive eval- uation of Bit- DDDM'sS effectiveness in the quantization of various DGDMs and their applications. For 2D structured graphs, where node relationships are represented using adja- cency matrices, we employ the widely used discrete DGDM, Digress (Vignac et al., 2023), which utilize FiLM (Perez et al., 2018) and Graph Transformer (Dwivedi & Bresson, 2020) backbones. These models applied to molec- ular synthesis datasets, including QM9 (Wu et al., 2018)
Latency Profiling. We use the Torch CUDA profiler to measure the latency and peak memory usage for generating graphs with a batch size of 16 on single NVIDIA RTX- 3090 GPU, complemented by two 2.20GHz Intel Xcon Gold 5220R CPU, and 512GB of CPU memory. The node size n and cardinalities of nodes and edges |x'), |E| is decided based on the dataset settings.
'Our code is publicly available here.
7

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Quantization Performance on CATH
Table Inverse protein folding performance of pree-precision
3.5
GRADE-IF (Yi et al., 2023) (i... baseline) and quantized GRADE-
S6.D Activation
Perplexity Speedup
with different quantization methods on CATH dataset.
3.0
QuantiFation (Sec 4.20
[I]
Perplexity
(+ [e-endiiioned W Vight Decoppsition
3.0
Speedup
Protein Folding
CATH
2,8%
Sec 4.3)
0b+ Wight Quartization
2.6%
Perplexity Recovery
Speedup Mem.
Method
Precision
2.5
(4)
(%)
(7)
(GB !)
2.2
FP32
4.4
52.2
1x
12.8
baseline
BF16
4.4
52.1
1.3x
6.8
2.0
RTN
(i)
(II
BIt-DGDM
(iii) -S
RTN
W4A4
7.6
32.8
3.0x
3.7
GPTQ
W4A4
7.1
35.4
3.0x
3.7
Figure 4. Perplexity and speedup on CATH with different ablations of Bit-DGDM components. "S&D" denotes sparse-dense.
SVDQuant
W4A4
6.3
39.5
2.1x
4.6
DuQuant
W4A4
5.6
42.7
2.1x
4.7
SqueezeLLM
W4A16
4.7
48.6
1.0X
Q Quattization Performance on MOSES
Bit-DGDM
W4A4
4.5
51.6
2.5X
4.9
Valid
2.55
2.53
2.50
Speedup
82
81.33
81.11
80
8145
2.15
2.25
5.2. Graph Generation Results
0.01
0.1
0.2
0.5
Quantile of outhiers in (%)
The graph generation performance molecules and non- molecular benchmarks are shown in Table and Table A2, respectively. Overall, Bit-DGDM achieves graph generation quality closer to BF16 while delivering higher computa- tional speed and significantly reduced memory usage com- pared to quantization baselines. For instance, on the QM9 dataset, Bit-DGDM surpasses the state-of-the-art quanti- zation method DuQuant in graph generation quality, with 1.7% improvement on the Validity metric and 0.9% improvement on the Uniqueness metrie. Additionally, it achieves higher computational efficiency, with speedup of 2.5x compared to DuQuant's x. These results demon- strate the superiority of Bit-DGDM in quantizing DGDM models. Notably, while Sq stores weights as low-bit values using a lookup table, it still loads them as BF16 and performs weight-activation multiplications in BF16. SqueezeLLM does not achieve significant accel- eration these datasets, indicating that reducing model weight loading contributes only marginally to the acceler- ation of DGDMs. This finding supports the argument that DGDMs are low-parameter and high-computation models. Moreover, Bit-DGDM significantly outperforms SVDQuant, which mitigates the impact of outliers through smoothing and SVD, well DuQuant, which addresses outlerr ef- fects via rotation, by a large margin. This demonstrates that outlier-aware quantization methods designed for LLMs and IDMs are inefficient when applied to DGDMs.
2.55
2.50
Speedup
Valid
82.08
81.65
81.3
82.03
2.40
80.74
2.38
2.30
2.30
0.1
0.5
of epaapsitty for W (%)
Figure 5. Results of valid ratio and speedup on MOSES with dif- ferent hyperparameters.
Furthermore, compared to BF16, Bit-DGDM significantly reduces memory usage and achieves nearly 2 acceleration in computational speed. This result highlights the general- izability of Bit-DGDM across various DGDM generation models. It demonstrates that Bit-DGDM not only maintains high-generation quality also effectively balances com- putational efficiency and resource utilization, making it a practical choice for real-world applications.
5.4. Ablation Study
To validate the effectiveness of our proposed module, Figure presents performance comparison between Bit-DGDM and several variants on CATH dataset. (i) "RTN S&D acti- vation quantization" represents the RTN baseline integrated with our sparse-dense activation quantization (Sec.4.2). (ii) "(i) + S&D weight quantization" extends variant (i) by in- corporating our ill-conditioned low-rank weight decompo- sition module. (iii) Bit-DGDM-S,, Sw removes the term S Sw Eqn. Our findings reveal that: (1) Sparse- dense activation quantization significantly enhances model accuracy (i vs. RTN). (2) Ill-conditioned weight decompo- sition improves efficiency while maintaining comparable performance (ii vs. Bit-DGDM), (3) Overlapping outliers in weights and activations substantially degrade performance (iii vs. Bit- DGDM).
5.3. Inverse Protein Folding Results
Inverse protein folding is critical application of graph generation. In Table 2, we validate the effectiveness of Bit- DGDM in quantizing and accelerating DGDM. Bit-DGDM achieves the best performance among all quantization base- lines in terms of both Perplexity and Recovery metrics, remaining very close to the performance of BF16 precision.
8

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
5.5. Hyperparameter Analysis
References
In our proposed Bit-DGDMs, the quantile of activations with extreme values is defined as outliers, and the a S-aassitty configuration for weights serves as two critical hyperpa- rameters that balance the model's 's inference efficiency and predictive performance. Figure depicts our evaluation of diverse quantile of activation outliers and a-Sparsity config- urations on MOSES dataset. Experimental results indicate that selecting an appropriate number of outliers ensures comparable performance without significantly reducing in- ference speed. Excessive outliers slow down inference, while too few compromise performance.
Bai, H., Zhang, W., Hou, L., Shang, L., Jin, J., Jiang, X., Liu, Q., Lyu, M., and King, I. Binarybert: Pushing the limit of bert quantization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pp. 4334- 4348, 2021.
Bondarenko, Y., Nagel, M., and Blankevoort, T. Under- standing and overcoming the challenges of efficient trans- former quantization. In Proceedings of the 2021 Con- ference Empirical Methods in Natural Language Pro- cessing, pp. 7947-7969, 2021.
6. Conclusion
Brockschmidt, M., Allamanis, M., Gaunt, A. L.,, and Polo- zov, O. Generative code modeling with graphs. In Inter- national Conference on Learning Representations, 2018.
This study investigated a crucial research question: post- training quantization for DGDMs. The aim to reduce the memory usage and enable faster inference of DGDMs without compromising model performance or the quality of generated graphs. To tackle this issue, we proposed novel quantization framework for DGDMs called Bit-DGDM. Within this framework, we developed a sparse-dense ac- tivation quantization and an ill-conditioned low-rank weight decomposition to mitigate the outlier impacts in the quan- tization and improve computation efficiency. Furthermore, implemented equidistributed and adaptive sparse-dense kernels to accelerate sparse-dense matrix multiplication. Our comprehensive experimental results demonstrated the effectiveness of our framework in quantizing DGDMs.
Buluç, A., Fineman, T., Frigo, M., Gilbert, R.,, and Leiserson, C. E. Parallel sparse matrix- and matrix- transpose-vector multiplication using compressed sparse blocks. In Proceedings of the twenty-firs annual sympo- sium on Parallelism in algorithms and architectures, pp. 233-244, 2009.
Cai, H., Liu, J., and Yin, W. Learned robust pea: A scalable deep unfolding approach for high-dimensional outlier detection. Advances in Neural Information Processing Systems, 34:16977-16989, 2021.
Acknowledgement
Candès, E. J., Li, X., Ma, Y., and Wright, J. Robust principal component analysis? Journal of the ACM (JACM), 58(3): 1-37, 2011.
This work is partly supported by the National Natural Sci- ence Foundation of China (No. 62306255, 92370204), the National Key Research and Development Program of China (No. 2023YFF0725000, the Guangdong Basic and Ap- plied Basic Research Foundation (No. 2024A1515011883), the Fundamental Research Project of Guangzhou (No. 2024A0444233), and the Education Bureau of Guangzhou Municipality.
Chandrasekaran, V., Sanghavi, S., Parrilo, P. A., and Willsky, A. S. Rank-sparsity incoherence for matrix decomposi- tion. SIAM Journal on Optimization, 21(2):572-596 2011.
Chen, Y. Incoherence--ppimall matrix completion. IEEE Transactions Information Theory, 61(5):2909-29222 2015.
Impact Statement
This paper presents a machine learning method that en- hances computational efficiency. The technique aims to expand the accessibility of machine learning across various sectors, without anticipated negative social impacts. Our goal is to foster innovation and inclusivity, making advanced technologies available to a broader audience of developers.
De Cao, N. and Kipf, T. MolGAN: An implicit generative model for small molecular graphs. ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models, 2018
Dettmers, T., Svirschevski, R. A., Egiazarian, V., Kuznedelev, D., Frantar, E.,, Ashkboos, S., Borzunov, A., Hoefler, T., and Alistarh, D. Spqr: A sparse-quantized representation for near-lossless 1lm weight compression. The Tweifth International Conference on Learning Representations, 2024.

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Dwivedi, V. P. and Bresson, X. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 2020.
He, Y., Liu, L.,, Liu, J., Wu, W., Zhou, H., and Zhuang, B, Ptgd: Accurate post-training quantization for diffusion models. Advances in Neural Information Processing Systems, 36, 2024b.
Egiazarian, V., Panferov, A., Kuznedelev, D., Frantar, E., Babenko, A., and Alistarh, D. Extreme compression of large language models via additive quantization. In Forty-first International Conference on Machine Learn- ing, 2024.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Huang, M., Liu, Y., Ao, X., Li, K., Chi, J., Feng. Yang, H., and He, Q. Auc-oriented graph neural network for fraud detection. In Proceedings of the ACM Web Conference 2022. pp. 1311-1321, 2022.
Flegar, G. and Quintana-Ori Ortii, E. S. Balanced est sparse matrix-vector product on graphics processors. In Euro- Par 2017: Parallel Processing: 23rd International Con- ference on Purullel and Distributed Computing, Santiago de Compostela, Spain, August 28 Septemberr 1, 2017, Proceedings 23, pp. 697-709. Springer, 2017.
Ingraham, J., Garg, V., Barzilay, R., and Jaakkola, T. Gener- ative models for graph-based protein design. Advances in neural information processing systems, 32, 2019.
Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. GPTQ: Accurate post-training compression for generative pretrained transformers. In The Eleventh International Conference on Learning Representations, 2023.
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quantization and training of neural networks for efficient integer- arithmetic-one only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2704-2713, 2018.
Gholami, A., Yao, Z., Kim, S., Hooper, C.,, Mahoney, M. W., and Keutzer, K. Ai and memory wall. IEEE Micro, (01); 1-5, 2024.
Jeon, Y., Lee, C., Cho, E., and Ro, Y. Mr. biq: Post-training non-un non-uniform quantization based on minimizing the recon- struction error. Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 12329-12338, 2022.
Gong, Z. and Sun, Y. An category-free out-of-distributio node detection in graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 908-919, 2024a.
framework for
Ji, Y., Sun, Y., Zhang, Y., Wang, Z.,, Zhuang, Y., Gong, Z.., Shen, D., Qin, C.,, Zhu, H., and Xiong, H. A comprehen- sive survey on self-interpretabe neural networks. arXiv preprint 1xXi::250111633 2025.
Gong, Z. and Sun, Y. Graph reasoning enhanced language models for text-to-sql. In Proceedings of the 47th In- ternational ACM SIGIR Conference on Reseurch and Development in Information Retrieval, pp. 2447-2451, 2024b.
Jo, J., Lee, S., and Hwang, S. J. Score- generative modeling of graphs via the system of stochastic differen- tial equations. In International conference on machine learning, pp. 10362-10383, PMLR, 2022.
Gong. Z., Wang. G., Sun, Y., Liu, Q. Ning, Y., Xiong, H., and Peng, J. Beyond homophily: Robust graph anomaly detection via neural sparsification. In IJCAI, pp. 2104- 2113, 2023.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion- based generative models. Advances neural information processing systems, 35: 2022.
Haefeli, K. K., Martinkus, K., Perraudin, N., and Watten- hofer, R. Diffusion models for graphs benefit from dis- crete state spaces. In NeurlPS 2022 Workshop: New Frontiers in Graph Learning, 2022.
Kim, S., Hooper, C. R. C., Gholami, A., Dong, Z., Li, X., Shen, S., Mahoney, M. W., and Keutzer, K. Squeezellm: Dense-and-sparse quantization. In Forty-first Interna- tional Conference on Machine Learning, 2024.
Han, X., Cao, M., Xu, D., Feng, X., Liang. Y., Lang, X., and Guan, R. Seoe: an option graph based semantically em- bedding method for prenatal depression detection. Fron- tiers of Computer Science, 1866),1 2024.
Lee, W., Lee, J., Seo, J., and Sim, J. {InfiniGen]: Effi- cient generative inference of large language models with dynamic {KV) cache management. In 18th USENIX Symposium on Operating Systems Design and Implemen- tation ( (OSDI 24), pp. 155-172, 2024.
He, Y., Liu, J., Wu, W., Zhou, H., and Zhuang, B. EF- ficientdm: Efficient quantization-awan fine-tuning of low-bit diffusion models. In The Twelfth International Conference on Learning Representations, 2024a.
10

---

Outlier-Aware Post- fost-Training Quantization for Discrete Graph Diffusion Models
Li, K., Chen, Y., Liu, Y., Wang, J., He. Q., Cheng, M., and Ao, X. Boosting the adversarial robustness of graph neural networks: An ood perspective. In The Twelfth International Conference on Learning Representations, 2024a.
Liu, Y., Du, C., Pang, T., Li, C., Lin, M., and Chen, W. Graph diffusion policy optimization. arXiv preprint arXiv:2402. 2024.
Lv, C., Chen, H., Guo, J., Ding, Y., and Liu, X. Piq4sam: Post-training quantization for segment anything. In Pro- ceedings of the IEFE/CV Conference on Computer Vi- sion and Pattern Recognition, pp. 15941-15951, 2024.
Li, M., Lin, Y., Zhang, Z., Cai, T., Li, X., Guo, J., Xie, E., Meng, C., Zhu, J.-Y., and Han, S. Svdqunat: Absorb- ing outliers by low-rank components for 4-bit diffusion models, arXiv preprint arXiv:2411.0500 2024b.
Martinkus, K., Loukas, A., Perraudin, N., and Wattenhofer, R. Spectre: Spectral conditioning helps to overcome the expressivity limits of one-shot graph generators. In In- ternational Conference on Machine Learning pp. 15159- 15179. PMLR, 2022.
Li, X., Liu, Y., Lian, L., Yang, H., Dong, Z., Kang, D., Zhang, S., and Keutzer, K. Q-diffusion: Quantizing dif- fusion models. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 17535-17545, 2023.
Nagel, M., Amjad, R. A., Van Baalen, M., Louizos, C., and Blankevoort, T. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pp. 7197-7206. PMLR, 2020.
Li, Y., Gong, R., Tan, X., Yang. Y., Hu, P., Zhang, Q., Yu, E., Wang, W., and Gu, S. Breeq: Pushing the limit of post-training quantization by block reconstruction. In International Conference on Learning Representations, 2021.
Netrapalli, P., UN, N., Sanghavi, S., Anandkumar, A., and Jain, Non-convex robust pea. Advances in neural information processing systems, 27, 2014.
Liao, R., Li, Y., Song, Y., Wang, S., Hamilton, W., Du- venaud, D. K., Urtasun, R., and Zemel, R. Efficient graph generation with graph recurrent attention networks. Advances in neural information processing systems, 32, 2019.
Newman, M. E. Modularity and community structure in net- works. Proceedings of the national academy of sciences, 103(23):8577-858) 8582, 2006.
Niu, C., Song. Y., Song. J., Zhao, S., Grover, A., and Ermon, S. Permutation invariant graph generation via score-based generative modeling, In International Conference Ar- tificial Intelligence and Statistics, pp. 1474-4484, PMLR, 2020.
Lin, H., Xu, H., Wu, Y., Cui, J., Zhang, Y., Mou, L., Song, L., Sun, Z., and Wei, Duquant: Distributing outliers via dual transformation makes stronger quantized 1ms. In The Whirtt--ighth Annual Conference on Neural Infor- mation Processing Systems, 2024.
Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with general con- ditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Lin, Y., Zhang, T. Sun, P., Li, Z., and Zhou, S. Fq-vit: Post-training quantization for fully quantized vision trans- former. In Proceedings of the Thirty-Fiis Firrt-FFistttn International Joint Conference on Artificial Intelligence, IJCAI-22, pp- 1173-1179, 2022.
Podell, D., English, Z., Lacey. K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., and Rombach, R. Sdxl: Im- proving latent diffusion models for high- high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
Liu, J., Huang, 7.,, Ma, Z., Liu, Q. Chen, E., Su, T., and Liu, H. Guiding mathematical reasoning via mastering commonsense formula knowledge. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1477-1488, 2023a.
Polykovskiy, D., Zhebrak, A., Sanchez-Lengeling B., Golo- vanov, S., Tatanov, O., Belyaev, S., Kurbanov, R., Arta- monov, A., Aladinskiy, V., Veselov, M., et al. Molecular sets (moses): a benchmarking platform for molecular gen- eration models. Frontiers pharmacology. 11:565644, 2020.
Liu, J., Huang. Z., Zhai, C., and Liu, Q. Learning by ap- plying: A general framework for mathematical reasoning via enhancing explicit knowledge learning. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 4497-4506, 2023b.
Liu, Y., Ao, X., Feng, F., Ma, Y., Li, K., Chua, T.-S., and He, Q Flood: A flexible invariant learning framework for out- of-distribution generalization on graphs. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1548-1555 2023c.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.

---

Outlier-Aware Post- Post-Training Quantization for Discrete Graph Diffusion Models
Satorras, V. G., Hoogeboom, E., and Welling, M. E (n) equivariant graph neural networks. In International con- ference on machine learning, pp. 9323-9332. PMLR, 2021.
Wang, P., Chen, Q. He, X., and Cheng. J. Towards accurate post-training network quantization via bit-split and stitch- ing. In International Conference Machine Learning, pp. 9847-9856. PMLR, 2020.
Shang, Y., Yuan, Z., Xie, B., Wu, B., and Yan, Y. Post- training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1972-1981, 2023.
Wei, X., Gong, R., Li, Y., Liu, X., and Yu, F. Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization. In International Conference on Learning Representations, 2022a.
Wei, X., Zhang, Y., Zhang, X., Gong, R., Zhang, S., Zhang, Q. Yu, F., and Liu, X. Outlier suppression: Pushing the limit of low-bit transformer language models. Advances in Neural Information Processing Systems, 35:17402- 17414, 2022b.
Shen, D., Song, G., Xue, Z., Wang, F.-Y., and Liu, Y. Re- thinking the spatial inconsistency in classifier-fres diffu- sion guidance. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 9370-9379, 2024.
Westfall, P. H. Kurtosis peakedness, 1905-2014, rip. The American Statistician, 68(3):191-195, 2014.
Shi, C., Chen, J., Liu, J., and Yang. C. Graph foundation model. Frontiers of Computer Science, (66):886355, 2024.
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Ge- niesse, C., Pappu, A. S., Leswing, K., and Pande, V. Moleculenet: a benchmark for molecular machine learn- ing. Chemical science, 9()):133-553) 2018.
Simonovsky, M. and Komodakis, N. Graphvae: Towards generation of small graphs using variational autoencoders. In Artificial Neural Networks and Machine Learning- ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, Proceedings, Part I 27, pp. 412-422. Springer, 2018.
Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., and Han, S. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pp. 38087-38099. PMLR, 2023.
So, J., Lee, J., Ahn, D., Kim, H., and Park, E. Temporal dynamic quantization for diffusion models. Advances in neural information processing systems, 36:48686 2023.
Yi, K., Zhou, B., Shen, Y., Liò, P., and Wang, Y. Graph denoising diffusion for inverse protein folding. Advances in Neural Information Processing Systems. 36, 2023.
Sun, Y., Zhu, H., Wang, L., Zhang, L., and Xiong, H. Large- scale online job search behaviors reveal labor market shifts amid covid-19. Nature Cities, 1()):150-163, 2024.
Yi, X., Park, D., Chen, Y., and Caramanis, C. Fast algo- rithms for robust pea via gradient descent. Advances in neural information processing systems, 29, 2016
You, J., Ying, R., Ren, X., Hamilton, W., and Leskovec, J. Graphrnn: Generating realistic graphs with deep auto- regressive models. In International conference on ma- chine learning, pp. 5708-5717, 5717. PMLR, 2018
Team, G., Riviere, M., Pathak, S., Sessa, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024.
Tong, T., Ma, C., and Chi, Y. Accelerating ill- ill-conditione low-rank matrix estimation via scaled gradient descent. J. Mach. Learn. Res., 22:1 12:150 50:1 1150063,, 2021.
Zadeh, A. H., Edo, I., Awad, O. M., and Moshovos, A. Gobo: Quantizing attention braeetioo--bsedd nlp models for low la- tency and energy efficient inference. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitec- ture (MICRO), pp. 111-824. IEEE Computer Society, 2020.
Tseng, A.,
Sun, Q., Hou, D., and De Sa, C. Qtip: Quanti- zation with trellises and incoherence processing, arXiv preprint arXiv:2406. 11255, 2024.
Zang, C. and Wang, F. Moflow: an invertible flow model for generating molecular graphs. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 617-626, 2020.
Vignac, C., Krawezuk, I., Siraudin, A., Wang, B., Cevher, V., and Frossard, P. Digress: Discrete denoising diffu- sion for graph generation. In The Eleventh International Conference on Learning Representations, 2023.
Zhuang, B., Shen, C., Tan, M., Liu, L., and Reid, I. Towards effective low- bitwidh convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7920-7928, 2018.
Wang, C., Wang, Z., Xu, X., Tang, Y., Zhou, J., and Lu, J. Towards accurate post-traiming quantization for diffusion models. arXiv preprint arXiv:2305. 18723, 2023.
12

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
A. Related Work
A.1. Quantization for LLMs
Quantization methods can be categorized into two types based on whether retraining is required. The first type, Quantization- Aware Training (QAT), necessitates retraining the model after quantizing its parameters to maintain performance comparable to the original model (Bai et al., 2021). The second type, Post-Training Quantization (PTQ), directly quantizes model parameters without retraining (Kim al., 2024; Frantar al., 2023). Although QAT generally achieves superior performance, due to the substantial computational cost of training LLMs, most studies adopt PTQ.
Many LLM-specific quantization studies have identified that the primary bottleneck in LLM inference (Liu et al., 2023a:b) lies in the weight loading process rather than computation speed (Lee et al., 2024; Gholami et al., 2024). Given the vast number of model parameters, the time required to transfer weights from GPU/CPU memory to the computation cache often exceeds the actual computation time, meaning that merely accelerating computation does not significantly enhance inference speed. To address this issue, some works have proposed non-uniform quantization methods to accelerate weight loading, Unlike uniform quantization (Dettmers et al., 2024; Frantar et al., 2023), which partitions weight values into bins equal intervals and maps each bin to integer, non-uniform quantization (Zadeh et al., 2020; Kim et al., 2024; Jeon et al., 2022) employs clustering techniques such as K-means to group weight values into a limited set of representative values. These values are stored in a lookup table, and each weight value is replaced by the index of its closest match in the table. During model loading, only the integer indices and a small set of floating-pont values from the lookup table need to be retrieved, enabling rapid weight reconstruction.
Furthermore, to mitigate the weight loading issue, some works leverage Multi-Codebook Quantization retrieval methods, which quantize LLM weights at the vector level rather than as individual numerical values (Egiazarian et al., 2024; Tseng et al., 2024). In these approach, the lookup table stores poating-point vectors, and model parameters are represented as a operation of multiple vectors from the table, e.g., additive quantization (Egiazarian et al., 2024). However, these methods often require a small amount of labeled data to finetune the vectors stored in the codebook. Given that DGDMs have significantly fewer parameters than LLMs, these memory-oriented quantization techniques are inefficient when applied to DGDMs, as discussed in Sec. 5.2.
A.2. Quantization for IDMs
Some quantization works (Li et al., 2023; Shang et al., 2023; Wang et al., 2023) for image diffusion models (IDMs) (Shen et al., 2024) optimize quantized weights by minimizing the MSE between quantized and full pele-ieeciiion continuous outputs. (So et al., 2023) introduces time-step specific encoding for image diffusion models. (Lv et al., 2024) addresses bimodal distributions in post-key-lineat layers and post-softmax discrepancies, specific to Segment Anything Models. However, these approaches cannot be directly applied to Discrete Graph Diffusion Models (DGDMs) due to fundamental incompatibilities. As shown in Remark 3.1, in DGDMs, the intermediate node attributes and graph structures are obtained through discrete sampling from categorical distributions. While IDMs relay on Gaussian noise and continuous denoising processes. Our method advances existing method in three innovations. (i) Recognizing the significant outliers in model weights, we first propose an ill ill-conditioned low-rank weight decomposition. This contrasts with basic SVD approach, allowing our method to achieve better numerical stability. (ii) For the residual component derived from the raw weight and low-rank decomposition. our method enforces a-sparsity, enabling efficient sparse matrix multiplication during inference. (iii) For activation outliers, our approach eliminates the need for calibration data to select thresholds, making it more practical.
A.3. Outlier-Aware Quantization
In low-bit model quantization, critical challenge is the presence of outliers, which can unnecessarily expand the quantization range. To address this issue, outlier-aware quantization methods (Bondarenko et al., 2021) have been proposed. Wei et al. (2022b) alleviate outlier influences by transferring outlier factors to later layers without affecting model functionality. Xiao et al. (2023) introduce a method that shifts the quantization difficulty from activations to weights by applying channel- wise scaling to reduce the magnitude of activations while increasing the magnitude of weights. Li et al. (2024b) further extend this approach by employing SVD to decompose weights and quantize the residuals of its principal components. SqueezeLLM (Kim al., 2024) improves the robustness of non non-uniform quantization by enhancing K-means clustering to reduce sensitivity to outliers, proposing a sensitivity- clustering method. DuQuant (Lin et al., 2024) adopts a block- wise rotation strategy to redistribute outliers across adjacent channels. Unlike these approaches, our proposed Bit-DGDM
13

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
introduces two novel methods: (1) Sparse-Dense quantization, which provides simple and effective solution for handling activations, and (2) Ill-conditioned Low-Rank Decomposition, which decomposes the weight matrix into components, ensuring computational efficiency while maintaining precision. These innovations yield substantial improvements over existing outlier-awand aware quantization frameworks.
B. Theoretical Proof
In Sec. 4.3, we have introduced Theorem 4.5 to promise the recovery guarantee of our proposed weight decomposition approach with respect to the ideal dense low-rank matrix wy We now illustrate how it can be derived. Note that guaranteed recovery theorem follows the route built in (Tong al., 2021; Cai et al., 2021). We first introduce basic definitions. Let L, == U. U. and R. == V. where U, U.E.V. VT is the compact SVD of Di- The error metric for decomposed rank- matrices is defined as:
(luQ- L.) / + |IQQQQQT -R.) /
1/2
inf dist (L, R; L., R,) QeRexr,ranke
where the optimal alignment matrix Q exists and invertible if L and R are sufficiently close to L, and R. Particularly, the following lemma exists
Lemma B.1 (Lemma 9 in Tong et al. (2021)), For any L E Raxx and R Rmxr, if
dist (L, R: L, R.) < ca, (Dw)
for some < c < 1, then the optimal alignment matrix Q between [L, RJ and [L., R. exists and is invertible.
We then present the theorems local linear convergence and guaranteed initialization.
Theorem B.2 (Local Linear Convergence, Theorem 3 in Cai et al. (2021)). Let Dw L.R., be a rank-r matrix with incoherence parameter μu, and lei S. be an o-spurse matrix where a S 10 Define Q₂ as the optimal alignment matrix between Lg, R&] and [L,, R.). Assuming the initial guesses satisfy the prescribed conditions. we have the following:
dist (Lo, Ro: L., R.) S E0o, 500,, (Dir,
(L.Q0 L.) L.) E
2,00 v (RoQ0 - R.) /
ur
(Dw)
2,00
with co = 0.02, by using a fixed step size me = n where 1j E [0.1, 3]. the iterates of weight decomposition satisfy the following conditions:
dist (L, R4: L.. R.) S E0T (Di)),
[L L.) E.)
1//2
v QR - R.) 1/2
pr 2,00
S
,"a (Dw,,
2,00
n
where the convergence rate ; 0.67.
Theorem B.3 (Guaranteed Initialization, Theorem in Cai et al. (2021)). Suppose Div L,R., is a rank-r matrix with incoherence parameter μ, and S. an a-sparse matrix such that < prtx' where co a small positive constant with 1 Let Qo denote the optimal alignment matrix between [L. Ro] and [L,, R.). It has the following results: S 35
dist (Lo, Roi L.,, R.) 10c (Dir),
(L. - L.)EE
(Ro00 -R.) 1
S - o. (Dw).
ur
2,00
2,00
Besides, we cite conclusion from Tong et al. (2021) as Lemma B.4:
14

---

Outlier-Aware Post- Post-Training Quantization for Discrete Graph Diffusion Models
Lemma B.4 (Eqn. (48) in Tong et al. (2021)).
| Div llp S 1.5 dist (Lg, Rxi L., R.)
Given the above lemmas and theorems, we are ready to prove Theorem 4.5:
Proof of Theorem 4.5. The results of Theorem B.3 meet the conditions stipulated in Theorem B.2. By setting co = 10- we obtain the following:
dist(LR, R L", R*) - 0.02 -
00.n)) (Dw
QL Q - L.)..
RR Q -R.) Σ!/
S
pr
(1 0.6.) (Dw)
2,00
2,00
for all & 2 0. Based on Lemma B.4, we can proves that
| -Di I 1.5 (IR, Ra: L., RR) s 0.03(1 - 0.61)) (D
under the condition of [L -L,) -I) 1! 2,00 proved.
v T R..)
-R.) D!/2
S V=
_ or (Div). Thus our claim is
2,00
C. Backbones of Graph Diffusion Models
C.1. 2D Graph Diffusion Model
Regarding the input for the graph generation task, we have already introduced Sec. 3 of the main text. We will now proceed describe the model architecture.
Architecture. We employ a typical graph diffusion model, Digress (Vignac al., 2023), as the target for our model quantization acceleration. Digress utilizes the adjacency matrix of nodes to model the structure of graphs and adopts Graph Transformer architecture its backbone to represent nodes and structural features. The structure is illustrated in Figure A1. Notably, FiLM and PNA modules are critical components, and their computational processes are described as:
FiLM(M,, M2) = M1 W1 + (M, W2) O M2 + M2,
(15)
PNA(X) = [max X) min X) mean X) std (t) W,
where @ denotes the element-wise product and denote the concatenation.
C.2. Protein Inverse Folding Diffusion Model
Regarding the protein inverse folding task, we provide detailed descriptions of the input and the model architecture.
Input. The protein graph G = X, E comprises node feature X and edge features E. The node feature depicts the amino acid (AA) position, AA type and the spatial and biochemical properties to reflect its physicochemical and topological attributes. The local structure of a given node is defined by its spatial neighbors, determined by the k-nearest neighbor algorithm following (Yi et al., 2023). The edge attributes illustrate the relationships berween connected nodes, such as inter-atomic distances, local N-C positions and sequential position encoding scheme.
For protein-related tasks, modeling the 3D structure between nodes is crucial due to the importance of distances, angles, and other spatial properties to protein characteristics. We select GRADE-IF (?) the DGDM for protein tasks. GRADE-IF employs enhanced Equivariant Graph Convolution (EGC) layer to ensure SO(3) rotation equivariance and E(3) translation invariance. At the I-th layer, the EGC layer uses n node representations h describe each node's
Architecture.
15

---

Outlier-Aware Post- Best-Training Quantization for Discrete Graph Diffusion Models
E
X
E
Linnesr
Linear
FILM
FILM
Edgo-/ede MLP
Platton beads
Platten heads
Graph Transformer Laver
A Aggregation
X N
Graph Trunsformer Lower
E
FIL.M
X
Outer product
Node/Edge-werr MLP
Linear
Linear
X E
Structual Spectral festures
X
X
head
Figure AI. An illustration of graph transformer architectures used Digress.
amino acid (AA) type and geometric properties, edge embeddings m to connect nodes i and j. and x,"h to describe the coordinates of the nodes. The EGC layer updates the node representations as follows:
(1+1)
6e
h (1)
h
x - x101( m
(1+1)
(1)
(1)
x" @r (m!y (
x;
+
x;
n ifi
(16)
h (1+1) on
h
(1)
m
(111)
where de. ob, are the operations of edge and node propagations, respectively, and ou is an operation that projects the edge embedding scalar.
D. Algorithm and Complexity Analysis
Figure provides a succinet overview the comprehensive quantization process in Bit-DGDMM To enhance clarity and facilitate comprehension, we delineate the systematic protocol for the overall inference process of quantized model in Algorithm Besides, we extensively illustrate the procedure of ill-conditioned low-rank decomposition of weights (Sec. 4.3) in Algorithm 2. This section presents complexity analysis of the ill-conditioned low-rank decomposition. First, the initializations of the a-Sparsity matrix Sw. and low-rank component Lg R,J requires 2nm nmr nr² mr flops. The computational complexity w.rpp the iterative optimizations of these components is O + mr? + nr² + r²) flops. Since the rank is typically much less than the channel sizes n, m of weight W, the overall complexity of the ill-conditioned low-rank decomposition requires (nnrr).
E. Equidistributed and Adaptive Sparse-Dense Kernel Implementation
In this section, we will introduce the core concept of our designed Equidistributed and Adaptive Sparse-Dense kernel. For each sparse matrix, we utilize the CSR (Compressed Sparse Row) format (Buluç et al., 2009) to represent The CSR format is storage scheme that efficiently represents sparse matrices by storing non-zero elements along with their row and column indices, thereby reducing memory usage and improving computational efficiency.
16

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Algorithm 1 The inference process of quantized multiplication XW through Bit-DGDM
Input: X E I activations. Tmax Tmin- thresholds for filter activation outiies,, Dw: quantized dense part of weight
W, Sw: sparse component of weight W, LR." low rank component of weight W.
Filter the activation outliers: Sx- = [(x > Tmax (r Tmin Dx
X(Tiin < < Tmax.
Quantize the Dx_ low-bit Dx through Eqn. (4)
Calculating D Dw through low-bit multiplication
Calculating LR. through adaptive sparse-dense kernel
Calculating XSw through equidistributed sparse-dense kernel
Output: Dx Dw S.LR. + XSw
Algorithm 2 Ill-conditioned low-rank decomposition of weight W
Input: W E m weight matrix, r: the rank of underlying low-rank matrix, T,(+) (-): sparsification operator, {TH J: set
of step size.
Initialization:
Sw. = To (W)
+ nm flops
[Uo: Eo.
SVD, (W- Swo)
nmr + nm flops
Lo Uo D. Rg = V 2
nr² + my² flops
Iterative optimization:
repeat
Sw.- = (W - R)
+ +mmer) flops + Onmrr + +mr + r flops O(umr nvr2 + r²) flops
L++1 = Ls - 7k+1 RL + S. - W) R R R
R++1 = R4 - 114+11 (L SW - W) La (L. LR)
until || + Sw,k+1 W convergent
Output: a-Sparsity matrix Sw == Sw... r-rank component Dw = LR LRT(L= := Lg, R = R
Equidistributed Sparse-Dense Kernel. For the Equidistributed Kernel, since the sparse matrix Sw. which models the outliers of weights, satisfies the o-Sparsity constraint, we only need to assign one thread per row to handle the computation, ensuring load balancing.
The adaptive sparse-dense kernel is designed to optimize workload distribution on GPUs, particularly for matrices characterized by irregular row lengths. The core concept involves partitioning the non-zero elements into segments allocated to warps, rather than assigning entire rows to warps. The overall algorithm is shown in Algorithm 3.
Adaptive Sparse-Dense Kernel.
The pipeline initiates with a preprocessing phase, during which the matrix is analyzed to determine the optimal segmentation of non-zero elements. A specialized array, termed srow, is introduced to store the starting row index for each segment. This preprocessing step entails constructing a histogram of the rowptr array and executing an exclusive scan operation to compute the srow array. This preprocessing is performed once and stored, incurring negligible computational overhead.
During the execution phase, each warp is responsible for processing its designated segment of the val and colidx arrays. Each segment comprises contiguous block of non-zero elements, irrespective of row boundaries. The warp processes these elements in fixed-size chunks (e.g., 32 elements at a time, corresponding to the warp size). For each element, the corresponding thread multiplies the value by the respective element in the x vector and accumulates the result.
However, since elements from different rows may reside within the same segment, the algorithm must dynamically detect row boundaries. This achieved by examining the rowptr array. When a row boundary encountered within chunk, the warp employs warp shuffle instructions to execute a segmented reduction. This mechanism facilitates the efficient combination of partial sums from the same row without the need for shared memory.
Upon processing a chunk, if any thread transitions to a new row, segmented scan operation is performed to aggregate the results for each row. The first thread in the warp corresponding to each row then executes an atomic addition to update the global y vector. This approach minimizes the number of atomic operations by ensuring that each row is updated only once per warp after the accumulation of partial sums. Overall, the pipeline achieves load balancing by evenly distributing non-zero elements across warps and handles irregular row lengths through optimized intra- intra-warp communication.
17

---

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models
Algorithm 3 Adaptive Sparse- Denss Dense Kernel GPUs
Input:
val, colidx, rowptr: CSR-forma sparse matrix, x: input vector, a srow: segment starting rows, nsz number of non-zeros
Initialize
Divide val and colidx into I' warp-level segments
Segment & range: [[k,,/T], (k + )))
Precompute srow array via histogram of rowptr Execution on GPU
for each warp W in parallel do
startw f- W n2/T] // Start index of segment
endw < L(W + 1) R2/TT // End index
current_row [[]
local_sum +-
// Process chunk of 32 elements (warp size) repeat
Initialize i
for each thread warp do
Load valli + and colidx[i partial <- valfi + t] z/colidx(i + if column index crosses row boundary then
Detect row changes using rowptr Perform segmented scan within warp Update current.row via warp shuffle
end if
Accumulate partial to local if any thread changed then Warp-wide segmented reduction Reset local sums except first thread per row Atomic add completed rows to global end if i < + 32/ // Move to next chunk end for // Flush remaining local sums Perform final warp reduction and atomic updates to y until current index 2 endw
end for Output:Output vector y
F. Additional Experiment Details
F.1. Baseline Description
The detailed descriptions of the baselines used for comparison this work are as follows:
RTN (round-to-nearest) quantizes each value of weights and activations the nearest integer. Alter performing the multiplication, the values are dequantized back to floating point numbers.
GPTQ (Frantar al., 2023) leverages the Hessian matrix of activations for layer-wise error calibration and performs channel-wise quantization. The quantization loss is then propagated to other channels. Additionally, it introduces techniques such as arbitrary order, lazy- updates, and Cholesky reformulation, ensuring efficient quantization even with a large number of parameters.
SqueezeLLM (Kim et al., 2024) isolates a small number of weight outliers and employs a lookup table to focus all weight values on few cluster centers in a sensitivity- aware manner. Quantized weights are stored as integers and
18

---

Outlier-Aware Post- Post-Training Quantization for Discrete Graph Diffusion Models
loaded via the lookup table, significantly reducing the LLM weight loading time. Computation is still performed using FP16 weights and activations.
SVDQuant (Li et al., 2024b) reduces the quantization difficulty of activations by shrinking the activation magnitude and amplifying the weight magnitude. The amplified weights are decomposed into low-rank components using SVD, while the residual is quantized to low-bit precision.
DuQuant (Lin et al., 2024), based on specific outlier dimensions prior knowledge, employs rotation and permutation transformations to redistribute outliers to adjacent channels through block-wise rotation, thereby mitigating the impact of outliers.
E2. Metrics
Graph Generation Evaluation. For the molecules generation tasks, we employ three widely-used metrics to evaluate the quanlity of generated graphs: Validity, Uniqueness and Novelty. Specifically, Validity quantifies the fraction of generated molecules that satisfy fundamental chemical valency rules, ensuring their structural feasibility. Uniqueness evaluates the diversity of the generated molecules by calculating the proportion of distinct SMILES strings, which indicates the presence of non-isomorphic molecular structures. Novelty assesses the extent which the generated molecules differ from the training data by measuring the fraction of molecules that do not appear in the training set, thereby reflecting the model's ability to produce chemically novel compounds. Together, these metrics provide a comprehensive evaluation of the chemical validity, structural diversity, and innovation of the generated molecular outputs. Following (Vignac et al., 2023), we do not include novelty as reported metric for QM9 in the main table. This decision is based on the fact that QM9 represents an exhaustive enumeration of small molecules adhering to a specific set of constraints. Consequently, generating molecules outside this predefined set does not inherently indicate that the model has effectively learned the underlying data distribution. Instead, it may reflect deviations from the constrained chemical space that QM9 encompasses.
Protein Inverse Folding Evaluation. In our evaluations, we focus on the protein inverse folding as an important application for the graph generation. The quality of reconstructed protein sequence is evaluated using two key metries: perplexity and Recovery. Perplexity quantifies the degree to which the model's predicted amino acid (AA) probabilities align with the actual AA at each position in the sequence. A lower perplexity value indicates a stronger agreement between the model's 's predictions and the observed data, reflecting a better fit. On the other hand, the recovery rate measures the model's s ability to accurately reconstruct the correct AA sequence based on the protein's 's 3D structure. This metric is calculated as the percentage of AAs in the predicted sequence that match those in the original sequence. A higher recovery rate signifies superior capability of the model to infer the original sequence from the structural information. Together, these metrics provide a comprehensive assessment of the model's performance in protein inverse folding tasks.
E33 Quantization Cost Analysis
In Table A1, we report the memory and running time required for quantizing DGDMs. Note that we use the Digress model from the QM9 dataset as representative for validating quantized Digress, as the quantization time for Digress on other datasets is similar. The quantization overhead is mainly divided into two parts: first, determining activation outlier thresholds by randomly generating a certain number of graphs (set to 32 in our study); second, the iterative estimation required for ill-conditioned weight decomposition, including the sparse matrix and low-rank component. As shown in Table A1, the time and memory overhead of quantizing DGDMs is entirely acceptable, especially considering the significant improvement in inference speed, and does not require performeffrm hardware.
Table AI. Peak memory requirements and running time of Bit- Bit-DGDM for quantizing DGDMs. The running time is broken down into (i) activation thresholds computation (X comp.) and (ii) ill-conditioned low rank weight decomposition (W deco.).
QM9 (Digress)
CATH (GRADE-IF)
X Comp. (min)
W deco.
Peak Memory
X Comp.
W deco. (min)
Peak Memory
(min)
(GB)
(min)
(GB)
2.9
13.6
2.1
5.5
10.1
1.9
19

---

Outlier-Aware Post-Tiee most-Training Quantization for Discrete Graph Diffusion Models
Table A2. Non-molecule generation performance of fuel-precision Digress (Vignac al., 2023) (i.e., baseline) and quantized Digress with
different quantization methods on Stochastic Block Model (SBM) and Planar datasets.
Non-Molecule Generation
SBM
Planar graphs
Deg. (1)
Clus.
Orb.
Speedup
Mem. (GB 4
Deg.
Clus.
Orb.
Speedup
Mem.
Method
Precision
(L)
())
(t)
(U)
())
(L)
(t)
(GB J)
FP32
1.6
1.5
1.7
1x
12.6
1.4
1.2
1.7
1x
10.4
baseline
BF16
1.6
1.6
1.7
1.3x
6.8
1.4
1.2
1.7
1.3x
5.5
RTN
W4A4
15.6
3.4
2.8
2.9x
3.7
21.6
8.6
2231
2.8x
3.1
GPTQ
W4A4
4.6
2.7
3.2
2.9x
3.7
3.8
2.9
2.6
2.8x
3.1
SVDQuant
W4A4
1.9
1.9
2.1x
4.0
1.8
1.7
2.1x
3.5
DuQuant
W4A4
2.3
2.6
2.5
2,0x
4.1
3.8
3.6
2.4
2.0x
3,5
Lq
W4A16
2.0
1.8
2.0
1x
5.3
1.7
1.5
2.2
1x
5.1
Bit-DGDM
W4A4
1.8
1.7
1.9
2.7X
4.5
1.5
1.3
1.9
2.6x
3.9
F4. Graph Generation Results on Non-Molecule Graphs
In Table A2, we validate the graph generation performance our quantization method, Bit-DGDM, compared to other quantization approaches on the non- non-molecule benchmark (Martinkus al., 2022). The datasets used for evaluation include graphs from the Stochastic Block Model (SBM), where the training graphs sampled from the stochastic block model (with up to 200 nodes per graph), and planar graphs, where each graph consists of 64 nodes. Compared to molecule graphs, these two datasets contain graphs of larger scales. The evaluation metrics include node degrees (Deg.), clustering coefficient (Clus.), and orbit count (Orb.) (Liao et al., 2019). As shown in Table A2, our proposed Bit-DGDM not only achieves faster inference speed than most quantization baselines but also generates higher-quality graphs. This demonstrates the generalization capability our method for generating larger-scale graphs.
F.5. Kernel Efficiency Analysis
The analysis of Table A3 reveals distinet suitability of the Equidistributed and Adaptive kernels for handling the sparse components of weights (Sw) and activations A). The Equidistributed kernel particularly effective for processing Sw, as the spsasisyty constraint align well with its balanced thread allocation strategy. contrast, the Adaptive kernel is better suited for A its dynamic resource allocation can efficiently handle the irregular sparsity typically found in activations. The optimal configuration combines the Equidistributed kernel for Sw and the Adaptive kernel for S., leveraging their respective strengths to maximize computational efficiency.
F.6. Rank Selection Effect
The selection of rank is based systematic rrade-off analysis between computational efficacy and precision. Through experiments on CATH dataset (as shown in Table A4), we observed that low ranks (rank=8) led to significant degradation in graph generation quality, and higher ranks (rank=32) provided marginal quality improvements while incurring substantial latency overhead.
Table A3. A comparative study of MOSES dataset on the Speedup (compared with FP32) of different kernels applied to the sparse components of activations and weights, i.e., SA and Sw.
Table A4. The rank selection of effect evaluation on CATH datasets. The Speedup compared with the FP32 haseline.
Sw
Perplexity Recovery (%)
Speedup Mem, (GB)
Speedup
Equidistributed Adaptive
Rank=8
5.1
46.5
2.6
4.7
Equidistributed Adaptive
2.3x 2.5x
2.0x
Rank=16
4.5
51.6
2.5
4.9
SA
2.2x
Rank=32
4.5
51.8
2.2
5.4
20


Reviews:
[{"review_id": "Y0saP50WXv", "reviewer": "Reviewer_DgDZ", "summary": "This paper proposes post-training quantization (PTQ) methods to quantize discrete graph diffusion models (DGDM). The paper first analyzes outlier distributions of weights and activations in DGDM. For activations, the proposed method split activation matrices into high-precision sparse matrix (outliers) and low-precision dense matrix. Then, weights are split into sparse matrices and low-rank matrices which represent dense matrices. By utilizing low-bit and sparse-dense matrix multiplication CUDA kernels, the proposed method not only achieves better accuracy and perplexity, but also fast computation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We very much appreciate your positive comments and constructive suggestions. \n\n**Q1:** What are the differences between the proposed method and SVDQuant?\n\n**A1:** Thank you for your insightful question. Our method introduces two key innovations over SVDQuant. (i) Recognizing the presence of significant outliers in model weights, we propose an ill-conditioned low-rank weight decomposition method. This contrasts with SVDQuant's conventional SVD approach, allowing our method to achieve better numerical stability. (ii) For the residual component derived from the raw weight and low-rank decomposition, while SVDQuant maintains a dense structure, our method enforces $\\alpha$-sparsity. This constraint ensures that no more than $\\alpha$ proportion of elements in each row and column are non-zero, enabling efficient sparse matrix multiplication during inference. Consequently, our approach not only preserves precision but also achieves consistent acceleration, whereas SVDQuant’s dense residuals incur higher computational costs.\n\n**Q2:** How significant is the tradeoff between the efficiency and effectiveness of the proposed method? This question is related to the design of the method as it contains high-precision multiplications in addition to the low-bit quantized operations.\n\n**A2:** Thank you for your question. The trade-off between computational efficiency and model effectiveness is a central consideration in our model design. Our experimental results (Figure 4) demonstrate that while the high-precision multiplications do introduce some computational overhead, they are essential for maintaining the quality of generated graphs in precision-critical applications like molecular generation and protein inverse folding. Specifically, while removing the multiplication operations for sparse componenets(as in variant (iii) Bit-DGDM-$S_XS_W$) yields a slight speed improvement (2.5× → 2.6×), it comes at a significant cost in generation quality, as seen in the perplexity degradation (4.5 → 4.7). This confirms that the additional operations play a crucial role in handling high-magnitude outliers effectively. Moreover, compared to threshold-based variant (ii) removing weight outliers through thresholds, our method not only maintains competitive generation quality but also achieves better inference speed (2.5× vs. 2.2×). This gain stems from our use of ill-conditioned low-rank decomposition, which remains computationally efficient and model effective in significant outlier scenarios.\n\n**Q3:** The reviewer suggests using Times font in the figures helps to make the paper more consistent in format.\n\n**A3:** Thank you for your constructive suggestion. We have revised the font in all figures to Times font to ensure consistency with the paper’s format. The updated figures are available at the following link: [https://anonymous.4open.science/r/ICML_Refined_Figures_9670](https://anonymous.4open.science/r/ICML_Refined_Figures_9670). Please let us know if you have any additional feedback. We would be happy to make further improvements.", "claims_and_evidence": "The paper’s claim is convincing and supported by both empirical and theoretical evidence.", "methods_and_evaluation_criteria": "The proposed method makes sense and remedies the computation bottlenecks of DGDM. Also, the motivational studies show that the proposed method is valid and aims to solve realistic problems. The paper also shows extensive evaluation, ablation, and sensitivity studies.\n\nHowever, the reviewer finds that SVDQuant (Li et al., 2024) shares a similar idea regarding outliers and low-rank computation. Therefore, the reviewer suggests adding more comparisons with SVDQuant.", "theoretical_claims": "No issues.", "experimental_designs_or_analyses": "The experimental results show clear advantages of the proposed method in terms of accuracy and perplexity. However, the proposed method shows lower speedup and higher memory usage compared to some of the baselines. Therefore, as there exists some trade-off between accuracy and efficiency, the reviewer suggests adding more analysis regarding the tradeoff.", "relation_to_broader_scientific_literature": "The contribution of the paper is related to neural network quantization as the paper analyzes and reduces the impact of outliers in target matrices.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The paper is well-written and provides a detailed explanation of the preliminary and related work section.", "other_comments_or_suggestions": "The reviewer suggests using Times font in the figures helps to make the paper more consistent in format.", "questions_for_authors": "The questions of the reviewer can be summarized as follows:\n1. What are the differences between the proposed method and SVDQuant?\n2. How significant is the tradeoff between the efficiency and effectiveness of the proposed method? This question is related to the design of the method as it contains high-precision multiplications in addition to the low-bit quantized operations.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IaSfBc2bZI", "reviewer": "Reviewer_2aM4", "summary": "This paper presents Bit-DGDM, an advanced post-training quantization framework developed for Discrete Graph Diffusion Models. The proposed framework introduces two innovations, i.e., (1) a sparse-dense activation quantization mechanism and (2) an ill-conditioned low-rank weight decomposition technique, to effectively addresses two key challenges, i.e., the computational bottleneck in DGDMs and the outliers in weights and activations. The efficacy of the framework is rigorously validated through extensive experiments across a variety of graph generation tasks, demonstrating superior performance compared to existing quantization baselines.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We very much appreciate your constructive comments. For your concerns:\n\n**Q1:** The font in the figures of the paper should be consistent to make it easier to read.\n\n**A1:** Thank you for your helpful feedback. We have updated all figures to use Times New Roman font, ensuring formatting consistency. The updated figures are avaible here: [https://anonymous.4open.science/r/ICML_Refined_Figures_9670](https://anonymous.4open.science/r/ICML_Refined_Figures_9670)\n\n**Q2:** The time complexity analysis should be extensively introduced in the main paper.\n\n**A2:** We appreciate your valueble suggestion. Due to space limitations in the main manuscript, we have included the detailed complexity analysis in Appendix D. In our future version, we will add a concise summary of the key complexity results in Sec 4.3 of the main paper.\n\n**Q3:** The time cost for the quantization of DGDMs.\n\n**A3:** Thank you for your insightful question.  As detailed in Appendix F.3, the quantization process demonstrates practical efficiency. Specifically, for the DiGress model on the QM9 dataset, the ill-conditioned decomposition requires 13.6 minutes and memory usage remains manageable at 2.1 GB. We emphasize that this computational overhead is highly acceptable given the significant inference acceleration achieved through quantization.", "claims_and_evidence": "The paper's claims is supported by clear evidence.", "methods_and_evaluation_criteria": "The framework proposed in this study shows a high degree of alignment with the challenges intrinsic to DGDMs. The authors present an incisive analysis of prevailing issues, particularly emphasizing the outliers and computational intensity. The evaluation is rigorously designed, incorporating well-established benchmark datasets and scientifically validated metrics, thereby substantiating the efficacy of the proposed method.", "theoretical_claims": "I have checked the proofs and theoretical claims presented in the paper. The authors provide clear references for the work they rely on and explicitly delineate the underlying premises and assumptions that validate their theoretical proposition. The theoretical exposition is systematically organized.", "experimental_designs_or_analyses": "I have reviewed the paper's experimental design and results. The selected baselines are the most adavanced methods in the fields of LLM and image diffusion models, and the datasets utilized for validation are the mainstream benchmarks of graph generations. The experimental design is convincing, and the validation procedures demonstrate the efficacy of the proposed method.", "relation_to_broader_scientific_literature": "The contributions of this paper significantly extend the scope of prior quantization research by uncovering critial insights tailored to discrete graph diffusion models. The authors proves that computations, instead of memory loading, consitute the primary bottleneck in DGDMs. Besides, the study reveals the presence of outliers in both activations and weights within DGDMs. To solve this, the authors innovatively leverage robust PCA for weight decomposition and employ sparse matrix to handle outliers.", "essential_references_not_discussed": "The important references I familiar have been mentioned in this paper.", "other_strengths_and_weaknesses": "The other strengths of the paper are as follows:\n\n(1) The paper is well-structured and easy to follow.\n\n(2) The proposed quantization method is effectively in solving specific challenges in DGDMs.\n\n(3) The experiment results show the effectiveness of proposed method in both generation performance and inference speed. \n\n\nWeaknesses:\n\n(1) The font in the figures of the paper should be consistent to make it easier to read.\n\n(2) The time complexity analysis should be extensively introduced in the main paper.", "other_comments_or_suggestions": "I suggest that the authors use a consistent font in the figures of the paper to make it easier to read.", "questions_for_authors": "See above. Besides, I would like to know the time cost for the quantization of DGDMs.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nNXvKrl85E", "reviewer": "Reviewer_EbX6", "summary": "This paper focuses on the quantization of discrete diffusion models for graph data. To achieve this, the authors introduce sparse-dense activation quantization and low-rank decomposition with hardware support. Experimental results demonstrate that the proposed method enhances quantization performance while improving speed.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We highly appreciate your positive reviews and constructive suggestions.\n\n**Q1:** These methods [1,2,3] should be discussed in this context to provide a more comprehensive comparison. \n\n**A1:** Thank you for your constuctive suggestion. We note that these methods [1,2,3] were designed for image diffusion models (IDMs). Specifically, [1,2] optimize quantized weights by minimizing the MSE between quantized and full-precision continuous outputs. [3] introduces time-step specific encoding for image diffusion models. However, these approaches cannot be directly applied to Discrete Graph Diffusion Models (DGDMs) due to fundamental incompatibilities. As shown in Remark 3.1, in DGDMs, the intermediate node attributes and graph structures are obtained through discrete sampling from categorical distributions. While IDMs relay on Gaussian noise and continuous denoising processes. Our method advances existing method in three innovations. (i) Recognizing the significant outliers in model weights, we first propose an ill-conditioned low-rank weight decomposition. This contrasts with basic SVD approach, allowing our method to achieve better numerical stability. (ii) For the residual component derived from the raw weight and low-rank decomposition, our method enforces α-sparsity, enabling efficient sparse matrix multiplication during inference. (iii) For activation outliers, our approach eliminates the need for calibration data to select thresholds, making it more practical.\n\nWe will add detailed discussion of these works in our related work. We sincerely appreciate your valuable feedback.\n\n[1] Q-diffusion: Quantizing diffusion models. In CVPR, 2023.\n\n[2] Post-training quantization on diffusion models. In CVPR, 2023.\n\n[3] Temporal dynamic quantization for diffusion models. In Neurips, 2023.\n\n**Q2:** In lines 275--277, the authors state that the matrix is ill-conditioned but provide limited explanation of its impact. It would be helpful to clarify how this ill-conditioning affects the convergence of SGD and whether it introduces optimization difficulties. Additional theoretical or empirical analysis would strengthen this claim.\n\n**A2:** We sincerely appreciate the reviewer's insightful suggestion. To better illustrate how ill-conditioning affects the optimization of Eqn. (9) ($\\|LR^\\top + S_W - W\\|_F^2$), we have conducted additional **empirical analysis** examining the relationship between the number of outliers and the reconstruction error (measured by Frobenius norm between the reconstructed matrix and the original matrix). These results are provided in this [`link`](https://anonymous.4open.science/r/ICML_Rebuttal_9670/outlier_effect.png). We observe that the reconstruction error (Frobenius norm) increases monotonically with the number of outliers. This correlation indicates that stronger ill-conditioning (caused by more outliers) leads to greater optimization difficulties in matrix reconstruction. Moreover, in this [`link`](https://anonymous.4open.science/r/ICML_Rebuttal_9670/derivation.png), we present a comprehensive derivation of these two scale terms $(R^\\top R)^{-1}$ and $(L^\\top L)^{-1}$ in Eqn.(13).\n\nWe will include a more detailed description of this mechanism in the revised version to better clarify these points. Thank you for this valuable suggestion to improve the clarity of our work.\n\n\n**Q3:** It would be beneficial for the authors to provide a more detailed analysis of the computational complexity and conduct experiments to evaluate the actual impact on efficiency.\n\n**A3:** We sincerely thank the reviewer for raising this important point regarding the computational cost of our method. We agree that the additional multiplications introduced to compensate for quantization errors incur some computational overhead, but this overhead is marginal in practice.\n\nTo evaluate this, we conducted an ablation study (Figure 4 and Sec 5.4). The results show that the ablation variant (iii) Bit-DGDM-$S_XS_W$, which removes multiplications for sparse components, achieves only a modest speed improvement (from 2.5× to 2.6×) but suffers significant degradation in generation quality, as evidenced by the perplexity increase (4.5 → 4.7). This performance decline highlights the critical role of these operations in mitigating high-magnitude outliers. Furthermore, compared to threshold-based alternatives (variant (ii)), our method not only preserves competitive generation quality but also delivers better inference speed (2.5× vs. 2.2×). This efficiency gain stems from our use of ill-conditioned low-rank decomposition. Collectively, these experiments demonstrate that the additional computational overhead is negligible and does not substantially impact inference speed.\n\nIn the revised version, we will expand our empirical analysis to better demonstrate the impact of each proposed component in our work. Once again, we deeply appreciate this valuable feedback, which will undoubtedly help improve the clarity and rigor of our work.", "claims_and_evidence": "1. In lines 60 -- 65, the authors argue that existing quantization methods are insufficient to address the challenges posed by computation boundaries, citing several quantization techniques for LLMs. However, prior work specifically targeting quantization for diffusion models [1,2,3] also addresses these challenges. These methods should be discussed in this context to provide a more comprehensive comparison.\n\n2. In lines 275 -- 277, the authors state that the matrix is ill-conditioned but provide limited explanation of its impact. It would be helpful to clarify how this ill-conditioning affects the convergence of SGD and whether it introduces optimization difficulties. Additional theoretical or empirical analysis would strengthen this claim.\n\n[1] Li, Xiuyu, et al. \"Q-diffusion: Quantizing diffusion models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n[2] Shang, Yuzhang, et al. \"Post-training quantization on diffusion models.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n[3] So, Junhyuk, et al. \"Temporal dynamic quantization for diffusion models.\" Advances in neural information processing systems 36 (2023): 48686-48698.", "methods_and_evaluation_criteria": "1. The proposed method can be viewed as a combination of existing techniques, and its components lack sufficient novelty. As I understand, the authors decompose both weights and activations into outliers and a central part. For weight decomposition, the approach closely resembles techniques used in LoRA [1]. Additionally, the concept of decomposed outliers has been previously introduced in [2]. Given these similarities, the paper should better highlight its unique contributions beyond existing work. While the method primarily builds on existing techniques, I acknowledge its contribution, as the implementation of this combination is non-trivial. The approach requires careful design and integration, which adds value despite the lack of fully novel components.\n\n2. There exists increased computational cost due to the formulation. The proposed method relies on multiple integer multiplications to compensate for quantization errors, which may introduce additional computational overhead. It would be beneficial for the authors to provide a more detailed analysis of the computational complexity and conduct experiments to evaluate the actual impact on efficiency.\n\n\n[1] Hu, Edward J., et al. \"Lora: Low-rank adaptation of large language models.\" ICLR 1.2 (2022): 3.\n[2] Dettmers, Tim, et al. \"Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale.\" Advances in neural information processing systems 35 (2022): 30318-30332.", "theoretical_claims": "Correct", "experimental_designs_or_analyses": "## Strength ##\n\n1. The experimental evaluation is fair. The authors compare both memory usage and computation cost in terms of real acceleration, which requires CUDA implementation. This adds credibility to the experiments and strengthens their impact.\n\n2. The proposed method demonstrates good performance, as it effectively improves quantization accuracy. While there is a minor trade-off between memory and computation cost, such trade-offs are expected in this field and do not detract from the overall contribution.\n\n## Weakness ##\n\nWhile there are existing quantization techniques for diffusion models, they are not included in the experimental comparisons [1, 2, 3]. A direct comparison with these methods would provide a clearer understanding of the proposed approach's advantages and limitations.\n[1] Li, Xiuyu, et al. \"Q-diffusion: Quantizing diffusion models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n[2] Shang, Yuzhang, et al. \"Post-training quantization on diffusion models.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n[3] So, Junhyuk, et al. \"Temporal dynamic quantization for diffusion models.\" Advances in neural information processing systems 36 (2023): 48686-48698.", "relation_to_broader_scientific_literature": "None", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "## Strength ##\n\n1. The paper addresses an important but underexplored problem—quantization for graph discrete diffusion models. It also highlights the issue of weight outliers, which is crucial for improving quantization performance.\n\n2. The paper is well-structured, with a clear presentation of the motivation, background, methodology, and experiments, making it easy to follow.\n\n3. The inclusion of a CUDA implementation enhances the practical applicability of the method by supporting hardware acceleration.\n\n4. The paper provides a theoretical guarantee for the proposed method, strengthening its validity and reliability.", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JIic6Ntx7G", "reviewer": "Reviewer_Leoa", "summary": "This paper introduces **Bit-DGDM**, a post-training quantization framework for **Discrete Graph Diffusion Models (DGDMs)**, addressing the long inference times caused by huge computational load and the presence of outliers in weights and activations. It proposes decomposing activations into dense, easily quantizable parts and sparse, non-quantizable parts based on precomputes thresholds. For weights, it introduces a decomposition algorithm based on the assumption that weights can be split into a sparse part with 𝛼-Sparsity and a low-rank dense part. The approach ultimately enables low-bit dense computation and high-bit but sparse-dense computation, and the implementation of computational kernels achieves significant practical acceleration results.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We very much appreciate your positive comments of our paper.\n\n**Q1:** Some quantization methods [a,b,c,d], low-rank decomposition and importance-based weight (e.g. PB-LLM) dividing methods should be discussed.\n\n**A1:** We sincerely appreciate your valuable suggestion. Due to the page limit of the rebuttal, we have included the comparison with these related works in this [`link`](https://anonymous.4open.science/r/ICML_Rebuttal_9670/comparion_svd_impotance.md). We will further expand our related work section to incorporate a detailed discussion of these studies.\n\n**Q2:** The proposed method shows no advantage over the BF16 baseline in runtime memory.\n\n**A2:** Thank you for your valuable comment. Our Bit-DGDM shows notable memory usage advantages compared to the BF16 baseline. In Table 1, on the QM9 dataset, Bit-DGDM only requires 2.3GB memory compared to BF16's 3.4GB usage, with a 32% reduction. The results are consistent in other datasets. Since DGDM is primarily a computation-intensive model rather than a large-parameter model, our main focus remains on improving inference speed while maintaining performance. We will incorporate memory analysis in our experimental results.\n\n**Q3:** The idea of dividing weights and activations into difficult to quantize parts and easy to quantize parts is common.\n\n**A3:** Thank you for your feedback. Though the idea of separating weights and activations has been explored before, our method introduces several key innovations that distinguish it from prior work. (i) For easy-to-quantize weights, we are the first to propose ill-conditioned low-rank weight decomposition, enabling stable decomposition into low-rank components even in the presence of significant outliers. (ii) For difficult-to-quantize weights, we propose the use of α-Sparsity, a structured sparsity property that facilitates subsequent inference acceleration. (iii) For activation outliers, our approach eliminates the need for calibration data to select thresholds, making it more practical. \n\n**Q4:** Whether DGDM has practical and wide application is worth discussing.\n\n**A4:** Thank you for your insightful question. Compared to Gaussian denoising diffusion models, DGDM possesses unique capabilities in modeling **discrete** data, enabling multiple real-world applications, particularly in biological and chemical tasks. In this work, we validate DGDM’s effectiveness through two critical applications: molecular generation and inverse protein folding. Furthermore, our proposed Bit-DGDM is specifically designed to facilitate the practical deployment of DGDM under real-world computational constraints.\n\n**Q5:** What theoretical advancements the proposed ill-conditioned low-rank decomposition has compared to previous work.\n\n**A5:** The key theoretical advancement of our ill-conditioned low-rank decomposition lies in its fundamental difference from conventional SVD-based methods when handling matrices with significant outliers. In standard SVD, the presence of outliers introduces several critical limitations [1,2]. They distort the singular value spectrum by amplifying small singular values associated with high-frequency noise components, consequently degrading the signal-to-noise ratio in the resulting low-rank approximation. This distortion manifests most prominently in the impaired rank selection capability, where the outlier-contaminated singular value spectrum exhibits slowed decay, making traditional truncation criteria unreliable. Built on prior ill-conditioned decomposition works [3,4], our method establishs rigorous recovery guarantees for the ill-conditioned low-rank composition of weights. The theoretical framework ensures stable outliers isolation properties even under significant outlier, while simultaneously maintaining better control over the dense matrix condition number throughout the decomposition process.\n\n[1] Estimating the number of hidden neurons in a feedforward network using the singular value decomposition. 2006.\n\n[2] Robust PCA via outlier pursuit. Neurips, 2010.\n\n[3] Accelerating ill-conditioned low-rank matrix estimation via scaled gradient descent. JMLR, 2021.\n\n[4]Learned robust PCA: A scalable deep unfolding approach for high-dimensional outlier detection. Neurips, 2021.\n\n**Q6:** What is the basis of rank selection for a low-rank components? \n\n**A6:** Thank you for raising this important question. The selection of rank is based a systematic trade-off analysis between computational efficacy and precision. Through experiments on CATH dataset, we observed that low ranks (rank=8) led to significant degradation in graph generation quality, and higher ranks (rank=32) provided marginal quality improvements while incurring substantial latency overhead. We will include detailed studies on the rank selection in the revised manuscript. \n\n||Perplexity|Recovery(%)|Speedup|Mem.(GB)|\n|-|-|-|-|-|\n|Rank=8|5.1|46.5|2.6|4.7|\n|Rank=16|4.5|51.6|2.5|4.9|\n|Rank=32|4.5|51.8|2.2|5.4|", "claims_and_evidence": "The claims made in the submission supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed methods evaluation criteria make sense.", "theoretical_claims": "All proofs for theoretical claims are correct.", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "The parer design ill-conditioned low-rank decomposition of weight algorithm inspired by [1] and [2].\n\n[1] Candès, E. J., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis?. *Journal of the ACM (JACM)*, *58*(3), 1-37.\n\n[2] Tong, T., Ma, C., & Chi, Y. (2021). Accelerating ill-conditioned low-rank matrix estimation via scaled gradient descent. *Journal of Machine Learning Research*, *22*(150), 1-63.", "essential_references_not_discussed": "There are some quantization methods should be discussed and compared [a,b,c,d]\n\n[a] QVD: Post-training Quantization for Video Diffusion Models, in ACM MM\n\n[b] Ptq4sam: Post-training quantization for segment anything, in CVPR\n\n[c] Post-training Quantization on Diffusion Models, in CVPR\n\n[d] Towards Accurate Post-training Quantization for Diffusion Models, in CVPR", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and clearly presents its contributions. The reader can easily follow the author's logic.\n\n2. This article actually proposes a general quantization framework that can accelerate the model when there are outliers in both the activation values and weights.\n\n3. Compared with many other LLM-oriented baseline methods, the advantages of this method are demonstrated.\n\n4. The supplementary material is provided detailly, which enhance the reproducibility and transparency of the research.\n\nWeaknesses:\n\n1. The proposed method has no advantage in reducing the footprint of memory at runtime compare to BF16 baseline, although the memory usage is indeed relatively small.\n\n2. The idea of dividing weights and activations into difficult to quantize parts and easy to quantize parts is common.\n\n3. Whether DGDM has practical and wide application is worth discussing.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. I'm curious about what theoretical advancements the ill-conditioned low-rank decomposition proposed in this article has compared to previous work.\n\n2. The discussion of work on low-rank factorization should be added, along with other discussions that divide weights into quantized and unquantized (or higher bit precision) work by importance, e.g. PB-LLM[1].\n\n3. What is the basis of rank selection for a low-rank components? Is there any experimental verification?\n\n   [1] Shang, Y., Yuan, Z., Wu, Q., & Dong, Z. (2023). Pb-llm: Partially binarized large language models. *arXiv preprint arXiv:2310.00034*.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Bit-DGDM, a post-training quantization framework tailored for Discrete Graph Diffusion Models (DGDMs), addressing computational inefficiencies and outlier sensitivity in both weights and activations. The approach leverages a novel sparse-dense activation quantization and an ill-conditioned low-rank weight decomposition with α-sparsity to enable hardware-efficient inference without calibration data.

Pros:
* the paper addresses a practically important and underexplored problem—quantization in DGDMs—backed by solid theoretical motivation and well-executed empirical validation.
* all reviewers praised the methodological soundness, implementation rigor, and comprehensive evaluation across benchmarks and ablations.
* the rebuttal effectively clarified novelty over prior methods (e.g., SVDQuant, Q-Diffusion) and addressed concerns on theoretical contributions, computational trade-offs, and practical relevance.
* CUDA implementation and time/memory analyses enhance the work's real-world applicability.

Cons:
* some components (e.g., decomposition of quantizable/unquantizable parts) overlap with prior ideas.
* memory reduction over BF16 is modest, though justified by the compute-bound nature of DGDMs.
* additional references were requested and addressed in the rebuttal.

Overall, he paper makes a meaningful, well-substantiated contribution to the efficient deployment of DGDMs, combining theoretical depth with practical impact. All concerns raised during review were satisfactorily resolved.
Decision: Accept (poster)