Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
Authors: Kyo Kuroki
Yasuyuki Okoshi
Thiem Van Chu
Kazushi Kawamura
Masato Motomura
Venue: NeurIPS 2025 poster ()
Keywords: Quantization
Compression
Deep Neural Network
Primary area: deep_learning

Abstract: This paper proposes a novel matrix quantization method, Binary Quadratic Quan-
tization (BQQ). In contrast to conventional first-order quantization approaches—
such as uniform quantization and binary coding quantization—that approximate
real-valued matrices via linear combinations of binary bases, BQQ leverages the
expressive power of binary quadratic expressions while maintaining an extremely
compact data format. We validate our approach with two experiments: a matrix
compression benchmark and post-training quantization (PTQ) on pretrained Vision
Transformer-based models. Experimental results demonstrate that BQQ consistently achieves a superior trade-off between memory efficiency and reconstruction
error than conventional methods for compressing diverse matrix data. It also
delivers strong PTQ performance, even though we neither target state-of-the-art
PTQ accuracy under tight memory constraints nor rely on PTQ-specific binary
matrix optimization. For example, our proposed method outperforms the state-of-
the-art PTQ method by up to 2.2% and 59.1% on the ImageNet dataset under the
calibration-based and data-free scenarios, respectively, with quantization equivalent
to 2 bits. These findings highlight the surprising effectiveness of binary quadratic
expressions for efficient matrix approximation and neural network compression.

Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
Kyo Kuroki
Yasuyuki Okoshi
Institute of Science Tokyo
Institute of Science Tokyo
kuroki kyolartic....c. iir. iist.is..... ac. jp
okoshi. okoshi. yasguliyyauuukOartt.. iir isct. ac. jp
Thiem Van Chu
Kazushi Kawamura
Institute of Science Tokyo thiem@artic. iir. isct.ac..p..
Waseda University
kawamura. k. auBwaseda. jp
Masato Motomura
Institute of Science Tokyo
motomura@artic. lir isst. isctt.c..jj jp.
Abstract
This paper proposes novel matrix quantization method, Binary Quadratic Quan- tization (BQQ). contrast to conventional first-order quantization approaches- such as uniform quantization and binary coding quantization- approximate real-valued matrices via linear combinations of binary bases, BQQ leverages the expressive power of binary quadratic expressions while maintaining extremely compact data format. We validate our approach with two experiments: a matrix compression benehmark and post-training quantization (PTQ) pretrained Vision Transformer-base models. Experimental results demonstrate that BQQ consis- tently achieves superior trade-off between memory efficiency and reconstruction еrror than conventional methods for compressing diverse matrix data. also delivers strong PTQ performance, even though neither target state-of-the-e PTQ accuracy under tight memory constraints nor rely on PTQ-specific binary matrix optimization. For example, our proposed method outperforms the state-of- the-art PTQ method by up to 2.2% and 59.1 % on the ImageNet dataset under the calibration-bass based and fata-free scenarios, respectively, with quantization equivalent to 2 bits. These findings highlight the surprising effectiveness of binary quadratic expressions for efficient matrix approximation and neural network compression.
Introduction
Modern information systems increasingly demand efficiency in both computation and resource usage, driven by growing model sizes, data volumes, and deployment requirements across diverse hardware environments. In these systems, real-valued matrices frequently appear as weight parameters in deep neural networks (DNNs), as high-dimensional embeddings in retrieval systems, and training datasets. Because such matrices are central to a wide range of data processing and applications, their efficient representation and compression is crucial for reducing the costs of storage, computation, and data movement- an essential step toward deploying models on edge devices, reducing memory usage in retrieval systems, scaling to large datasets learning.
widely adopted strategy for this purpose quantization, which approximates continuous-valle valued data with discrete levels to save memory and enable faster computation. Most existing methods rely
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

on first-order scalar quantization approaches-suune as Uniform Quantization (UQ) or Binary Coding Quantization (BCQ) 23 that represent real-valued matrices as linear combinations of binary bases.
While effective under moderate compression, such first-order methods often struggle to accurately reconstruct the original matrix with ultra-low-bit quantization, as the number of possible values for each element becomes extremely limited. Beyond these scalar quantization approaches, alternative techniques such as Vector Quantization (VQ) [15] and its variants (e.g., Product Quantization (PQ) [31] and Lattice Vector Quantization (LVQ) 11 60)), as well as low-rank approximations (13), have also been explored. While these methods can capture correlations among dimensions more effectively, they typically rely on codebooks factorized components that contain unquantized real values. Consequently, although the index or low-rank representation is compact, their dependence on floating-poin vectors limits hardware efficiency, unlike scalar quantization methods. Although several studies [1]] 22 64 [6] have explored combining low-rank approximation with scalar quantization, the use of extremely low-bit (e.g., binary) representations for the factorized components has not yet been fully explored. Furthermore, to our knowledge, no prior work has applied, as in BCQ, independent scaling factors for each binary matrix in factorized representation. Incorporating such strategies could potentially enable even more efficient matrix approximation under extreme low-bit constraints. Motivated by these observations, we introduce Binary Quadratic Quantization (BQQ), a framework that represents matrices using quadratic combinations of binary variables, with independent scaling factors assigned each binary matrix. Specifically, the target matrix is represented as a sum of binary matrix products, enabling expressive nonlinear approximations while maintaining an exceptionally compact data format. This approach pushes the boundaries of matrix quantization by addressing the limitations of traditional methods and offering a fundamentally new perspective on matrix approximation.
In this paper, we demonstrate the effectiveness of BQQ through comprehensive evaluations: (i) measuring the trade-off between quantization error and memory usage across various matrix datasets, (ii) assessing performance when applied to post-training quantization (PTQ) of DNNs. Beyond these applications, the generality of our framework suggests its potential for other scenarios where efficient matrix approximations are essential, such as accelerating approximate nearest neighbor (ANN) [2 47 bbseed retrieval systems and improving the scalability of large-scale learning powered by abundant training data [70 73
Our main contributions are:
We propose BQQ, a novel matrix quantization framework based on quadratic expressions of binary matrices, offering new perspective on extreme matrix compression.
Minimizing the quantization error under the BQQ formulation naturally leads to an NP-hard optimization problem. To address this issue, we develop efficient solution based on polynomial unconstrained binary optimization (PUBO) and convex quadratic programming.
We demonstrate that BQQ consistently achieves an excellent trade-off between memory usage and quantization error for compressing diverse matrix data.
We show that BQQ also delivers sfate-of-the-art (SOTA) performance in weight PTQ of Vision Transformer 12 -based models (ViTs), even though our PTQ method is based mainly on minimizing weight reconstruction error, rather than explicitly minimizing activation error to achieve SOTA performance.
To the best of our knowledge, this is the first study to achieve practical accuracy-such as 72% ImageNet top-1 accuracy on the DeiT-base model-using data-free PTQ for VITs at a model size equivalent to 2-bit quantization.
2 Related Works
Quantization is fundamental technique for reducing the precision of real-valued parameters, widely used for model compression and efficient processing. It converts continuous values into a limited set of discrete levels, with methods varying in granularity and complexity depending on hardware constraints and the acceptable sccuracy-pefformance trade-off. UQ is the most commonly used quantization method, which approximates a real-valued matrix W E Ruxn as:
Quantization
R Σ
2 61
(I)
where B, E 10 a E IR is the scaling factor, and b E R is a bias (or zero-point). This form corresponds to p-bit quantization with uniform step sizes. On the other hand, non-uniform

---

quantization assigns quantization levels in data-aware manner, allowing better alignment with the underlying distribution of matrices and reducing quantization error. One example is BCQ [23), which approximates real-valued matrix sum of binary bases with individual scaling factors:
W ~
B,
(2)
where a; E R and B, E 10,1) mx 66 or (-1,
1 m
662 29
23
65
5
53
By introducing
bias term, the {0, } pmx and (-1,1] jmxx representations can be made equivalent and encompass UQ This enables flexible quantization levels that can better capture the characteristics of W. Alternatively, some methods apply a nonlinear transformation before quantization. One such method is logarithmic quantization [51]. which approximates the logarithmic scale of the original weights: logzl W| s Σi 2 b1, where B; E {0, 1|mxn and b RR. This technique particularly effective when the distribution of matrix values is highly skewed. It also offers a hardware- -aiaafffffddlyyyy implementation of matrix multiplication, as the powers-of-twe representation allows the operation to performed using efficient bit-shift operations. However, because the sign information is lost in the logarithmic transformation, an additional bit is required to retain the original sign of each element.
Matrix Factorization Matrix factorization expresses matrix Wixn exactly approximately product W x YZ, and fundamental tool in signal processing, machine learning, and data analysis. The target matrix Wmx", and the factor matrices Ymxl and Zix" are subject to different constraints depending on the specific method. For example:
Singular Value Decomposition (SVD) 19 W Rmxx, Y E R. Z E R
Note that singular value diagonal matrix and orthogonal matrix are described as one.
Non-negative Matrix Factorization (NMF)
E ERRO", Y R
Real/Binary Matrix Factorization (RBMF)
58
W WER"XX Y E B m Z E {0. 13 1x
Non-negative/Binary Matrix Factorization (NBMF)
52
W E
R
Y E Ryxi Z
{0, 1)
Binary Matrix Factorization (BMF) 69
W E (0. 1) mXn Y mx m)x Z E (0, 1
Boolean Matrix Factorization (BoolMF)
49
50
W 10, 1m Ye E (0, 1 jxxl zixn
{0, 13. with Boolean product: W.y ~ Vi Az,))
Such matrix factorization techniques are widely used not only in data analysis but also for matrix compression through low-rank approximation (i.e., 1 & min(m, n)) [3] Building on this idea, Low-Rank Adaptation (LoRA) [25) enables efficient fine-tuning of large pre-trained models by restricting weight updates to low-rank subspace. More recently, [I]] 64, (22) combines LoRA with
quantization, further reducing memory usage while preserving model quality, and has become a widely adopted approach for resource-efficiett fine-tuning.
Preliminaries
Polynomial Unconstrained Binary Optimization (PUBO) Polynomial Unconstrained Binary Optimization (PUBO) I1 [8]
class of combinatorial optimization problems defined as the minimization of a multivariate polynomial over binary variables. Formally, it can be expressed as:
L(s) = Σ 21
(()) Set
Σ $15322
S Se, S1g
Σ
II
SKys
(3)
$1 $1.552 SE52K---E
where s E {0, 1 is binary vector and (k) denotes the k-thh order interaction coefficients. In the special case where the degree k = 2, the problem reduces to the well-known Quadratic Unconstrained Binary Optimization (QUBO) formulation 54 35 QUBO equivalent to minimizing the energy function of the Ising model [27] and has been widely studied in various fields, including physics, computer science, and artificial intelligence. is ha-hard, and thus, broad range of heuristics [33 20 21 36 46), optical computing (26 24
general, solving PUBO, including QUBO, problems
and quantum computing [32 30 have been proposed to tackle them efficiently.
Annealed Mean Field Descent One of the recent promising approaches for solving QUBO prob- lems is Annealed Mean Field Descent (AMFD) (36), which is based on Mean Field Annealing [3]. It

---

aims to find minimum solutions by gradually annealing the temperature while optimizing mean mean-field
L(s) approximation to the canonical distribution: Pc(s) = 3 exp T Z = Z=E. exp T
L(a)
Since computing the distribution is generally intractable due the exponential number of configura- tions, it is approximated by an independent distribution for each variable (mean-field approximation): PMr I Ps(); where (s;); denotes the probability of taking value s,. AMFD minimizes the Kullback-Leeblee (KI.) divergence between PM (8) and (c(() by gradient descent- based updates. minimum states, thereby allowing
At low temperatures, the canonical distribution concentrates the extraction of approximate minimum solutions. While AMFD derived an explicit form the KL. divergence for QUBO problems. this work extends the framework to general PUBO problems. We show that the KL divergence between PMF(s) (8) and (s()) can be written as:
DKL (PMF(s) || Pc()) = L(x) T
+ h Σ (1-- x1) x.) + In
(4)
where 2; = E
= S.P:(s,) (1() is the expectation of s, under the mean-field distribution. Please refer to the App. A. for a detailed derivation of this formulation. Note that xr is a real-valued variable I, rather than a binary variable. Therefore, the KL divergence is differentiable with respect to rę. One iteration of the AMFD algorithm is illustrated in Alg. Note that the term [11 - 2;) In(1 - x;) + x; In In in Eq. (4) is approximated by second-order Taylor expansion around H; 0.5 to prevent numerical overflow when I; is close to 0 or 1. This paper applies the
ranging from
extended AMFD to optimize quantized representations under the general PUBO setting.
4 Proposed Method
4.1 Binary Quadratic Quantization (BQQ)
A primary limitation of conventional first-order quantization methods like UQ and BCQ is the limited number of distinct values each element can take. For example, 1-bit quantization allows only two levels (e.g., +1} or {0, 1)), and 2-bit quantization increases this to just four. Such coarse granularity restricts representational flexibility, especially under aggressive compression. While such methods are limited in expressiveness, binary matrix multiplication can yield outputs with wider value range, enabling multi-bit representations even though each binary matrix individually encodes only minimal information. This property suggests previously underexplored potential for approximating real-valued matrices through compositions of binary matrices, offering a new perspective beyond traditional quantization methods. Nonetheless, existing matrix decomposition approaches operate within fixed numerical domains either real-to-real (e.g.. SVD, NMF) or binary- to-binary (e.g.. BMF, BooIMF). Hybrid methods like RBMF and NBMF bridge these domains but stop short of fully binary decompositions of real-valued matrices.
Motivated by this gap. we explore a novel quantization scheme that, unlike BCQ which uses linear combinations of binary matrices, is based on linear combinations of binary matrix products:
p-1 W
Σ a. Y. + B.1)) 6,12), 2m0
(5)
where W E mmx Y, E {0, 10 and Z, E 10 are binary matrices, while a;, 7i E R are scaling factors, and 3; d; E IR are bias terms. Also, 1y, 1z, denote all-ones matrices with the same shape Y, Z, W. respectively. Notably, this formulation subsumes BCQ as special case.
Algorithm One Iteration of AMFD [36]
Input: L. Kolds Beurs T. AT, 7)
Output: Tears Knew T
1: Piwd &- Beur + (Xeur- Vola)
Forward point
2: d <- V [((rwa))
Gradient of the first term in Eq. 4 scaled by T
3: +- T (xeur 0.5)
T-
Gradient of the other terms in Eq. (4) (approx. expr.) scaled by T
4: Enew +- clip (2x Fold 7 (F + 1,
> Descent with accelcration and constraints > Anncaling
5: T f-1 AT

---

Specifically, when I = max(m, n), setting Y 5,1y as the identity matrix (if m 2 n.), or setting 7; + 8,12 z the identity matrix (if m n), recovers the standard BCQ structure. We turn our attention the generalized form of Eq. ( referred as Binary Quadratic Quantization (BQQ):
p-1 W 2
Σ (n.. 4,1 Z.) +ul,
(6)
where Tt, St, I are scalar coefficients. Note that the all-one matrices used only for notational convenience and do not need to be stored explicitly; only the binary matrices and scalar coefficients must be preserved. Also, the intermediate dimension I can be arbitrarily set, allowing the number of binary elements to be adjusted independently of the original matrix size.
4.2 Mixed Integer Programming for BQQ
To realize BQQ formulation (Eq. (6). consider minimizing the squared error between the original real-valued matrix and its approximation. The objective function given by:
Laqe
Σ + + 41v Z.)
where the goal is to optimize 3p + 1 real-valued coefficients and the elements of 2p binary matrices. This is -ixed-integer optimization problem and is NP-hard, making analytical solutions intractable. To address this, we adopt the following strategy:
1. We apply greedy optimization independently to each index i in Eq. to mitigate the increasing complexity from a growing number of binary variables.
2. We decouple the optimization of real-valued and binary variables, and alternate between convex quadratic optimization and PUBO.
To perform greedy optimization for each index in Eq. (7). we first define the residual matrix (1) as the difference between the original matrix W and the partial reconstruction using variables D
up to index 1: w W
(r + sj sjY,ız + t, Z, + u,1)] Notably, as an W. Then, the i-th subproblem can be formulated as the minimization of res
exception, set W (0) following objective:
W (r:Yi + s,Y, 4, Alyy Z: =.:1
(8)
sub
Next, to minimize the objective function (Eq. (8), we adopt an alternating optimization approach that separates continuous and binary variables. When the continuous coefficients are fixed, the problem
Algorithm 2 Subproblem Solving via AMFD
Input:
Input matrix W E Rmxn, initial temperature Tinis final temperature Tin, steps Natep learning rate 7/, accelerating rate Ç, intermediate dimension
Output: Binary matrices Y {0,1)
100.1)mx mxx Z {0, 16.1 1x scaling factors T.s S,, U4 R
1: Let 1 E {1)mxn, ly [1)mxx] and 1z [1)
2: Sample Youa, Zold 1))
Initial expectation values
3: Y + Youd 17( old 0.5). Z Zold 1/(Zold 0.5)
4: W + W/ (maxx W) min W))
> Normalization
5: AT (Tim- T // (Natep 1)
6: P+Tiii
7: (ri, 84, li, us] SFO(Y 2, W)
> SFO: scaling factors optimization using Eq. 1T
8: for / = to Naop do
(Y. 2 I 21 T AMFD (1 (W, (ro, 81, 4, 4,,,,)) us]),, (Y old, Zota]; [Ý, TT,
9:
> AMFD: AMFD iteration using Alg
10: [ri, 8;, 4, u;] f- SFO(Y, 2, W)
Y step Y 0.5), Z step 2 - 0.5)
Binarization the higher probability
12: [re, 84s 4, Wz] (max W) - min W)) SFO Y, 2, W)
5

---

Algorithm 3 Greedy Binary Quadratic Quantization
Input: Matrix W, learning rate accelerating rate Ç. initial temperature Tini,, final temperature Tin.
steps Nawp intermediate dimension I, binary matrix stacks p
Output: Binary matrices [Yo. Yi.... 1p- [Zo, Zi...., Z 1). scaling factors s, IP, u
Wies W
2: for = to (p 1) do
3:
Y, Zi, Ts, 81, bi Us SS WS(W res, mia Tim, Naep 7. (,1) f. SS: subproblem solving via Alg.
4: W WTS res (r.Yi + Y,12 f, Zi + u;
5: u
Ei= U4
Original
First-order Quantization
Binary Quadratic Quantization
Input PRiooo Large Real-valued Matrix
Input
Input
P Large Binary Matrix
h
Small Blinary Matrix
69
las on
us 25 13 23r
us
Small Binary Matrix
Output wx
Output
Output
2'D +b1
U.V. + aT., 142)
x
Figure 1: Comparison between BQQ and first-order quantization in DNN layer.
reduces optimizing over binary variables only. In this case, the objective can be reformulated as a PUBO by reducing powers of binary variables using y² = y, making each such term linear. The resulting objective takes the following form:
(8) pubo
((0)
Σ Y.Z. (Y (Z, Z,)
+ s
Σ 17%-0% Yi] + + m Σ Z. (Z: Z,)
(9)
+ 2r.8. Σ (Y: @ K. 2r. Σ Yiz. Y.(z, @ Z:)]
where Σ denotes the elementwise
all entries of the corresponding matrix.
On the other hand, fixing the binary matrices, the continuous coefficients r;, 84, 14, u, can be optimized in closed form via the convexity of the & norm.
Tri S6, L, ux] Bryy Uags Uc, Unz H Eydt
(10)
where H R4x4 and e E R24 are Hessian matrix and the first-order coefficients Eq. with respect to rıs St,t, u4 respectively. By incorporating the optimization of the scaling factors into a single iteration of the AMFD algorithm for PUBO, we aim to solve the subproblem. The complete procedure is presented in Alg. 2 Using the solution of this subproblem, we then approximate the original real-valued matrix in a greedy manner. This overall approach is described in Alg.
4.3 Post-Training Quantization via BQQ
This subsection presents a model compression technique for deep neural networks (DNNs) based on BQQ. particular, we focus on ViT-based models. which have recently achieved remarkable success in image processing but still struggle with ultra-low-bit quantization. Quantization methods for ViTs, as well for general DNNs, can be categorized into two types: Quantization-ANe Aware
Training (QAT) 44 39 63], which integrates quantization into the training process using labeled
data, and Post- Trainingn Quantization (PTQ) 43 68 71 72), which applies quantization
pretrained model using either unlabeled or limited data. Especially, in situations where training data is unavailable due to privacy constraints or data access limitations, has driven interest in data-free quantization techniques 40) 42 Our work focuses PTQ under two different scenarios. the data-free setting, we perform only data-free weight quantization. In the calibration- based setting, first apply data-free weight quantization, followed by correction
need address these challenges

---

of bias and normalization parameters using a small amount of unlabeled calibration data. Previous studies have explored weight quantization to reduce model size and inference costs, as well as quantization of both weights and activations to further reduce inference costs. In this study, we focus exclusively on weight quantization to clarify the standalone effectiveness of BQQ. Note that, unlike standard first-order quantization, BQQ approximates the original matrix using combination of binary matrices with altered shapes, as illustrated in Fig. [
(a): channel-wise (column-wise) scaling factors
Data-Free Quantization First, we apply data-free quantization approach, directly quantizing the weights. Specifically, we formulate weight quantization as op- timization problem that minimizes the squared reconstruc- tion error between pretrained weight matrix W and its quantized counterpart via BQQ, as shown in Eq. ( While prior methods often use channel-wise (i.e., column- wise) scaling factors to maintain accuracy, they result in a large number of scaling parameters. Instead, we adopt a group-wise quantization strategy [65]. in which each weight matrix is divided into smaller submatrices, and BQQ is applied independently to each with its own set of scaling factors (see Fig2). This can reduce the number of scaling parameters, thereby shrinking the model size.
W1.1 coli W
W1a
W Wre-
W Wzo col
Wxa c
col
W
W
W
Wm.
m22
Mmn-1
Num. of scaling factors:
(b): group-wiss soaling factors
W. W12 W group
Win W Bro
W
W. W. group AV
W
W group W Map-1
mn
Num. of scaling factors: v/ (group size m)
Figure 2:
Weight scaling methods.
Correction of Bias and Normalization Parameters After quantizing all weight matrices, we optionally apply a lightweight correction step using small set of unlabeled calibration inputs. Similar to (5), we refine only the bias and layer normalization parameters-we parameters fixed-by minimizing the mean squared error between the output logits of the original fog and quantized models fq. as a form of knowledge distillation:
keeping all other
mine fl.oore (Owg)) - fq (0)1 / [.ooog (Owr)), where 0 denotes the bias and normalization parameters. This correction step compensates for quantization-induced errors and helps recover lost accuracy without requiring full fine-tuning or access to labeled data.
Evaluation
Implementation Details As described Eq. (6) BQQ decomposes real-valued matrix of size v x n into binary matrices Y; E {0, 1)mxi and Z; {0. 1 To ensure a fair comparison with baseline methods like UQ and BCQ, we fix the intermediate dimension 1 = round all binary matrices. This ensures the total number of binary parameters matches that of UQ and BCQ. making p in Eq. (6) the pseudo bit width. Another way to match the number of binary parameters is to adjust the ratio between the intermediate dimension / and the number of stacks p in Eq. (6); however, this paper adopts the approach described above. Unless otherwise noted, the hyperparameters used in Alg. are set to the following values throughout all experiments: Tim 0.2, Tim 0.005, 17 0.06, < = 4, and Naup 50,000. Also, the scaling factor and the bias for UQ are optimized via grid search to minimize the mean squared error (MSE), as described in App. A.2 For BCQ, we implement the method based on (62 referring to parts of the open-source code provided in [65
+ -z)) for
Matrix Data Compression We evaluate the trade-off between approximation error and memory size across five types of real-valued matrices: (i) random matrix sampled from a Gaussian distribution, (ii) weight matrix from the DeiT-S model (59), (iii) an inter-city distance matrix from the TSPLIB dataset [55] (iv) matrix composed of multiple 128 188dimennional feature vectors extracted from the SIFT dataset (28), commonly used in ANN search, (v) red channel matrix of an image from the ImageNet dataset [10]. Each matrix is standardized to have zero mean and a variance of one prior to quantization. We compare nine methods: (1) SVD, a low-rank approximation using SVD; (2) SVD + p-bit UQ, SVD low-rank approximation followed by p-bit UQ of the factorized matrices; (3) UQ; (4) BCQ: VQ, vector quantization where groups of values are clustered using k-means and encoded as indices of codebook; (6) VQ + p-bit UQ VQ followed by p-bit UQ of centroids; (7)Eg LVQ, lattice vector quantization using the Eg lattice with 240 centroids of norm V2. representing each 8-dimensional input as a linear combination of centroids stored in 8 bits; (8) 8-bit UQ + JPEG; a combination of 8-bit UQ and JPEG-style compression (61. where discrete cosine transformation is
applied before quantization to exploit spatial redundancy in images; (9) BQQ. The performance measured in terms of MSE and the memory size of the quantized matrices.
7

---

Random
DeiT
Distance
SIFT
ImageNet
SVP
SVD
0.0
5.6
0.2
NSE
2/1
0.2
0.2
0.0
8-bi UQ -PGG
Memory (KB)
Momory Size [KB)
Memory Size (KB)
Memory (KB)
Memory Size [KD)
DOC
1903
500
4380
Requericy
600
30000
Frequency
Frequency
3380
Proquency
2003
7500
2380
992
$300
0.0
9:30
Value
Value
Valuee
Via
Velue
135
Frequncy
Preauscc
Frequency
30
Frequincy
103
73
Singular
Singular
Singular Value
Singuiar Value
Value
value
Velue
Sg Ara Row Indixx
Row Fndeas
L9ce Row tedie
o 1o Row Irdix
-0g Rank Whitex
Col Indox
Cal Indlxx
2003
Col Indexx
Col trdex
Col Index
20
Figure 3: Comparison of the trade-off between reconstruction error (MSE) and memory size for five matrix datasets. Rows 1-4 show the trade-off curves, value distributions, singular value distributions, and 3D plots of the matrices, respectively.
Fig. 3 presents trade-off curves along with visualizations of value distributions. singular value distributions, and 3D surface plots derived from the original matrices. Across all datasets, BQQ consistently achieves a superior trade-off between compression rate and reconstruction accuracy, demonstrating its general effectiveness for matrix data approximation. Notably, the advantage of BQQ over UQ BCQ, and LVQ becomes especially pronounced for matrices whose singular value spectrum dominated by a few large components. In contrast, for matrices with relatively flat singular value distributions-i.. This suggests that BQQ particularly benefits from matrices with concentrated spectral energy. On the other hand, when compared to SVD, SVD UQ, VQ, and VQ UQ the opposite trend is observed: BQQ exhibits greater advantage for matrices with more uniform singular value spectra. Admittedly, the ImageNet dataset, BQQ yields less favorable trade-off between memory size and reconstruction error compared to JPEG. However, since JPEG combines discrete cosine transform with quantization, integrating BQQ with transform-based approaches could potentially lead to further improvements in compression efficiency.
those lacking dominant components-tth gain over them is smaller.
Post-Training Quantization for ViTs Following the methodology outlined in Sec. 4.3 evaluate the performance of BQQ on pretrained ViTs. Specifically, we compare BQQ with several leading PTQ methods, including COMQ (68), FQ-VIT [43), PTQ4ViT [66]. RepQ-ViT [41] ERQ [71), and PSAQ- ViT [40] using two representative ViT architectures: DeiT (59 and Swin Transformer [45). All methods tested under the 32-bit activation setting to ensure fair comparison. Additionally, to
appropriately assess the effectiveness of BQQ, we also compare it with variants that use the exact same process but replace the quantization method with UQ or BCQ, as baselines. In our quantization framework, we apply group-wise quantization with submatrices of size 384 x 384 for DeiT' and 96 x 96 for Swins based on the first block's channel number. If a weight matrix matches the group size, it is not subdivided further (i.e., layer-wise quantization). As a special case, the final classification layer uses group size of 100 x 96. Also, the linear patch embedding layer in DeiT is grouped based on its embedding dimension (384 x 384 for DeiT-S and 784 x 784 for DeiT-B), while Swin's 's patch embedding layer, implemented as a convolutional layer, uses channel-wise UQ instead of BQQ. The same group sizes are applied in the UQ and BCQ baselines for fair comparison. In the case of bias and normalization parameter correction (denoted as c-UQ, c-BCQ, and c-BQQ for each quantization method), we optimize them using the Adam optimizer 34] with a learning rate of 0.001 for 15 epochs
8

---

Table I: Comparison of ImageNet top-1 accuracy across various quantization methods on ViTs.
Top-1 Accuracy [%)
Method
W bit
Data Free
W scale
DeiT-S
DeîT-B
Swin-T
Swin-S
COMQ
column- wise
67.19
77.14
74.05
78.02
ERQ
column-we wise
31.95
63.67
45.97
35.44
RepQ-ViT
column-wii wise
0.31
0.12
e-UQ
group-wise
52.21
60.37
67.49
74.16
c-BCQ
group-wise
60.13
73.37
68.09
73.97
cBQQQ
2
group-wise
69.41
77.94
74.03
78.47
PSAQ-VIT
column-wis
0.27
0.15
0.14
UQ
V
group-wise
3.23
2.45
14.69
30.69
BCQ
V
group-wise
10.83
12.99
18.62
34.84
BQQ
2
V
group-wise
58.25
72.09
57.37
68.17
COMQ
column-wise
77.47
80.47
79.31
81.95
ERQ
column-wist
75.56
79.73
77.99
80.87
RepQ-ViT
column-wise
58.26
68.80
21.41
69.57
FQ-VIT
column-wise
51.06
65 64
65.38
71.88
PTQ4VIT
layer-wise
70.22
75.42
70.74
73.46
<UQ
group-wist
72.08
78.85
78.11
80.91
cBCCQ
group-wise
75.53
79.78
78.60
81.19
BBQ
3
group-wise
77.33
80.81
79.34
81.86
PSAQ-VIT
3
column-wise
52.76
66.40
65.87
72.53
UQ
group-wisc
42.28
58.56
70,90
75.72
BCQ
V
group-wise
63.46
69.09
72.99
76.18
BQQ
3
V
group-wise 75.61
79.90
77.33
80,36
COMQ
column-wise 78.98
81,40
80.89
82.85
ERQ
column-wise 78.95
81.46
80.85
82,99
RepQ-ViT
column-wise 75.39
78.77
75.08
81,53
FQ-VIT
column-vee wise
76.23
79.92
78.81
81.89
PTQ4VIT
X
layer-wise
77.50
80.07
78.46
80.24
c-QQ
group-wise
78.15
81.01
80,42
82,40
cBBCQ
group-wise
78.67
81.22
80.46
82,47
cBBQQ
4+
X
group-wise
79.12
81.47
80.57
82.72
PSAQ-VIT VIT
V
column-wise
76.59
80.23
79.15
81,94
UQ
4
V
group-wise
73.53
77.49
79.17
81.47
BCQ
V
group-wise
75.82
78.58
79.63
81.72
BQQ
4
V
group-wise
78.76
81.20
80.21
82.21
Full Precision
32
79.83
$1.80
81,37
83.21
*: Pseudo p" BQQ a model size matching that p-hit quantized model, despite each matrix being bit.
via minibatch size 16, and calibration data are randomly selected from the ImageNet [0 training
dataset, with 2048 samples for DeiT's and 1024 samples for Swins, in accordance with 68
setting.
Tab. summarizes the experimental results. Here, W bit denotes the bit width for weight quantization, while W scale indicates the granularity of scaling factors (c.g., per-layer, per-group, or per-column). Note that although each weight matrix in BQQ binary, its configuration is designed
match the information content of a first-order p-bit quantized model, which we refer to as pseudo p--bit. The results for COMQ
FQ-VIT [43] and PTQ4ViT [66 are cited from 688 while those for RepQ- ViT [I), ERQ 711. and PSAQ-V VIT 40 were obtained using publicly available implementations. As shown in the experimental results, BQQ consistently achieves SOTA performance regardless of whether calibration data is used, demonstrating compelling trade-off between accuracy and model size. In particular, it shows notable improvements both in data-free settings and configurations with model size equivalent to 2-bit quantization. To the best of our knowledge, this is the first study to achieve practically usable accuracy with model size equivalent to 2-bit quantization of any data. In addition, while most existing methods preserve accuracy by using column-wis wise scaling factors- in larger model size-BQQ adopts group-wise scaling, which can reduce parameter overhead. Despite using more compact scaling strategy, it still achieves competitive accuracy.
the absence
6 Discussion
BQQ Effectiveness and Characteristies As shown in the matrix compression experiments, the advantage of BQQ over UQ, BCQ, and LVQ becomes more pronounced for matrices with skewed singular value distributions, where few dominant singular values capture most of the spectral energy. Conversely, when the singular values are more uniformly distributed, the performance gap between these methods and BQQ becomes smaller. On the other hand, BQQ shows a clear advantage over SVD- and VQ-based methods when the singular value spectrum is relatively flat. This is likely because SVD and VQ are designed to capture and compress redundant patterns in the matrix, which works

---

well for low-rank or structured data. When such redundancy is absent- as in random- like matrices with weak spectral maas-their performance tends degrade. Unlike SVD and VQ, UQ, BCQ, and IVQ quantize each element or vector independently to its nearest representative value. This makes it difficult to exploit pattern redundancy, but also allows these methods to remain relatively stable across different spectral shapes. In fact, when the data lacks significant structure, such independent quantization can lead to more efficient compression than pattern-based baseer approaches. Overall, BQQ integrates the strengths of both pattern-oriented and element-wise quantization strategies. It leverages the ability to capture structural cellunddacy-smmiin to SVD and VQ-while also benefiting from the stability and granularity of scalar quantization methods like UQ, BCQ, and LVQ. As result, it achieves robust compression performance across a wide range of singular value distributions.
In addition, PTQ experiments on ViT's demonstrate that BQQ achieves SOTA performance in both data-free and calibration-based settings. While COMQ or ERQ slightly outperforms it in some cases, they adopt channel-wise quantization with more scaling parameters, whereas our group-wiss approach yields more compact model. Moreover, in contrast to most PTQ methods that optimize discrete parameters by minimizing output error, our approach optimizes binary parameters by simply minimizing the reconstruction error from the original weight matrix (i.e, no PTQ-specific binary variable optimization is performed). Despite this, BQQ matches or even surpasses PTQ-specialized methods, which is a noteworthy outcome. These results are likely due to BQQ's ability to capture structural redundancy often overlooked by first-order methods in overparameterized layers. It is also noteworthy that the matrix multiplication between weights and inputs can be performed using only addition operations, resulting in minimal computational overhead for inference (see App A.4 This suggests that significant acceleration could be achieved with specialized hardware.
Further Potential and Limitations Despite the demonstrated effectiveness of BQQ, there remains significant room for further improvement. In the current implementation. we adopt a greedy op- timization strategy as described in Alg. 3 which is suboptimal from a global perspective. Jointly optimizing all binary matrices and scaling factors could potentially lead to further reductions in quantization error. In addition, although our PTQ framework with BQQ is based minimizing weight approximation error- exceet for the correction of bias and normalization parameters it is generally more effective minimize output quantization error, as demonstrated in many previous studies. Adapting BQQ to optimize binary matrices with respect to output error could therefore lead to even greater PTQ accuracy. Nevertheless, it is noteworthy that BQQ already achieves SOTA performance. Moreover, in our experiments, the intermediate dimension is fixed, as described in Sec. [5] However, this configuration may not be optimal under a fixed binary parameter budget. Ex- ploring the optimal ratio between the intermediate dimension and the number of binary matrix stacks (p in Eq. (6)) could further improve approximation error (see App. A.9.. While BQQ holds such potential, this work has certain limitations. Specifically, while an upper bound on the approximation error is provided (see App. A. it does not yet establish a theoretical guarantee that our method outperforms first-order quantization under specific conditions. Additionally, the quantization process still incurs a neglliiiee computational cost (see App. App.A..p. Nonetheless, we believe that BQQ has the capacity to contribute to wide range of applications beyond the experiments presented in this study, and could have a significant impact.
7 Conclusion
We introduced Binary Quadratic Quantization (BQQ), a novel quantization framework that approx- imates real-valued matrices as linear combinations of binary matrix products. Across both matrix compression and ViT-based PTQ tasks, BQQ consistently outperforms existing methods in terms of accuracy and compression ratio. These findings highlight the remarkable capability of second-order binary representations in capturing complex structures beyond the reach of first-order schemes, while maintaining an extremely compact data format. By providing an expressive and versatile framework for compressing real-valued matrices using binary bases, BQQ opens new possibilities for building efficient, scalable systems across a wide range of machine learning and information processing applications. We believe this work lays the groundwork for future research into quadratic binary representations and their role hirh-peeformance model compression, retrieval systems, and large-scale learning on massive training data.
10

---

Acknowledgements
This work was supported by IST-ALCA-Next (Grant JPMJAN24F3), by JST PRESTO (Grant JPMJPR23PI), and by JST SPRING (Grant JPMISP2180).
References
[1] Erik Agrell and Thomas Eriksson. Optimization of lattices for quantization. IEEE Transactions on Information Theory, 44(5):1814-1828 1998. doi: 10.1 10111009/1799900
[2] Sunil Arya, David M. Mount, Nathan S. Netanyahu, Ruth Silverman, and Angela Y. Wu. An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM, 45(6):891-923, November 1998. ISSN 0004-5411. doi: 10.1 10.1145/93347.29933488 URL
nttps://doi.org 1145/293347 293348
[3] Griff Bilbro, Reinhold Mann, Thomas Miller, Wesley Snyder, David den Bout, and Mark White. Optimization by mean field annealing. In D. Touretzky, edi- tor, Advances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann, 1988. URL https::/proceeddiiie inwrreeeeddddiinn..nn.......p.pp nwwrins.rccoom wwtpcpcpppppppp...cme wwtp:ffflessseeeeeeeeeaemmmameee pttp///pppe/////eeeeeee
pdf
[4] Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In AAA/ Conference on Artificial Intelligence,
2019. URL http: http://////pp..eemmmaccccccooooo
org wtw.:oouai...rr/ooouuuI.... : 208290939
[5] Adrian Bulat, Yassine Ouali, and Georgios Tzimiropoulos. QBB: Quantization with binary bases for LLMs. The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httptttt//peeeeeeeeeeee/// htt.::/w..ttt.oooommmooooooooone
[6] Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, and Dacheng Tao. DB-LLM: Accurate dual-binarization for efficient LLMss. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACI. 2024, pages 8719-8730 9-730, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 18.58//3/v www.24.inniings-ei... 516. URL http/ttt////aallathoooggy/ .eceeey..rr//ee 2024. 2tt..004....ii..i findings-acl.552 516/
[7] Dmitry A. Chermoshentsev, Aleksei O. Malyshev, Mert Esencan, Egor S. Tiunov, Douglas Mendoza, Alán Aspuru-Guzik, Aleksey K. Fedorov, and Alexander I. Lvovsky. Polynomial unconstrained binary optimisation inspired by optical simulation. 2021.
[8] Christopher Clark, Kenton Lee, Ming-Wet Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. ArXiv, 100044,0.10044, 2019. 165163607
URL https://api semanticscholar org/CorpusID:
[9] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reason- ing challenge. ArXiv, abs/1803.00443 2018. URL htttptt/s://ppi www.ppi..emmanncccool.... org/
CorpusID:392281
[10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009. doi:
(009/CVVR.200..52000
[11] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. In Thirth-seventh Conference on Neural Information Processing Systems, 2023. URL httpt::::pppeeeeeeiiie .et http://eettototttt//eie-aa-e
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition scale. In International Conference on Learning Representations, 2021. URI. https://openrevies.nee http://reww...eeeoroooooooaoaaaa

---

[13] Carl Eckart and Gale Young. The approximation of one matrix by another of lower rank. Psychometrika, 133):111-218, Sep 1936. ISSN 1860-0980. doi: 10.1007/BFO2288367 URL
https://doi.orr htt.://6a...rr///..000///8222220
[14] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate quantization for generative pre-trate pre-trained transformers, In The Eleventh International Conference on Learning Representations, 2023. URL httpt:///penrreeeiee
[15] Allen Gersho and Robert M. Gray. Vector quantization and signal compression. Kluwer Academic Publishers, USA, 1991. ISBN 0792391810.
[16] Jerry D. Gibson and Khalid Sayood. Lattice quantization. volume 72 of Advances in Electronics and Electron Physics, pages 259-330. Academic Press, 1988. doi: hutps///oo..or//10..00 $0065-2539 6056600. URL
www.ssceeeeeeeeccccomm. com htt.:/acceereetttccccccccceeeeaa
p:1/$0066253988666600
[17] Fred Glover, Jin-Kao Hao, and Gary Kochenberger. Polynomial unconstrained binary optimisa- tion - part 1. Int. J. Metaheuristics, 1(3):222-266, July 2011. ISSN 1755-2176.
[18] Fred Glover, Jin-Kao Hao, and Gary Kochenberger. Polynomial unconstrained binary optimisation-pan 2. Int. J. Metaheuristics, 1(4):317-34, December 2011. ISSN 1755-2176. doi: 10. 10.500//MMEERR.AA 201 2011..43356 1.043556. URL httpp://doii org/0 1tt.:/10..1111111111 1504/IJMHEUR. 2011 044356
[19] G. H. Golub and C. Reinsch. Singular value decomposition and least squares solutions. Nu- merische Mathematik, 14(5))403-422 April 1970. ISSN 0945-3245. doi: 10.1007/8FO21133027.
httthttp::/ddee org/ 10. 10001000//BBB /BP0216022)
[20] H. Goto, K. Tatsumura, and R. Dixon. Combinatorial optimization by simulating adiabatic bifurcations in nonlinear hamiltonian systems. Science Advances, 5(eaav2372), 2019. doi: 10.1 1/26//sciadv.aav23722
[21] H. Goto, K. Endo, M. Suzuki, Y. Sakai, Kanao, Y. Hamakawa, R. Hidaka, M. Yamasaki, and K. Tatsumura. high-performance combinatorial optimization based on classical mechanics. Science Advances, 7(eabe7953), 2021. doi: 10. 126/sciaav
[22] Han Guo, Philip Greengard, Eric Xing, and Yoon Kim. LQ-IoRA: Low-rank plus quantized matrix decomposition for efficient language model finetuning. In The Twelfth International Conference on Learning Representations, 2024. URL htttps://openrevieee wwwpiiiwwweeeffoooo net/forum?
1d=xw29VvOMMJ
[23] Yiwen Guo, Anbang Yao, Hao Zhao, and Yurong Chen. Network sketching: structure in deep enns. In 2017 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 4040-4048, IEEE Computer Society, 2017. doi: 10. www.11199VVP22..2.11 1109/CVP 2017001722 430
Exploiting binary
URL
http:/poo....eemmmuutrrrrrocee// org/10
[24] T. Honjo, T. Sonobe, K. Inaba, T. Inagaki, T. Ikuta, Y. Yamada, T. Kazama, K. Enbutsu, Umeki. R. Kasahara, K. Kawarabayashi, and H. Takesue. 100,000-spin coherent Ising machine. Science Advances, 7(cabh0952), 2021.
[25] Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. WRLphts::///perteee set//forumiid-nZeeWkeee
[26] T. Inagaki, Y. Haribara, K. Igarashi, T. Sonobe, S. Tamate, T. Honjo, A. Marandi, P. McMahon, T. Umeki, K. Enbutsu, O. Tadanaga, H. Takenouchi, K. Aihara, K. Kawarabayashi, K. Inoue, S. Utsunomiya, and H. Takesue. A coherent Ising machine for 2000-node optimization problems. Science, 354(6312):603-6M) 2016. doi: 10. wtt.///iienc..aa42222.
[27] Ernst Ising. Contribution to the Theory of Ferromagnetism. Z. Phys., 31:253-258, 1925. doi: 10.1007/BF009805777
12

---

[28] Hervé Jégou, Matthijs Douze, and Cordelia Schmid. Product Quantization for Nearest Neigh- bor Search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117- 128, January 2011. doi: 10.1009/PAMI..210.577 URL URL.tttss:: htttps//iintt hal. science/ inria-00514422
[29] Yongkweon Jeon, Chungman Lee. Eulrang Cho, and Yeonju Ro. Mr.biq: Post-training non- uniform quantization based minimizing the reconstruction error. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), puges 12329- 12338, June 2022.
[30] M. W. Johnson, M. H. S. Amin, S. Gildert, T. Lanting, Hamze, N. Dickson, R. Harris, A. J. Berkley, J. Johansson, P. Bunyk, E. M. Chapple, C. Enderud, J. P. Hilton, K. Karimi, E. Ladizinsky, N. Ladizinsky, T. Oh, I. Perminov, C. Rich, M. C. Thom, E. Tolkacheva, C. S.
Truncik. S. Uchaikin, J. Wang. B. Wilson, and G. Rose. Quantum annealing with manufactured spins. Nature, 473(7346): 1997446):994-988 May 2011. doi: 10. 10010///auure11002..
[31] Herve Jégou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEFE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117-128, 2011. doi: 10.1 P0.1100/PAAM..210......
[32] T. Kadowaki and H. Nishimori. Quantum annealing in the transverse Ising model. Physical Review E, 58:5355-53300 5363, Nov. 1998. doi: 10.1 10.100/Phyy.eee..85555.
[33] Taro Kanao and Hayato Goto. Simulated bifurcation for higher-order cost functions. Applied Physics Express, (6(1), January 2023. ISSN 1882-0778. doi: 10. 10.35848/1832-0086/aabba Publisher Copyright: 2022 The Japan Society of Applied Physics.
[34] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors. ICLR (Poster), 2015. URL http://///dblp. db/ conf /imr:/icrrcc1011/ hww.hhmmlkkkn Atm1#KingmaB14
de/
[35] Gary Kochenberger, Jan Hao, Fred Glover, Mark Lewis, Zhipeng Lü, Haibo Wang, and Yang Wang. The unconstrained binary quadratic programming problem: survey, Journal of Combina- torial Optimization, 28( 28()):88-8I, July 2014. doi: 10.100777110778-01449774444 URL https:// htmll
ideas.repec repec http:pooog.aa//p///oooooo/////// http:///000411110000000000000800
1007
[36] Kyo Kuroki, Thiem Van Chu, Masato Motomura, and Kazushi Kawamura. Annealed mean field descent is highly effective for quadratic unconstrained binary optimization, 2025. URL
https://arxiv org/ mooss/2004.033
[37] Se Jung Kwon, Dongsoo Lee. Yongkweon Jeon, Byeongwook Kim, Bae Scong Park, and Yeonju Ro. Post-training weighted quantization of neural networks for language models, 2021. URL https:://ppenrevie .ett:foorummdd-116xxxxTii1
[38] Daniel Lee and Sebastian Seung. Algorithms for non-negative matrix factorization. In
T. Leen, T. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems, volume 13. MIT Press, 2000. URL httppttttts:/prooedddngg iew.roeeeediinnnnnnnnii...... wwt.:ee....eerpaaaeaeeeee ccc/paper..
pdf
[39] Yanjing Li, Sheng Xu, Baochang Zhang, Xianbin Cao. Peng Gao, and Guodong Guo. Q-vit: accurate and fully quantized low-bit vision transformer. In Proceedings of the 36th International Conference Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 97817 1177771088
[40] Zhikai Li, Liping Ma, Mengjuan Chen, Junrui Xiao, and Qingyi Gu. Patch similarity aware data-free quantization for&nnsp:vssion transformers, In Computer Vision ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XI, page 154-170, Berlin, Heidelberg, 2022. Springer Verlag. ISBN 978- 1200022-. doi: 1007/ 978-3-031-20083-0 10. URL http:: org/ 10. 1007 0-778883--0122200---22
[41] Zhikai Li, Junrui Xiao, Lianwei Yang, and Qingyi Gu. Repq-vit: Scale reparameterization for post-training quantization of vision transformers. In Proceedings of the IFEF/CVF International Conference Computer Vision (ICCV), pages 17227-17236, October 2023.
13

---

[42] Zhikai Li, Mengjuan Chen, Junrui Xiao, and Qingyi Gu. Psaq-vit v2: Toward accurate and general data-free quantization for vision transformers. IEEE Transactions on Neural Networks and Learning Systems, 35(12):17277-17238 2024, doi: (0000//TNN..S.22233300000
[43] Yang Lin, Tianyu Zhang, Peiqin Sun, Zheng Li, and Shuchang Zhou. Fq-vit: Post-training quantization for fully quantized vision transformer. In Lud De Raedt, editor, Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 1173-11 1179. International Joint Conferences on Artificial Intelligence Organization, 2022. doi: 10.24963/ijcai. 2022/164 64. https:/: org/ 10. 24 /iaaii/jjai... 2022/164 Main Track.
[44] Shih- Shih-Yang Liu, Zechun Liu, and Kwang- Ting Cheng. Oscillation quantization for low-bit vision transformers. In Proceedings of the 40th International Conference on Machine Learning. ICMI'23. JMLR.org. 2023.
[45] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9992-10002, 2021. URL https
7/api. www.ssseaaccccchlla.... org/ Perg/CorpusID:.. :32352877
[46] Hengyuan Ma, Wenlian Lu, and Jianfeng Feng. Efficient combinatorial optimization via heat diffusion. In The ehirty-eighth Annual Conference on Neural Information Processing Systems, 2024. httpt::///ppenreeee
net/ wttp///oorrddddparreee
[47] Yu A. Malkov and D. A. Yashunin. Efficient and robust approximate nearest neighbor search using hicrarchical navigable small world graphs. IEEE Trans. Pattern Anal. Mach. Intell., 42(4):824-836 836, April 2020. ISSN 0162-8828. doi: 00.1100/PAAL2201.28884444 URL
Sctps://doi.org// 1109/TPAM 2018 2889473
[48] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mix- ture models. ArXiv, abs/1609.077 07844, 2016. URL https:///pptt www.stimmccoooleeee...e org/
CorpusID: 112999111111111
[49] Miettinen, T. Mielikäinen, A. Gionis, G. Das, and H. Mannila. The discrete basis problem. In 10th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD) 2006, Berlin, Germany, September 18-22, 2006, 2006.
[50] Pauli Miettinen, Taneli Mielikäinen, Aristides Gionis, Gautam Das, and Heikki Mannila. The discrete basis problem. Knowledge and Data Engineering, IEEE Transactions on, 20:1 1348- 1362, 11 2008. doi: 00.1109/KKDE.2008.55..
[51] Daisuke Miyashita, Edward H. Lee, and Boris Murmann. Convolutional neural networks using logarithmic data representation. CoRR, abs/1603.01025,, 2016. URL http: btttp://ars //tp:///aivvvv/./ org/ abs/1603. 01025
[52] Daniel O O'Malley, Velimir V. Vesselinov, Boian S. Alexandrov, and Ludmil B. Alexandrov. Nommegativv//inarry matrix factorization with a d-wave quantum annealer. PLOS ONE, 13(12): c0206653, 2018. doi: 10.133//juurnalppp 02066.0. URL https://doi. org/ 1371/
journal. pone. 0206653
[53] Gunho Park, Baeseong Park, Minsub Kim, Sungjae Lee, Jeonghoon Kim, Beomscok Kwon, Se Jung Kwon, Bycongwook Kim, Youngjoo Lee, and Dongsoo Lee. Lut-gemm: Quantized matrix multiplication based on luts for efficient inference in large-scale generative language models. In ICLR, 2024. URL htpp::/o////o/eeieeeee wwtp:///ww....tooommddidd//////e gLARhFLEOF
[54] Abraham P. Punnen. The quadratic unconstrained binary optimization problem theory. algorithmms, and applications. Springer, Cham, Switzerland, 2022. ISBN 9783031045202.
[55] G. Reinelt. TSPLIB-a traveling salesman problem library. ORSA Journal on Computing, 3(4): 376-384, 1991.
14

---

[56] Rajarshi Saha, Naomi Sagan, Varun Srivastava, Andrea Goldsmith, and Mert Pilanci. Com- pressing large language models using low rank and low precision decomposition. In The Thirty-eigtth Annual Conference on Neural Information Processing Systems, 2024. URL
htttps://peerreves. http://ppppaeeeeeeeeeeeerrreeeaa
[57] Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. An adversarial winograd schema challenge scale. 2019. URL httph:://paapp semanticschoola org/
CorpusID: 199370376
[58] Martin Slawski, Matthias Hein, and Pavio Lutsik. Matrix factorization with binary compo- nents. In C.J. Burges, Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, ed- itors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, wwwppppeerreleeeeeeeeeeeeeeeeeee
Inc.. 2013. URL URL httt /ttttppe///prooceddinggss wtwp://wwwoo...onoo..ee.oee.ep.e wwwrres..ccom wttpppp..cpppppeeeeepppppppppppe ffless_fieeeeeeeeeeeeeeeeeeeeeer pptppp/pp//////eee [BBdff.becsBffffddaaa00cccc3
pdf
[59] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers &amp; distillation through attention. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10347-10357. PMLR, 18-24 Jul 2021. URL hatttps:/ppoceeete ieaidiinnssm..........m. mlr paa...pressvv39 touvron21a html
[60] Vinay A. Vaishampayan, N. J. A. Sloane, and Sergio D. Servetto. Multiple-description vector quantization with lattice codebooks: Design and analysis. IEEE Trans. Inf. Theory, 47:1718- 1734, 2001. URL httph: /apii semanticscholaatt.......ccccc http://i......aaoooooooooo/aaaaa
[61] Gregory K. Wallace. The jpeg still picture compression standard. Commun. ACM, 34():330 April 1991. ISSN 0001-0782. doi: 10. 10.1145/003885.10088) URL https:///ooiorrg/10
1145/103085.1 103089
[62] Chen Xu, Jianqiang Yao, Zhouchen Lin, Wenwu Ou, Yuanbin Cao, Zhirong Wang, and Hongbin Zha. Alternating multi-bit quantization for recurrent neural networks. In International Con- ference on Learning Representations, 2018. URL https:// www./eeruwffiiii...m
S19dR9x0b
[63] Sheng Xu, Yanjing Li, Teli Ma, Bo-Wen Zeng, Baochang Zhang, Peng Gao, and Jinhu Lv. Tervit: An efficient ternary vision transformer. ArXiv, abs/2201.080555 2022. URL https
//api.semantiin www.sssemmiiitcii.c. wrg/Corpus :260637566
[64] Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, XIAOPENG ZHANG, and Qi Tian. Qa-lora: Quantization-avee low-rank adaptation of large language models. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, editors, International Conference on Representation Learning, volume 2024, pages 52401- 52418, 2024. URL htttps://proceeeeee Papeeeeaaee-Pae------eeeeeeeeeee
iclr www.c...ccppaeeeee wwwpcppppeeeteeeeeeeeee wwllss/papr//00/00
pdf
[65] Haoran You, Yipin Guo, Yichao Fu, Wei Zhou, Huihong Shi, Xiaofan Zhang, Souvik Kundu, Amir Yazdanbakhsh, and Yingyan Celine Lin. ShiftaddLLM: Accelerating pretrained LLMs via post-training multiplication-less reparameterization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httpt::://ppeerreeee wwwpwewwwwwweeeeeeeeeeeeee
id=JN16h3U301)
[66] Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptt4vit: Post-training quantization for forsnnsp transformers with&nnsspttwin uniform quantization. In Computer Vision ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XII, page 191-207, Berlin, Heidelberg, 2022. Springer- Verlag. ISBN 978- 3-031-19774-1. doi: 10. (007/978-3-001-1999 978-3-031-19775-88
12. URL httpps///do:: orggioorr111000 1007/
[67] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence? In Annual Meeting of the Association for Computational Linguistics, 2019.
wtt.pp..maaatssoo....... org/ Corp//orppi::D : 159041722.
15

---

[68] Aozhong Zhang, Zi Yang, Naigang Wang. Yingyong Qi, Jack Xin, Xin Li, and Penghang Yin. Comq: backpropagation free algorithm for post-training quantization, 2024. URL
https://arxiv... wtwp://////....././///
[69] Zhongyuan Zhang, Tao Li, Chris Ding, and Xiangsun Zhang. Binary matrix factorization with applications. In Seventh IEEE International Conference on Data Mining (ICDM 2007). pages 391-400, 2007. doi: 10.1 109/1CDM.2007.999 .99.
[70] Zhenghao Zhao, Yuzhang Shang, Junyi Wu, and Yan Yan. Dataset quantization with active learn- ing based adaptive sampling. In Computer Vision ECCV 2024: 18th European Conference, Mi- lan, Italy, Seplember 29-0ctt 93-October 2024, Proceedings, Parl LX, page 346-362, Berlin, Heidel- berg, 2024. Springer- Verlag. ISBN 7-77----001--772--9 doi: 10. 10077998-3-001-775558 --00777-----11--777----6---- URL httpt:tt/p//iii... org/ 10.
[71] Yunshan Zhong, Jiawei Hu, You Huang, Yuxin Zhang, and Rongrong Ji. Erq: error reduction for post-training quantization of vision transformers. In Proceedings of the 41st International Conference on Muchine Learning, ICML'24. JMLR.org. 2024.
[72] Yunshan Zhong, You Huang, Jiawei Hu, Yuxin Zhang, and Rongrong Ji. Towards accurate post- training quantization of vision transformers via error reduction. IEEE Trans. Pattern Anal. Mach. Intell., 47(4):2676-269) 2692. January 2025. ISSN 0162-8828, doi: 10.1009TPAML.22253522002.
URL TRLhhtps::0 https://doi....... org/ 10. 119//TPMMII 2025 3528042.
[73] Daquan Zhou, Kai Wang, Jianyang Gu, Xiangyu Peng, Dongze Lian, Yifan Zhang, Yang You, and Jiashi Feng. Dataset quantization. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 1220-117216 October 2023.
16

---

Appendix
A.1 Derivation of Equation
In order extend the QUBO formulation, which is the domain of application for AMFD, to general PUBO formulation, we prove that the KL divergence between the mean-field approximate distribution and the canonical distribution for the PUBO formulation is given by Eq.
Proof.
From the definition of KI. divergence:
Par (s)
DxL (PMr (8) || Pc Pc (a)) =
Σ PMr (s)
(s)
( II pa (es) ( (0) In (Z) ( (p (st) in ]
Σ
(In)))+n(22))
(S.1)
For the first term in Eq. S.I
Σ
I ps (8:) L(s)
Σ
Pj (ss)
Σ
1
Σ
S1z
+
Σ
( S1 + +....
81
11500
Σ
II jtil
Ps (8x)
Σ
sip (sts)
+ Σ
II Ps (58)
Σ
(2) Su 812Pr (812)
II (s,)
Σ
Pitiis Pr, (8 StaPe (812) BegDes (s6s) +
II Σ
(0,)
Σ
(
i1
Σ
Stpi (8t))
144 4y
$1 m
II Σ
Ds (6,)
Σ
(2)
Σ
St, (8n) Σ
staPrs (812)
3112
fff1.22 By
11
$12
ps (+s)
Σ
(3)
Σ
sipi (8i))
Σ
siglis (#12)
Σ
sig (sis)
Ciiizii
*ig
Mig
(2: 1122
((k
Tigz
kCj
L(x).
(S.2)
For the second term in Eq.
S.I
I (s.)) (8.) In(z)
N
|I Σ
Ps (81) In(2)
1m 0,00
( 1
(s)))
= I2((2)
(S.3)
17

---

For the third term in Eq. (S.1:
Σ
pi (s:)
Σ In (ps (si))
ps (55)
pi (s.)
Σ pc (5 (p) (p)
Ps (83)
17 5q
(: 5,=0
(EEN(()))) =1
Σ Pi (ps)
=
(.. Σ scp((ss) = =p(())
h 24) 21) x1)]
(S.4)
From Eq. (S.1 (S.4). we obtain the KL divergence expression in Eq. (4) Therefore, the claim is proven.
A.2
Uniform Quantization with Grid Search
Here, we provide detailed explanation of UQ algorithm introduced in Sec. As shown in Alg. S.1 the scaling factor determined via grid search so as to minimize the MSE between the original matrix and the quantized matrix. We optimized with Napli- 100 for all experiments.
Algorithm S.1 MSE-Aware Uniform Quantization with Grid Search
Input: Input matrix W, bit width Noir. the number of grid divisions Nsplit Output: Quantized matrix Q, scaling factor a, bias b, dequantized matrix W
1: L 4- 2.
The number of quantization levels Search range setting
2: a mean W
3: Wmin + min W)
4: Wmax ma)
5: Initialize = + 00, Q 5 None, (a,, 9) + None
6: for each Tmax E &linspace(u, Wmaxx N split do
7: for each Mmin E linspace Wmin: Nspli) do
8:
W. + clip(W, Tminn 7 mmax)
Clip W to [riin T Tmax
9:
Q. -
W.mimi (L (1--))
Quantize to 00 L - 1]
Tmax -imin
W.
Q.
(Fmax Ymin + Pmin
> Dequantize > Compute quantization error
1-1
11:
+
W
[( - W.)
12:
if e < then
8 +-
Save best quantization setting
14:
Q - Qu
15:
a + I=m-ii f- Tmin
16:
W, +- W.
return Q, a, b. W
A.3
Eg Lattice Vector Quantization Using 240 Centroids of Norm v2
Here, we describe the Fg LVQ algorithm introduced in Sec. In this work, we construct a codebook consisting of all 240 Eg lattice vectors with Euclidean norm v2 as centroids. For given 8- minimize the MSE, after
dimensional vector, the nearest centroid selected to approximate selecting the centroid based on cosine similarity, the optimal scalar scaling factor is computed. Each 8-dimensional vector is thus associated with single 8-bit index, which equivalent to approximately one bit of scalar quantization. By iteratively applying this procedure to the residual errors, the
18

---

quantization can be extended to multiple bits. However, maintaining separate scaling factor for each 8-dimensional vector would make the memory overhead non-negligible Therefore, in this work, the scaling factors are quantized using 2-bit uniform quantization, which was empirically found to achieve the best trade-off between memory efficiency and quantization error. The resulting residual-based multi-bit LVQ algorithm is summarized in Alg. S.2
Algorithm S.2 Es Lattice Vector Quantization with Residual Greedy Search
Input: Input matrix W RMMN, number of quantization bits Nbas scale bits Sbits
Output: Quantized and reconstructed matrix Wa q. code vector indices k", scaling factors or
1: Flatten W W Ita
2: Pad Wfat fat with zeros so that its length is a multiple of 8
3: Reshape W flr D R
4: Construct the Es codebook C C 2R200x8 consisting of all lattice vectors with norm v2
5: Initialize reconstruction Duxal f- 0
6: for bit 1 to Nuiss do
7: Normalize codebook and data: Cnorm = C/1 Dnorm = D/
8: Compute cosine similarity matrix:
D nomm CT
norm
9: Select closest code vector index for each row:
k arg max Sr.ks r= 1=1,.... k=
10:
Select corresponding codes: Ceeeectd + Clk*]
Compute scalar coefficients a E R" for each row of D: ar 2i==1 Dru Ssolected.
11:
Σ5=1 C2
Sellcced,
Quantize using uniform quantization with Sbits
13:
Reconstruct partial matrix: Dhat < a Cyelected
Update residual: D D Daat Accumulate reconstruction: Dusal +- D + Dhat
15:
16:
Reshape D to original shape of W: W < D
17:
return Wq. k", a
A.4 Inference Computational Cost Analysis of the BQQ Layer in DNN
Since p-bit quantization typically incurs approximately times the computational cost of 1-bit quantization, we analyze the computational cost based on the 1-bit case for both the conventional first-order quantization and the proposed BQQ layer. We focus where weights in ViTs are concentrated. Let the input be a veal-velued matrix X E Rad and the 1-bit quantized weight for the first-order baseline be W, E {0, 1)m,,,, For the BQQ method, the weights are represented as Y E {0, 1 mmxx and Z (0, 1) fixm, where / is the intermediate rank.
single linear layer, which is
Here, AND refers to bitwise operations between binary weights and real-valued inputs (not binary- binary operations), ADD denotes real-valued addition, and MUL represents real-valuce multiplication.
As described in the experimental setting in Sec. 5 we set = round m+t for the BQQ layer. Note
mn
that the cost of computing the zero-point bias omitted because it common term and does not change as the bit width increases.
First-Order 1-Bit Quantization
OUT = aW, x The matrix multiplication involves the following computational cost:
COSTroq mnd AND + md(n 1) ADD + MdL
BQQ 1*-Bit Quantization
The output is computed as:
OUTBgo Yr + sY1z + Z)
19

---

The computational steps are broken down as follows:
A ZX Ind AND + (n 1)ld ADD
B + 1gX: : (n 1)d ADD
C Y(rA + sB): MUL + ld ADD AND (1 1) ADD
- 1y. fA (1 - 1)d ADD + IdMUL
Out C+D D md ADD
The total computational cost becomes:
COSTBQQ ld(mm + n) AND dl(m + n 1)1 + +n-
m ADD MUL
Relative Cost Ratio and the BQQ method.
We compare the computational cost between the first-order 1-bit quantization The relative ratio is given by:
COSTBQQ COSTroQ
[d(m + 12) AND + d[(m
1) + n2 2] ADD + M/L
mnd AND md(n 1) ADD md MUL
d[(n + 1 2) ADD + (31 - (3) MUL]
where - mn
= =1 mnd AND md(nn 1) ADD mn MUL'
+
yielding a computational complexity ratio of O(1), since the first-order quantization has ((mnd) operations while the BQQ method has ((m ni)ld operations, and for x ann their ratio becomes of order one:
m+n"
COSTB0O
of (m + nildd
= (1), when / RJ
m7
COST
mnnd
+
Practical Examples For the DeiT model in Sec. 5 with
384, / 192:
574 ADD + (3 192 384) MUL
+
1.0052
3842 AND 384 383 ADD + 384 MUL
For the Swin model with m = 96, | 48:
14 142 ADD + (3 48 96) MUL
+
1.0207
962 AND
96 95 ADD + 96 MUL
These results demonstrate that the inference computational cost of the BQQ layer is nearly equivalent to that of the conventional first-order 1-bit quantization.
A.5 BQQ Execution Time for PTQ
We report the quantization time required by proposed method, BQQ, under data-free setting. All experiments were conducted using the following environment:
Python 3.9.19
PyToorch 2.6.0 with CUDA 12.4
Four NVIDIA GeForee RTX 4090 GPUs
AMD EPYC 7313 16-Core Processor
During quantization, we parallelized the process by assigning each matrix to a separate GPU thread, enabling concurrent quantization of multiple layers. Quantization time measured using Python's time module. We evaluated quantization time using a small model (DeiT-S, 22M parameters) and large model (DeiT-B, 86M parameters).
Tab. S.T summarizes the computation time for pseudo 2*t 3 and -itb quantization using BQQ. The reported times indicate the total time required to quantize the entire model. Note that the larger the pseudo bit width, the longer it takes because of greedy optimization for each pseudo bit index, as shown in Alg. Since processing is performed in parallel on each GPU, speedup is possible by increasing the number of GPUs, but computation time a barrier to scaling up the applicability of PTQ to large-scale models such large language models.
20

---

Table S.1: Execution time BQQ on DeïT-S and DeiT-B.
BQQ Time [min]
Model
#layers
#parameters
2* bit
3* bit
4 bit
DeiT-S
12
22M
13
17
21
DeïT-B
12
86M
32
45
57
A.6 Effect of Nstep on Accuracy and Computation Time
We investigate the effect of the number of optimization steps Nstep (and the total quantization time) on the final ImageNet top-1 accuracy and computation time under the data-free quantization setting. The results on DeiT-Small (DeiT-S) and DeiT-Base (DeiT-B) models are shown in Fig. S.I As expected, increasing stepp generally results in longer quantization time but also enables more precise optimization, which tends to improve final accuracy. In particular, it noteworthy that increasing Naep beyond 50,000 the setting used in the main manuscript manuseript-yyellss For instance, with DeiT-S, an accuracy of 60.24% achieved at pseudo 2*-bit precision, which is 1. 1.99% higher than the accuracy reported in the main manuscript. Similarly, with DeiT-B, an accuracy of 72.91 % is obtained at pseudo 2*-bit precision, representing a 0.82% improvement. As shown in the figure, in highly compressed settings such pseudo - quantization, large Nstep is essential to prevent significant accuracy degradation. In contrast, for pseudo 3*-bit or 4*-bit quantization, accuracy remains relatively stable even with smaller Nstep values.
more accurate results.
These observations suggest that more aggressive compression schemes are more sensitive to the quality of optimization, Therefore, more rigorous optimization is required in such cases. Notably, the time required for accuracy to reach saturation appears largely independent of the pseudo bit-width, with convergence observed in approximately 10 minutes for DeiT-S and 20 minutes for DeiT-B durations that are not prohibitive in practical applications.
small quantization errors have greater impact final accuracy.
DeiT-S
DeiT-B
Accuracy [%]
60.24%
70
60
60
72.91%
50
pseudo
pseudo 2 bit
pseudo
40
pseudo bit
pseudo
pseudo
Precision
Precision
20
10a
10a
10a
10a
N Nstep
N Sttep
80
80
Accuracy [%]]
70
50
pseudo 2 bit
pseudo 2 bit
40
pseudo 3" bit
40
pseudo 3" bit
30
pseudo 4" bit
30
pseudo 4" bit
Full Precision
Full Precision
20
10a Quantization Time [min]
10a
Quantization Time [min]
Figure S.I: Effect of the number of optimization steps Nstep (top) and total quantization time (bottom) on final ImageNet top-1 accuracy in the data-free quantization setting.
21

---

A.7 Accuracy Under Extreme Compression
We present additional results beyond those the main manuscript, focusing on further compression- specifically, setting the model size to pseudo 1.5 bit. In the main experiments, the factorization was performed using binary matrices Y E {0, 1] mxx and Z, E {0. 1)xx", where the intermediate Here, we increase the intermediate dimension to I dimension was set 1 round m++
(i.e., 1.5 times the original parameter count), and fix the number of stacks round (1.5- monn
(denoted as in Eq. b to 1. This results in an effectively 1.5-bit quantized model, noting that all weight elements remain binary (1-bit). Tab. [S.2 reports the top-1 accuracy on ImageNet after compression. While the accuracy significantly deteriorates in the data-free setting, it is remarkable that with only a small amount of calibration data, the accuracy remains reasonably high. Notably, for DeiT-B, the top-1 accuracy exceeds 70%, demonstrating that even under such severe compression, practical accuracy can be retained-an impressive outcome.
Table S.2: ImageNet top-1 accuracy of BQQ under extreme compression.
ImageNet Top-1 Accuracy [%]
Method
W bit
Data Free
DeîT-S
DeiT-B
Swin-T
Swin-S
c-BQQ
1.5*
53.67
71.36
61.25
69.89
BQQ
1.5*
7.41
35.21
10.82
20.06
A.8 Evaluation on Language Models
Here, we evaluate our quantization methods on several compact language models that are well-suited
for edge deployment: Qwen2.5-0.5B,, Qwen2.5-1.5B, and DeepSeek-R1. ---tek-----t-l----- I.B. We compare three approaches: (1) BQQ, which performs data-free quantization; (2) tuned-BQQ (t- (t-BQQ), which first applies BQQ and then fine-tunes only the continuous parameters; and (3) GPTQ [14, standard post-training quantization (PTQ) method. For both t-BQQ and GPTQ, the calibration data consist of the full training split of WikiText-2.
Tab. S.3 reports the perplexity on WikiText-2 488 and the accuracy on six downstream tasks: PIQA 4 Winogrande (WinoG) (57) ARC-Easy (ArcE) and ARC-Challenge (AreC) [9], HellaSwag (HellaS) [77,, and BoolQ 8. The table also includes the average accuracy across all tasks. BQQ achieves higher average accuracy than GPTQ on the 1.5B models, particularly under low-bit settings (2-bit and 3-bit). Notably, even without any calibration data, BQQ attains comparable or superior performance to GPTQ. In contrast, for the smaller 0.5B model, BQQ struggles to maintain accuracy. We attribute this the medee-ddeennent discrepancy between the quantization distributions that minimize activation error and those that minimize weight error. When this discrepancy becomes large, the performance of BQQ tends to degrade.
Although the performance of neural network quantization is not always guaranteed to be superior, we emphasize that the current work proposes BQQ as general binary quantization framework rather than dedicated quantization technique for neural networks. In our implementation, BQQ quantizes weights by minimizing reconstruction error in the weight space, without relying on output-based error signals that are commonly used in many neural network quantization methods. This implies that BQQ does not yet exploit task-specific loss functions or activation statistics. We therefore believe that adapting BQQ to incorporate such feedback-especillly minimizing the downstream output error-could lead to significant further improvements model performance, and this represents a highly promising avenue for future research.
A.9 Quantization Error vs. Binary Matrix Stack-to-intermeeiiate Dimension Ratio
In the main manuscript, we fixed the intermediate dimension as round
mn
for all experiments.
However, this setting is not necessarily optimal. The same compression ratio can also be achieved by varying the number of stacked binary matrices and the intermediate dimension. this subsection, we investigate how the quantization error (MSE) changes with respect to the number of stacks p and
22

---

Table S.3: WikiText-2 perplexity and downstream task aceuracy.
Model
Method
PPL
ArcE
AreC
BoolQ
HellaS
PIQA
WinoG
Avg.
BQQ
17106.9
28.3
20.3
37.8
26.3
54.0
50.5
36.2
tBQQ
39.7
39,9
GPTQ
4392.5
25.3
21.3
43.2
25.7
52.7
49.0
36.2
BQQ
1808.8
31.5
19.1
38.0
26.5
54,4
50.5
36.7
t-BQQ
19.4
45.2
27.2
43.0
39.7
62.1
56.1
45.6
Qwen2.5-0.58
GPTQ
21.8
45.8
21.0
56,5
34.0
63.6
56.0
46.1
BQQ
71.7
38.8
21.6
43.9
30.5
61.2
51.2
41.2
tBQQ
14.7
51.4
30.5
52.7
45.6
66.6
54.3
50.2
GPTQ
14.4
62.9
27.9
$7.4
38.9
68.2
56.2
51.9
baseline
53.4
BQQ
688.3
33.8
19.1
57.7
27.2
56.6
51.9
41.0
t-BQQ
23.6
45.1
28.5
62.0
40.2
61.2
53.8
48.5
GPTQ
922.4
26.0
45.0
26.0
51.6
37.0
BQQ
14.6
63.6
32.0
63.9
41.7
71.1
$7.9
55.0
BBQQ
11.8
61.8
38.3
60.6
57.0
72.0
59.0
58.1
Qwen2.5-1.5B
GPTQ
30.0
60.7
70.0
53.5
BQQ
10.5
72.4
38.5
69.1
47.7
74.8
60.7
60.5
t-BQQ
9.9
73.0
45.1
65.5
73.8
GPTQ
9.7
74.5
40.0
70.8
49.3
75.2
63.1
62.1
baseline
16
9.3
71.5
45.1
73.0
67.8
76.1
63.4
66.1
BQQ
921.2
29.8
47.9
26.5
53.6
49.3
37.6
t-BQQ
46.1
41.1
22.7
52.5
31,0
58.8
52.6
43.1
GPTQ
872.4
26.6
20.2
47.6
25.7
52.9
49.1
37.0
BQQ
59.2
56.1
30.6
$1.2
33.8
63.2
$2.9
48.0
Deepseek-RR Reae--R--Dii R1-Distill
tBQQQ
28.5
50.7
32.9
59.3
41.2
61.7
$5.7
50.3
Qwen-1.K
GPTQ
59.6
52.3
25.6
54.0
32.8
62.4
$1.9
46.5
BQQ
39.4
59.3
31.8
66.5
36.0
64.8
57.5
52.7
BBQQQ
21.7
54.1
31.9
65.2
43.9
63.8
56.0
52.5
GPTQ
43.4
32.2
66.9
36.0
65.9
52.6
baseline
16
40.4
56.1
34.6
68.6
44.8
65.8
55.6
54.2
the normalized intermediate dimension lucaes where the actual intermediate dimension defined as I round (ucate m2n We conducted experiments by sweeping over and lscale and measuring the resulting MSE.
Tab. S.4 summarizes the experimental results. These results show that the optimal balance between number of stacks and the intermediate dimension varies across datasets, indicating that fixing is not always optimal. Therefore, adaptively determining this ratio can potentially
I = round mn m++n lead to more efficient compression.
A.10 Theoretical Upper Bound of BQQ
In this subsection, we provide theoretical analysis of the approximation error inherent in the BQQ formulation. derive concrete upper bound, we consider a particular case Eq. (5) by setting 3 100=0055mm 6, = 1, and 7; -0.5. Although this specific case does not yield a closed-form optimal solution, an approximate one can be obtained by aligning the binary components with the sign patterns of the singular vectors obtained from SVD, combined with appropriate scaling. This leads theoretical upper bound on the square root of the subproblem error, denoted by
V L sub (Eq. ( The overall error of BQQ is then obtained by summing over all stack indices i:
BQQ total error Σ3 (8) Based the Eckart- Yountt-ouung---a Mirsky theorem and the triangle inequality,
23

---

Table S.4: Quantization (MSE) with varying the number of stacked binary matrices p and the
intermediate dimension scaling seale Note that the MSE values are scaled by a factor 103.
Random
SIFT
ImageNet
#stucks (p)
Luale
Size [KB]
MSE
Size [KB]
MSE
Size [KB]
MSE
Size [KB]
MSE
Size [KB]
MSE
2.1
324.3
18.4
298.1
1.3
14.6
2.1
97.8
6.3
-2.7
0.5
329.9
18.5
309.4
1.3
2.1
108.1
6.3
29.7
12.2
2
106.4
36.9
101.9
2.5
7.7
4.1
46.3
12.6
20.5
105.3
36.9
95.8
2.5
4.1
30.0
12.6
10.9
4
0.5
4.1
108.4
36.9
100.8
2.6
2.5
4,1
34.4
12.6
8,9
8
0.25
4.2
114.2
57.0
105.0 6
2.5
3.5
4,2
35.9
12.6
14.1
33.2
30.6
3.8
6.2
6.2
36.0
55.4
33.2
3.8
6.2
11.5
12
0.25
38.4
55.4
35.7
3.7
6.3
12.2
19.0
4.7
2
2
8.2
11.5
73.8
10.9
5.0
0.6
8.2
4.9
25.1
2.1
8.2
11.2
73.8
10.0
5.1
0,2
8.2
25.1
0.5
11.9
73.8
10.9
8.3
25.2
we obtain the following bound:
min
(3 sub
<
min
W (
[a. - 0.5 1y)
0.5 - 12)1
(S.5)
min
W
W
w
|Y, 0.5 1r) 0.5 12)
(S.6)
min
W 0
svd
[8)
w sxd
- (a. - 0.5 1r)( - - 12)
(S.7)
min(m,,
min
Σ
of
w
ax
sgn
U
sgn
vi(
(S.8)
J=41
min(m,,)))
Σ
of
W
W
sud
sgn
(t())
svd
sgn
v
svi
sgn
(r(+))
sgn (v((t)
(+)
V(
sgn
sgn
(S.9)
where o, and w denote the singular values and the reconstructed matrix after applying l-rank approximation to the SVD of W respectively. Furthermore, U(1) syd and vi( represent the left and right singular vectors obtained from the SVD of W where U already incorporates the top l singular values.
This result indicates that the approximation error of the BQQ formulation is closely related the magnitude the discarded singular values. Consequently. BQQ achieves higher representational fidelity for matrices with rapidly decaying singular spectra, while its approximation quality degrades when the low-rank truncation leaves substantial residual energy. This theoretical property can also be observed in the trends shown in Fig.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions of this work, including the proposed quantization framework, the associated optimization method, and its empirical performance. The claims are consistent with the theoretical and experimental results, and appropriately scoped reflect the evaluated settings and assumptions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations of this work are discussed in the subsection FFurther Potential and Limitations" in Section 6 We specifically address the suboptimality of the greedy optimization strategy, the lack of a theoretical upper bound on the approximation error, and the relatively high quantization time. Additionally, we consider possible directions for improving the method through broader adaptation and structural refinement, in line with the NeurIPS Code of Ethics on transparency and reprodueibility.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
25

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
This work has derived the KL divergence between the mean-field approxima- tion distribution and the canonical distribution in the PUBO cost function (Equation 4 to extend the previous AMFD algorithm from QUBO to PUBO, as shown in Appendix A.1
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided all necessary details for experimental reproducibility, in- cluding the optimization algorithms used and the full set of hyper-parameters.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
26

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The code is not currently publicly available. We plan to release it after publication for reproducing the main results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
cublic/guidess htte://ooouusissionnooooo oniioiioolly for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
7 //nips wttp:c/pcpppp...dd...oosppps htapl/ic/guides//ooeeeiee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided the experimental settings and details in Section 5
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
27

---

The paper reports results from single run with fixed random seed, following common practice in recent quantization studies. While error bars and statistical signifi- cance are not reported, the results were consistently observed across multiple settings and benchmarks. We leave more extensive statistical analysis future work.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The experiments were run on local CPU/GPU machines. Appendix A.5 includes hardware details and measured runtimes for several experiments, providing sufficient indication of the compute requirements for reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research presented in the paper fully conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
28

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: None of social impacts we feel must be specifically highlighted here.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeetm of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: No high-risk data or models are released in this work, so safeguards are not applicable.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-us should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
29

---

This work makes use of existing assets, including datasets, pretrained models, and quantization methods. All such assets are credited in the paper, and their licenses and terms of use are properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, wapprsai thcode. com /daaaset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This work introduces new code, which is not released publicly at this time. However, relevant implementation details are clearly described in the paper. Therefore, the question regarding documentation alongside released assets is not applicable.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
30

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects were involved in this work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: No LLMs used in the development of the proposed method, experiments, or analysis, and they had no influence on the contributions of this work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to our LLM policy //neur cc. for what should or should not be described.
31


Reviews:
[{"review_id": "r8tC6hhix3", "reviewer": "Reviewer_Qheg", "summary": "The paper proposes a matrix quantization method called Binary Quadratic Quantization (BQQ). BQQ approximates real-valued matrices as a sum of binary matrix products, which intuitively enables expressive nonlinear approximations while maintaining a compact data format. \n\nThe experimental results demonstrate that BQQ  achieves a better trade-off between compression rate and reconstruction accuracy compared to uniform quantization, binary coding quantization, and low-rank approximation using SVD.", "questions": "Can the author address the mentioned weaknesses? Specifically, when considering methods that require a lot of time for quantization (such as the proposed BQQ), how does the accuracy-compression tradeoff of BQQ compare to such methods (e.g., k-means, Hadamard transform + quantization, Lattice-based quantization)? What advantages does BQQ offer over these methods?", "limitations": "Yes, the paper discussed limitations (Sec. 6, \"Further Potential and Limitations\").", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros:\n\n- BQQ achieves a good trade-off between memory efficiency and reconstruction error; it seems to capture structural redundancy via the nonlinear approximations while maintaining a compact data format\n\n- BQQ outperforms several existing methods in matrix compression and PTQ tasks.\n\nCons:\n\n- BQQ requires a non-negligible amount of time and resources for the quantization process\n\n- BQQ seems not to have a theoretical upper bound on the approximation error.\n\n- BQQ uses a greedy optimization strategy with an unclear compromise\n\n- The choice of alternatives leaves more to be desired. Specifically, it's unclear how BQQ compares to other quantization methods that cannot be applied online (e.g., k-means, Hadamard transform + quantization, Lattice-based quantization)", "was_revised": true, "final_justification": "I have decided to increase my score from 4 to 5 based on the new experimental and theoretical results provided by the authors and their intent to include these in the paper.", "rebuttal": "---\n\n# Response to Reviewer Comments\n\nWe sincerely appreciate the positive feedback and constructive suggestions. Below, we address the main concerns raised in the review.\n\n---\n\n## (1) Comparison with Other Quantization Methods\n\nWe thank the reviewer for suggesting additional comparisons with a broader range of quantization methods. While our original experiments focused on scalar quantization baselines, we agree that extending the evaluation to a more diverse set of techniques can provide valuable insights.\n\nIn response, we conducted additional experiments that compare our method against the following approaches:\n\n* **Vector Quantization using k-means (VQ)**\n* **$E_8$ Lattice Vector Quantization (LVQ)**\n* **Hadamard Transform (HT) + Uniform Quantization (UQ)**\n* **Discrete Cosine Transform (DCT) + UQ**\n* **JPEG (DCT + quantization + entropy coding)**\n\nTo ensure fair comparisons, we clipped all benchmark matrices to the nearest lower power-of-two size. This process was necessary to enable the application of HT, which requires input dimensions to be powers of two. All methods were evaluated on these clipped matrices for consistency.\n\nFor VQ, we also evaluated a variant using quantized centroids (VQ+UQ) to account for codebook storage.\n\nIn LVQ, we used the $E_8$ lattice with 240 norm-$\\sqrt{2}$ centroids, representing each 8-dimensional input as a linear combination of $n$ centroids, each stored in 8 bits, making it comparable to $n$-bit scalar quantization in memory.\n\nFor the HT and DCT methods, we experimented with two levels of energy preservation by retaining only a portion of the low-frequency components after transformation. Specifically, we evaluated settings that preserve approximately 50% and 100% of the transformed components, denoted as **HT50**, **HT100**, **DCT50**, and **DCT100**, respectively. Uniform quantization (UQ) was then applied to the retained components for compression.\n\n---\n\nWe summarize the performance of different methods across 5 datasets. For readability, we show results as “Size \\[KB] / MSE $\\times 10^{3}$” pairs.\n\n\n| Method           | Random      | DeiT         | Distance    | SIFT        | ImageNet    |\n| ---------------- | ----------- | ------------ | ----------- | ----------- | ----------- |\n| **(Size ≈ 2bits scalar quantization)** |             |              |             |             |             |\n| BQQ              | 4.1 / 105.0 | 16.4 / 91.7  | 1.1 / 4.0   | 4.1 / 30.6  | 4.1 / 5.6   |\n| DCT100+UQ        | 4.1 / 117.9 | 16.4 / 120.2 | 1.0 / 193.2 | 4.1 / 359.4 | 4.1 / 318.2 |\n| DCT50+UQ         | 4.1 / 496.8 | 16.6 / 499.9 | 1.1 / 289.0 | 4.1 / 324.9 | 4.1 / 144.7 |\n| HT100+UQ         | 4.1 / 119.8 | 16.4 / 123.0 | 1.0 / 131.0 | 4.1 / 209.3 | 4.1 / 244.3 |\n| HT50+UQ          | 4.1 / 505.2 | 16.4 / 505.1 | 1.0 / 535.5 | 4.1 / 558.1 | 4.1 / 157.3 |\n| JPEG             | 4.1 / 428.6 | 16.3 / 360.8 | 1.0 / 378.9 | 4.1 / 147.4 | 4.1 / 1.6   |\n| LVQ              | 4.6 / 85.9  | 18.4 / 109.3 | 1.2 / 74.4  | 4.6 / 80.6  | 4.6 / 17.1  |\n| VQ               | 3.7 / 917.5 | 15.6 / 893.9 | 0.8 / 337.1 | 3.7 / 318.6 | 3.7 / 133.0 |\n| VQ+4bit UQ       | 4.2 / 448.2 | 16.5 / 415.4 | 1.1 / 16.8  | 4.2 / 53.2  | 4.2 / 7.6   |\n\n---\n| Method           | Random      | DeiT         | Distance    | SIFT        | ImageNet    |\n| ---------------- | ----------- | ------------ | ----------- | ----------- | ----------- |\n| **(Size ≈ 3-bit scalar quantization)** |             |              |             |             |             |\n| BQQ              | 6.2 / 33.9  | 24.6 / 29.9  | 1.6 / 1.3   | 6.2 / 10.0  | 6.2 / 1.8   |\n| DCT100+UQ        | 6.2 / 36.4  | 24.6 / 38.3  | 1.5 / 67.7  | 6.2 / 192.3 | 6.2 / 211.6 |\n| DCT50+UQ         | 6.2 / 491.7 | 24.9 / 494.5 | 1.6 / 276.9 | 6.2 / 269.0 | 6.2 / 49.5  |\n| HT100+UQ         | 6.2 / 37.2  | 24.6 / 39.6  | 1.5 / 43.0  | 6.2 / 72.9  | 6.2 / 142.6 |\n| HT50+UQ          | 6.2 / 499.7 | 24.6 / 499.6 | 1.5 / 527.3 | 6.2 / 546.9 | 6.2 / 123.9 |\n| JPEG             | 6.1 / 147.3 | 24.3 / 123.5 | 1.5 / 152.8 | 6.1 / 53.2  | 5.7 / 0.7   |\n| LVQ              | 6.9 / 25.3  | 27.7 / 40.5  | 1.7 / 21.5  | 6.9 / 23.1  | 6.9 / 5.3   |\n| VQ               | 5.8 / 876.7 | 23.8 / 847.2 | 1.3 / 195.3 | 5.8 / 282.2 | 5.8 / 83.8  |\n| VQ+4bit UQ       | 6.2 / 216.5 | 24.7 / 190.5 | 1.6 / 8.4   | 6.2 / 14.7  | 6.2 / 4.4   |\n\n---\n\nThe above results on the five benchmark matrices show that BQQ outperforms most other compared methods—including VQ, LVQ, and transformation-based schemes—on the majority of datasets. The only exception is JPEG on ImageNet, where BQQ performs slightly worse. These results demonstrate the strong empirical performance of BQQ and underscore its practical advantages over a diverse and competitive set of baselines.\n\nMethods such as the Hadamard Transform (HT) and Discrete Cosine Transform (DCT), when used with quantization, serve as preprocessing steps to decorrelate or concentrate signal energy. Their compression efficiency and accuracy depend on the subsequent quantizer. From a methodological perspective, these hybrid schemes are not standalone quantizers and thus not directly comparable to BQQ, which is itself a quantizer. Nevertheless, BQQ's ability to outperform these popular combinations is notable and underscores its effectiveness. Moreover, combining such transforms with BQQ may further improve compression, which we consider a promising direction.\n\nWe will incorporate a formal version of this table into the main paper if the paper is accepted.\n\n---\n## (2) Upper Bound of BQQ\n\nWe appreciate the reviewer's insightful comment regarding the lack of theoretical approximation bounds. In response, we analyzed a simplified version of Equation (5) by assuming $\\beta_i = -0.5\\alpha_i$, $\\delta_i = 1$, and $\\gamma_i = -0.5$. Although this setting does not yield a closed-form optimal solution, we found that an approximate solution can be constructed by aligning the binary components with the sign patterns of the singular vectors obtained from a truncated SVD, combined with appropriate scaling. This approximation can then be used to provide an upper bound on the error of BQQ.\nSpecifically, we found an upper bound on the residual error in the greedy error minimization process of BQQ, denoted by $L_{\\text{sub}}^{(i)}$, as given below.\nThis bound is derived based on the Eckart-Young-Mirsky theorem and the triangle inequality.\n\n---\n\n$$\n\\min \\sqrt{L_\\mathrm{sub}^{(i)}} \\le \\min \\left\\lVert \\mathbf{W}^{(i)} - \\mathbf{W}\\mathrm{svd}^{(i)} + \\mathbf{W}\\mathrm{svd}^{(i)}- \\left[ \\alpha_i(\\mathbf{Y}_i - 0.5 \\cdot \\mathbf{1}_Y)(\\mathbf{Z}_i - 0.5 \\cdot \\mathbf{1}_Z) \\right] \\right\\rVert  \\le A + \\left\\lVert \\mathbf{W}\\mathrm{svd}^{(i)} - c B \\right\\rVert\n$$\n\n$$\nA := \\sqrt{ \\sum_{j = l+1}^{\\min(m,n)} \\sigma_j^2 }, \\quad\nB := \\mathrm{sgn}({\\mathbf{U}}\\mathrm{svd}^{(i)})  \\mathrm{sgn}({\\mathbf{V}}\\mathrm{svd}^{(i)}), \\quad\nc := \\frac{ \\langle {\\mathbf{W}}\\mathrm{svd}^{(i)}, B \\rangle }{ \\| B \\|^2 }\n$$\n\n---\nwhere $\\sigma_j$ and $\\mathbf{W}\\mathrm{svd}^{(i)}$ denote the singular values and the reconstructed matrix after applying an $l$-rank approximation to the singular value decomposition (SVD) of $\\mathbf{W}^{(i)}$, respectively. Furthermore, $\\mathbf{U}\\mathrm{svd}^{(i)}$ and $\\mathbf{V}\\mathrm{svd}^{(i)}$ are the singular vectors obtained from the SVD of $\\mathbf{W}^{(i)}$. Note that $\\mathbf{U}\\mathrm{svd}^{(i)}$ is represented as a single matrix that already incorporates the top $l$ singular values.\n\nThis bound is connected to the magnitude of the discarded singular values, which explains the empirical observation (e.g., Figure 3 and Figure S.1) that BQQ outperforms better on matrices with rapidly decaying singular spectra, and underperforms when the low-rank approximation leaves significant residual energy.\n\n---\n## (3) Greedy Optimization and Its Computational Cost\n\nWe acknowledge the limitation of using a greedy strategy. However, solving polynomial unconstrained binary optimization (PUBO) problems to global optimality quickly becomes intractable as problem size grows. In BQQ, joint optimization would involve $(m \\times l + l \\times n) \\times p$ binary variables for $m \\times l$ and $l \\times n$ matrices and $p$ BQQ terms, leading to significant memory and time costs even for moderate-sized matrices.\n\nTo mitigate this, we adopt a greedy residual minimization strategy inspired by Binary Coding Quantization (BCQ), which incrementally reduces the approximation error by adding one component at a time. In contrast to global optimization methods that require re-running the entire procedure from scratch when the number of stacks ($p$ in Eq.~(5)) is modified, the greedy approach preserves previously optimized components. This enables efficient and flexible adjustment of model capacity without discarding past computations.\nNonetheless, as the reviewer rightly noted, applying this transformation to many matrices in neural networks incurs non-trivial computational costs. Reducing this overhead remains an important direction for future work.\n\nWhile we acknowledge that our current optimization strategy is relatively simple and could certainly be improved with more sophisticated or learned methods, we would like to emphasize that the core contribution of this work lies in the formulation of the BQQ framework itself. The novelty of BQQ as a binary matrix approximation scheme is independent of the specific optimization algorithm used, and we hope the reviewer understands that our primary goal is to establish the expressiveness and potential of the framework, rather than to claim advances in optimization methodology.\nWe will further clarify this point and add future directions regarding optimization improvements in the final manuscript.\n\n---\n\n## Final Remarks\nWe hope these responses adequately address the reviewer's concerns. We thank the reviewer again for the valuable feedback, which has helped strengthen the paper both theoretically and experimentally."}, {"review_id": "Z8w1LHXfTt", "reviewer": "Reviewer_Bvhm", "summary": "This paper introduces a new quantization method called Binary Quadratic Quantization (BQQ), designed mainly for ultra-low-bit quantization. BQQ represents real-valued matrices as linear combinations of binary matrix products, surpassing previous first-order methods, such as uniform and binary coding quantization. To get an accurate quantized binary matrix, this paper also leverages a combination of polynomial unconstrained binary optimization (PUBO) and convex quadratic programming to address the optimization problem. The authors demonstrate BQQ’s effectiveness through matrix compression benchmarks and post-training quantization (PTQ) of Vision Transformer (ViT) models, showing that BQQ consistently achieves a superior trade-off between memory efficiency and reconstruction error. On ImageNet, BQQ outperforms state-of-the-art PTQ methods by up to 2.2% and 59.1% with 2-bit equivalent quantization.", "questions": "- In Table S.1, why does higher-bit quantization take longer? Did they use the same number of optimization iterations? For lower-bit quantization, I think more iterations are needed for optimization. Is my understanding correct?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths\n\t- This paper is well-written and easy to follow.\n\t- The proposed method achieves the SOTA low-bit quantization results in terms of accuracy and compression ratio.\n- Weakness\n\t- The paper shows FLOPs reduction and weight bit counts, but doesn’t provide real speedup data.\n\t- The experiments only evaluated the proposed method with end-to-end models on image classification benchmark, i.e. ImageNet. Since it’s applicable to various models and tasks, it would be better to evaluate on LLMs as well.\n\t- It lacks analysis on the parameters used in BQQ, including the low rank dimension $l$ and the pseudo bit width $p$. Adding more analysis on these under the same compute/memory constraints would help readers understand BQQ better.", "was_revised": true, "final_justification": "BBQ can also work well for LLMs, especially for the extremely low-bit quantization. So I'm going to increase my rating.", "rebuttal": "# Response to Reviewer Comments\n\nWe thank the reviewer for the constructive and thoughtful comments. We are encouraged that the reviewer found our paper well-written and recognized the strength of our method in achieving state-of-the-art low-bit quantization results in both accuracy and compression.\n\nBelow, we address each of the concerns one by one:\n\n---\n\n## (1) Lack of Real Speedup Measurements\n\n\nWe agree that measuring actual runtime performance is important for practical deployment. Our primary contribution, however, lies in proposing the BQQ framework and demonstrating its theoretical and empirical benefits in terms of memory efficiency and approximation quality. While we provide FLOPs and bit-level analyses as proxies for computational cost, real-world speedup on GPUs depends on hardware specifics and implementation details beyond FLOPs alone.\n\nCurrent GPU architectures include optimized cores for integer-only (INT) operations, enabling efficient low-bitwidth computations when both operands are integers. However, because mixed-precision operations involving floating-point (FP) and integer (INT) data—such as those required when 1-bit weights are cast to floating-point formats before processing—lack dedicated optimized cores, 1-bit weights must be converted to FP for computation. As a result, the actual amount of computation performed on the GPU remains effectively the same as in unquantized models, despite reductions in memory access and data transfer. Consequently, despite reduced memory bandwidth and data transfer, our GPU implementation did not show latency improvements.\n\nRecent research, for example LUT Tensor Core (Zhiwen Mo et al., ISCA 2025), shows that specialized hardware supporting optimized tensor operations between 1-bit and 16-bit data types can achieve over 7.5× speedups compared to standard GPU execution. Such innovations suggest that substantial acceleration is possible with appropriate hardware support for mixed-precision computations.\n\nAt present, we have not yet developed a hardware-optimized implementation tailored to BQQ and thus cannot provide actual runtime acceleration results. We sincerely wish the reviewer to value our theoretical contribution on memory and computation cost efficiencies. Implementing and evaluating practical runtime improvements on optimized hardware is an important direction for future work.\n\n\n---\n\n## (2) Evaluation Limited to Vision Transformer\n\nWe thank the reviewer for pointing out that the experiments were conducted only on end-to-end image classification models (i.e., ImageNet) and for suggesting evaluation on large language models (LLMs) to strengthen the generality of our method. We agree that **BQQ is broadly applicable to a wide range of architectures and tasks**—including not only various neural networks but also non-neural applications such as **Approximate Nearest Neighbor (ANN) search**, where binary representations are highly beneficial.\n\nIn this work, we focused on **Vision Transformers (ViTs)**, which are known to be particularly fragile to quantization—especially under low-bit settings where small perturbations in weights can lead to significant degradation. Demonstrating strong performance on ViTs thus highlights the **robustness of BQQ** in one of the most challenging quantization scenarios. While applying BQQ to LLMs is a promising direction, the sheer size of these models makes our current optimization pipeline computationally expensive, and **full-scale evaluation remains an ongoing effort**.\n\nThat said, to provide the reviewer with more concrete evidence of BQQ’s broader applicability, we additionally applied our method to **compact LLMs suitable for edge devices**.\nSpecifically, we evaluate our method on **Qwen2.5-0.5B**, an efficient and high-performing LLM designed for edge deployment. Compared with **GPTQ**, which is a standard quantization method, **BQQ demonstrates competitive performance** in terms of **perplexity and 0-shot downstream task accuracy** under the same bit-width, as summarized in the table below.\n\n\n| Method   | #Bits | Wikitext2 PPL | PIQA | ARC-E | ARC-C | HS   | WGe  | BoolQ | Avg. |\n|---|---|----|---|---|---|---|---|---|---|\n| GPTQ     | 3     | 26.2 | **62.2** | 48.9  | **25.4** | **35.3** | **55.9** | **51.7** | **46.6** |\n| c-BQQ      | 3     | **20.5** | **62.2** | **51.4** | 23.1  | 33.6 | 52.1 | 47.1 | 44.9 |\n| GPTQ     | 2     | 12523.6  | 52.7 | 25.3  | **21.4** | 25.6 | 51.9 | 43.6 | 36.8 |\n| c-BQQ      | 2     | **51.0** | **57.5** | **39.8** | 18.0  | **28.3** | **52.6** | **58.8** | **42.5** |\n| Baseline | 16    | 13.0| 70.2 | 64.6  | 29.5  | 40.6 | 56.4 | 62.4 | 54.0 |\n\n\nHowever, we emphasize that the current work proposes BQQ as a general binary quantization framework, rather than a dedicated quantization technique for neural networks. In our implementation, BQQ quantizes weights by minimizing reconstruction error in the weight space, without relying on output-based error signals that are commonly used in many neural network quantization methods. This implies that BQQ does not yet exploit task-specific loss functions or activation statistics. We therefore believe that adapting BQQ to incorporate such feedback—especially minimizing the downstream output error—could lead to significant further improvements in model performance, and this represents a highly promising avenue for future research.\n\n---\n\n\n## (3) Lack of Analysis on Intermediate Dimension and the Number of Stacks (\\$l\\$ and \\$p\\$ in Eq.(5))\n\nWe appreciate the reviewer’s insightful comment regarding the need for a more detailed analysis of BQQ’s internal parameters, such as the intermediate dimension $l$ and the number of binary matrix stacks $p$. In our main experiments, we fixed the intermediate dimension as:\n$\nl = \\mathrm{round}\\left(\\frac{mn}{m + n}\\right)\n$\nto maintain the same number of elements as the original $m \\times n$ matrix, and adjusted the model size by varying the number of stacked binary matrices $p$. However, we agree that varying $l$ under fixed model size constraints can provide a deeper understanding of the trade-off between compression and reconstruction accuracy.\n\nTo address this point, we conducted additional experiments where we **fixed the total model size** and **varied the ratio between $l$ and $p$**. We used the same set of matrix datasets as in Fig. 3. The results are summarized in the following table, where the “$l$-scale” denotes the relative value of $l$ normalized by:\n$\n\\mathrm{round}\\left(\\frac{mn}{m + n}\\right)\n$\n| #stacks (\\$p\\$) | \\$l\\$-scale | Size \\[KB] (Random) | MSE (Random) | Size \\[KB] (DeiT) | MSE (DeiT) | Size \\[KB] (Distance) | MSE (Distance) | Size \\[KB] (SIFT) | MSE (SIFT) | Size \\[KB] (ImageNet) | MSE (ImageNet) |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | ---- | ---- | --- |\n| 1 | 1   | 2.1| 324.3| 18.4  | 298.1 | 1.3  | 14.6  | 2.1 | 97.8   | 6.3| 42.7 |\n| 2  | 0.5  | 2.1 | 329.9| 18.5  | 309.4| 1.3 | 9.2| 2.1 | 108.1| 6.3  | 29.7  |\n| 4  | 0.25| 2.1| 337.1 | 18.5| 317.3 | 1.3| 12.2 | 2.1 | 110.8 | 6.3 | 49.6 |\n\n| #stacks (\\$p\\$) | \\$l\\$-scale | Size \\[KB] (Random) | MSE (Random) | Size \\[KB] (DeiT) | MSE (DeiT) | Size \\[KB] (Distance) | MSE (Distance) | Size \\[KB] (SIFT) | MSE (SIFT) | Size \\[KB] (ImageNet) | MSE (ImageNet) |\n| ---- | ---- | ----- | ---- | --- | ---- | ---- | ---- | --- | ---- | ---- | ---- |\n| 1 | 2  | 4.1 | 106.4 | 36.9 | 101.9 | 2.5  | 7.7  | 4.1   | 46.3 | 12.6| 20.5 |\n| 2 | 1  | 4.1| 105.3  | 36.9  | 95.8 | 2.5  | 2.3  | 4.1 | 30.0  | 12.6 | 10.9  |\n| 4  | 0.5| 4.1 | 108.4| 36.9 | 100.8 | 2.6| 2.5| 4.1 | 34.4 | 12.6 | 8.9|\n| 8 | 0.25 | 4.2 | 114.2 | 37.0| 105.6| 2.5 | 3.5 | 4.2| 35.9 | 12.6  | 14.1 |\n\nOur findings show that the **optimal value of $l$** can depend on the **spectral characteristics of the data**. For instance, in datasets with **rapidly decaying singular values** such as *Distance* and *ImageNet* (see Fig. S.1), a smaller $l$ (e.g., $l = 0.5$) yielded better performance at small model sizes. On the other hand, in other cases, using $l = 1$ produced consistently strong results.\nThese results suggest that while our default choice of:\n$\nl = \\mathrm{round}\\left(\\frac{mn}{m + n}\\right)\n$\nworks well in general, **adapting $l$ to the specific data characteristics** could lead to **improved trade-offs between reconstruction error and memory usage**.\n\nWe thank the reviewer again for prompting this important analysis.\n\n---\n\n## (4) Runtime and Optimization Iterations\nThank you for your insightful question. The reason why higher-bit quantization takes longer is due to our **greedy optimization approach** applied to each binary matrix component. In our main experiments, the intermediate dimension is fixed as:\n$l = \\mathrm{round}\\left(\\frac{mn}{m+n}\\right)$ and the model size is controlled by the number of stacked binary matrices $p$.\n\n---\n\nImportantly:\n* When $p = 1$, only one binary matrix optimization subproblem is solved.\n* When $p = 2$, two such subproblems are solved **sequentially**.\n* Therefore, increasing $p$ **directly increases the number of optimization steps**, proportional to the number of binary matrices.\nAs a result, **higher-bit quantization** leads to **longer computation times**.\n\n---\n\nRegarding the runtime results presented in the appendix:\n\n* All experiments were conducted with a **fixed 50,000 optimization iterations**.\n* Your understanding is correct that **lower-bit quantization tends to be more sensitive**, and generally benefits from **a larger number of iterations**.\n\nAs shown in the supplementary material:\n\n* For **2-bit quantization**, more iterations are required to reach satisfactory accuracy.\n* For **3-bit and 4-bit quantizations**, accuracy plateaus well **before** 50,000 iterations, indicating that **fewer iterations suffice** to maintain stable performance.\n---\n## Final Remarks\n\n We sincerely appreciate the reviewer’s thoughtful feedback and hope the final version of our work will address these concerns more fully."}, {"review_id": "qdUcOoCNqA", "reviewer": "Reviewer_XJUw", "summary": "This paper proposes a novel method for quantizing a matrix to represent it using a small number of bits. The proposed method is named Binary Quadratic Quantization (BQQ). BQQ generalizes an existing approach known as Binary Coding Quantization (BCQ), which approximates a matrix as a weighted sum of binary matrices. In BQQ, the matrix is approximated by a sum of \"products of two binary matrices.\" Conceptually, while BCQ represents a matrix using a linear expression like `aX + b`, BQQ approximates it with a quadratic expression such as `(aX + b)(cY + d)`.\nTo perform this approximation in practice, the paper introduces a framework consisting of: (1) a greedy approximation that iteratively reduces the residual error of the original matrix, (2) a formulation that becomes a PUBO problem when real-valued variables are fixed, and (3) an analytic closed-form solution when the binary variables are fixed.\nThe proposed method is applied to both matrix approximation tasks and transformer layer approximation tasks, and it has been reported to outperform existing methods.", "questions": "The proposed method reduces the approximation error greedily. That is, in the loop of Line 2 in Algorithm 3, I would expect that $r_i$ and $s_i$ take larger values when $i$ is small, and gradually become smaller as $i$ increases. Does the proposed method actually exhibit this behavior?\n\nIn the context of SVD, the Eckart-Young theorem guarantees that selecting the most significant singular values in descending order yields the best low-rank approximation. In contrast, no such intuitive theorem generally holds for tensor decompositions such as the CP decomposition.\n\nHow does this compare to the proposed method? Specifically, when processing in order of $i = 0, 1, \\dots$ and terminating at some point, can we say --- empirically or theoretically --- that the resulting approximation is good?", "limitations": "None", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\nThe strengths of the proposed method are summarized as follows. Overall, I find the proposed approach interesting and well-motivated, and I have a favorable impression. Therefore, I assign a borderline accept score to this paper.\n\n## Straightforward Extension\nThe proposed method is a straightforward extension of the existing BCQ approach. While BCQ approximates a matrix using a linear form (`aX + b`), the proposed method expresses the matrix using a quadratic form (`(aX + b)(cY + d)`). This mathematical extension is elegant. It is evident that a quadratic form has greater expressive power than a linear one, and the effectiveness of this formulation is intuitively convincing.\n\n## Well-Structured Optimization\nThe optimization strategy described in Section 4.2 is well-structured: a greedy approach that iteratively reduces the objective function and performs optimization under two configurations: (1) fixing the real-valued variables and (2) fixing the binary variables. This approach is clear, and it makes sense that the BQQ outperforms BCQ if the optimization succeeds. The experimental results verify this point and report BQQ's superior performance over BCQ.\n\n## Extensive Experiments\nThe proposed method is thoroughly evaluated in the matrix approximation task and the Transformer layer approximation task. It demonstrates strong approximation performance across various settings, and I am satisfied with these results.\n\n\n# Weaknesses\n\nThere are no particularly strong weaknesses to point out, but I offer the following comments as suggestions to improve the manuscript further.\n\n## On Mathematical Expressions\n\nThere are two suggestions for improving the mathematical notation.\n\nIn Line 3 of Algorithm 2, the expression $\\mathbf{\\hat{Y}}_{\\mathrm{old}} - 0.5$ appears. Here, the scalar value \"0.5\" is used as if it were a matrix. A more precise notation would be to typeset it in boldface as $\\mathbf{0.5}$ (as done in Line 3 of Algorithm 1), or to represent it using an all-one matrix as $0.5 \\cdot \\mathbf{1}$. Similar expressions also appear in other places, so it is recommended that all such instances be revised consistently.\n\nIn Line 1 of Algorithm 3, the notation $\\mathbf{W}_\\mathrm{res} \\gets \\mathrm{clone}(\\mathbf{W})$ is used. Personally, I find the term `clone` to be overly Python-specific and possibly unnecessary. The symbol \"$\\gets$\" typically implies a deep copy unless otherwise stated in pseudocode. If `clone` is explicitly used here, then similar clarity should be maintained elsewhere. For instance, Line 14 of Algorithm S.1 might also need to state $\\mathbf{Q} \\gets \\mathrm{clone}(\\mathbf{Q}_c)$.\n\n## On Computational cost\n\nThe computational cost of the proposed method is not fully discussed in the main manuscript. Specifically, the discussion section refers to Appendix A.5. However, computational cost is an important factor in matrix quantization and decomposition. Therefore, it is recommended that a more explicit summary of the computational results be included directly in the main manuscript.", "was_revised": true, "final_justification": "I have a positive impression of this paper, which remains strong even after the rebuttal. Therefore, I will maintain my original BA score.", "rebuttal": "---\n\n# Response to Reviewer Comments\n\nWe thank the reviewer for the insightful and constructive feedback. We are encouraged by the overall positive assessment and appreciate the valuable suggestions. Below, we address each comment in turn.\n\n---\n\n## (1) On Mathematical Expressions\n\nWe appreciate the reviewer's careful reading of our algorithms and the helpful suggestions on improving notational consistency. We agree that the expression `0.5` in Algorithm 2, Line 3, should be more precisely written as a matrix, e.g., in boldface or an all-one matrix scaled by 0.5, as was done in Algorithm 1. If the paper is accepted, we will revise all such instances to improve clarity and consistency.\n\nRegarding the use of `clone`, we understand the concern that it is Python-specific. If the paper is accepted, we will revise the pseudocode to either eliminate such implementation-specific terminology or annotate operations like deep copying consistently throughout, including Line 14 in Algorithm S.1.\n\n---\n\n## (2) On Inference Computational Cost\n\nThe reviewer is absolutely right that the computational cost of DNN inference is an important consideration in quantization methods. While this point is currently discussed in Appendix A.5, we agree that it deserves to be presented directly in the main text. If the paper is accepted, we will incorporate a concise summary of the inference-time computational cost analysis into the main body to improve clarity and accessibility.\n\n---\n\n## (3) On the Behavior of the Greedy Strategy\n\nThe reviewer correctly observes that in Algorithm 3, the scaling parameters $r_i$, $s_i$, and $t_i$ tend to have smaller magnitudes as the index $i$ increases. This is indeed the case. Since our method reduces the residual error in a greedy manner, later steps operate on smaller residuals, which naturally results in terms with smaller absolute values.\n\nTo further illustrate this behavior, we include an additional experimental result showing how the values of \\$r\\_i\\$, \\$s\\_i\\$, and \\$t\\_i\\$ decrease progressively during the greedy process. The table below summarizes this trend:\n\n| \\$i\\$ | \\$s\\_i\\$ | \\$t\\_i\\$ | \\$u\\_i\\$ |\n| ----: | -------: | -------: | -------: |\n|     0 |     0.61 |    -0.27 |    -0.26 |\n|     1 |     0.35 |    -0.19 |    -0.19 |\n|     2 |    -0.20 |     0.09 |     0.09 |\n|     3 |    -0.12 |     0.06 |     0.06 |\n|     4 |    -0.07 |     0.03 |     0.03 |\n|     5 |     0.04 |    -0.02 |    -0.02 |\n\n---\n\n## (4) On the Analogy with SVD and the Eckart–Young Theorem\n\nWe appreciate the reviewer's thoughtful question regarding the relationship between our greedy approach and the Eckart–Young theorem. As correctly observed, our greedy optimization procedure—where scaling coefficients are determined in a decreasing order of significance—bears a conceptual resemblance to the construction of low-rank approximations in SVD.\n\nHowever, unlike SVD, our method does not guarantee optimality in a theoretical sense. Nevertheless, we empirically observe that it achieves a favorable trade-off between approximation error and bit-width compared to existing methods. In this sense, while heuristic, our method resonates with the spirit of the Eckart–Young theorem, suggesting that greedy, non-optimal procedures can still yield near-optimal approximations under certain conditions.\n\nIn addition, although this slightly departs from the question of greedy optimality, we have found that an upper bound on the approximation error for each subproblem in our greedy process can be derived using the Eckart–Young–Mirsky theorem and the triangle inequality:\n\n$$\n\\min \\sqrt{L_\\mathrm{sub}^{(i)}} \\le \\min \\left\\lVert \\mathbf{W}^{(i)} - \\mathbf{W}\\mathrm{svd}^{(i)} + \\mathbf{W}\\mathrm{svd}^{(i)}- \\left[ \\alpha_i(\\mathbf{Y}_i - 0.5 \\cdot \\mathbf{1}_Y)(\\mathbf{Z}_i - 0.5 \\cdot \\mathbf{1}_Z) \\right] \\right\\rVert  \\le A + \\left\\lVert \\mathbf{W}\\mathrm{svd}^{(i)} - c B \\right\\rVert\n$$\n\n$$\nA := \\sqrt{ \\sum_{j = l+1}^{\\min(m,n)} \\sigma_j^2 }, \\quad\nB := \\mathrm{sgn}({\\mathbf{U}}\\mathrm{svd}^{(i)})  \\mathrm{sgn}({\\mathbf{V}}\\mathrm{svd}^{(i)}), \\quad\nc := \\frac{ \\langle {\\mathbf{W}}\\mathrm{svd}^{(i)}, B \\rangle }{ \\| B \\|^2 }\n$$\nwhere $\\sigma_j$ and $\\mathbf{W}\\mathrm{svd}^{(i)}$ denote the singular values and the reconstructed matrix after applying an $l$-rank approximation to the singular value decomposition (SVD) of $\\mathbf{W}^{(i)}$, respectively. Furthermore, $\\mathbf{U}\\mathrm{svd}^{(i)}$ and $\\mathbf{V}\\mathrm{svd}^{(i)}$ are the singular vectors obtained from the SVD of $\\mathbf{W}^{(i)}$. Note that $\\mathbf{U}\\mathrm{svd}^{(i)}$ is represented as a single matrix that already incorporates the top $l$ singular values.\n\nThis bound highlights a theoretical connection between our method and classical SVD-based low-rank approximation, offering additional justification for the effectiveness of our greedy strategy.\n\nIf the paper is accepted, we plan to include this theoretical insight in the revised version, along with a more detailed discussion of its implications for total approximation quality.\n\n---\n\n## Final Remarks\n\nOnce again, we sincerely thank the reviewer for the thoughtful questions, constructive suggestions, and positive evaluation. The feedback has helped us better articulate the contributions and implications of our work, and we believe it will significantly enhance the clarity and impact of the final version, should the paper be accepted.\n\n---"}, {"review_id": "SW1BpTpPjE", "reviewer": "Reviewer_UDji", "summary": "This paper presents Binary Quadratic Quantization (BQQ), a novel matrix quantization framework that compresses real-valued matrices using quadratic combinations of binary matrices. BQQ extends the established Binary Coding Quantization (BCQ) approach—which relies solely on linear combinations—by introducing binary matrix products, enabling more expressive nonlinear approximations while preserving a highly compact representation. The authors demonstrate compelling accuracy–compression trade-offs, particularly on Vision Transformer models.", "questions": "1.  Missing latency analysis: The paper does not include a discussion on how the proposed quantization impacts inference latency, which is a critical aspect of practical deployment. The quantization scheme likely affects latency in at least three ways:\n(a) Weight loading time — using binary representations significantly reduces the model size, alleviating the weight loading bottleneck, look for “Memory Bandwidth and Compute Bottlenecks in LLM Inference.”\n(b) Low-bit computation efficiency — if supported by hardware, binary or low-bit operations can execute substantially faster.\n(c) Theoretical computational complexity — it would be useful to discuss the effect on the algorithmic complexity (in Big-O notation), especially compared to full-precision operations. (this is also in relation to comment 2.)\n\n2. Hyperparameter selection for intermediate dimension $l$: The dimension $l$, which governs the size of intermediate binary matrices, has a significant effect on both model accuracy and computational cost. While the paper adopts a heuristic choice $l=round(m n/(m+n)) for fair comparison, it is unclear whether this is optimal. A more detailed explanation or sensitivity analysis around this hyperparameter would strengthen the paper. It seems the authors are familiar with this issue and have noted on it within the discussion. I highly advise to add a short section to address this.\n\n3. Relation to LoRA and related work: The proposed method shares conceptual similarities with LoRA (Low-Rank Adaptation), which also introduces low-rank structure to improve representation, enable efficient fine-tuning, and reduce memory and runtime costs. It would be beneficial to explicitly discuss this connection in the main text and cite recent works that combine LoRA with quantization, such as QA-LoRA or LQ-LoRA, to position the current approach within the broader landscape of model compression and adaptation techniques.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\nA. The paper builds upon the Binary Coding Quantization framework by introducing a method that compresses model weights using quadratic combinations of binary matrices, extending beyond traditional linear formulations. \nB. The mathematical formulation is clear, concise, and well-explained, making the core ideas easy to follow. \nC. The experimental results are compelling, demonstrating that the proposed quantization approach maintains competitive accuracy compared to existing quantization techniques, while also refering to the vanilla model\n\nWeakness\nA The experimental evaluation is relatively limited, focusing only on a single task and a single model family (Vision Transformers). To strengthen the empirical claims and support the generality of the method, it would be valuable to include additional results—ideally in the appendix—across a broader range of tasks or architectures.\nB. The paper does not address the effect of the intermediate size $l$, a new hyperparmeter, on the accuracy, compression and computation complexity", "was_revised": true, "final_justification": "I thank the authors for their detailed reply and addressing some of my concerns\n\nMy score stays on 4\n\nThe rebuttal addressed some of my concerns -- namely, the additional evaluation and other issues related to clarity (for which the paper already got the highest score)\nNevertheless, my main concern regarding latency analysis is not addressed.\nMy score shows it is a good paper but not an outstanding one in its originality or significance.\n\nFinally, looking at concerns and scores provided by my peer reviewers strengthen my decision (my score is an avreage of the others)", "rebuttal": "---\n\n# Response to Reviewer Comments\n\nWe thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of the clarity of our formulation, the novelty of our proposed quantization framework, and the strength of our experimental results. Below, we respond to each of the concerns raised.\n\n---\n\n## (1) On the Scope of Experimental Evaluation\n\nWe agree that broader evaluation would strengthen our claims. We focused on ViTs due to their known sensitivity to quantization, where even small weight perturbations can cause large accuracy drops. Demonstrating strong performance in such settings highlights our method’s advantage.\n\nTo further demonstrate the broader applicability of BQQ, we evaluated our method on a compact large language model (LLM) optimized for edge deployment: **Qwen2.5-0.5B**. This model has recently gained attention for its strong performance despite its small size. We compared BQQ against a standard quantization baseline, **GPTQ**, under equal bit-width settings. As shown in the table below, BQQ achieves comparable results in both perplexity and downstream task accuracy, highlighting its effectiveness even in highly resource-constrained environments.\nNotably, in the extremely low-bit regime (2-bit quantization), BQQ demonstrates a remarkable advantage—achieving a 5.7% average improvement in downstream accuracy and reducing perplexity by a factor of 200 compared to GPTQ. This underscores BQQ’s robustness and superior performance in challenging low-bit scenarios. If accepted, we plan to include a broader range of experimental results to further showcase the versatility and effectiveness of BQQ across diverse model architectures and tasks.\n\n| Method   | #Bits | Wikitext2 PPL | PIQA | ARC-E | ARC-C | HS   | WGe  | BoolQ | Avg. |\n|---|---|----|---|---|---|---|---|---|---|\n| GPTQ     | 3     | 26.2 | **62.2** | 48.9  | **25.4** | **35.3** | **55.9** | **51.7** | **46.6** |\n| BQQ      | 3     | **20.5** | **62.2** | **51.4** | 23.1  | 33.6 | 52.1 | 47.1 | 44.9 |\n| GPTQ     | 2     | 12523.6  | 52.7 | 25.3  | **21.4** | 25.6 | 51.9 | 43.6 | 36.8 |\n| BQQ      | 2     | **51.0** | **57.5** | **39.8** | 18.0  | **28.3** | **52.6** | **58.8** | **42.5** |\n| Baseline | 16    | 13.0| 70.2 | 64.6  | 29.5  | 40.6 | 56.4 | 62.4 | 54.0 |\n\nAdditionally, we emphasize that the current work proposes BQQ as a general binary quantization framework, rather than a dedicated quantization technique for neural networks. In our implementation, BQQ quantizes weights by minimizing reconstruction error in the weight space, without relying on output-based error signals that are commonly used in many neural network quantization methods. This implies that BQQ does not yet exploit task-specific loss functions or activation statistics. We therefore believe that adapting BQQ to incorporate such feedback—especially minimizing the downstream output error—could lead to significant further improvements in model performance, and this represents a highly promising avenue for future research.\n\n---\n\n## (2) On the Effect of the Intermediate Dimension\n\nThank you for suggesting deeper analysis of the intermediate dimension $l$, which affects both accuracy and computation. In our main experiments, we used the heuristic\n$l = \\mathrm{round}\\left(\\frac{mn}{m + n}\\right)$\nto match the original matrix size and adjust overall capacity via the number of binary stacks $p$.\n\nTo explore this further, we conducted additional experiments varying $l$ and $p$ while keeping total bit budget fixed. Results on the same benchmark matrices from Fig. 3 show that optimal $l$ depends on data characteristics. For instance, datasets with fast-decaying singular values (e.g., \"Distance\", \"ImageNet\") benefit from smaller $l$ in low-capacity regimes. In others, our default setting remained robust.\n\n| #stacks (\\$p\\$) | \\$l\\$-scale | Size \\[KB] (Random) | MSE (Random) | Size \\[KB] (DeiT) | MSE (DeiT) | Size \\[KB] (Distance) | MSE (Distance) | Size \\[KB] (SIFT) | MSE (SIFT) | Size \\[KB] (ImageNet) | MSE (ImageNet) |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | ---- | ---- | --- |\n| 1 | 1   | 2.1| 324.3| 18.4  | 298.1 | 1.3  | 14.6  | 2.1 | 97.8   | 6.3| 42.7 |\n| 2  | 0.5  | 2.1 | 329.9| 18.5  | 309.4| 1.3 | 9.2| 2.1 | 108.1| 6.3  | 29.7  |\n| 4  | 0.25| 2.1| 337.1 | 18.5| 317.3 | 1.3| 12.2 | 2.1 | 110.8 | 6.3 | 49.6 |\n\n| #stacks (\\$p\\$) | \\$l\\$-scale | Size \\[KB] (Random) | MSE (Random) | Size \\[KB] (DeiT) | MSE (DeiT) | Size \\[KB] (Distance) | MSE (Distance) | Size \\[KB] (SIFT) | MSE (SIFT) | Size \\[KB] (ImageNet) | MSE (ImageNet) |\n| ---- | ---- | ----- | ---- | --- | ---- | ---- | ---- | --- | ---- | ---- | ---- |\n| 1 | 2  | 4.1 | 106.4 | 36.9 | 101.9 | 2.5  | 7.7  | 4.1   | 46.3 | 12.6| 20.5 |\n| 2 | 1  | 4.1| 105.3  | 36.9  | 95.8 | 2.5  | 2.3  | 4.1 | 30.0  | 12.6 | 10.9  |\n| 4  | 0.5| 4.1 | 108.4| 36.9 | 100.8 | 2.6| 2.5| 4.1 | 34.4 | 12.6 | 8.9|\n| 8 | 0.25 | 4.2 | 114.2 | 37.0| 105.6| 2.5 | 3.5 | 4.2| 35.9 | 12.6  | 14.1 |\n\nThese results validate our heuristic while suggesting that tuning $l$ to specific data or architectures could yield further gains.\n\n---\n\n## (3) On Latency and Computational Complexity\n\nWe appreciate the reviewer’s thoughtful comment regarding the lack of latency analysis. While our primary contribution lies in proposing the Binary Quadratic Quantization (BQQ) framework and demonstrating its advantages in terms of memory compression and approximation quality, we acknowledge that inference latency is a critical factor for practical deployment. Below, we address the three aspects the reviewer mentioned:\n\n**(a) Weight Loading Time:**\nWe fully agree that binary representations greatly reduce model size, which directly alleviates I/O bottlenecks during weight loading. This is particularly relevant in scenarios where memory bandwidth is the limiting factor. In such cases, BQQ's compressed binary format allows faster transfer from storage or DRAM to compute units, offering potential latency gains in loading time. Since I/O transfer time is approximately proportional to model size, reducing the model to 1/16 its original size—as in the case of 2-bit BQQ—can theoretically lead to up to 16× faster weight loading, assuming the I/O subsystem is the primary bottleneck.\n\n**(b) Low-Bit Computation Efficiency:**\nAlthough binary and low-bit operations can be highly efficient on hardware that supports them (e.g., INT cores on modern GPUs), our current implementation involves casting 1-bit weights to floating-point values before matrix operations. This is due to the lack of native mixed-precision (e.g., INT-FP) kernel support on standard GPU backends. As a result, despite memory savings, the actual compute kernel is still performed in floating point, leading to no measurable improvements in raw latency under current software-hardware stacks. \n\nHowever, recent work such as *LUT Tensor Core* (Zhiwen Mo et al., ISCA 2025) illustrates that with dedicated support for mixed-precision tensor operations (e.g., 1-bit × 16-bit), speedups over 7.5× are possible. We believe that BQQ is well positioned to benefit from such hardware innovations, and integrating with emerging low-bit inference libraries or hardware designs is a promising avenue for future research.\n\n**(c) Theoretical Computational Complexity:**\nIn terms of algorithmic complexity, the per-layer FLOPs of BQQ are on the order of $\\mathcal{O}((m + n)ldp)$, where $m$ and $n$ are the dimensions of the original weight matrix, $l$ is the intermediate dimension, $p$ is the number of binary matrix stacks, and $d$ is the input dimension of the layer.\n\nIn comparison, standard scalar $p$-bit quantization requires $\\mathcal{O}(mndp)$ FLOPs per layer. When $l \\approx \\frac{mn}{m + n}$, BQQ achieves the same asymptotic computational complexity as scalar quantization. While our current implementation does not fully leverage this due to software and hardware constraints, we emphasize that BQQ is structurally amenable to such optimizations.\n\nWe hope the reviewer recognizes that while we have not yet developed a latency-optimized implementation, our analysis highlights the conditions under which latency gains are likely to be realized. We thank the reviewer again for highlighting this important aspect and agree it is a crucial direction for future work.\n\n---\n\n## (4) On the Relation to LoRA and Related Work\n\nThank you for pointing out the connection to LoRA. While LoRA is primarily designed for efficient *adaptation* of large models by introducing lightweight low-rank updates, our method targets *compression* of the original real-valued matrices themselves. Despite this difference in objective, there is a structural resemblance in the use of low-rank components. We agree that clarifying this distinction—and positioning BQQ within the broader context of related methods, including LoRA-based techniques and hybrid approaches such as QA-LoRA and LQ-LoRA—will help readers better understand the contributions and novelty of our work.\n\nIf accepted, we will mention the structural similarities between our method and LoRA-based approaches, such as the use of low-rank components, to help situate our method within the broader landscape of efficient model design.\n\n---\n\n## Final Remarks\nOnce again, we thank the reviewer for their valuable insights. Your suggestions have significantly improved both our presentation and analysis."}]

Author rebuttal:

Meta-review: This paper presents Binary Quadratic Quantization (BQQ), a matrix quantization framework that represents real-valued matrices as sums of binary matrix products. Empirical results indicate that BQQ achieves  a better balance between compression efficiency and reconstruction accuracy relative to uniform quantization, binary coding quantization, and SVD-based low-rank approximations.

All reviewers recognize the method as original, interesting, and demonstrably stronger than existing approaches in both matrix compression and post-training quantization. During the author–reviewer discussion, the authors provided a compelling rebuttal and supplemented their work with further evidence, including broader comparisons to alternative quantization methods, evaluations on Qwen-0.5B, theoretical upper bounds, and analysis of intermediate dimensions.

In light of these contributions, I recommend acceptance of the submission. I highly encourage the authors to include the additional results in the final version. It will make the paper significantly stronger.
Decision: Accept (poster)