Boosting Entropy with Bell Box Quantization
Authors: Ningfeng Yang
Tor M. Aamodt
Venue: ICLR 2026 Poster (2026)
Keywords: Quantization
Quantization-Aware Training
Pre-Training
Primary area: foundation or frontier models, including LLMs

Abstract: Quantization-Aware Pre-Training (QAPT) is an effective technique to reduce the compute and memory overhead of 
Deep Neural Networks while improving their energy efficiency on edge devices. Existing QAPT methods produce models stored in compute-efficient data types (e.g. integers) that are not information theoretically optimal (ITO). On the other hand, existing ITO data types (e.g. Quantile/NormalFloat Quantization) are not compute-efficient. 
We propose BBQ, the first ITO quantization method that is also compute-efficient. BBQ builds on our key insight that since learning is domain-agnostic, the output of a quantizer does not need to reside in the same domain as its input. BBQ performs ITO quantization in its input domain, and returns its output in a compute-efficient domain where ITO data types are mapped to compute-efficient data types. Without sacrificing compute efficiency, BBQ outperforms prior SOTA QAPT methods by a perplexity reduction of up to 2 points for 4-bit models, up to 4 points for 3-bit models, up to 5 points for 2-bit models, and up to 18 points for 1-bit models. Code is available at https://github.com/1733116199/bbq.

Published as a conference paper at ICLR 2026
BOOSTING ENTROPY WITH BELL BOX QUANTIZATION
Ningfeng Yang University of British Columbia nxyangêece. ubc. abc.c
Tor M. Aamodt
University of British Columbia aamodt êece. ubc.ca
ABSTRACT
Quantization-Awan Pre-Training (QAPT) an effective technique to reduce the compute and memory overhead of Deep Neural Networks while improving their energy efficiency on edge devices. Existing QAPT methods produce models stored in compute-efficient data types (e.g. integers) that are not information the- oretically optimal (ITO). On the other hand, existing ITO data types (e.g. Quan- tile/NormalFloat Quantization) are not compute-efficient. We propose BBQ, the first ITO quantization method that is also compute-efficiet. BBQ builds on our key insight that since learning is not need to reside in the same domain as its input. BBQ performs ITO quanti- zation in its input domain, and returns its output in compute-efficient domain where ITO data types are mapped to compute-efficien data types. Without sac- rificing compute efficiency, BBQ outperforms prior SOTA QAPT methods by a perplexity reduction of up to 2 points for 4-bit models, up to 4 points for 3-bit models, up to points for 2-bit models, and up 18 points for 1-bit models. Code is available at https:://////m Dtttps://qithute /gtphgttttthhgggg/ com /etmm/133111111 /ttp////7311669bbb
the output of quantizer does
INTRODUCTION
Quantization is an effective method reduce the compuuation/neoory/ennnyy consumption of Deep Neural Networks (DNNs), allowing DNNs to be deployed to edge devices with limited hardware re- sources. However, quantization often degrades model quality. While Post- Tost-Training Quantization
(PTQ) methods (Lin et al.) 2024 (hao et al.) 2024 Ma et al. 2024b Liu et al. 2024a Kim et al.) 2024] can mitigate quality degradation without re-training for higher precisions (4+-bit), they strug- gle to maintain quality when weights and activations are quantized to 4-bit and below (Panferov)
cct al. 2025 Esser et al. 2020 Ma et al. 2024a] Kumar et al.| 2025). Quantization-
Training (QAT) methods (Panfcrov et al. 2025; Esser ct al. |020 Ma et al.] 2004a Kumar et al.) 2025 Shen
ct al. 2024 Liu et al. [0255] Yamamoto 2021), 1), on the other hand, can achieve higher accuracy than
PTQ methods under the same precision (Du et al. 2024 Panferov al. 2025 Liu al.
2024b) by
introducing quantization in the training loop.
QAT can be further divided into Quantization-Aware Pre-Training (Panferov et al. 2025 Yang & Aamodi 2025) (QAPT) and Quantization- Aware Fine-Tume Taning (Malinovskii et al.] 2024 Du et al. 2024) (QAFT). QAFT initializes low-precision model from full-precision pre-trained checkpoint, and trains the model for a short duration to fit downstream task, which is typically much smaller than the pre-training dataset (Du al. 2024). On the other hand, QAPT initializes a low-precision model from scratch, aims to fit a much larger dataset, and typically trains for much longer durations. Compared to first pre-training in full precisson and subsequently applying PTQ/QAFT, QAPT may have higher pre-training speed (Kumar et al.) 2025 Xi et al. 2023 Castro et al. 2025) as the forward pass of training may be performed in low precision.
This work aims to improve the accuracy of QAPT without compromising the resulting model's ef- ficiency on edge devices. Specifically, we target the case when low-precision model initialized randomly, trained for long durations large datasets, and is intended to be deployed to edge de- vices with constraints on memory capacity, inference latency, and energy consumption. The energy constraint requires the model to be compute-efficient, or that expensive high-pree nigh-precision matrix mul- tiplications be substituted with low oww-iiiciioon arithmetic, while the memory and latency constraints require the model to be small in size, to have low parameter count and precision. A main chal-

---

Published as a conference paper at ICLR 2026
Signed INT4 MX FP4
-8, 47, 16, +5, +4, +3, =2, +1,0
+6, +4, +3, +2, =1.,, =1, +0.5, +0
Table I: Possible values of the INT4 and the MX FP4 data type.
lenge of QAPT is that models with limited memory footprint cannot fit large datasets well due to a lack of learning capacity (Kumar et
2025).
Using the Shannon entropy (Shannon 1948 Cover & Thomas 2006) of quantized weights as a proxy for the amount of information/ wwwooo////wwwweggeoo present model trained with QAPT. we ob- serve that SOTA QAPT methods QuEST (Panferov et al.] 2025) and L.SQ (Esser et al. 2020) pro- duce quantized models that under-utilize the available learning capacity. The under-utilization because LSQ and QuEST use compute--fficient data types (e.g. integers and floats) which are not information theoretically optimal (ITO). While existing ITO data types (Dettmers et al.) [023 2022) can reduce this under-utilization of learning capacity, they lack compute efficiency modern CPUs/GPUs, which limits their applicability on edge devices with limited energy.
To maximally utilize the limited learning capacity while preserving compute efficiency, we pro- pose Bell Box Quantization (BBQ). BBQ is designed based on our key insight: since learning is domain-agnostic, the output of quantizer does not need to live in the same domain its input. BBQ performs ITO quantization in its input domain to maximally preserve information, but returns its output in different domain, where ITO data types are mapped to compute-efficient data types. BBQ achieves higher capacity utilization and better prediction quality than QuEST and LSQ. Unlike existing ITO data types, quantizzd model can still accelerate expensive matrix multiplica- tions with low- wwwpprecision arithmetic.
2 BACKGROUND AND MOTIVATION
In this section, we discuss compute- efficient data types on modern GPUs, existing ITO quantization methods, and lastly, the domain- domain-agnostic property of learning which is the key inspiration of BBQ.
Quantization is the discretization of a continuous randon!] variable æ to a discrete random variable i, such that è is sufficiently close to x. We present a generic template for quantization as follows:
f = fq), and r(f(x))
(1)
where R IRd is transform function that converts E R. to an intermediate domain suitable TRd T rounding function that takes real-valued inputs and returns outputs q in set of 2 real values, i.e. q T T C I. and [T] 29; and f-1 the inverse function of f. is responsible for converting the rounded variable g back to the original domain of Throughout this work, 2, 9. and å represent neural network weights or activations that raw, quantized, and de-quantized, respectively.
for quantization;
2.1 COMPUTE EFFICIENT DATA TYPES
instructions that directly operate on members of T without first decoding to high-precision data types. Table I shows members of T for INT4, which is supported by the Nvidia Ampere (Nvidial 202TT and Turing (Nvidia) 2018) architectures, and members of T for MX FP4 (Rouhani et al. 2023). which supported by the Blackwell architecture (Nvidia 2025). When f is linear or affine, a compute- efficient data type can be used to accelerate matrix multiplication and convolution (Yao al. 202 For example, if f (q) = sq for a constant scalar s, then the multiplication of activation matrix X = = =Qz and weight matrix W == (QQ = sQ.. can be simplified to s² Qw)). In other words, the matrix multiplication can be computed using solely low-precision representations of activations and weights, and later de-quantized to the original domain by multiplication of s?.
data type T compute-
hardware offers low- oo-precciioon
Prior SOTA QAPT methods QuEST (Panferov et al. 2025) and LSQ (Esser et al. 2020) use lin- ear/affine f and compute-efficient data types. We show the definition of f, and f for LSQ
We assume x: is random variable, or that is sampled from some distribution, enable the use of statistical properties like the Shannon entropy.
2

---

Published as a conference paper at ICLR 2026
and QuEST as follows:
LSQ: f(x) = r(s) |clip(e,
20 1 - -20-1,
sq
QuEST: f(z) a 2" r(e) | -20-1 ,0-1,2--11 1)], f)) HT 0 2 2)
HT(x)
where and a² are scalars; all division operations are element-wise; HT is the Hadamard trans- form, a linear operation; and the clip operation and the the output of to be a b-bit signed integer. Since f-1 is linear or affine in LSQ and QuEST, matrix multiplications can be accelerated with low-precision arithmetic.
operation [-] enforce
In short, compute- -efrutiteffiiieent data type T and linear/affine dequantization function f-1 are two key features that enable quantization methods like LSQ and QuEST to run efficiently on modern GPUs.
2.2 INFORMATION THEEORETICALYY OPTIMAL QUANTIZATION METHODS
An ITO quantization method is one that ensures each member of T is used equally often (Dettmers) et al., (203)) For example, if b 2, then 25% of elements of z should be assigned to each of the possible values T. While there are many possible mappings that evenly divide the elements of trivial mapping is to use the 25th, the 50th, and the 75th percentiles as rounding boundaries. In other words, elements of x less than the 25th percentile of x are rounded to the smallest value in T; elements larger than the 25th percentile but less than the 50th percentile are rounded to the second smallest value in T, etc. all d elements of x are i.i.d. sampled from N(, o²) and is sufficiently large, this trivial mapping be expressed as:
f(z) = x/0, =T (2")
=q
(3)
where [i is the ith smallest element in T. Equation 3 only requires that contains 2 unique real values, regardless of what these real values may be. As such, Equation 3 describes a set of ITO methods rather than unique one. By evenly dividing the elements of x, all ITO data types maximize the empirical Shannon entropy of defined as
(4))
Σ PP4,,)) log2 P(g.1) SE
where P(g. t) is the probability that a (uniformly) randomly chosen element of is equal to f. Note that entropy was shown to positively correlate with model accuracy/prediction quality (Cheng al.) 2025 Shen et al., 2024).
Inspired by ITO quantization methods, Dettmers et al. 2023) proposed NF4, a 4-bit NormalFloat whose values are a list of mes-max-normmllnn Gaussian quantiles (Yoshida 2023 Dotzel et al. 2024). Due to lack of hardware support, NF4 must be de-quantized to full-precision floats and then used in computation, which limits its applicability
corrgy-constrained edge devices.
This presents a dilemma: while ITO quantization methods maximally preserve information, they are not compute-efiie -fffiiitefffiiintt compute-efficient quantization methods, on the other hand, are not ITO for Gaussianly distributed weights/activations. Can we obtain the best of both worlds?
2.3 LEARNING IS DOMAIN-AGNOSTIC
Since neural networks are universal function approximators (Hornik et al.) 1989). they are capable of learning from transformede transformed/angmmeted data, or even data encoded in latent spaces that are not human- understandable. To list few examples: DNNs can correctly classify images that are rotated (He) et al. 2015); DNNs can classify images that are transformed to the frequency domain (Xu et al. 2020 When autoencoders (Gao et al. 2025) are trained just to reconstruct the data, the resulting latent embedding of the data can be used by other DNNs to complete downstream tasks. These examples are all evidence that, as long as information is preserved, simply projecting/raastormnngg data to different domain does not prevent learning.
The domain- angasticc property of learning provides an opportunity to circumvent the inefficiency of ITO quantization methods. Rather than treating quantizers data compressors and reconstructors,
3

---

Published as a conference paper at ICLR 2026
HT(x) /
((v)
[2n0(v)] -2
-3
3
0.5
Weights Activations
(a) HT & RMS
(b) PIT
Uniform Quantization
Figure I: The three steps of the BBQ quantization formula (Equation 5 for b = 4. Step is the Hadamard Transform (HT) followed by RMS Normalization. Step the probability integral trans- is uniform quantization. We name our method Bell Box Quantization because Figure Ta looks like a bell and Figure Tb like a box (rectangle) which is quantized Figure Ic
form (PIT). Step X
we could treat them feature extractors that return set of compact latent features of the original data. Critically, these latent features reside in an output domain, that is, not necessarily the same do- main as the input, but designed to be a compute-efficee -fffccieen domain, where features can be efficiently used by matrix multiplication operators. While N2UQ (Liu et al., (2022) also returns outputs in compute-efficient but diistict-from-inppe domain, BBQ is the first method to apply such technique to ITO quantization for both activations and weights. In addition, N2UQ assumes raw weights are uniformly distributed, while BBQ is ITO without making any assumptions on the distribution shape of weights
activations. In Section 4 quantitatively compare BBQ against N2UQ, showing BBQ achieves lower perplexity under the same precision. Inspired by the domain domain-agnostic property of learning, ask the research question: During QAP',, can ITO quantization outperform non-ITO methods if we return outputs in a compute-eficieen domain?
3 BELL BOX QUANTIZATION
In this section, we present the Bell Box Quantizer (BBQ). BBQ is designed to ITO, or to maxi- mally preserve information from its input æ (activations weights), while also returning compute- efficient outputs that can be accelerated by modern hardware. The BBQ quantization and dequanti- zation formulas shown in Equations 5 and 6 respectively:
q = [2))(u)]
2b-1 2, where v = -T(2)//
(5)
$
26-T
(6)
where HT Rd Rd is the Hadamard transform operation; the root-mean-square (RMS) nor- malization factor, a, is defined as (1/d [ (H v is a variable introduced for
notational convenience; + Rdd + (0, 1)d the element-wise Gaussian CDF operation: [.] is the element-wise floor operation; E {0, -0.5) is hyperparameter zero point; and is a learn- able scaling factor. Figure [
illlstrrtee the three major steps of Equation Hadamard Transform combined with RMS Normalization (step a). probability integral transform (step b). and uniform
quantization (stepla). We discuss steps a and c in Sections 3.1 3.2 and 3.3 respectively, and the dequantization formula Equation in Section
3.1 STEP HADAMARD TRANSFORM AND RMS NORMALIZATION (FIGURE TA
Since ITO quantization is ill-defined without knowing distribution, the first step of BBQ is to convert input æ from an unknown distribution to a Gaussian by applying the Hadamard Trans- form (Panferov et al. 2025 Yang et al. 2025). After the transform, we assume elements of HT(x) behave like samples from N (0, a following Panferov et al. (025) and Yang et al. 2025). Instead of transforming all elements of Jr we follow QuEST and perform the Hadamard Transform on every H elements of x along the input channel dimension. During training, the Hadamard Transform is a differentiable operation, and simply use autograd frameworks perform its backward pass. Dur- ing inference, the Hadamard transform is implemented with a vector vector-matre matrix multiplication between an Heelement slice of and the pre-computed H H Hadamard matrix.
4

---

Published as a conference paper at ICLR 2026
p(clip(v)
cliptv>
p(
oiv
clip(v)
((v)
(a) Graph of clip and Q. Both functions take input in (-o0, oo) and return output in finite range, and therefore both qualify clipping functions.
(b) PDF of clip(u) and (()) for o [00 1). 2(v) has maximum (diffcrential) entropy as its probabil- ity mass is evenly spread out.
Figure 2: Comparison of clip (blue) and the standard Gaussian CDF (orange).
Next, we apply RMS normalization by multiplying the Gaussian-like data by 1/a. As result, elements of = (xxx)) behave like samples from N 1). During training, we use autograd frameworks to perform the backward pass of RMS Normalization. Following standard quantization practice (agel et al., 2021), perform channel-wil wise RMS normalization for weights, and per- tensor RMS normalization for activations.
We discuss optional variant of BBQ, named BBQ-Fast, which achieves identical perplexity (Sec- tion A.3) but has better inference speed. During training, BBQ-Fas has identical behaviour to BBQ, except BBQ-Fast additionally keeps track of the exponential moving average of 1/a, denoted as E1/o. using the update rule in Equation 7 which is executed every training iteration with 3 = 0.99.
F1jo < B. F1jo + (1 B) (1/a)
(7)
During inference, BBQ-Fast performs activation RMS normalization by multiplying HT by the final value of Ei/o. instead of by 1/0r, since measuring requires the execution of the root-mean- square operation, which often incurs expensive cross-thread-bee communication for large activa- tion tensors.
3.2 STEP B PROBABILITY INTEGRAL TRANSFORM (FIGURE
The probability integral transform David & Johnson 1948 Fisher 1932) converts any continuous distribution a uniform one by applying its CDF on the data. We apply the standard Gaussian CDF & to e which behaves like samples from the standard Gaussian N(0. 1), creating ((v) which behaves like samples from UO, 1). Function & is designed clip data within a finite range (Figure 2ah while maximizing entropy (Figure 2b) for Gaussianly distributed data, and is meant to replace the vanilla clip function in QuEST and LSQ Figure 2a shows that & is much smoother than clip. This is because O is infinitely differentiable and clip is piecewise linear. A smoother operation more suitable (Nesterov) 2014) Bottou et al.] 2018) for optimization methods like Gradient Descent. For example, the GELU function (Hendrycks & Gimpel (002)) defined GELU(2)) a ((x)) smoother than the piecewise linear ReLU (Agarap) 2018), and GELU can outperform ReLU empirically [Hendrycks Gimpel| 2023). During training, de is differentiable operation and use the autograd framework to perform BackProp. During inference, we combine & with the subsequent floor operation using the following property, which is Yu E, Vi E {0, 1, 2.... 2 1]:
be
...
(i+1
(2)(1)) =i Φ
28
26
Property 8 allows us to pre-compute Φ a binary search that requires floating point comparisons. Our profiling results in Section 4.2 show this binary search incurs negligible runtime overhead. The pseudocode of 3-bit inference kernel implementation of BBQ is shown in Algorithm
(i/2²) for all 2hh possible values of and runtime perform
3.3 STEP UNIFORM QUANTIZATION (FIGURE)
After applying the Probability Integral Transform, since ((v) behaves like samples from (o, 1), achieve ITO quantization by applying uniform quantization, obtaining [2*)(n) which can
5

---

Published as a conference paper at ICLR 2026
Algorithm Example 3-bit Inference Quantization Kernel for Thread Block
1: function BBQ(block id, x.pir, h_pir, q-ptr, Eija)
2:
..prr/block..cd.loock.dkk # x-ptr is an M x N matrix reshaped to (MN/H, H) 3: ptr[:. pir pre-computed Hadamard matrix with shape (H, H)
4: xh x @ h## Step vector-matrix multiplication between shapes (1, H)) and (H, H)
5: v Ei Step element-wise multiplication by pre-computed scalar (BBQ-Fast)
6:
q= array with H INT4/MX FP4 elements
7:
for < 0 H -1do # can be done in parallel across warps and threads
8:
if v[][] > Φ (4/8) then # Step b Binary search with pre-computed Ø-1 values
9:
v[t] 2 ① (6((/8)) then
10:
q[t] vft] Φ (7/8) else 2 # Step quantize to INT4/MX FP4
11:
else
12:
q[t]= if v(t) 2 Φ (5/8) else 0 Step quantize INT4/MX FP4
13:
end if'
else
15:
if v(t] 2 (2/8) then
q vft] 2 -- (3/8) else Step
quantize to INT4/MX FP4
17:
else
18:
q[t]= if v(t] 2 (1/8) else Step quantize to INT4/MX FP4
end if 20: end
19:
21: end for
22:
q.ptr[block id:block id+1, :] = # store quantized INT4/MX FP4 results to global memory
23:
end function
Precision b
Zero Point
Possible Values of
INT4
MX FP4
-8, -7, 6, 5, 4, -3, -2, -1, 0, 1, 2, 4,5,,,7
X
3
0
4, -3 2, -1, 2,
/
2
-0.5
-1.5, -0.5, 0.5, I.5
X
-0.5
-0.5, 0.5
X
Table The BBQ data type: zero point 2, possible values of q, and whether BBQ can be encoded as INT4 and MX FP4 for each precision b E {1,2,
be stored as a b-bit unsigned integer. Since zero-mean activations can contribute to faster conver- gence (Bjorck al. 2018; loffe & Szegedy 2015), we convert the non-negative data (2)) to symnettic data by subtracting 20 and the hyperparameter zero point 2, producing the final stored as signed compute-efficient data type. Inspired by NF4 (Dettmers) ct al. (023), ) we use 2 = 0 for b E {3. 4}, allowing the data type to represent zero exactly while sacrificing value on the positive side. We use
| (122 2
0.5 for E (1,2) to ensure activations remain zero-mean. Table 2 lists the values supported by the BBQ data type. For E {1, 3). BBQ can be represented using the MX FP4 data type on existing Blackwell GPUs For E {3,4] BBQ can be represented using INT4 on existing Ampere and Turing GPUs. During training, we use the Straight- Through Estimator (Bengio et al. 2013) for the floor operation, and the autograd framework for all other differentiable operations.
3.4 DEQUANTIZATION
Inspired by LSQ, the dequantization formula of BBQ, Equation 6 includes a learnable scaling pa- rameter & = 7/20-1, learnable scaling factor enables control of the magnitude of á. To ensure the learnable scaling factor can be initialized to the same value regardless of precision b, we do not use directly but instead decouple s into the ratio
Iearnable scaling factor and constant precision-dependerr scaling factor 2 Following LSQ, we apply gradient scaling (Esser et al., 2020) to reduce the gradient of y by factor of vd, where is the number of weights/activationn in æ. Following Nagel et al. (2021), I). we use channel-wise y for weights and per-tensor 7 for activations.

---

Published as a conference paper at ICLR 2026
BBQ
QuEST
L.Q
Params
Tokens
Full
Bits
Entropy
Perplexity
Entropy
Perplexity
Entropy
Perplexity
95M
3B
24.75
4
3.93
25.51
3.61
26.37
3.59
27.46
95M
38
24.75
3
2.96
26.55
2.78
29.04
2.74
30.27
95M
3B
24.75
2
1.97
31.34
1.92
35.58
1.69
36.58
95M
3B
24.75
1
1.00
49.22
1.00
67.78
125M
5B
21.51
4
3.93
22.15
3.61
22.98
3.60
23.77
125M
5B
21.51
3
2.96
23.22
2.78
25.21
2.74
26.28
125M
5B
21.51
2
1.98
27.34
1.93
31.32
1.81
31.42
125M
5B
21.51
1.00
44.58
1.00
72.54
200M
TOB
17.93
4
3.93
18.79
3.61
19.06
2.73
1778
200M
10B
17.93
3
2.96
19.74
2.78
20.82
2.50
140.9
200M
10B
17.93
2
1.98
23,08
1.93
25.46
1.63
78.19
200M
10B
17.93
1
1.00
38.27
1.00
52.37
300M
208
15,43
4
3.93
16.10
3.61
16.26
300M
20B
15,43
3
2.96
16.90
2.78
17.67
300M
20B
15.43
2
1.98
19.75
1.93
21.53
Table 3: Entropy and Perplexity of LLaMA models pre-trained on the C4 dataset. The headers are the number of parameters (Params), the number of tokens (Tokens), perplexity without any quan- tization (Full), activation/weight precision (Bits), and the entropy and perplexity of BBQ, QuEST, and L.SQ.
The initialization of plays a critical role, overly large initialization lead to gradient explosion while overly small initialization can lead to gradient vanishing, We initialize to (* To where do is the measured at the first iteration of training, and (* is defined as:
5* arg min E F( ( 1))
See Section A.2 for detailed explanation of (* and Equation 9 In short, by initializing 7 to (* can have approximately the same magnitude as æ (in the first iteration) to prevent the activation magnitude from exploding or vanishing tokens travel deeper into the network. After the first iteration, we simply let y update itself based on gradient descent. We not apply weight decay to y, an overly decayed may lead to gradient vanishing.
EVALUATION AND DISCUSSION
We run experiments to evaluate the perplexity of BBQ against other SOTA QAPT methods on representative LLM architectures. We the publicly available source code of QuEST (Pan- ferov et al. 2025), add the implementation of BBQ, and train on ILaMA (Touvron et al. 2023] Vaswani al. (017) models with n non-embedding parameters plus embedding parameters, where E {30M, 50M, 100M, 200A/ } and E {65M, 75M, 100M, 100M). }. For each model, pre-train with 100n C4 tokens while quantizing the weights and activations of all linear layers to b-bit following QUEST. We compare BBQ against QuEST/LSQ for each b E f1, 4), and present the evaluation quality (perplexity) and weight entropy (bits) in Table 3 The omitted entries in Table 3 are because LSQ does not support 1-bit quantization and LSQ diverges for larger models (unlike BBQ which works well for 1-bit and converges even for larger models). We show zero-shot results in Table 9 and training loss curves in Section A.7 Each experiment for + conducted on one Nvidia RTX 5090 and lasts for up to day. Each experiment for n + > 200M is conducted on one Nvidia A100 80GB and lasts for 3.5 days. In total, all experiments took approx- imately 1.5 GPU months. Lastly, we evaluate BBQ against QuEST, LSQ, ParetoQ-SEO (Liu et al, GPT (Brown et al.
200M is
2025)
N2UQ (Liu al. 2022) and NormalFloat (Dettmers al. 2023 models and present our results in Table
2020
Results in andleeaan44 BBQ can consistently achieve higher entropy and lower perplexity than QuEST and LSQ at the same precision, and that entropy can be a good proxy to prediction quality. A special case is when b 1. In this case, both BBQ and QuEST achieve the maximal
Note that when weights quantized to b-bit, the maximum achievable weight entropy is h hits. In other words, entropy upper bounded by precision.
7

---

Published as a conference paper at ICLR 2026
Params
95M
95M
95M
95M
95M
95M
95M
95M
95M
95M
95M
95M
Bits WA
16
4
4
4
3
3
3
3
3
1
Method
BBQ
QuEST
LSQ
NF4
BBQ
QuEST
LSQ
NF3
N2UQ
BBQ
QUEST
Perplex
25
26.7
26.83
27.55
28.5
28.61
29.7
30.85
36.16
29.87
53.8
77.96
Params
95M
95M
95M
95M
95M
95M
125M
125M
125M
125M
125M
125M
Bits WA
2
2
2
3
3
2
2
Method
BBQ
QQEST
LSQ
NF2
SEQ
N2UQ
BBQ
QuEST
BBQ
QuEST
BBQ 50.46
QuEST
Perplex
34.36
36.7
38,98
246.6
40.07
35.59
24.78
26.05
29.38
31.47
69.78
Table 4: Perplexity of GPT models pre-trained on the C4 dataset.
2.00
Entropy (Bits)
1.93+
Theoretical Maximum
QuEST
BBQ
1.90
10k
20k
30k
40k
50k
60k
70k
Time (Iterations)
Figure 3: Quantized weight entropy vs. training iterations for LLaMA-300M with 2-bit weight and activations, pre-trained on billion C4 tokens (batched into 80 thousand training iterations).
entropy of bit. We attribute the performance gain of BBQ in the 1-bit case to the fact that & from BBQ is smoother than the clip operation from QuEST (as discussed in Section B.2). In addition, we note that LSQ diverges for L.aMA-200M, which reflected its entropy. Notably, when LSQ does not diverge, like in the case of LLaMA-95M and LLaMA-1 115MM, we see that LSQ can achieve an entropy of 3.6 bits for 4-bit precision, 2.7 bits for 3-bit precision, and bits for 2-bit precision. However, when LSQ diverges, like the of LLaMA-200M, we see that its entropy drops to 2.73 bits for 4-bit precision, 2.50 bits for 3-bit precision, and 1.63 bits for 2-bit precision. This shows that entropy good indicator of quantized model's prediction quality.
4.1 ENTROPY AND LEARNING CAPACITY
In this section, we discuss, based on empirical measurements of entropy during training, the im- provement of BBQ over QuEST T in terms of learning capacity utilization. Figure 3 shows the be- haviour of weight entropy of BBQ and QuEST during 2-bit training. We note a few observations. First, BBQ can maximize entropy it can achieve the theoretical maximum entropy of 2 bits at the beginning training. Second, during training, BBQ-quantized model can adjust (increase or decrease) its entropy. Third, excluding the first 8 thousand iterations when learning rate is warming up. BBQ tends to increase weight entropy for the last 72 thousand iterations. This suggests, as more training tokens presented to the model, more information is accumulated in the weights. Lastly, the entropy of QuEST seems to have an empirical ceiling at around 1 1.93 bits. We attribute the empir- ical entropy ceiling of QuEST to the following: QuEST applies uniform quantization to HT(x)/ and therefore QuEST only ITO if HT(x) follows uniform distribution, which unlikely as the Hadamard transform has tendency Gaussianize (Yang et al., 2025 Panferov et al. 2025).
See Section A.5 for a visualization of the difference between BBQ and QUEST.
By having a lower empirical entropy upper bound of 1.93 bits, QuEST limits the learning capacity of the model, while BBQ does not suffer from such capacity under-utilization. Suppose a model has d parameters, each quantized 6 bits. This means the model has a total of 2db possible unique states. Some of the 2 states have higher entropy, while others have lower entropy. QAPT's objective is to find a (locally) optimal state out of all 2dh possible states. However, if QuEST enforces its weight entropy to be lower than 1.93 bits, then some of the high-entropy states cannot be explored by the training algorithm. Therefore, the model's learning capacity effectively reduced.
4.2 INFERENCE SPEEDUP
In this section, we discuss our profiling results the inference latency of BBQ, compared against FP16 and NF4. We perform latency measurements NVIDIA RTX 5090, Blackwell GPU that
8

---

Published as a conference paper at ICLR 2026
0.3
bbc
fp4
int8
fp16
ms
0
2000
1000
6000
8000
2000
1000
6000
8000
(a) BBQ quantization kernel of an N x N matrix
(h) Matrix multiplication two N x N matrices
Figure 4: Kernel latency on Nvidia RTX 5090 (y-axis) vs. matrix size N (x-axis).
element-we wise ops
bbq kernel
nf4 kernel
others
LLaMA 700M on RTX 5090
LLaMA 2.4B on A100 80GB
300
800
Latency (ms)
200
600
400
100
200
bbq-fp4
fo16
nf4
bbq-int4
fp16
nf4
Quantization Methods
Quantization Methods
Figure 5: End-to-end LLMMA inference latency of BBQ, FP16 and NF4 on RTX 5090 and A100 GPUs. For each linear layer, BBQ launches activation quantization kernel (green region), fp4/int4 matrix multiplication kernel (part of blue regions), and an element-wise scaling kernel (part of orange region).
supports MX FP4 matrix multiplication, and NVIDIA A100 80GB, Ampere GPU that supports INT4 matrix multiplication. At inference time, for each linear layer, BBQ launches an activation quantization kernel to compute the quantized activation matrix Qz. low low-precision matrix multipli- cation kernel between Q₂ and the quantized weight matrix Qw. and lastly an element-wise scaling kernel scale the result Q₂Qu. by factor of 8. We note that BBQ performs its weight quan- tization offline and therefore weight quantization does not incur inference latency overhead. Since compute-efficienge the feature that distinguishes BBQ from NF4, manifests latency savings dur- ing the execution of compute- bouud kernels, we focus on profiling the prefill phase of inference. However, we note both BBQ and NF4 can achieve latency reduction during the memory-boue -ound de- code phase both can reduce memory bandwidth consumption. In addition, during the decode phase, BBQ may save energy compared to NF4 as computation is performed in low precision.
We first show at the kernel level, the latency of the BBQ activation quantization kernel is negligible compared to the time saved by performing matrix multiplication low-precision. Figure 4a shows the latency of Triton (Tillet et al.) 2019) implementation (shown in Section A.I of Algorithm on
N N activation matrices, for N E {256-1 ie Z,2 < < 32). Note that 256 32 = 8192 corresponds to the dimension of LLaMA-65B (Touvron al. 2023). important LLM benchmark. Figure 4a shows that when = 8192, our BBQ quantization kernel takes 0.3 milliseconds to quantize N x N activation matrix to FP4. However, Figure 4b shows the latency saving of FP4 matrix multiplication over FP16 matrix multiplication is 4 milliseconds when N = 8192. Thus, for LLaMA-65B, BBQ's activation quantization adds latency overhead that only one- tenth the latency savings enabled by BBQ's reduced precision matrix multiplication. We attribute the low latency of the activation quantization kernel to the following: since quantization kerness element-wise and highly arrive from global memory, and therefore additional computation (such as the Hadamard transform and binary search) can be fused into the quantization kernel without incurring latency overhead.
comoory-bound compute units GPUs are often idle waiting for data to
We next show BBQ can outperform FP16 and NF4 in terms of end-to-end inference latency, using LLaMA-700M and LLaMA-2.44 2.B.BB our benchmarks. Figure illustrates Fillutttlssstratees the overall inference la- tency, with contributions from different types of kernels shown in different colours. While BBQ adds

---

Published as a conference paper at ICLR 2026
Row
HT
RMS
PIT
Learn 7
Y
Init
Perplexity
Entropy
Notes
/
/
/
31.34
1.97
BBQ
8
/
35.79
1.98
7
/
/
35.93
1.98
4
/
x
x
x
35.58
1.92
QuEST
/
/
x
x
138.3
1.92
X
31.46
1.98
Table 5: Ablation BBQ features: Hadamard Transform (HT), RMS Normalization (RMS), Prob- ability Integral Transform (PIT), making y learnable parameter (Learn ?), and initialization (* Init). A // means a feature is present, and means the feature not included.
additional latency overhead quantize activations (green region) and perform element-wise scal- ing (orange region), BBQ significantly reduces the latency of matrix multiplication (blue region), which is the latency bottleneck both FP16 and NF4. In general, BBQ shows 40% speedup over FP16 and 48% speedup over NF4. While both BBQ and NF4 can compress large models, BBQ compute-efficient and has shorter latency than FP16. In contrast, during prefill, NF4 has longer latency than FP16, since NF4- NF4-quantized activations and weights must be de-quantized before being consumed by full full-precision matrix multiplication kernels.
4.3 ABLATION
In this section, we present ablation studies of BBQ's individual features in Table 5 Specifically, we study the effects of Hadamard Transform, RMS Normalization, probability integral transform (4) instead of the regular clipping function, having learnable scaling factor y and initializing ? to ("op instead of a dummy value (e.g. 1). We evaluate on a 2-bit LLaMA-95M pre-trained with 3B tokens. Results show Hadamard Transform and RMS normalization
two solid features BBQ inherited from QuEST, as removing them from BBQ results in perplexity increase of 4,45 (row 2 compared to 1) and 4.59 (row compared to 1). The probability integral transform ($) is meant to replace QuEST's clip function. We see that naively substituting clip with & (row compared row 5) without ? initialization leads divergence (a perplexity increase from 35.58 to 138.3), but the combination of & and y initialization (row 6) can outperform QuEST (row 4) by a perplexity decrease of 4.12. Lastly, making ? learnable (row compared to row 6) can further reduce the perplexity by smaller margin of 0.12.
5 LIMITATIONS AND FUTURE WORK
While BBQ can achieve lower perplexity than QuEST and LSQ for QAPT, BBQ is, by design, a quantizer that makes no attempt to reduce/bound the Euclidean distance between I and ê, since they live in separate domains. Therefore, when applied to QAFT, BBQ's unbounded quantization error will drastically reduce model quality and cannot catch up to "same-domain" quantizers (ones that follow Equation TT like QuEST and LSQ within the short time frame of QAFT. For similar reasons, BBQ is not suitable for PTQ.
BBQ can maximize entropy under the assumption that the Hadamard Transform can Gaussian- ize Pannfrovv et al. 2025 Yang et al. 2025) BBQ's input data. Empirically, we find HT(x) to be exactly Gaussian at the beginning of training but not perfectly Gaussian afterwards. As future work, may be possible to replace with more accurate smooth approximation of the non-differentiable empirical CDF of HT(x), to more closely approximate true probability integral transform.
6 CONCLUSION
In this work, we identify existing SOTA QAPT methods under-utilize learning capacity. While ex- isting ITO quantization methods can maximize entropy, they are not compute-efficient which limits their applicability to edge devices with energy constraints. Utilizing our key insight that learning is Nomain-agnostic returning outputs in compute- compute-efficient domain. Empirical results show BBQ outperforms QuEST and LSQ without sacrificing compute efficiency.
propose BBQ, which performs ITO quantization in the input domain, while
10

---

Published as a conference paper at ICLR 2026
7 REPRODUCIBILITY STATEMENT
link to our source code is provided in the abstract of the paper. Our code should allow readers to reproduce tables and figures in the paper.
ACKNOWLEDGMENTS
This research was supported by the Natural Sciences and Engineering Research Council of Canada through a Discovery Grant. We thank Renjie Liao for providing access to additional GPU resources UBC. Tor Aamodt has recently served as consultant for NVIDIA, HPE, Cisco and IBM.
REFERENCES
Abien Fred Agarap. Deep learning using rectified linear units (relu), 2018. URthttp::/aaxxx org/ /b///bb///// 1803. 08375 1xxi:18803.83355oommee. 7 pages, 11 figures, 9 tables.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation, 2013. URL
abs/ /oos/138..0 3432
Johan Bjorck, Carla Gomes, Bart Selman, and Kilian Q. Weinberger. Understanding batch normal- ization. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 7705-7716, Red Hook, NY, USA, 2018. Curran Associates Inc.
Léon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning, 2018 URL /ttttt:ss//aaxxtt wwtexiii.org///i. hasr//bs/11600 04838
Tom B. Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL fttps://aae /ttp///aivvvv////////aa//ea. o abs 2005. 14165
Roberto L. Castro, Andrei Panferov, Soroush Tabesh, Oliver Sieberling, Jiale Chen, Mahdi Nikdan, Saleh Ashkboos, and Dan Alistarh. Quartet: Native fp4 training can optimal for large language models. CoRR, abs/2505. 14669, May 2025. UtLhttpss:// /ttttpp://doi.or//110000 0tt.://.....rr1...885555555
2505 14669
Feng Cheng, Cong Guo, Chiyue Wei, Junyao Zhang, Changchun Zhou, Edward Hanson, Jiaqi Zhang, Xiaoxiao Liu, Hai Li, and Yiran Chen. Ecco: Improving memory bandwidth and capacity for ilms via entropy-aware cache compression. In Proceedings of the 52nd Annual International Symposium Computer Architecture, ISCA '25, pp. 793 807, New York, NY, USA, 2025. As- sociation for Computing Machinery, ISBN 7798400712616
doi: 10.1455/665003.333300444
URL httphhtp////oooo wrw.orggg..... 10.00.114/35 /111/1195553..... 3731024
Yubei Chen Chun-Hsiao Yeh. IN100pytorch: Pytorch implementation: Training resnets on imagenet-1000 https://gii 2022.
/ttt:////ittuub.to tetiihubb... com/ ctm/ http//dnielcchh...m Imte:/etmmaeeeeaee 100 Pytorch,
Thomas M. Cover and Joy Thomas. Elements of Information Theory (Wiley Series Telecom- munications and Signal Processing). Wiley- Viter-enterscceece, USA, 2006. ISBN 0471241954.
F. N. David and N. L. Johnson. The probability integral transformation when parameters are estimated from the sample. Biometrika, 38(1-2)1182-190, 05 1948. ISSN 0006-3444. doi: 10.10933biomet/35..22 182. URL doi www.ooogroo........ htt.://00////////333
182
Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 8-bit optimizers via block- quantization. In International Conference on Learning Representations, 2022. URL https:
//openrevi ew. net. watt/fooumm/ ttt.forrmmmmm.iiippppxxxxx..

---

Published as a conference paper at ICLR 2026
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: efficient finetuning of quantized llms. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, USA, 2023, Curran Associates Inc.
Jordan Dotzel, Yuzong Chen, Bahaa Kotb, Sushma Prasad, Gang Wu, Sheng Li, Mohamed S. Ab- delfattah, and Zhiru Zhang. Learning from students: applying t-distributions to explore accurate and efficient formats for Ilms. Proceedings of the 41st International Conference on Machine Learning. ICML'24, JMLR. 2024.
Da Du, Yijia Zhang, Shijie Cao, Jiaqi Guo, Ting Cao, Xiaowen Chu, and Ningyi Xu. BitDis- tiller: Unleashing the potential of sub-4-bit LLMS via self-distillation.
Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 102-116, Bangkok, Thailand, Au- URL
gust 2024. Association for Computational Linguistics. doi: 16653//11202..alllooo
Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dhar- mendra S. Modha. Learned step size quantization. In International Conference on Learning Representations, 2020. URL https: /ttp///pp/reeveee//e///e ww. http://forummm--rkko0500
Ronald A. Fisher. Statistical methods for research workers. Oliver and Boyd., 1932.
Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In The Thirteenth Inter- national Conference on Learning Representations, 2025.
Pi 9ZNKD.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CoRR, 151.03385, 2015.
wtwpp//gg/.//..../.....aa 03385
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), 2023, URL https://
arxiv. abs / 1606. 08415
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks uni- versal approximators. Neural Networks, 2(5):359-366, 1989. ISSN 0893-6080. doi: https:
URL https: //ww://wwww............ /ttp//wwceencddreeetttttt.eeeeee com/
Sergey loffe and Christian Szegedy. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on ternational Conference Machine Learning Volume 37, ICML' pp. 448-454 JMLR.org, 2015.
Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, and Kurt Keutzer. Squeezellm: dense-and-sparse quantization. In Proceedings of the 41st International Conference on Machine Learning, ICML'22. 24. JMLR.org, 2024.
Tanishq Kumar, Zachary Ankner, Benjamin Frederick Spector, Blake Bordelon, Niklas Muen- nighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Re, and Aditi Raghunathan. Scaling laws for precision. In The Thirteenth International Conference on Learning Representations, 2025.
URL
net./
Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui. Yingtao Zhang, Linzhan Mou, Linqi Song. Zhenan Sun, and Ying Wei. Duquant: Distributing outliers via dual transformation makes stronger quan- tized L.Mss. In The Thirty-eighth Annual Conference on Newral Information Processing Systems, 2024. URL http:::::://ppeeevvvi..mm www.n/ee/eete.. forum? d=wp8u2ppppppmmmmm cmqz
Jing Liu, Ruihao Gong, Xiuying Wei, Zhiwei Dong, Jianfei Cai, and Bohan Zhuang. QLLM: Ac- curate and efficient low-bitwidth quantization for large language models. In The Twelfth Interna- tional Conference on Learning Representations, 2024a. URL https: /ttppp:///peeee http///ppereveeeeeeeeeeee reveew..ww...... net/ Eorum? wwi.ddFFIppmmmmm.
12

---

Published as a conference paper at ICLR 2026
Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric Xing, and Zhiqiang Shen. Nonuniform-to uniform quantization: Towards accurate quantization via generalized stright-througe estimation. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 4932-
4942, 2022. doi: 10.11099CVR552688... 2022.00489
Zechun Liu, Barlas Oguz, Changsheng Zhao, Emii Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. LLM-QAT: Data-free quantization aware training for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 467-484, Bangkok, Thailand, August 2024b. Association for Computational Linguistics. doi: (0.18653/v1/2224 andingg-acl..26. atRLattts///caaattooooo wtw.:/ooooooooogg/eeee/ wwtp////./22// fincing acl.26/
Zechun Liu, Changsheng Zhao, Hanxian Huang, Sijia Chen, Jing Zhang, Jiawei Zhao, Scott Roy, Lisa Jin, Yunyang Xiong, Yangyang Shi, Lin Xiao, Yuandong Tian, Bilge Soran, Raghuraman Krishnamoorthi, Tijmen Blankevoort, and Vikas Chandra. Paretoq: Improving scaling laws in ex- tremely low-bit LLM quantization. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https: /ttpt::s:///pprreeeeeee www.reveerreeeeeeeeeeee fart/fforum? http://rrrmmmmpppssdaem3e d=PMSNdBxTHP
Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang. Shaohan Huang, Li Dong. Ruiping Wang, Jilong Xue, and Furu Wei. The era of 1-bit llms: All large language models are in 1.58 bits, 2024a. httttttt://aaaa org abs 2402.
Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang. Shilei Wen, Fei Chao, and Rongrong Ji. Affinequant: Affine transformation quantization for large language models. In ICLR, 2024b. URL
net wwt.e//oooommmooo2aaaaa.a..
Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis Kuznedelev, Konstantin Pavlovich Burlachenko, Kai Yi, Dan Alistarh, and Peter Richtárik. PV-tuning: Beyond straight-througe estimation for extreme LLM compression. In The Tiirty-eighth Anmual Conference on Neu- ral Information Processing Systems, 2024. BRLhttps: htttp:::/pperreeeertt http///pppeeee..eeeeeppeeeeee.ep wwt.leewweeeeeeeeeeee wwt/ffrunnrrn forum? did=
YvA8UFOT31
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart van Baalen, and
Tijmen Blankevoort. A white paper on neural network quantization, 2021. URL https://
wtx.w..rrg/abs/210..000
Yurii Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Springer Publishing
Company, Incorporated, edition, 2014. ISBN 1461346916.
Nvidia. Nvidia turing gpu architecture, 2018. URL https::/imagss..rrtt
com/ aem- http://nnzz//Soiitee hut.ppaeecaiiiitttaaaa.aaaa.aaaa
Nvidia. Nvidia ampere ga102 gpu architecture, 2021. URL
pdf
Nvidia. Nvidia blackwell gpu architecture, 2025.
URL https:/ //
com https//www...aa.aoooaooaoaaaa/aa wyt.bb-teebbaaaaalll-grrrrreeeee
Andrei Panferov, Jiale Chen, Soroush Tabesh, Roberto L. Castro, Mahdi Nikdan, and Dan Alistarh.
QuEST: Training accurate LLMs over highly-compressed weights and activation. In Sparsity in
LLMs (SLLM): Deep Dive into Mixuure of Experts, Quantization, Hardware, and Inference, 2025.
URL https: /ttp//eeeeeeeeeeeeeeeeeeee net
forum? www.uoorou.u..o...oooor iid.lddleeegnn
Bita Darvish Rouhani, Nitin Garegrat, Tom Savell, Ankit More, Kyung-Nam Han, Ritchie Zhao, Mathew Hall, Eric Chung Jasmine Klar, Yuan Yu, Michael Schulte, Ralph Wit- tig. Ian Bratt, Nigel Stephens, Jelena Milanovic, John Brothers, Pradeep Dubey, Marius Coreea, Alexander Heinecke, Andres Rodriguez, Martin Langhammer, Summer Deng, Maxim Naumov, Paulius Micikevicius, Michael Siu, and Colin Verrilli. Ocp microscaling for- mats (mx) specification, 2023. URL https: www. http://wwww....p.ttt..e/o/ org//ocumentss/ oop-mi 1oo-microsca crosca ing formats-
spec pd
13

---

Published as a conference paper at ICLR 2026
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Fei- Fei. Imagenet large scale visual recognition challenge. Int. J. Compur. Vision, 115(3)2111-522, doi: 10. 10.1077/111223--15-0881666 URL htt.t::////o
December 2015. ISSN 0920-5691.
00e/10.100770 01777/11163--11 126-1101111000011-----6-
C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27(3):379 423, 1948.
7305. 0b.. www.rr.www...................... wttp://ewwccoo//o/aaaa//....0..0 wttp://.111110/.oo.
doi:
URL https: http:///mmmmmeeelree 1948. t.b 1001338..
Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. Omniquant: Ommidirectionally calibrated quantization for large language models. In The Twelfth International Conference Learning Representations, 2024.
net. (-
Xuan Shen, Zhenglun Kong, Changdi Yang, Zhaoyang Han, Lei Lu, Peiyan Dong, Cheng Lyu, Chih hsiang Li, Xuehang Guo, Zhihao Shu, Wei Niu, Miriam Leeser, Pu Zhao, and Yanzhi Wang. Edgeqat: Entropy and distribution guided quantization-aware training for the accelera- tion of lightweight iims on the edge. CoRR, abs/2402.10787, 2024. BtLLhttps:///ot org 10. 48500/ wrwiv.vv..... 2402. 10787
Philippe Tillet, H. T. Kung, and David Cox. Triton: intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop Machine Learning and Programming Languages, MAPL 2019, pp. 10-19, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 97814503671 1966. doi: URL UtRb:tt:: petttps:/ddi..rgg// 111.11111111311100000 3329973
00.1145/3315508.33999
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers & distillation through attention. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 10347- 10357. PMLR, 18-24 Jul 2021. URL https://prooceedings mlr. prils//1res//119/ Eouvron21a. html
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation
language models, 2023. URL https: wawxiv.vvv.v../avv./aaaaaaaaaavv org.or///.o....... org/abs////eeeooooooeo...aaaaaar 2302. 13971
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention all you need. Luxburg, S. Bengio, H. Wallach, R. Fergus. S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https::/proceedd.ntet neurips......ppp..p....... cc/paper. files/paper/2017 orce44445aaaaeeeeooododoccaaaaaa
Guyon, U. Von
Paper.
Haocheng Xi, CaangHao Li, Jianfei Chen, and Jun Zhu. Training transformers with 4-bit integers.
In Thirly-sevnnth Conference on Neural Information Processing Systems, 2023. URL https:
7/ o/pperevieww www.eeeoorrdddaieeeeeee 1MI
Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren. Learning in the frequency domain. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1737-1746, 2020. doi: 10.1 (009CCVPR2220022220.000
Kohei Yamamoto. Learnable Companding Quantization for Accurate Low-bit Neural Net- works In 2021 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5027-5036. Los Alamitos, CA, USA, June 2021. IEBE Computer Society. doi: 10. 1109/CVPR46437.2001 .0049999 1109/CVPR45437 2021. 00499
00499.
URL httpp:://dat http:////.oo.eeecccpptttteerotos jeeeccnptterrecccieey..o///////
14

---

Published as a conference paper at ICLR 2026
Kaicheng Yang, Xun Zhang, Haotong Qin, Yucheng Lin, Kaisen Yang, Xianglong Yan, and Yulun Zhang. Robuq: Pushing dits to w1.58a2 via robust activation quantization, 2025. URL hetps:
//axxiv.crg/aaaa....... abs 2509. 23582
Ningfeng Yang and Tor M. Aamodt. Improving the straight-through estimator with zeroth-order information. In The Thirt Annual Conference on Neural Information Processing Systems, 2025. URL https: /tttps://ppeneevee... ottp:///www.eteooooooooooooooooo
Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang. Qi- jing Huang, Yida Wang, Michael Mahoney, and Kurt Keutzer. Hawq-v3: Dyadic neural network quantization. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 11875-11886. PMLR, 18-24 Jul 2021. URL https: bettps://pxoceedinng mlr. .tt.sssss./epp..e//ommm......s vr39/yao21a.10 a.taaa21ahha... htmml
Davis Yoshida. Nf4 isn't information theoretically optimal (and that's good), 2023. URL https:
7/ www.iii..r/aabaa...a..a 2006 06965
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 KERNEL IMPLEMENTATION
In this section, we present an example implementation of Algorithm using the Triton (Tillet al. 2019) programming language.
Nvidia GPUs provide extensive support for low- low-precision tensor core operations, allowing expen- sive bigh-precision matrix multiplications to be replaced with low-precision alternatives, leading to higher compute throughput and lower energy consumption. For example, Nvidia Blackwell ar- chitecture (Nvidia 2025) provides tensor core support for the MX FP4(Rouhani al. (023) data type. As another example, the Turing (Nvidia 2018 and Ampere (Nvidia (021)) architectures pro- vide support for the INT4 data type. We list in Table the representable values for both data types.
Binary Representation
Signed INT4
MX FP4
0b0000
0
0
0b0001
0.5
ObOOTO
2
0b0011 1
3
1.5
0b0100
0b0101
5
3
Оь01T0
6
4
0b0111
0b1000
-8
0b1001
-7
-0.5
0b1010
-6
-1
0b1011
-5
-1.5
0b1 100
-4
-2
0b1101
-3
-3
0b1110
-2
0b1111
-1
-6
Table 6: Signed INT4 vs. MX FP4
Table 7 shows the values that BBQ can represent.
Precision
Possible Values of
4,,1,3,21,,,112233
2
-1.5,-..5.0551155
-0.5,0.5
Table 7: Values that BBQ can represent. As shown, for b = 4, BBQ can be encoded as INT4. For
b 3, BBQ can be encoded INT4 or FP4. For b (1,2), BBQ can be encoded FP4.
16

---

Published as a conference paper at ICLR 2026
Since b can be encoded by both INT4 and FP4, present an example triton kernel for BBQ with 3-bit quantization following Algorithm !
@triton.jit
def bbq( ( x_tr,, htp,, rsigma_ptr, qm2_ptı,
ROW_SIZE: consteeex,,
BLOCKSIZE: tl. constexpr=1128
DTYPE: t1 constexxr = "int4"
):
rid - t1 www.proormmmooo iidinidddddiiiiii_
cid = t1. wt..prrrrmmmice -id (axis=1)
xrids rid BLOCKSI2E tl.arange (0, BLOCKSI2E) [:, None]
xcids = cid BLOCKSIZE tl.arange (0, BLOCKSIZE) [None, :]
xids xrids ROW_SIZE xcids
hrids tl. tr.arange (0, BLOCKSI2E) [:, None]
heids = tl.arange (0, BLOCKSI2E) [None, :]
hids hrids BLOCKSIZE heids
load a block of x
= tl. (x.1oad(xxpt) + xids)
# load pre-computed hadamard matrix shared across the entire kernel h t1 toad (t_ptr + hids)
# load reciprocal of sigma rsigma tl load (rsigma_ptr)
hadamard transform v tl.dot (x, h) * rsigma
# accelerated normal CDF and rounding
gmz aormal_
(v, DTYPE)
pack two 4-bit data type into a single byte
shift ar.are (0, 2) [None, None, qmz = te.reshape (qmz, (BLOCXSIZE, BLOCKSIZE // 2, 2) << shift qmz tl tl.xor_sum (qmz, axis=-1, feep_dims=False)
write quantized data memory qmzrids rid * BLOCKSIZE + t1. (0, BLOCKSIZE) I:, None]
qmzcids cid (BLOCKSIZE // 2) t1.arange (0, BLOCKSIZE // 2) [None, :]
qmzids qmzrids * (ROW SIZE // 2) + qmzcids
tl.store (qmz_pt_. qmzids, qmz)
17

---

Published as a conference paper at ICLR 2026
Next, show the definition of function normal wnwdaaldddddddddduuuuddddd round33ii... We pre-compute -- for all possible values i, and use a binary search method to decide which should be the result of quan- tization, and lastly, directly the binary representation of 2 according to Table 6 For 3-bit quantization, the pre-computed values of t --
for E {0. 2, 3, 4, 6. 7), are -000-
1. 1.15034938037600833 -0.6744895501968811 -0.3 0.3186993339443522 0.0, 0.31863933336445522 0,6744897501960888 8, 1.1503493807600083 3-,,-3, 22 -1, 3.3.
The corresponding values of i - 2
are
@triton. jit def normal_edf round_3bil (v: tl.tensor, DТYPE: tl. c1.cosstexpr) : if DTYPE "
qmz = t1.where(
0.0,
wherh
v C. C.674489751166818
th.where (v >- 1.1503493303750083, 0b0011, 0b0010), ti.where (v 0.318699633643355,, 0b0001, 0b0000),
),
tl. where
>= -0. 016644997701968888.8
ti.where (v >= -0.3186393639643375,, 0b1111, 0b1110),
ti.where (v >= -1.1503493803760083, 0b1101, 0b1100).
)oto (t1..int8)
return qmz elif DТYPE "fp4":
qmz = tl. (
v 0.0, t1. eheree
v. >= 0. 0.674897501960818, ti.where (v >= 1.150349380376083. 0b0101, 0b0100),
th.where (v 0.318699833663755,, 0b0010, 0b0000),
s1.where(
>> -0. 00086444977001668818,,
ti.where (v >= -0.3186399639543775,, 0b1010, 0b1100),
tl.where (v -1.1503493803760088 0b1101, 0b1110),
to (oo.iinnt)
return qmz
18

---

Published as a conference paper at ICLR 2026
A.2 EXPLANATION EQUATION
9
In this section, we discuss why Equation 9 is useful in ensuring that elements of x and elements of & have approximately the same average magnitude. We first re-iterate the definitions of a::
HT(z))/a
9 -20-))
(10)
2 25-14
We first assume elements of (x) has approximately the same average magnitude as elements of since the Hadamard matrix is orthogonal. This means elements of v HT(z)/a has an average magnitude that is a times smaller than that of We next remove the floor operation and the zero point for simplicity, arriving at the following simplified equation:
C HT
q (1) - 2
(1I)
2 26-19 ((2b(()
- (v)
We assume the average magnitude of elements of (0) to be 5 times smaller than that of v. In other words, the average magnitude of clements of 2 is approximately Ç"o times smaller than that of Therefore, compensate by setting ((2 1) has approximately the same average magnitude as x.
("a, ensuring elements of ø
To estimate the value Ç*
Equation 9 which expresses 5* the optimal that minimizes the MSE between and ((2 - 1) when v follows a Gaussian distribution. Using Monte- Carlo sampling to estimate the MSE, and gradient descent to solve for the arg min, estimate
Ç" ~ 1.694.
A.3 ABLATION ON EXPONENTIAL MOVING AVERAGE OF 1/a
In this section, we show BBQ and BBQ- achieve identical perplexity.
4-bit
3-bit
2-bit
1-bit
BBQ
25.51
26.55
31.35
49.80
BBQ-Fast
25.40
26.55
31.43
49.72
Table 8: Perplexity of BBQ vs BBQ-Fast, on LLaMA-95M pre-trained with 3 billion C4 tokens.
19

---

Published as a conference paper at ICLR 2026
A.4 ZERO-SHOT RESULTS
In this section, we present the zero-shot evaluation perplexity in Table 9 which corresponds to the models pre-trained in Table
Model
Dataset
Params n+e
Method
Bits WA
Perplexity
LLaMA
wikitext
95M
None
16
56.11
LLaMA
wikitext
95M
BBQ
4
57.87
LLaMA
wikitext
95M
QuEST
4
58.91
LLaMA
wikitext
95M
LSQ
4
61.62
LLaMA
wikitext
95M
BBQ 3
60.18
LLaMA
wikitext
95M
QuEST 3
64.21
LLaMA
wikitext
95M
LSQ 3
68.35
LLaMA
wikitext
95M
BBQ
2
70.19
LLaMA
wikitext
95M
QuEST 2
77.32
LLaMA
wikitext
95M
LSQ
2
80.11
LLaMA
wikitext
95M
BBQ
109.66
LLaMA
wikitext
95M
QuEST
172.77
LLaMA
wikitext
125M
None
16
50.26
LLaMA
wikitext
125M
BBQ
4
50.50
LLaMA
wikitext
125M
QuEST
4
51.31
LLaMA
wikitext
125M
LSQ
4
53.72
LLaMA
wikitext
125M
BBQ
3
50.94
LLaMA
wikitext
125M
QuEST
3
54.37
LLaMA
wikitext
125M
LSQ
3
58.61
LLaMA
wikitext
125M
BBQ
2
60.47
LLaMA
wikitext
125M
QuEST
2
69.45
LLaMA
wikitext
125M
LSQ
2
70.61
LLaMA
wikitext
125M
BBQ
97.34
LLaMA
wikitext
125M
QuEST
180.33
LLaMA
wikitext
200M
None
16
40.42
LLaMA
wikitext
200M
BBQ
4
42.56
LLaMA
wikitext
200M
QuEST
4
41.67
LLaMA
wikitext
200M
LSQ
4
4133
LLaMA
wikitext
200M
BBQ
3
41.82
LLaMA
wikitext
200M
QuEST
3
43.95
LLaMA
wikitext
200M
LSQ
3
426.8
LLaMA
wikitext
200M
BBQ
2
49.53
LLaMA
wikitext
200M
QuEST
2
54.12
LLaMA
wikitext
200M
LSQ
2
208.5
LLaMA
wikitext
200M
BBQ
80.34
LLaMA
wikitext
200M
QuEST
123.19
LLaMA
wikitext
300M
None
16
34.95
LLaMA
wikitext
300M
BBQ
4
38.57
LLaMA
wikitext
300M
QuEST
4
35.26
LLaMA
wikitext
300M
BBQ
3
36.77
LLaMA
wikitext
300M
QuEST 3
37.26
LLaMA
wikitext
300M
BBQ
2
41.24
LLaMA
wikitext
300M
QuEST
2
45.28
Table 9: Zero-shot wikitext perplexity.
20

---

Published as a conference paper at ICLR 2026
A.5 BBQ vs. QUEST
In this section, we illustrate the difference between BBQ and QuEST in Figure 6
BBQ and QuEST share the Hadamard transform ()) and RMS normalization (2) to re- shape the distribution better match the standard Gaussian.
In step 3. QuEST scales by a", shifts by 0.5, and uses the clip function to limit values within a finite range. BBQ uses the + function to limit values within a finite range, while simultaneously maximizing the entropy.
In step 4 both methods uniform quantization quantize data. Both methods ensure the quantized data can be stored in a compute-cffiiien data type (integer).
step 5 BBQ applies linear scaling, while QuEST reverses all of the operations did before (RMS Normalization, scaling by a", and shifting by 0.5), as QuEST is "same- domain" quantizer. BBQ is "cross-domain" quantizer, so BBQ does not reverse its pre- vious operation such
4 shifting by 2 and multiplication by 2 division by etc. Instead, BBQ applies linear scaling to scale its range to be within [->,7] learnable parameter.
where 7 is
In step @, QuEST reverses its Hadamard transform (a unitary operation). BBQ does not reverse the Hadamard transform because it is "cross-domain" quantizer.
HT(@)
00(@ 0.5)
round
clip( a 0.5, -8, 7)
B HT())
= 0//
QUEST
388
-30
@ (b-1)
floor(2"b x (3) 22
(
Figure 6: BBQ QUEST.
21

---

Published as a conference paper at ICLR 2026
1000
Frequency
800
Frequency
600
400
400
200
200
0.0
0.1
0.2
0.3
0.0
0.1
0.2
0.3
MY
|vl
Language model: 2-bit ILLMA-95M.
(b) Vision model: 2-bit DeiT-Tiny.
Figure 7: Histogram of the absolute value of learned in a language model versus vision model.
A.6 EXTENSION TO VISION MODELS
While BBQ is designed for language models, discuss and evaluate potential extension of BBQ to vision models. When we naively apply BBQ to vision models without any modification, we no- tice, as shown in Figure [ a subset of learned y have magnitudes extremely close to O. Since every is assigned to weight channel, if'r is small, weights from the corresponding channel suffers from gradient vanishing. As shown in Figure 7a language models do not suffer from such collapsing. Instead of making learnable parameter and initializing Ç" as discussed in Section 3.4 we propose BBQ variant, BBQ-Visio that instead dynamically calculates the value y as (ar on every forward pass, where ( is constant hyperparameter. BBQ Vision prevents 7 from collapsing to 0, since for to 0, all weights in that channel must also be In addition, BBQ- Vissiiiion still preserves some degree of Icarnability
the network can modify weights/activatioons to indirectly control the value of ar. We compare BBQ- Vision against QuEST and LSQ on DeiT (Touvron et al,
2021) and Resnet (He et al.) 2015) with various sizes and show results in Table TO] We use Ç = 2.45. Our preliminary results show BBQ Vision can consistently outperform QuEST and LSQ.
especially lower precisions (1-bit and 2-bit).
BBQ Vision
QuEST
LSQ
Model
Params
Bits WA
Ent
TL
VA
Ent
TL
VA
Ent
TL
VA
DEIT-T
5M
4
3.85
2.84
74.68
3.61
2.91
74.3
3.67
2.88
74.88
DEIT-T
5M
2.93
2.89
74.24
2.78
2.98
72.2
2.81
3.01
71.36
DEIT-T
5M
2
1.94
3.05
70.16
1.93
3.18
67.24
1.90
3.23
65.32
DEIT-T
5M
1.00
3.52
53.54
1.00
3.67
47.55
DEIT-S
20M
4
3.82
2.53
79.46
3.61
2.51
80.00
3.57
2.52
80.04
DEIT-S
20M
3
2.91
2.48
80.88
2.78
2.62
79.34
2.74
2.64
78.96
DEIT-S
20M
2
1.93
2.76
77.02
1.93
2.82
76.18
1.83
2.87
74.42
DEIT-S
20M
1.00
3.18
66.16
1.00
3.37
59.94
Resnet- 10
5M
1.00
3.37
1.00
3.40
Resnet-18 18
11M
1
1.00
3.12
73.13
1.00
3.14
72.28
Table 10: Entropy (Ent), Training Cross Entropy Loss (TL), and Validation top-1 Accuracy (VA) of
vision models pretrained Imagenet- 100 (Chun-Hsiao Yehl 2022
Russakovsky et al. 2015).
22

---

Published as a conference paper at ICLR 2026
A.7 VALIDATION LOsS VS. TRAINING PROGRESS
In this section we present validation loss vs. training iteration curves for all of our experiments in Section[4 For visualization purposes, only show the validation loss for the last 90% iterations, hide the first 10% iterations when learning rate is warming methods that diverged. For all figures, the y-axis is the validation cross entropy loss, and x-axis is iterations (batches). For all experiments, we quantize both weights and activations of linear layers to b bits. addition, for all QuEST experiments, we use trust factor of T a² /(20 1) as discuss in QuEST (Panferov et al. 2025
and don't t show loss curves for
val/loss
LLaMA-95M-LSQ-
LLaMA-95MM 95M-BBQ LLaMA-95M 45M-QUEST-4
3.8
3.6
3.4
iter
2k
4k
6k
8k
10k
Figure 8: LLaMA-95M (4-bit) pre-trained 3 billion C4 tokens (batched over 12 thousand itera- tions). LSQ is green, QuEST is gray, and BBQ is blue
val/loss
LLaMA-95M-LSQ-
LLaMA-95M-BBBL
4
3.8
3.6
3.4
iter
2k
4k
6k
8k
10k
Figure 9: LLaMA-95M (3-bit) pre-trained on 3 billion C4 tokens (batched over 12 thousand itera- tions). LSQ pink, QuEST is red, and BBQ is green.
23

---

Published as a conference paper at ICLR 2026
val/loss
-LLaMA-95M-LQQ-2 LLaMA-95M-BBQ- LLaMA-95M LLaMA-95M-QUEST-2
4
3.8
3.6
iter
2k
4k
6k
8k
10k
Figure 10: LLaMA-95M (2-bit) pre-trained on 3 billion C4 tokens (batched over 12 thousand itera-
tions). LSQ yellow, QuEST is purple, and BBQ orange.
val/loss
LLaMA-955 BBQ-1
LLaMA- 11L-999M--QUUUEEE QuEST-1
4.6
4.4
4.2
4
iter
2k
4k
6k
8k
10k
Figure 11: LLaMA-95M (1-bit) pre-trained on 3 billion C4 tokens (batched over 12 thousand itera- tions). QuEST is purple and BBQ is orange.
24

---

Published as a conference paper at ICLR 2026
val/loss
LLaMA-125M-LSQ-4 LLaMA-125M-BBAA LLaMA-125M 1-----ESS-
3.6
3.5
3.4
3.3
3.2
iter
3.1
5k
10k
15k
Figure 12: LLaMA-125M (4-bit) pre-trained on 5 billion C4 tokens (batched over 20 thousand iterations). LSQ is red, QuEST is brown, and BBQ is pink.
val/loss
LLLMA-125MMLLS 125M-LS--3
11AMA-125M-BBB
LLaMA- 11MM--25M--M QuEST-3
3.7
3.6
3.5
3.4
3.3
3.2
iter
5k
10k
15k
Figure 13: LLLMA-125M (3-bit) pre-trained on 5 billion C4 tokens (batched over 20 thousand iterations). LSQ is green, QuEST is gray, and BBQ is blue.
25

---

Published as a conference paper at ICLR 2026
val/loss
LLaMA-125M-LSQ- LLaMA-125M-BBQ2 LLaMA-125M LLAMA-125M-QUEST-2
3.8
3.7
3.6
3.5
3.4
iter
5k
10k
15k
Figure 14: LLaMA-125M (2-bit) pre-trained on 5 billion C4 tokens (batched over 20 thousand iterations). LSQ is purple, QuEST orange, and BBQ green.
val/loss
LLaMA-125M 125M-BBQ-1
LLaMA 1-E-1125MM----E- QuEST-1
4.3
4.2
4.1
4
3.9
iter
3.8
5k
10k
15k
Figure 15: ILLMAA-15M (1-bit) pre-trained on 5 billion C4 tokens (batched over 20 thousand iterations). QuEST is pink and BBQ is orange.
26

---

Published as a conference paper at ICLR 2026
val/loss
LLaMA-200M-BBQ- LLaMA-20OM-QUEST
3.3
3.2
3.1
3
iter
5k
10k
15k
20k
25k
30k
35k
Figure 16: LLaMA-200M (4-bit) pre-trained on 10 billion C4 tokens (batched over 40 thousand iterations). QuEST is green and BBQ brown.
val/loss
LLaMA- 200M-BBQ-3 LLaMA-2000 3-MM-QUEST-
3.4
3.3
3.2
3.1
3
iter
5k
10k
15k
20k
25k
30k
35k
Figure 17: LLLMA-200M (3-bit) pre-trained on 10 billion C4 tokens (batched over 40 thousand iterations). QuEST is pink and BBQ is gray.
27

---

Published as a conference paper at ICLR 2026
val/loss
LLaMA-200M-BBQ- LLaMA-200M-QUEST1
3.8
3.6
3,4
3.2
iter
10k
20k
30k
Figure 18: LLaMA-200M (2-bit) pre-trained on 10 billion C4 tokens (batched over 40 thousand iterations). QuEST is orange and BBQ is pink.
val/loss
LLaMA-200M 200M-BBQ-1- LLaMA-200M QQEST-1
4
3.9
3.8
3.7
iter
5k
10k
15k
20k
25k
30k
35k
Figure 19: LLLMA-200M (1-bit) pre-trained on 10 billion C4 tokens (batched over 40 thousand iterations). QuEST is green and BBQ is brown.
28

---

Published as a conference paper at ICLR 2026
val/loss
LLaMA-300M-BBQ- LLaMA-300M-QUESTT.
3.2
3.1
3
2.9
2.8
iter
10k
20k
30k
40k
50k
60k
70k
Figure 20: LLaMA-300M (4-bit) pre-trained on 20 billion C4 tokens (batched over 80 thousand iterations). QuEST is green and BBQ brown.
val/loss
LLaMA-3000 300M-BBQ-3 LLaMA-3000 3000-QUEST-3
3.2
3.1
3
2.9
iter
10k
20k
30k
40k
50k
60k
70k
Figure 21: ILLMAA300M (3-bit) pre-trained on 20 billion C4 tokens (batched over 80 thousand iterations). QuEST is orange and BBQ is green.
29

---

Published as a conference paper at ICLR 2026
val/loss
D5MA-300M-QUESTE. LLaMA-300M-BBQ-
3.3
3.2
3.1
3
iter
10k
20k
30k
40k
50k
60k
70k
Figure 22: LLaMA-300M (2-bit) pre-trained on 20 billion C4 tokens (batched over 80 thousand iterations). QuEST is green and BBQ red.
30


Reviews:
[{"review_id": "O6USuDieTb", "reviewer": "Reviewer_24io", "summary": "The paper proposes to improve LLM quantized training by transforming the input to enable desired properties (entropy) in the network layers.  The quantization method aims to utilize all of the quantized levels equally often. The input quantization consists of seven steps and includes Hadamard transform, RMS normalization, probability integral transform, and so on. Even though no formal proof of equal utilization has been provided, empirical evidence shows that entropy increases compared to SOTA. Empirical evidence also indicates that perplexity is decreased, resulting in improved performance.", "questions": "1. Overall, the paper is acceptable to me. I am curious to know if there is a straightforward way to achieve something similar with non-transformer models, such as CNNs.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes to transform the input for improved performance. This idea is novel and does lead to better-performing LLMs.", "weaknesses": "1. It is not known if just transforming the input would also lead to deeper layers, also preserving the \"utilize all of the quantized levels equally often\" property. Though the transformer layer norm would bias it to do just that. So it's not that big of a problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SHcu1cGPy7", "reviewer": "Reviewer_qRhd", "summary": "This paper introduces BBQ (Bell Box Quantization), an information-theoretically optimal (ITO) quantization method for quantization-aware pre-training (QAPT) of large language models. BBQ builds on prior work, such as QuEST and NormalFloat, by introducing a Gaussian CDF on the normalised Hadamard transform before applying a learnable bit-dependent scaling, resulting in a quantizer that is both ITO- and compute-efficient. The key idea is that quantization can occur in one domain while producing outputs in a different, hardware-efficient domain, thus preserving information content without losing computational advantages.\n\nThe authors conduct experiments on LLaMA models of various sizes, demonstrating improvements in quantized weight entropy and C4 perplexity over QuEST and LSQ. The proposed quantizer also supports efficient implementations on emerging low-precision formats, such as MXFP4.", "questions": "* Table 1: How is MXFP4 integrated into the method, given that MXFP4 is a dynamic quantisation scheme with an E8M0 scale and FP4 data?\n* line 281: A smoother function can be: any experiments or reference to support this statement?\n* Table 3: Please provide a description of the heads in the caption\n* Table 3: It would be easier to read the table if the precision were presented as the difference from the full-precision\n* The argument that quantised weight entropy is a good proxy for the final ppl is fundamental to the results section. However, there are no ablation studies or experiments to validate this hypothesis.\n* Please add legends to all figures with training curves, cause they are hard to read.\n* Section 4.2 requires more work: the latency overhead of the quantization function should be given as a fraction of the total latency. I would recommend showing the median latency over several runs for the full LLama models and using profiling to illustrate the proportion of the quantization kernel. In addition, it would be nice to quantify the total speed-up of the method compared to NormalFloat and then the full-precision baseline in a table\n* A competing QAT method: Learnable Companding Quantization for Accurate Low-bit Neural Networks by Kohei Yamamoto introduces a similar concept where the transformation function is learnt. Although it is an older paper, it follows a similar logic. Have yοu considered comparing to that method too? They have awe-inspiring results for ResNets \n* Appendix A2: From the main text, I understood that EMA of the reciprocal of $\\sigma$ is part of the method. What do you compare against here? \n* Appendix A3: I don’t think that adding all these validation losses adds much value to the paper. Expanding your results section to include more architectures and zero-shot results would be more impactful", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The authors correctly note that existing QAPT methods are limited in their representation capacity, as they implicitly constrain the entropy of the quantised weights. They ground this hypothesis clearly and use it to motivate the development of the first information-theoretically optimal (ITO) quantizer for QAPT. The resulting method, BBQ, is conceptually straightforward yet effective, consistently improving over strong baselines such as QuEST and LSQ across multiple bit widths.\n\nA key strength of the paper is its demonstration that introducing the Gaussian CDF transformation to achieve ITO quantization does not incur additional computational overhead, owing to an efficient implementation that maintains compatibility with low-precision arithmetic. \nIt is also interesting to see how the authors explore and integrate emerging numerical formats, such as MXFP, highlighting the method’s forward-looking relevance to upcoming GPU architectures.", "weaknesses": "In general, the premise of the paper is sound but requires significant work in presentation, ablations and experimental results to qualify for the conference: \n\n* The experimentation section is lacking. I would like to see zero-shot results and one more family of models, rather than just increasing the size of the same architecture. Please include a comparison to other SOTA methods, such as ParetoQ.\n\n* From a methodological standpoint, the contribution feels incremental relative to QuEST: most of the quantization pipeline (Hadamard transform, normalisation, uniform rounding) is directly inherited, and the main novelty — applying a Gaussian CDF and learning a scaling factor — would benefit from deeper ablations against QuEST’s corresponding components. This would make it more straightforward to determine how much of the gain comes from the CDF transformation itself.\n\n* More results and information are required regarding inference speed-up (see questions below)\n\n* The presentation of the paper has issues (see details in the question below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z1UUcB9J9t", "reviewer": "Reviewer_BCbT", "summary": "This paper presents Bell Box Quantization (BBQ), a novel quantization method that aims to achieve information-theoretically optimal (ITO) quantization while remaining compute-efficient for modern hardware. The method introduces a seven-step quantization pipeline involving the Hadamard Transform, RMS Normalization, Gaussian CDF-based Probability Integral Transform, and precision-scaled quantization operations. BBQ is applied in Quantization-Aware Pre-Training (QAPT) settings, targeting low-precision training (1–4 bits) for large-scale models such as LLaMA. Experimental results show that BBQ outperforms existing quantization schemes like LSQ and QuEST in both entropy (capacity utilization) and perplexity (model quality), while maintaining hardware efficiency. The paper further demonstrates that entropy serves as a strong proxy for quantized model quality.\n\nThe paper presents a technically sound and conceptually novel quantization framework that advances the state of low-bit quantization for large-scale models. However, its current limitations in adaptability and architecture diversity need to be addressed in future work.\n\nRecommendation: weak accept (with added generalization experiments and ablation analysis).", "questions": "- section 3.4 : Uniform quantization and unsigned to signed conversion : Does it make any difference for ReLU based models where output activations are always positive and unsigned values are preferred for better accuracy. ? I support this sentence \"we subsequently convert the positive on data to symmetric data by subtracting 2b−1 and the hyperparameter zero point z,\" is to take care of ReLU activation outputs ? \n*  How well does BBQ perform on architectures beyond transformers, such as CNNs or Vision transformers?\n* Can the authors provide results isolating the contribution of each step (Hadamard Transform, Φ, RMS normalization, etc.) to the final accuracy and entropy gain?\n* Does BBQ introduce any gradient instability during early training, especially in extremely low-precision (1–2 bit) setups?\n* Could the authors discuss whether the binary search for Φ−1 values introduces any measurable latency overhead in hardware implementations?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The introduction of ITO-based quantization combined with hardware-compatible output domains is a well-motivated and original contribution to quantized training research.\n* The step-by-step quantization process (Hadamard Transform → Gaussian CDF → Uniform Quantization → Scaling) is well explained and systematically justified.\n* Across multiple model sizes and bit-widths, BBQ consistently achieves lower perplexity and higher entropy compared to LSQ and QuEST, validating the proposed method.\n* Using entropy as a measurable proxy for model capacity utilization is a meaningful contribution that provides interpretability beyond standard performance metrics.\n* The profiling experiments demonstrate that BBQ can achieve real-world speedups on hardware supporting FP4, showing its deployment potential for large models.", "weaknesses": "- section 2.3, \"learning is domain agnostic\" : the two example to support this is very simplistic assumption. For eg. rotated, cropped and color jitter image is still image and in the same domain, images transformed to frequency domain is still an image but encoded in different format. In both cases, domain is still image processing. Autoencoder is a bit closer but they can't be used to train cross domain models. Authors are suggested to add more concrete examples and possibly quantification of this assumption\nLimited scope of evaluation: Experiments are conducted exclusively on LLaMA-based architectures; it remains unclear whether BBQ generalizes to convolutional or vision transformer models.\n* Since BBQ operates in separate input/output domains and does not minimize reconstruction error, it may not perform well in fine-tuning or post-training quantization scenarios — limiting its versatility.\n* Lack of ablation analysis: The impact of individual steps (e.g., Hadamard Transform, Φ smoothness, or γ initialization) is not quantitatively analyzed, leaving unclear which components are most critical.\n* Implementation details underexplored: Although latency improvements are discussed, comparisons with more hardware-optimized kernels/different GPU generations  or newer architectures like NPUs could provide stronger evidence of scalability.\n* No comparison to information-theoretic quantizers beyond LSQ/QuEST: Including recent ITO-based methods (e.g., [NF4](https://chatgpt.com/c/690279a0-9b50-8320-a3bc-93decef608ce#:~:text=neural%20network%20weights.-,arXiv,%2B2,-FP4%20(4%2Dbit), [FP4](https://arxiv.org/html/2501.17116v1?utm_source=chatgpt.com), or adaptive quantizers) would strengthen the empirical claim of superiority.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LKf7Oqv5oL", "reviewer": "Reviewer_VA2y", "summary": "This paper proposes BBQ, a novel quantization method for low-bit training of large language models. The authors improve the quantization process via adaptive scaling factors and demonstrate its performance on the LLaMA model under 1-4 bit quantization, comparing it with QuEST/LSQ methods.", "questions": "(1) The authors mention that \"LSQ does not support 1-bit quantization and LSQ diverges when n+ e= 300M\". Does BBQ also face similar limitations? Will BBQ remain effective for models with larger parameter scales (e.g., >300M)?  \n(2) The paper states that \"each experiment for n+ e= 300M is conducted on one Nvidia A100 80GB and lasts for 3.5 days\". For larger-scale models (e.g., 1B+ parameters), will the computational cost of BBQ increase significantly? Is it still practically viable?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "(1) Propose the BBQ quantization method, which optimizes low-bit quantization through adaptive scaling factors  \n(2) Systematically evaluate quantization performance across different bit widths (1-4 bit) on the LLaMA model", "weaknesses": "(1) Limited experimental scale and diversity: The method is tested only on LLaMA, without validation on other mainstream architectures (e.g., GPT, Bloom), making it hard to confirm its generalizability  \n(2) Insufficient theoretical analysis: No in-depth explanation is provided for why BBQ performs better at certain bit widths (e.g., 3-bit), relying solely on experimental observations without theoretical support", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)