Highly Efficient and Effective LLMs with Multi-Boolean Architectures
Authors: Ba-Hien Tran
Van Minh Nguyen
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Boolean neural networks
Primary area: foundation or frontier models, including LLMs
TL;DR: A novel multi-Boolean framework for low-bit LLMs

Abstract: Weight binarization has emerged as a promising strategy to reduce the complexity of large language models (LLMs). Existing approaches fall into post-training binarization, which is simple but causes severe performance loss, and training-aware methods, which depend on full-precision latent weights, adding complexity and limiting efficiency. We propose a novel framework that represents LLMs with multi-kernel Boolean parameters and, for the first time, enables direct finetuning LMMs in the Boolean domain, eliminating the need for latent weights. This enhances representational capacity and dramatically reduces complexity during both finetuning and inference. Extensive experiments across diverse LLMs show our method outperforms recent ultra low-bit quantization and binarization techniques.

Published as a conference paper at ICLR 2026
HIGHLY EFFICIENT AND EFFECTIVE LLMs WITH MULTI-BOOLEAN ARCHITECTURES
Ba-Hien Tran & Van Minh Nguyen
Huawei Paris Research Center
Paris, France ba . .hien. trran Stra@@uuuwwwr com (corresponding author)
ABSTRACT
Weight binarization has emerged as a promising strategy reduce the complexity of large language models (LLMs). Existing approaches fall into post-training bina- rization, which simple but causes severe performance loss, and training-awe aware methods, which depend on full-precisin latent weights, adding complexity and limiting efficiency. We propose a novel framework that represents LLMs with multi-kernel Boolean parameters and, for the first time, enables direct finetuning LMMMs in the Boolcan domain, climinating the need for latent weights. This en- hances representational capacity and dramatically reduces complexity during both finetuning and inference. Extensive experiments across diverse LLMs show our method outperforms recent ultra low-bit quantization and binarization techniques.
INTRODUCTION
Large language models (Brown al., 2020; Touvron al., 2023a; Liu et al., 2024a) have demonstrated unprecedented capabilities, largely due to the continuous growth in both model and dataset sizes. A key area of focus in optimizing these models is lower- precision computation, which offers substantial benefits in terms of memory and computational efficiency. One prominent approach to achieving this is through the quantization of weight parameters, which reduces the model size by lowering the precision of the weight values. Recent studies on scaling laws (Dettmers & Zettlemoyer, 2023; Kumar et al., 2025) have highlighted the potential of using low-precision techniques for large language models (LLMS).
Perplexity (+)
40
25.
15-
0.0
5.0
7.5
Model Size (GB)
Figure 1: Finetuning OPT mod- els (Zhang et al., 2022) using our Boolean kernels pared to OPTQ (Frantar et al., 2023) ( ), which quantizes the models to 3 bits, and the FP16 on the C4 dataset.
Binarization represents one of the most extreme forms of quantiza- tion for LLMS. While significant progress has been made, challenges remain (Yuan et al., 2024; Huang et al., 2024; Li et al., 2025). Even with advanced techniques like Quantization-Awarn which fine-tunes the model extensively after binarization (Xu et al., 2024; Jo et al., 2024), or trains it from scratch (Wang et al., 2023), performance still lags behind that of full- mill-precision (FP) models. This performance gap be attributed to the limited representation capacity of binary weights and the heavy reliance on FP latent weights for binarization. This reliance not only makes the approach computationally expensive but also suboptimal, as it requires gradient approximation. Meanwhile, recent advances in 4-bit quantization have achieved remarkable com- pression with minimal accuracy loss, but further compression or applying these methods to smaller models has yielded unsatisfactory results (Frantar et al., 2023; Lin et al., 2024).
com-
Training (QAT),
haseline
In this paper, we aim to push the boundary of low-precision LLMS by proposing novel method named Multiple Boolean Kernels (MBOK). We extend the work in Nguyen et (2024), which proposes training neural networks with native Boolean weights directly in the Boolean domain, However, effectively applying this approach to LLMS remains a key challenge. particular, our contributions are:
We propose the framework MBOK, which employs multiple Boolean kernels, each using distinct Boolean weights ($ 4.2). This allows for flexibly representing LLMS with low bits, while approaching to FP performance with minimal both finetuning and inference cost. The Boolean weights are directly trained in Boolean domain, avoiding the need for FP latent weights and gradient approximations.

---

Published as a conference paper at ICLR 2026
We propose novel successive method that effectively transfers knowledge from FP LLM into the Boolean model ($ 4.3), followed by further fine-tuning using knowledge distillation ($ 4.3.2).
We introduce a method for automatically allocating the number of kernels for each weight ($ 5), supporting any average bit-widthh, including fractional values.
We provide comprehensive empirical analysis and benchmarks, demonstrating our method's superior performance over recent binarization and quantization approaches (see & 6) with much lower memory and computational overhead. For example. Fig. shows that our method achieves the best accuracy-compresson trade-off, outperforming FP and cxisting quantization techniques.
RELATED WORKS
LLMs quantization. Quantization techniques are commonly used to reduce the memory and latency of LLMS. They fall into two categories: QAT, which involves retraining or finetuning in quantized form, and Post- Training Quantization (PTQ), which can be applied directly without retraining. Due to the difficulty of retraining such large models, most work focuses PTQ (Frantar al., 2023; Sheng al., 2023; Lin et al., 2024; Lee al., 2024), though recent efforts also explore QAT via data-free free methods (LLM-QAT (Liu et al., 2024c)), or parameter-efficien fine-tuning like LoRA (Dettmers et al., 2023). promient PTQ method OPTQ (Frantar et al., 2023), which introduces one-shot low-bit weight quantization using approximate second-order information. Follow-up work refines this by addressing outliers (Kim et al., 2024; Dettmers et al., 2024), accounting for activation effects (Lin et al., 2024; Lee al., 2024), and optimizing quantization parameters (OmniQuant (Shao et al., 2024)). However, effective LLMS quantization is still challenging (Xu et al., 2025).
Binarization. This represents the most extreme form of quantization, typically using the sign(.) (STE) (Bengio et al., 2013). Early
function with gradients estimated via the straighi-through--stime work focused small Transformer models (Vaswani et al., 2017) trained or fine-tuned on labeled data (Bai et al., 2021; Qin et al., 2022; Liu et al., 2022; 2023). Recent efforts have extended binarization LLMS. Methods like BiLLM (Huang al., 2024), PB-LLM (Yuan et al., 2024), STBLLM (Dong et al., 2025), and ARB-LLM (Li et al., 2025) adopt hybrid PTQ approaches, binarizing con-salient weights while using higher precision for important ones, with calibration data used to adjust scaling factors. BitStack (Wang et al., 2025), QBB (Bulat et al., 2024), DB-LLM (Chen et al., 2024) further improve this with multiple binary bases, either through a training-free method or via knowledge distillation. In contrast, BitNet (Wang et al., 2023) replaces linear layers with custom 1-bit weight structure, BitLinear, and trains the model from scratch. One (Xu et al., 2024), which decomposes weights into 1-bit components and scaling vectors for QAT, further enhanced by MoS (Jo et al., 2024) using a mixture of scalings. Despite progress, these methods remain costly due to their dependence on FP latent weights during training. Table summarizes the key characteristics of these methods in comparison to ours.
Table 1: A summary SOTA binarization methods LLMS compared to our method.
Train Method from Scratch
Post-training Binarization
Finetune from Calibration
Weight Update
Higher: hit Binury Bases Salient Weights
Multiple
FP Model
Data
(Wang al., 2023)
NA
BiLLM (Huang 2024))
x
NA
PB-LLM (Yuan at, 2024) 2024)
NA NA NA NA
STRLIM (Dong al., 2025)
X
ARB-LLM (Li at, 2025)
/
BitStcck (Wang al., 2025)
X
DB-LIM (Chen el., 2024)
/
FP Iasens weights
QвB (Bulat et 2024)
2024)
FP latent- weights
OneBit (Xu al., 2024)
X
FP leteht-wigghts
MoS (Jo et al., 2024)
FP Jatcnt- weights
MBOK [Ours]
X
Native Boolena weights
PRELIMINARIES
We use standard notation for vectors (a), matrices (A), and scalars (a). The i-th element of vector a is a(j) and the element at the i-th row and j-th column of matrix A is A Aall The symbol @ denotes element-wise multiplication, with broadcasting if needed.
Notations.
3.1 PITFALLS OF FULL-PRECISSON LATENT WEIGHTS FOR BINARIZATION
Binarization an effective technique for reducing both the size and computation deep learning models by converting high-precision weight parameters into 1-bit values (Hubara al., 2016;
2

---

Published as a conference paper at ICLR 2026
Courbariaux et al., 2015; Rastegari et al., 2016). For linear layer, Y XWEP FP + b, where XFP E RK*" the imput data, and W E Rm** with the input size n and output size m, and b E R" are the FP weights and bias. Binarization results in Y = a XW bin + b, with Win = sign (Fr) and a as a scaling factor (e.g.. a [Wre (Rastegari et al., 2016).
During training, the FP weights must be retained for learning the binarized weights. In vanilla gradient descent, binarized weights are updated as W bin = sign(WEr -7- re where is the learning rate and Gw is the gradient of the FP weights. This leads high memory usage, especially with optimizers like Adam (Kingma & Ba, 2015). which require storing two additional FP momenta for each parameter. Moreover, the gradient approximation for binarized weights often uses a differentiable proxy, like the STE (Bengio et al., 2013), but this introduces performance drops due proxy gradient noise. This noise can cause oscillations and instability during training.
3.2 NATIVE BOOLEAN FRAMEWORK FOR NEURAL NETWORKS
To address the issues associated with latent-weight-based approaches, Nguyen et al. (2024) recently proposed principled framework for directly training Boolean neural networks in the Boolean domain. Consider the I-th Boolean linear layer; in the forward pass, the output of the next layer is defined as:
( [RJ bi
b
+
L(X
( [K:)) (1.)))
1 i m,
(I)
where & denotes the sample index in the batch, and is a logic gate such as and, or, xor, or xnor; Hereafter, for clarity, we consider T. = xnor concrete example. The weights w(') (!)
[sa] are Boolean
values {TRUE, FALSE] or {-1, +1). as typically used in practical implementations.
The logic gate L can be extended to handle mixed-type data. In this paper, we focus on the case where the input data is realevalued, and the weights are Boolean. Specifically, for an input element x E R, define bboot TRUE
2 0, and Pboot FALSE + O, and [xr] its magnitude. The logic operation between real input
TR and Boolean weight E B is defined xnor(w, 2x) _ such that Sbool xnor snor(WWool-- X) and |s| = |z|]
Backward pass. This layer receives the backpropagation signal from the downstream layer. Specif- ically,
2 A
Sc denotes the variation of the loss function L w.r.t. the output at layer (.
To optimize the Boolean weights, we need to compute the corresponding loss signal, denoted as (1)
SC which aggregated over the batch dimension as:
[55]
aw [F12]
b
1QQ [kitJ] = TRUE) Q (Riia)
Σ
(1) 1(Q
= FALSE) Q Peia
(2)
where xnor(Z) (2)
(es) (R.)). and 1-)) is the indicator function. The backpropagation signal be computed in similar manner.
(1) (1)
[A...K] for the upstream layer, P( A 6C
[k.j] 5x
Boolean optimizer. Given the loss signal, the rule for updating the Boolean weight w!!) to [Kj]
minimize the loss function is as w!() [1]]
[J]] ( if xnoo(QQ (1) w(l) = TRUE. Based on this
[A.) [J]]
update rule, we can develop an optimizer that accumulates the signal Q (t)
[a.j] over training iterations.
Specifically, let w(),, (f).
[Fa] denotes the weight at iteration and ())
[.j] represents its accumulator,
initialized as (1). The update rule for the accumulator is then defined as:
b.j]
(),t- [J
< 3 3'M (0,t (PJ)
(0)t + 1/ [c.]
where 17 is the accumulation factor acting as a learning rate, and 3" is a regularizing factor that reflects the system's 's state at time t. In our work, use brain plasticity (Fuchs et al., 2014) and Hebbian theory (Hebb, 2005) to adaptively set Bt. We encourage the reader check Appendix A for details.
Remarks on complexity and applicability to LLMs. This Boolean framework optimizes Boolean parameters ((())
[R.]] directly in the Boolean space, eliminating the need for FP latent weights. As shown in Eq. 3, the Boolean optimizer is more lightweight than common LLM optimizers like Adam,
3

---

Published as a conference paper at ICLR 2026
requiring only FP momentum per parameter. This reduces both training and inference complexity and avoids gradient approximation induced from STE. As shown Proposition A. in Appendix, xnor(w, s) w x s, mathematically enabling direct application to existing linear algebra operations. Practically, native logic operations are much faster than multiplication.
MULTIPLE BOOLEAN KERNELS
4.1 BOOLEAN REFORMULATION FOR LINEAR LAYERS
LLMS (Brown et al., 2020) are mostly based on the Transformer architecture (Vaswani al., 2017), in which linear layers are the core elements. Inpsired by Xu et al. (2024), employ www--aainleeeeddeeee decomposition (SVID) such that an FP input matrix W E m of linear layers decomposed into Boolean matrix W hool sign W) and two FP
bool
fignt extraction of W punll1 approx. of W
Figure 2: Illustration of SVID.
vectors Sin and Sout- Precisely, let [W] be the element-wise absolute value of W, write [
UEV its singular value decomposition (SVD) (Beltrami, 1990). Using rank- I approximation of
[W], Sin and Sout are given as: Sin = /on V and Sout V
[1.j.. Then, the input matrix is
approximated as W bool @ [ == bool O ( This procedure is illustrated in Fig. 2.
Proposition 4.1. (Xu et al., 2024) For W E immxn, write W
UEV its SVD. Let Vōi (
and b Vi.) With the notations as described above, we have:
W W Whool bool
Sout
W - ab
(4)
Remark 4.2. Proposition 4.1 re-states Proposition 2 of Xu et al. (2024) with its precise assumption of vectors and which is necessary for its proof provided in Appendix therein.
Proposition 4.1 shows that using W bool together with value matrix approximation is better than a direct rank-1 approximation of W terms of Frobenius-orrm. This emphasizes the important role of W bool in approximating the original FP matrix. Moreover, our following Proposition 4.3 shows that the SVID approximation as described above is optimal for approximating the original matrix W bool- Proposition 4.3. For W E Rmxn and the notations described above, we have:
W
W bool @ S < W Wbool O cd
F² Vc E Ri AA E Rux1 (5)
The proof is given in Appendix D.3. The linear layer can be then reformulated (Xu al., 2024):
XWT FP ~ [[X @ ( W Woool] @ S
(6)
4.2 ENHANCED EXPRESSIVITY WITH MULTIPLE BOOLEAN KERNELS
We have shown that SVID provides a good approx- imation of the original weights, its expressivity can be still limited to capture well the original FP parameters of complicated models, which were trained on large-scale datasets over extended pe- riods of time. To overcome this limitation, we propose the use of a multi-Boolean kemel struc- ture for the weights. Specifically, we employ K kernels, where each kernel utilizes distinct Boolean weights and scaling factors, to better represent the original weight parameters. This leads to the approximation: WFP ~ W appprox
INT
will
Wild
fnoes
inpont
Figure The computation a linear layer approxi- mated using multi kernels of Boolean.
Σ [A]
bool @ (s [ ). The computation of a linear layer can then be approximated as follows (see Fig. for an illustration):
[k]]]
K
X FP
X
[A] Sin
W
bool
Sout
T
(7)
4

---

Published as a conference paper at ICLR 2026
Here, the computational costs associated with the FP scaling factors, Sin and Sout, small because they only involve dlement-wise multiplications. The dominant computational cost arises from the matrix multiplication between the scaled imput data, X Sin, and the weights. However, thanks to the use of Boolean weights, the complexity is significantly reduced, as these multiplications can be replaced by additions. Moreover, are required to achieve a reasonable result. Additionally, we find that, after the successive extraction process from the model ($ 4.3.1), we only need to train the Boolean weights for the last kernel and the scaling factors, further significantly reducing the overall complexity.
will demonstrate in 6.1.1, only small number of kernels
4.3 EFFECTIVE KNOWLEDGE TRANSFER INTO BOOLEAN MODELS
We have introduced our proposed multi-Boolean kernel structure for effectively representing the linear layers of LLMS. In this section, we outline the process for transferring knowledge from source FP model to Boolean model. This process consists of two steps: (1) data-free initialization to maximize information retention from the source, and (2) data-dependent finetuning. where the Boolean model is further trained on target dataset with guidance from the FP model.
4.3. SUCCESSIVE EXTRACTION USING SVID
For each linear layer, to initialize the values of the Boolean weights and scaling factors for all kernels, we successively apply SVID the given weights. The goal here to further proceed sVID process to approximate the residual error introduced by the previous step. Specifically, after each step of decomposing the weight matrix using SVID, we obtain a residual matrix, which is defined as:
W PES
W!!]
input
W]]k] bool
Sout in
[A]
(8)
Here, W[A]] is the residual matrix, and W bool out
[k] and si are the extracted parameters for the k-th kernel, while w!lki input represents the input FP matrix for step k. For the first step, this is the original weight matrix, and for subsequent steps, it is the residual matrix obtained from the previous step.
Fig. 4 illustrate this process. Although using multiple kernels effectively captures the original weight matrix, residual eтror still remains at the end of the process. While this residual error is small, it can accumulate as it propagates through the layers, finally leading to predictions that diverge from those of the original FP model. To address this issue, it is necessary further finetune the resulting model to compensate for these errors and make it better suited to target task. We will discuss this in 6.1.2. In the following section, will introduce knowledge distillation to achieve this goal.
P
(sucooesivvlly)
(successively)
WFF
W
W
W
boot
1st lerrel extraction
2nd kernel extraction
K-th kernel extraction
Figure 4: Illustration of successive extractions of Boolean kernels from given weight matrix.
4.3.2 FINETUNING WITH KNOWLEDGE DISTILLATION
Knowledge distillation (KD) (Hinton et al., 2015) trains a student network to mimic a more powerful teacher, usually with greater efficiency. The student learns from the teacher's output distribution and/or intermediate states as "soft targets". Here, the FP model the teacher and the Boolean model the student. Specifically, the output probability distribution of an LLM for token X is:
exp(X ()/T)
P(X (JiT)
Σn Nv
X ()
where Nv is the vocabulary size and t is the softmax temperature. The logit-based knowledge distillation (KD) loss across the sequence of all output tokens is defined as follows:
5

---

Published as a conference paper at ICLR 2026
loggit
Diogits (PFp(X(3) T), P (X []:
(10)
2=1
Here, (F and Pbool PX
(]: denote the distributions over the j-th token from the FP and Boolean models, respectively, with L as the sequence length. We find that 7 = 1 works best in practice. Among possible measures for Diagiss (Ko al., 2024), the forward Luile (KI.)
divergence gives the strongest results; further discussion is in Appendix G.2.
To further reduce distributional discrepancies in intermediate layers, we additionally employ an intermediate state-based KD loss over sequence of hidden states:
Lis ΣΣ j=
a.h OF
(1I)
bool
where Q and Q represent the h-th hidden states the FP and Boolean models for the j-th token, repsectively; H is the set of chosen intermediate states. Finally, the overall loss then computed as C = Llogita + TCis. where is a weighted factor that balances the contribution of the two losses. We empirically found that y 10 works best.
KERNEL ALLOCATION
Using more kernels enhances the Boolean model's representational capacity but also increases its size. We propose method to automatically allocate kernels per weight under fixed budget. Let Nw be the number weights in the FP teacher model, and K; for I E [1, Nw] the number of Boolean kernels for the 1-th weight. Our goal to determine k & [Kihe([,,N)) subject to design constraints. Key factors include:
(1) Residual error: Let ef R denote the approximation error from applying the successive SVID extraction to the k-th kernel the I-th weight, measured by the Frobenius norm of W[A
res (Eq. 8). (2) Weight importance: Let hu denote the importance of the I-th weight in the FP teacher model. Higher scores indicate the need for more Boolean kernels. We propose estimating hy using projection weighted canonical correlation analysis (PWCCA) (Morcos et al., 2018), reliable method for analyzing deep model representations. Details are provided in Appendix E.
(3) Weight size: The size of the /-th weight is denoted by s; and pt & s1/ Σ represents its relative size in the model.
For a given k, the size of the target Boolean model, in terms the number of weights, is NN Kist- Relative to the source FP model, this repersents expansion ratio, defined as:
p(k) & EiSr Nw
Nw Kist
Nw
Kip-
(12)
Optimization objective. To control model size, we constrain the expansion ratio a target > and limit the kernel size by Kmax, with T < Kmax I co. The optimization space is thus K. A [1, Kmax] Nw and the problem is formulated as:
kc arg Eemiinnk s.t. p(k) ST where E(k)
[Ki] hig
f(P))-
(13)
kEK'
Here, E(k) is the objective (energy) function, and f(-) is monotonically decreasing function. In practice, we use (p:) (1/pi) log(1/pi) Intuitively, the goal is to minimize residual error while prioritizing weights with higher importance and smaller size, balancing accurate knowledge transfer with model efficiency.
Optimization algorithm. The problem has complexity O(K)W), which is prohibitive for LLMS.
To tackle this NP-hard problem efficiently, we note that e decreases with ls for all and {(k) is maximized at k = 1, with any increase in k reducing (k). This motivates a heuristic iterative approach: at each step, increment the K2 that yields the largest reduction in E(k). The full algorithm is given in Algorithm 9 in the Appendix. We will demonstrate in & 6.5 the practicality of our method.

---

Published as a conference paper at ICLR 2026
6 EXPERIMENTS
Setups. In all experiments, we follow the protocol from Jo et al. (2024), without quantizing activations. The training set combines WikiText2 (Merity et al., 2017) and a selected partition of C4 (Raffel et al., 2020) data, using sequences of length 2048. We apply cosine decay learning rate with a 3% warm-up over epochs and batch size 8. Boolean parameters use a maximum learning rate of 5 x 10 while remaining FP parameters are optimized with AdamW (Loshchilov & Hutter, 2019) at a maximum learning rate of 2 x 10 with 31 = 0.9 and B2 = 0.999. Following standard practice (Jo et al., 2024), performance is evaluated via perplexity on WikiText2 and C4 (lower is better).
6.1
ABLATION STUDIES AND ANALYSIS
6.1.1
EFFECT OF THE NUMBER OF KERNELS
Initialization
Finctuned
Perplexity (4)
Normalized Deviation
1.0
1.0-
#Kemels
0.8
0.8
0.6
0.6
Wiki2
39.38
31.47
29.10
28.52
35.44
28.62
26.48
25.90
#Kernels
5
0.0
5
8
Wikiz 28.19 28.16
28.13
Number of Kernels
Number of Kernels
25.65 25.50
Figure 5: Normalized LI norm difference between the approximated weights at initialization and after finetuning against the FP weights (/W spprux Wrrlh///Wrlll and the final results.
We begin by examining the effect of the number of Boolean kernels on OPT-125M model (Zhang et al., 2022). Fig. 5 shows the normalized difference between weights approximated via successive SVID and the original FP weights, both at initialization and after finetuning. Increasing the number of kernels reduces approximation error and improves perplexity, unlike MoS (Jo et al., 2024), where adding more experts does not always help and can even hurt performance. Using 3-4 kernels yields good approximation, with diminishing improvements beyond that. Interestingly, the normalized difference relative to the full FP weights is larger after KD finetuning. We hypothesize that KD compensates the errors due to the lower expressiveness of a small number of kernels, further emphasizing its role in adapting the model to approximate the model rather than exactly replicating each weight.
6.1.2 OPTIMIZATION STRATEGY
Ligis
Cn.
Number of Flips (x
Optim.
Wiki2
C4
1.5
Komel
10
1st
33.90
30.70
2nd
30.29
27.55
3rd
29.00
26.36
20000 40000
80000
20000 40000
60000
20000 40000 60000
4th
28.60
25.93
All
32.04
29.08
Iteration
Iteration
Iteration
Figure 6: The progression of training losses, number of flips, and perplexity of the resulting models (OPT-125M) examined with respect to the optimization of different kernel configurations.
Next, we study the effect of optimizing kernels on the OPT-125M model. We consider four Boolean kernels but train only one at a time, keeping the others frozen. Fig. shows the loss convergence. Training the first kernel converges slowest, while higher-order kernels improve progressively. As shown in Proposition 4.1 and Proposition 4.3, the SVID effectively extracts optimal Boolean weights and scaling factors. In our successive SVID framework, the first kernel is well extracted and captures the most important information, and higher-order kernels approximate residuals. Since the kernels are related in a successive manner, modifying lower-order kernels affects higher-order ones. We observe that training only the first kernel results in many weight flips, indicating optimization difficulty, whereas fine-tuning only the last kernel efficiently compensates for residual errors, showing the lowest flip rates and best performance. This is in line with the observation by Liu et al. (2024b), where they compress "delta" induced by the finetuning process by using 1-bit weights. This further
7

---

Published as a conference paper at ICLR 2026
highlights the advantage of approach, training complexity is significantly reduced by only optimizing the last kernel. Thus, we apply this strategy in all our experiments.
6.2 MAIN BENCHMARK RESULTS
Table 2 compares our method with recent baselines in binarization and 2-bit quantization, evaluating perplexity and accuracy on zero-shot tasks including Winogrande (Sakaguchi et al., 2021), HellaSwag (Zellers al., 2019), PIQA (Bisk al., 2020), BoolQ (Clark et al., 2019), and ARC (Clark al., 2018). For our method, we use 2 Boolean kemels, an ultra low-bit setting. Due to space constraints, the results for LLaMA2-7B and LLaMA2-13B (Touvron al., 2023b) and different number of Boolean kernels are provided in Appendix G.4 and Appendix G.3. We note that our method is close to scalar quantization while being completely orthogonal to vector quantization (VQ) which adds substantial overhead (Gray, 1984). For completeness, encourage the reader refer to Appendix G.11 for VQ comparisons, and Appendix G.6 for further baselines.
Our method consistently and significantly outperforms the baselines in both perplexity and zero-shot accuracy, achieving results close to the FP16 baseline despite using only a budget of 2 bits for weight. As expected, QAT methods like OneBit and MoS perform better than PTQ methods, but this comes at the cost of extensive finetuning. In contrast, our approach efficiently address this problem by optimizing parameters directly in Boolean space, avoiding the need for optimizing in FP latent sapce.
Table 2: Perplexity and sero-shot accuracy results of Float16, quantized and binarized L.LSS.
Perplexity (1) Wiki2
Zero-shot Accuracy (1)
Model
Method
Whits
C4
BoolQ
PIQA
Hella.
WinoG. ARC-c ARC-c Average
FP16
14.62
14.72
57.82
72.42
53.70
59.51
50.97
29.52
53.99
PB-ILLM
1.7
272.83
175.42
62.17
54.24
27.25
50.27
27.98
23.72
40.94
BiLLM
1.11
69.45
63,92
61.92
59.52
33.81
49.32
34.38
22.35
43.55
20.36
20.76
57.85
66.53
39.21
42.80
23.97
47.50
OPT-1.38
Mo5
18.45
18.83
60.34
68.66
41.99
$3.99
44.87
26.19
49.34
OPTQ
9.5c3
3.8c3
39.60
52.07
25.57
49.33
26.68
23.63
35.15
LLM-QAT
4.9c3
2.1c3
37.83
50.05
25.72
49.72
25.76
25.09
34.07
OmniQuant
42.43
55.64
56.45
60.94
33.39
51.85
38.76
23.38
44.13
MBOK [Ours]
16.13
16.61
58.53
70.67
48.11
56.75
48.19
27.90
51.69
FP16
16
5.68
7.08
73.21
77.42
72.99
66.85
52.53
41.38
64.06
PB-ILLM
198.37
157.35
53.53
27.23
27.48
26.02
40.66
BiLLM
1.11
41.66
48.15
62.23
58.65
34.64
33.08
25.68
44.24
OneBit
8.48
10.49
62.50
70.40
54.03
55.32
41.07
30.88
52.36
LLaMA-7B
MoS
7.97
9.72
64.59
71.82
58.18
58.88
42.09
31.31
54.48
OPTQ
1.9e3
7.8c2
43.79
49.95
25.63
49.41
25. 25.84
27.47
37.02
LLM-QAT
2
7.1c2
3.0e2
37.83
50.87
24.76
51.78
26.26
25.51
36.17
OmniQuant
2
15.34
26.21
58.69
62.79
43.68
52.96
41.54
29.35
48.17
MBOK [Ours]
2x1
6.83
8,53
69.20
74.32
64.80
60.30
49.05
34,90
58.76
FP
16
5.09
6.61
68.47
79.05
76.24
70.17
59.80 39.85
44,54
66.39
PB-LLLM
1.7
35.83
39.79
62.17
58.70
33.99
$2.17
31.86
23.63
43.75
BiLLM
1.11
14.56
16.67
62.53
68.17
52.24
59.43
41.91
29.94
52.37
OneBit
7.65
9.56
63.30
71.98
60.61
59.43
42.85
32.42
55.10
LLaMA-138
MoS
7.16
8.81
63.82
73.88
64.05
60.93
44.28
33.11
56.68
OPTQ
2
3.2e3
9.9c2
42,39
50.00
25.27
50.67
26.14
27.39
36.98
LLM QAT
2
1.8c3
1.2c3
37.83
50.33
25. 25.40
51.62
27.02
26.87
36.51
OmniQuant
13.43
19.33
62.20
68.99
54.16
53.83
45.50
30.38
52.51
MBOK [Ours]
2x1
6.17
7.88
68.10
76.33
69.88
64.17
$2.34
37.88
61.45
6.3 ACCURACY-COMPRESEEE TRADE-OFFS
We further investigate the accuracy- compressio trade-offs of our method, quantization methods, and the FP model. Specifically, we compare 3-bit quantization using round-to-nearest (RTN) (Yao et al., 2022; Dettmers et al., 2022) and OPTQ (Frantar et al., 2023) methods against our approach using Boolean kermels. We evaluate these methods on OPT models of varying sizes. The results, presented in Table 3 and Fig. 1, show that with 3 kernels, our method closely approaches the performance of the FP model. Given the same weight budget, our method clearly sits on the Pareto frontier, delivering the best performance for the same model size.
8

---

Published as a conference paper at ICLR 2026
Table 3: OPT perplexity results (lower is better) on WikiTex12 and C4. The results of FP, rount-to- nearest (RTN) and OPTQ are taken from (Frantar et al., 2023).
OPT Model
WBits
Wiki2
C4
125M
350M
1.38
2.78
6.78
125M
350M
1,38
2,78
6.78
FULL-PRUCISION
16
27.65
22.00
14.63
12.47
10.86
26,56
22.59
16.07
14.34
12.71
RTN (Yao al., 2022; Detimers 2022)
3
1.3c3
64.57
1,3e4
1.6e4
5,8c3
834
55.49
5.2c3
1.1e4
5.3e3
OPTQ (Frantar al, 2023)
53.85
33.79
16.88
42.41
31.33
18.17
MBOK [Ours]
3x1
29.10
23.12
15.30
13:09
11.03
28.62
22.10
15.68
14,00
12.33
6.4 COMPARISON WITH LATENT- APPROACHES
We compare our method with latent-weight approaches on OPT models, using MoS with 3 experts and our method with 3 Boolean kernels. We also introduce baseline using our SVID framework to construct 3 binary weights that rely on FP latent weights for training. Results in Fig. 7 show that our method converges much faster, as it directly optimizes Boolean parameters without the need for STE to approximate gradient signals. Both our approach and the latent latent-weight weight method outperform Mos, demonstrating the benefit of using additional Boolean kernels and our successive SVID framework. Our method is also more efficient, avoiding the need for FP latent weights and extra momentum.
OPT-125M
OPT-350M
10
OPT
Method
Wiki2
C4
10ah
Training loss
MoS' (3 experts)
38.62
34.72
125M
Latent weights
29.47
27.18
10a
MBOK kernels) [Ours]
MoS (3 experts)
29.93
28.25
20000
40000
60000
20000
40000
60000
350M
3 Latent weights
23.58
22.65
MBOK kernels) [Ours]
Iteration
Iteration
Figure 7: Comparions between our method and latent-weight approaches.
6.5 KERNEL ALLOCATION AND COMPARISON TO BITNET B1.58
We next evaluate our kernel allocation method on the OPT-125M
20
FD2
model. supports bit allocation any granularity, including frac- tional averages, providing practitioners with a flexible model selec- tion tool under deployment constraints. Fig. 10 reports results for varying average bit budgets, showing consistent improvements as the budget increases. Fig. 9 illustrates kernel allocation with 3.5- average, where more kernels assigned to FC2 and output projec- tion layers in the final blocks. This aligns with prior observations (Bondarenko al., 2023; Frantar et al., 2023) that these layers are particularly important and sensitive to compression.
aKemeels
FC1
proj
10
proj
proj
proj
Transfomer Block
Figure 9: Allocated kernels for OPT-125M.
Parplextt
In addition, our framework's 's flexibility enables direct comparison with BitNet-bl.58 (Ma et al., 2024), which employs ternary weights. With a 1.58-bit budget, our model achieves reasonable results, whereas BitNet- b1.58 reaches a C4 perplexity of 10199.89 due to finetuning instability. consistent with Xu et al. (2024). We also compare against ShiftAddLLM (You et al., 2024), a PTQ method supporting bit allocation. Our approach performs substantially better (32.23 with a 2-bit budget vs. 435.84 for their mixed 2.2-bit allocation, see Table 17 in Lhi..
30-
28
286
1.382 2.6 à 3.5 4.5
Average bit budget
Figure 10: OPT-125M per-
formance w.r.t. bit budget.
6.6 DISCUSSION ON COMPLEXITY AND LATENCY
OPT-6.7B
We emphasize the efficiency of our method during finetuning by comparing MoS (Jo et al., 2024) with our approach using 3 Boolean kernels on the OPT-6.7B model. Because we optimize directly in the Boolean domain, each weight requires only I bit, whereas MoS relies on 16-bit latent weights. Moreover, we finetune only the last Boolean kernel, with the optimizer storing single 16-bit momentum per weight. contrast, Adam (Kingma & Ba, 2015) for latent weights needs two 16-bit momenta per weight. Fig. 11 shows the estimated
Mos.
MBOK
10
20
30
Mermory (GB)
Figure 11: Estimated memory for finetuning for weights and optimizer states

---

Published as a conference paper at ICLR 2026
memory for minibatch of one, highlighting the substantial memory savings of our method. These gains could be further amplified by incorporating optimizer state compression techniques such as GaLore (Zhao et al., 2024).
Beyond our theoretical analysis of training complexity (see Appendix F), we provide empirical evidence of practical GPU latency gains (see Appendix G.11 for detailed analyses). Leveraging the BitBLAS library (Wang et al., 2024) for 1-bit matrix multiplications (INTI weights, FP16 activations) on an A100 GPU, MBOK achieves up an 8.7x speedup for LLaMA-13B linear layers compared to FP16 baselines at batch size of (Table 4). Our method significantly outpaces existing binarization and scalar quantization techniques, and runs much faster than SOTA 2-bit vector quantization baselines like QUIP# (Tseng et al., 2024a) and QTIP (Tseng et al., 2024b) while delivering comparable performance. Taken together, our native Boolean approach a highly efficient alternative to VQ methods, with even greater performance expected on dedicated Boolean hardware.
Table Measured latency (ms) of linear layers LLaMA-13B, with values in parentheses denoting speed-up relative to the FP16 baseline.
WEIGHT SIZE
FP16
QUIP# (Tseng et al., 2024a)
QTIP (Tseng al., 2024b)
MBOK (Ours)
5120 5120
0,16540
0.62260 (0.27x)
1.96368 (0.08x)
0.05074 (3.25x)
5120 x 13824
0.42830
0.62836 (0.68x)
5.23681 (0.09x)
0. 0.05098 (8.40x)
13824 5120
0.43411
0.62840 (0.69x)
5.21193 (0.08x)
0.04987 78.70x)
7
CONCLUSIONS
We introduced Multiple Boolean Kernels (MBOK), a novel framework for low-bit finetuning LLMS. By utilizing Boolean weights and optimizing them directly in the Boolean domain, our framework significantly reduces both memory and computation costs during both finetuning and inference. The flexible multi-Boolean structure, along with the proposed successive SVID, effectively transfers knowledge from a source FP model. Through extensive experiments on LLMS of various sizes, we demonstrate that our method approaches FP performance while achieving the best accuracy- compression trade-off compared to existing quantization and binarization methods.
Limitations. Our method, like other binarized neural networks, could not be assessed on native Boolean accelerators due to hardware being optimized for real arithmetic. Nevertheless, demon- strated strong results even on modern hardware, underscoring the promise of our approach and motivating future development of accelerators tailored Boolean computation.
REFERENCES
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-Policy Distillation of Language Models: Learning from Self- Generated Mistakes. In The Twelfth International Conference on Learning Representations, 2024.
URL https: http://pppppeeee.eeeeepeeeeeeeep net/ forum? 1d=11d=33mm 32kt a
net
Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. BinaryBERT: Pushing the Limit of BERT Quantization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4334 4348. Online, 2021. Association for Computational Linguistics. URL /tatps::/acllatteee org/2021. acl- long. 334/.
Beltrami. Sulle funzioni bilineari, giomale di mathematiche ad uso studenti delle uninersita. 11, 98-106.(ar english translation by d bolcy is available as university of minnesota, department of computer science). Technical report, Technical Report 90-37, 1990.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint arXiv: 3332.3338. 2013.
10

---

Published as a conference paper at ICLR 2026
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, al. PIQA: Reasoning about Physical Com- monsense Natural Language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Removing Outliers by Helping Attention Heads Do Nothing.
Quantizable Transformers: In Advances in Neural Curran Associates, Inc.,
Information Processing Systems, volume 36, pp. 75067-75096.
2023. URL https: file http::/tttt///ttt https////cccceceeeeeeeeee wtwp:eedddinnnnnnnuuueeeeeeeeeee neurips cc/ cwpppperccppppppppppepeee wttp:/fttssseeeeeeeeeeeeeeee .df..
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler. Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Rad- ford. Ilya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners. Ad- vances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Asso- ciates, Inc., 2020. URL U
cc wttppppeeftleee///////eaeeeeecee files/paper/ Pap.aaad-apppp.......
Adrian Bulat, Yassine Ouali, and Georgios Tzimiropoulos. QBB: Quantization with Binary Bases for LLMs. In The ehihty- Anmual Conference on Neural Information Processing Systems, 2024. forum? i 1d=kwdddwd 6MRGFxOR. URL httph::t//// http:///pppeeeeeeeeemmm net/ wwwe//ee////tttee
Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. QulP: 2- Bit Quantization of Large Language Models With Guarantees. In Advances Neu- ral Information Processing Systems, volume 36, pp. 4396-4429. Curran Associates, Inc., 2023. URL https: http:///pccccccccceee/oc//ccccce newrips.r..... cc/papeseppapeeeeeeeeeeeeea ccppaper_fieeespeeeeeeeeeees files/ Files/paperr20233 file B:iee//ffBccd13322277eeeeee22222
Paper- Conference pdf.
Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding. Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, and Dacheng Tao. DB-LLM: Accurate Dual-Binarization for Efficient LLMs. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 8719-8730, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/ v1/2024.findnng acl.51. URL hwtps::tttttaaaaeeeeeeee httpas///acaaaahhlaaeceeeoo htttes:://eeaathoolooooo//////// 516/.
org/ findings-anne
Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions. In Pro- ceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguisties: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2924-2936, Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10.18653/v 1300. URL https: haapaaaaaaannntaooaaaaaaoa htt.a/ttooooooo.ooo///1111000000
org/NN9-1300.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have Solved Question Answering? Try ARC, the A12 Reasoning Challenge. arXiv preprini arXiv: (e:::0503305 2018.
Matthieu Courbariaux, Yoshua Bengio, and Jean- Pan--i David. BinaryConnect: Training Deep Neural Networks with Binary Weights during Propagations. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
Tim Dettmers and Luke Zettlemoyer. The case for 4-bit precision: k-bit Inference Scaling Laws. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 7750-7774. PMLR, 23-29 Jul 2023. URL https: httpss:/ppccceeedeee wttpesssmrr.pprrere wwtp://ettteee22555222 html.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. GPT3.int00: 8-bit Matrix Multiplication for Transformers at Scale. In Advances in Neural Information Processing Systems, 2022. URL http::/:////eeeeieieee net forum? d=dxi GWqBoxaD.
11

---

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Effi- cient Finetuning of Quantized LLMs. In Advances in Neural Information Process- ing Systems, volume 36, pp. 10088-10115. Curran Associates, Inc., 2023. URL https::/proceediigg www.ggg....uuueee cc http:/ppperrfilleeeppeee/e////// Published as a conference paper at ICLR 2026
Tim Dettmers, Ruslan A. Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. SpQR: A Sparse-Quantzze Representation for Near-Lossless LLM Weight Compression. In The Twelfth International Confer- ence on Learning Representations, 2024. URL htttps://opezzeviwww http:/ewwwweee...oeeeeeeeeeeoeoe forum? d Qlu25ahSuy.
Peijie Dong, Lujun Li, Yuedong Zhong, Da Du, Ruibo FAN, Yuhan Chen, Zhenheng Tang, Qiang Wang. Wei Xue, Yike Guo, and Xiaowen Chu. STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs. In The Thirteenth International Conference on Learning Representations. 2025. URL https: / openrevi lew. net/ forum? d= 6XUSDVBFKV.
C. Eckart and G. Young. The Approximation of One Matrix by Another of Lower Rank. Psychome- trika, 1936.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. OPTQ: Accurate Quantization for Generative Pre-trained Transformers In The Eleventh International Conference on Learning Representations, 2023. URL https: httpp:://penreeeeeee http///pppeeeeeeeeeeeeeeeeeeeeee net. forum? wwwteeitttttttttteeooo
Eberhard Fuchs, Gabriele Flügge, al. Adult Neuroplasticity: More than 40 Years of Research. Neural plasticity, 2014, 2014.
Robert Gray. Vector Quantization. IEEE ASSP Magazine, 1(2):4 1984.
Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Dan Roberts. The Unrea- sonable Ineffectiveness of the Deeper Layers. In The Thirteenth International Conference on Learn- ing Representations, 2025, URL https: oow.neeveeww.v.eeooe/ee wwt.wwwwwwweeeeeeeeeeeeeeeeeee
net forum tteeetoooemmemeeeeeeeeeeerr
Donald Olding Hebb. The Organization of Behavior: A Neuropsychological Theory. Psychology press, 2005.
Geoffrey Hinton, Vinyals Oriol, and Jeff Dean. Distilling the Knowledge in Neural Network. arXiv preprint arXiv: 025311 1, 2015.
Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. BiLLM: Pushing the Limit of Post- Dost-Training Quantization for LLMs. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 20023-20042. PMLR, 21-27 Jul 2024. URL https:// proceedings www.mrrpp.... wal.sppees//ve v255 huwngu 4q. html.
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El- El-Yani,, and Yoshua Bengio. Binarized neural networks. Advances in neural information processing systems, pp. 4107-4115, 2016.
Dongwon Jo, Taesu Kim, Yulhwa Kim, and Jae-Joon Kim. Mixture of Scales: demory-Efficien Moken-Adaptic Binarization for Large Language Models. In The Thirty-eighth Annual Confer- ence on Neural Information Processing Systems, 2024. URL httpt::s:///peeeeeee...... forum?1d==GGBBYcck2
Charles Jordan. Calculus of Finite Differences. Chelsea Publishing Company, New York, 2nd edition, 1950. doi: htpp:///doi.rr/110.. wttp://..///11111100001010100000 10177800225572000027771
Sehoon Kim, Coleman Richard Charles Hooper, Amir Gholami, Zhen Dong. Xiuyu Li, Sheng Shen, Michael W. Mahoney, and Kurt Keutzer. SqueezeLLM: Dense-and Sparse Quantization, In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 23901-2399 PMLR, 21-27 Jul 2024. URL https:// proceedings .ml...r.....m prwss/eeesseeeeee... paress/v35.. 10/35/km2222 html.
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In International Conference Learning Representations, 2015.
12

---

Published as a conference paper at ICLR 2026
Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and Se- Young Yun. DistiLLM: Towards Streamlined
Distillation for Large Language Models. In Proceedings of the 41st International Conference Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 24872- 24895. PMLR, 21-27 Jul 2024. URL htw.s:tp.... perroceddingg..... wtt.:/ml.....aa//e/..... wtt.ppppws///kooeosess html.
Tanishq Kumar, Zachary Ankner, Benjamin Frederick Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Re, and Aditi Raghunathan. Scaling Laws for Precision. In The Thirteenth International Conference on Learning Representations, 2025. URL. net/
https: coppereeiee..eeeeoorut.......... forum221d=w11PC33CCU
Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. OWQ: Outlier-aware Weight Quantization for Efficient Fine-tuning and Inference of Large Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 13355-13364, 2024.
Zhiteng Li, Xianglong Yan, Tianao Zhang, Haotong Qin, Dong Xie, Jiang Tian, Zhongchao Shi, Linghe Kong, Yulun Zhang, and Xiaokang Yang. ARB-LLM: Alternating Refined Binarizations for Large Language Models. The Thirteenth International Conference on Learning Representations, 2025. URL https: wwwpreveee......////ppp....pp..w net forum? 2082000.DT
Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang. Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation- aware Weight Quantization for On-Device LLM Compression and Acceleration. In Proceedings of Machine Learning and Systems, volume 6, pp. 87-100, 2024. URL ffilee/ Paper Cowfe-eeceeeeeeeeee..... pdf.
https: httpps:/prroceedings mlsys org wtwpa//w....opppp..../e////eeeee (2absccaafsedddeeaaaaaaaaa
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng. Chenyu Zhang, Chong Ruan, et al. DeepSeek-Ve V3 Technical Report. arXiv preprint arXiv:2412.19437, 2024a.
James Liu, Guangxuan Xiao, Kai Li, Jason D. Lee, Song Han, Tri Dao, and Tianle Cai. BitDelta: Your Fine-Tune May Only Be Worth One Bit. In Advances in Neu- ral Information Processing Systems, volume 37, pp. 13579-13600. Curran Associates, Inc., 2024b. URL https: ittps::/proceedings neurips wewrips.cc/pappeeeeeeeeeeeeeeeee
pdd..
Zechun Liu, Barlas Oguz, Aasish Pappu, Lin Xiao, Scott Yih, Meng Li, Raghuraman Krish- namoorthi, and Yashar Mehdad. BiT: Robustly Binarized Multi-distilled Transformer. In Ad- vances in Neural Information Processing Systems, volume 35, pp. 14303-14316 14316. Curran Asso- ciates, Inc., 2022. URL https: http:::://ccceopeeeeeeeeeo neurips....... cc httpc//aaer__re/peeeeeeeee files/paper/ 2022 waaffffeeScc1111
Conerrrreeeeeeeeeencceeeeeeeeeee pdf.
Conference,
Zechun Liu, Barlas Oguz, Aasish Pappu, Yangyang Shi, and Raghuraman Krishnamoorthi. Binary and Ternary Natural Language Generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pp. 65-77, Toronto, Canada, 2023. Association for Computational Linguistics. doi: 10. 1c223 long.5. URL https: // /aaclantholoyy org/ 2023.
long. 5/.
Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. LLM-QAT: Data-Free Quantization Aware Training for Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2024. pp. 467-484 Bangkok, Thailand, 2024c. Association for Computational Linguistics. URL https::/p///. / /acl antho org.hhhooooooooooooooooooooo 2024. f nd mddiggs-acl...
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. In International Conference on Learning Representations, 2019. URL https::t////ectt http:::///peeeeview net forum?
id=Bkg6RiCqy/
Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang. Shaohan Huang, Lifeng Dong, Ruiping Wang, Jilong Xue, and Furu Wei. The Era of 1-bit LLMs: All Large Language Models in 1.58 Bits. arXiv preprint arXiv:2402. /7764, 1, 2024.
13

---

Published as a conference paper at ICLR 2026
Stephen Merity, Caiming Xiong, James Bradbury. and Richard Socher. Pointer Sentinel Mixture Models. In International Conference on Learning Representations, 2017. URL https:// openreview net net//oouumidd-Byy22dddl
Ari Morcos, Maithra Raghu, and Samy Bengio. Insights on Representational Simi- Advances in Neural In- formation Processing Systems, volume 31. Curran Associates, Inc., 2018. URL
larity in Neural Networks with Canonical Correlation.
https:// httpps///proceeccngg..nee....i.e ottppppceeeaaaoereeeerro...ppepa aTa38F0cc6d777111101119966dddd wtteeeeccff-Pappp Pa-eaper.
[ htpppppppp/001// Bflee//
Van Minh Nguyen. Variation and Boolean Logic BackPropagation. arXiv preprint arXiv:2311.07427, 2023.
Van Minh Nguyen, Cristian Ocampo, Aymen Askri, Louis Leconte, and Ba-Hien Tran. BOLD: Boolean Logic Deep Learning. In The Tiihth-eighth Annual Conference Neural Information Processing Systems, 2024. URL wwwpt:ttp//.oc.
net/ forum? dddDO ZCP jk.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High- Peeforreffo Deep Learning Library. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Haotong Qin, Yifu Ding, Mingyuan Zhang, Qinghua YAN, Aishan Liu, Qingqing Dang, Ziwei Liu, and Xianglong Liu. BiBERT: Accurate Fully Binarized BERT. In International Conference on Learning Representations, 2022. URL httpst:::.. wwwp/epeeeeeeeeeeeeeeeeeeeee new. net/ forum? 6=5xEgr1 5FAJ.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research, 21(140):1-67 2020. URL
http:://mmr... wwwp:/.rggpppppprppee (etpprss/21/221 22222222000 074 html.
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. XNOR-Net ImageNet Classification Using Binary Convolutional Neural Networks. In Proceedings of the European Conference on Computer Vision (ECCV), October 2016.
Keisuke Sakaguchi. Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An Adversarial Winograd Schema Challenge Scale. Communications of the ACM, 64(9):99-106.
2021.
Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. OmniQuant: Ommidirectionally Calibrated Quantization for Large Language Models. In The Twelfth International Conference on Learning Representations, 2024. URL https: / wwwp/eeeeee....././/// net forum? d= 8WuhhhOLY.
Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christopher Re, lon Stoica, and Ce Zhang. FlexGen:
Generative Inference of Large Language Models with Single GPU. In Proceedings the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 31094-31116. PMLR, 23-29 Jul 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and Efficient Foundation Language Models. arXiv preprint 13X1v.2302.13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open Foundation and Tinee Chat Models. arXiv preprint 200.:22307009888 2023b.
14

---

Published as a conference paper at ICLR 2026
Albert Tseng, Jerry Chee, Qingyao Sun, Volodymyr Kuleshov, and Christopher De Sa. QulP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks. In Proceedings of the 41st International Conference Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 48630-48656. PMIR, 21-27 Jul 2024a.
Albert Tseng, Qingyao Sun, David Hou, and Christopher De. QTIP: Quantiza- tion with Trellises and Incoherence Processing.
In Advances in Neural Infor- mation Processing Systems, volume 37, PP. 59597-59620. Curran Associates, Inc., 2024b. URL https:tt///t. https::/proceeeeeeeeee ngs neuri wwt.enueu....pppee.e.eee..... ptcc/paper_fileee http:/ffllsspppre//e2/// pappr/2024//////// file/ 2eeee/de2e88255
Papprbo--ppper---feeeeeeeeeeee pdf.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is All you Need. In Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: http://///ooooeee..eo.o.. www.ins.ucccceeee.. cc/paper_ffieeeeep sapper_flles htt.ffllssppppr22017 #flee3355e22334420 9 fbd053c1 44a4aaaa-444455aa55
Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. BitNet: Scaling 1-bit Transformers for Large Language Models. arXiv preprint arXiv:2310. /0. 1453, 2023.
Lei Wang, Lingxiao Ma, Shijie Cao, Quanlu Zhang, Jilong Xue, Yining Shi, Ningxin Zheng, Ziming Miao, Fan Yang. Ting Cao, Yuqing Yang, and Mao Yang. Ladder: Enabling Efficient Low- Precision Deep Learning Computing through Hardware-awarc Tensor Transformation. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pp. 307- 323, Santa Clara, CA, 2024. USENIX Association. ISBN 978-1-999113--4--3 URL https: //www. uswnix.e............... org/ con wwwencee /ttp://eeeeeeeeeeeeettaaattoooo/ witpeiattoooo/anngeeeeeel lei.
Xinghao Wang, Pengyu Wang, Bo Wang, Dong Zhang, Yunhua Zhou, and Xipeng Qiu. BitStack: Any- Size Compression of Large Language Models in Variable Memory Environments. In The Thirteenth International Conference on Learning Representations, 2025. URL net/forum: id= 1Bt jGbyv0.
Yuqiao Wen, Zichao Li, Wenyu Du and Lili Mou. f-Divergence Minimization for Sequence- Level Knowledge Distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 10817-10834, Toronto, Canada, 2023, Association for Computational Linguistics. doi: 10. 10.18653/w1/2223.adee acl-long.660 605. URL https: httpe///aaaaaaahaaaeeceee wwt.oor.g........ooe. a01- ooo 605/.
Yuzhuang Xu, Xu Han, Zonghan Yang. Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. OneBit: Towards Extremely Low-bit Large Language Models. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL https:
/ooperrviiew.
Zifei Xu, Saych Sharify, Wanzin Yazar, Tristan Webb, and Xin Wang. Understanding the Difficulty of Low-Precision Post-Training Quantization for L.Mss. In ICLR 2025 Workshop on Sparsity in LLMs, 2025. URL https: / www.rwview........ net/ forum? dx KeWKKkk.
Zhewei Yao, Reza Yazdani Aminabadi, Minjia MinjiaZaang, Xiaoxia Wu, Conglong Li, and Yuxiong He. ZeroQuant: Efficient and Affordable Post- -rainTrannng Quantization for Lcrge-Scale Transformers. In Advances in Neural Information Processing Systems, 2022. URL https: petetps:/oppnniee http///ppeeeeieeeeeeeeeeeeeeeeee net/ / forum? id= EVCE12-G1.
Haoran You, Yipin Guo, Yichao Fu, Wei Zhou, Huihong Shi, Xiaofan Zhang, Souvik Kundu, Amir Yazdanbakhsh, and Yingyan (Celine) Lin. ShiftAddLLM: Accelerating Pretrained Reparaneterization. In Advances in Neural Information Processing Systems, volume 37, pp. 24822-24848. Curran Associates, Inc., 2024. URL https: http::////ooceddee ngs neurips wwcpppppprailiiecce /t1les//pper/2224/ file file/2c30a37c777 /ttp://2200277770000000000022920
LLMs via Post- Dessn-Trannn Multiplication-Lan
Paper- www--Cooffeeeeeeeeeeeee pdf.
Zhihang Yuan, Yuzhang Shang, and Zhen Dong. PB-LLM: Partially Binarized Large Language Models. In The Twelfth International Conference on Learning Representations, 2024. URL https: http:////peeeeeieeee net forum? feRRhikU.
15

---

Published as a conference paper at ICLR 2026
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can Machine Really Finish Your Sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-48000 Florence, Italy, 2019. Association for Computational Linguistics. doi: 10. 1/155//1 URL https:/ //acllacccccc anthology
arg/P19-14722
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. OPT: Open Pre-trained Transformer Language Models. arXiv preprint arXiv:2205. 00060 2022.
Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. GaLore: Memory-Efficienn LLM Training by Gradient Low-Rank Projection. In Proceedings of the 41st International Conference on Machine Learning. volume 235 of Proceedings of Machine Learning Research, pp. 61121-6114 113.. PMLR, 21-27 Jul 2024. URL https: / prooeed ngs. ml w.w..ppes//ee.. / v235 zhao24s 8. html.
16

---

Published as a conference paper at ICLR 2026
Appendix
TABLE OF CONTENTS
A Primer Boolean Neural Networks
18
A.I Neuron Design
18
A.2 Mathematical Foundation of Boolean Variation
19
A.3 Boolean Backpropagation.
23
A.4 Boolean Optimizer
24
Discussion on Hardware Considerations
25
B.1 Computation Proposed in $ 4.1
25
B.2 Multi-core Computation Strategy 4.2
25
Code Samples of Core Implementation
26
C.1 Boolean Linear Layer and Optimizer
26
C.2 Successive SVID for Kernel Extraction
28
Proof of Propositions
29
D.1 Proof of Boolean Linear Reformulation using SVID
29
D.2 Proof of Proposition 4.1
30
D.3 Proof of Proposition 4.3
31
Details on Kernel Allocation
33
E.1
Weight Importance Estimation
................................
33
E.2 Kernel Allocation Algorithm
34
Theoretical Analysis of Training Complexity
34
Additional Experiemental Results
35
G.I Additional Information of Experiemental Settings
35
G.2 On the Choice of KD Loss
................................
35
G.3 Results of Different Number of Kernels on LLMs
................................
35
G.4 Additional Results on LLaMA-2
36
G.5 Generation Quality
36
G.6 Additional Baselines
37
G.7 Effects of Knowledge Distillation
39
G.8 Analysis of Scaling Values
39
G.9 Convergences of OPT Models
39
G.10 Effects of Successive SVID Initialization.
G.11 Discussion on Latency and Comparison with Vector Quantization
H Ethics Statement
42
I Reproducibility Statement
42
The Use of Large Language Models
42
17

---

Published as a conference paper at ICLR 2026
A PRIMER ON BOOLEAN NEURAL NETWORKS
For completeness, this section reviews the concepts and methodology of Boolean neural networks as proposed by Nguyen (2023); Nguyen al. (2024).
A.1 NEURON DESIGN
Boolean Neuron. Consider the (-th Boolean linear layer; in the forward pass, the output of the next layer is defined as Nguyen al. (2024):
(!) b2l)
i [k.]]
b +
L(X
1 S m,
(14)
[k,s]]
where k denotes the sample index in the batch, and L. is logic gate such and, or, xor, xnor; The weights (i() [r.]] are Boolean values {TRUE, FAI.SF) or {-1, +1}, as typically used in practical implementations. n and m are the number of input and output neurons, respectively. As the most extreme use case, the input data are also Boolean values. The above summation is understood the counting of TRUE values. We emphasize that the framework is flexible, as it allows Boolean linear layers to be connected through activation layers, layer normalization, arithmetic layers, other types of layers.
Mixed Boolean-Real Neuron. enable flexible integration and coexistence of Boolean designs with real-valued components in deep models, we consider two cases of mixed-type data: (i) Boolean weights with real-valued inputs, and (ii) real-valued weights with Boolean inputs. This paper focuses on the first case. These scenarios are addressed through extension of Boolean logic to accommodate mixed-type data. To proceed, we introduce the essential notations and definitions. Specifically, we define B A (TRUE, FALSE] as the Boolean domain, equipped with standard Boolean logic operations.
Definition A.1 (Three- logic). We define the mixed logic domain as WI & B BU{0) where O represents undefined or neutral value. The logic connectives in Ml are defined in alignment with standard Boolean logic, as follows. First, the negation operator is extended as: -TRUE FALSE, -FALSE TRUE, and -0 O. Next, let denote generic logic connective (e.g., AND, OR). We distinguish its use in M and B by writing LM and Lg, respectively. The extended connective LM is defined by:
LM(a, 6) = [Ts(a, b) for a, B,
otherwise.
Notation A.2. Denote by IL logic (e.g., B Mi), R the real set, E the of integers, N numeric set (e.g., R or 2), and ID a certain set of L or N.
Definition A.3. For x E N, its logic value denoted by Flogic given as Flogic Flogie FALSE + < O, and Plogic 0 + 0.
TRUE
+
Definition A.A. The magnitude of a variable x, denoted by [x), defined as follows. If e N, then [z] is the standard absolute value. For IL, the magnitude is given by:
0 ifx= 0,
xxc
otherwise.
Definition A.5 (Mixed-type logic). For I, logic connective of L. and variables b, operation e = L(a, b) is defined such that [c| = |a|[b] and Glogic = L(alogies blogio).
18

---

Published as a conference paper at ICLR 2026
A.2 MATHEMATICAL FOUNDATION OF BOOLEAN VARIATION
In this section, we present the mathematical foundation of Boolean variation which is the corner stone of the method for training Boolean weights directly within the Boolean domain, without relying on FP latent weights (Nguyen et al., 2024).
A.2.1 BOOLEAN VARIATION
Definition A.6. Order relations '<' and '>' in B are defined as follows:
FALSE < TRUE, TRUE > FALSE.
(15)
Definition A.7. For a, b B, the variation from to b, denoted 8(a - b), is defined as:
TRUE,
if > a,
8(a + b) 2
0, FALSE,
if a, if <
(16)
Definition A.8 (Type conversion). Define:
p: L
TRUE,
if 0,
+ p(x)
if 0,
(17)
FALSE,
if O.
Proposition A.9. (Nguyen, 2023; Nguyen et al., 2024) The following properties hold:
1. Vx, y N: p(zy) xnor(p(z)) p(y)).
2. Va, E L. e(xnor(a, (a, b)) = e(a) e(b).
3. Vx, E N: x y + | |y| and p(æ) = p(y).
In particular, property Proposition A.9(2) implies that by the embedding map e(.), we have:
((TRUE, FALSEJ, xor) (=1)-x]]
(18)
((TRUE, FALSE], xnor) = ({=1).x),
(19)
where _ and stand for isomorphic relation, and the real multiplication, resp. A consequence is that by e(-), computing sequence of pointwise XOR or XNOR, counting, and majority vote is equivalent sequence of pointwise multiplications and accumulation performed on the embedded data.
Proposition A.10. The following properties hold:
1. T., N: xnor(a, .)) e(a)æ.
2. x, y e N: xnor(z, y) ry.
3. x e N}, y. E N: xnor(x, (I, y + 2) = xnor(x,g) + xnor(x, 2). N: xnor(æ, Au) = Axnor(æ, v).
4. x e [L, N}, v. N:
5. x E (L,
N: xor(z, y) -xoorrr,))) y).
Proof. The proof follows definitions A.5 and A.8.
Following Definition A.1 we have Vt E M, xnor (nOU(T t) = xnor t) = =t,
and xnor(0, (0, t) = Put v xnor(a, z). We have |u| |x] and p(v) xnor (a, p(x)).
Hence, = () + p(v) = 0 = = 0: TRUE 3 p(v) = p(())
= FAI.SE
p(v) = pp))
F z. Hence (1).
The result is trivial if x = 0 or y = For z, * 0, put e = xnor(z, y). we have [e|
|<||y| and p(u) xnor(p(æ)) p(y))- According to Definition A.8, if sign(x)
19

---

Published as a conference paper at ICLR 2026
sign(y), have p(u) TRUE |2| y zy. Otherwise, i.e., sign(x) sign(y). p(u) = FALSE = -|2|| zy. Hence (2).
(3) and (4) follow (1) for and follow (2) for x E N.
For (5), write u = xor(x, y) and v xnor(a, (z, y), we have |u| = |a| and p(4) xor(p(æ), p(y)) = -xnor(p(x)) p(v)) -p(e). Thus, sign(u) sign(v)
Notation A.11. We denote F(S, T) the set of all functions from source image T.
Definition A.12. For E F(B, D), Vo E B, write 5f(x + -x) = =(f(x) f(-x:)) variation of f w.r.t. x, denoted (z)),, , defined as:
The
fx(x) - xnor + -z), 8f + -z)).
Remark A.13. For convenience and consistency of notation, we intentionally adopt the standard
symbol for the continuous derivative, f', also denote Boolean variation The intended meaning
whether it represents continuous derivative or a Boolean variation can be inferred from the context which the function f is defined. Intuitively, the variation of w.r.t TRUE if f varies in
the same direction with x.
Example A. 14. Let B. f(x) = xor(x, a) for x E B, the variation of w.n.t can be derived by establishing truth table (see Table 5) from which we obtain f'(x:) -0.
Table Variation truth table of f(x) a))
a
x
T
8(x - -x) f(a,2)
fla,
6f(x + +-x) f(x)
TRUE
TRUE
FALSE
FALSE
FALSE
TRUE
TRUE
FALSE
TRUE
FALSE
TRUE
TRUE
TRUE
FALSE
FALSE
FALSE
FALSE
TRUE
FALSE
FALSE
TRUE
FALSE
FALSE
TRUE
FALSE
FALSE
TRUE
TRUE
FALSE
TRUE
TRUE
TRUE
A.2.2 BOOLFAN VARIATION CALCULUS
Below rules of Boolean variation which necessary for training Boolean neural networks.
Proposition A.15. (Nguyen, 2023; Nguyen al., 2024) For f.g F)B.B), Vz, following properties hold:
B the
1. (f + y) = xnor(5(x-+ - (x))
2. -=)x)'===f'( =
3. (g o f)((x) xnor(g' (f(
Proof. The proof is by definition:
1. Va, y E B. there are two cases. If y then the result is trivial. Otherwise, i.e., y w. by definition we have:
f'(x) = xnor (fx - -x), 8f(x - -z))
8f(x + -x) xnor (6(x + -z), (x)).
Hence the result.
20

---

Published as a conference paper at ICLR 2026
2. Vz, y B, is easy to verify by truth table that 0(-f(1 - v)) -5f(x (rr v). Hence, by definition,
(-f)((x) = xnor(ó(# + - + +-1)))
xmor (0( - -df( + -x))
xnor(ó( + vx), f + -x)
=-)(aa).
3. Using definition, property (i), and associativity xnor, Vz B we have:
(g f)(x)) xnor(ō(x )g( + ((=)))
+ -1
(mor(g(f((z)) xnor(o(x
(n)r) =0)) + -)))
6f
Proposition A.16. (Nguyen, 2023; Nguyen et al., 2024) For f E F(B,N), the following properties hold:
1. x, e B: -3) xnor(ō(#
+ =nor((l f))
2. a E N: =af(s))
3. g E F(B,N): (f+ = (xx)+ gz))
Proof. The proof is as follows:
1. For y, E B. Firstly, the result is trivial if = x. For + x, i.e., = -x by definition:
f((x) = xnor (8(x + -z), 6f -z))
Hence, + --]]== [=(()) since 16(x - -w) = 1, and
pf xnor (ō(x + -z). + xz))
(x + -x) xnor (6(x + -z)
where p(-) is the logic projector Eq. 17. Thus, 8f(x -x) xnor(ō(x -x), (x)). Hence the result.
2. Firstly Vz, y E B, we have
- affl)-
Hence, by definition,
(af)(z)= = xnor (8(x + - 5(af(x + -z))
xnor(ō + -x), a05(2 -r)
-x))
a vexorrōl(x + 05 + w)), due to Proposition A. 10(4)
= af((x)
3. For f.g F(B,N).
(f g))))(z) = xnor(8 (ō(z - -z), + g)(z + -z))
= snor(8(2 + -r), ff + =x) + ōg(g + -x))
xnor + -x), 6f(r + xx) + xxor + -z),5g(z + +-r)), fz(x) tg((x),
where (x) is due to Proposition A.10(3).
21

---

Published as a conference paper at ICLR 2026
For f E F(Z, N), its derivative, also known in terms of finite differences, has been defined in the literature as (x) = f(z + 1) - f(z), see e.g. Jordan (1950). With the logic variation as introduced above, make this definition more generic as follows.
Definition A.17. For F(2, D), the variation of I w.r.t x E Z is defined as f'(x) & 65 I - 1), where o is in the sense of the variation defined in B.
Proposition A.18. (Nguyen, 2023; Nguyen et al., 2024) The following composition rules (chain rules) hold:
1. For B
z 9 D. (xor(())=)xx((((f(())()(())))
1 B 9 D: (g =
2. For B
g(
1). then:
f)(x = xfor(d
Proof. The proof is follows.
1. The case of IB 4 B 2 IB is obtained from Proposition A.15(3), For B 4 B 2 N. by using Proposition A. 161), the proof is similar to that of Proposition A.15(3).
2. By definition, we have
f)(x) xnor(ō(x + -z).8 + f(-x)))
(20)
Using property (1) of Proposition A.16, we have:
f(-x) = f(x) öf(r -x)
= f(x) + xnor(ō(x + f
(21)
Applying Eq. 21 back to Eq. 20, the result is trivial if f'(x) = 0. The remaining case
[f'xx) 1 for which we have xnor(ō(x + -x), 11. First, for xnor(ō(r xx fx))) = 1, we have: 6g(f(x) f(-=x) 6g(f(z) f(x) +1)
((((a)
xxor(g (f)
xnor(g' (f (nor(((((()))xxxoo(((====- xnor(((= -x),
(22)
Substitute Eq. 22 back to Eq. 20, we obtain:
(g o f)(()) = xnor (5 + -z), fg(f(z) + f(-x)))
xnor(ō(æ + -x). xnor(g (f(x)), xnor(g'(f(z)), xnor())())6 + -x), f(x)))
where that last equality is by the associativity of xnor and that xnor (x, x) True for x E IB. Similarly, for xnor(ō(æ +
= 11, we have:
8g(f(x) + f(-x)) = f()) 1))
=-g(((()-11
(f(((z)))
())(=)))))xoor(8 + fz))
(23)
Substitute Eq. 23 back to Eq. 20 and use the assumption that g have:
= -'((()))= 1),
o f)(()) anor(8(2 (2).). f((=))) xnor(ō(x xnorlg (f(x) 1), xnor(6(x -2), -xx,,(()xx))
xnor(g' xnor(g'([=))), (f(x)), (x)))
Hence the preposition is proved.
Example A.19. From Example A.14, we have 8xor a)/8x = -a for a, a E B. Using Proposi- tion A.15-(2) we have: 8xnor(æ, a)/8x since xnor(x, a) -orr(x, (r,
22

---

Published as a conference paper at ICLR 2026
A.2.3 MULTIVARIATE CASE
The properties of Boolean variation described above can be extended the multivariate case in straightforward manner. For example, in the case of multivariate Boolean functions, the extension is as follows.
Definition A.20. For = (I1...., Zn) E B", denote x-4 (#1..... Ii-1, Is 24t1..... Inn for n 2 1 and sissiii or (x)) is defined as: f((x) = (x) ((xx)//8, 8z, & xnor(ō(x; + -x;), 85(x x-)).
ii n. For F(B", B). the (partial) variation of f w.T.t. Iu denoted f. (x(
The composition rule then becomes:
Proposition A.21. (Nguyen et al., 2024) Let f E F(B",B) n 2 1, and g E F(B, For isism n:
f((x) = (no) E
(24)
Example A.22. Apply Proposition A.16-(3) to Y!
[kal from Eq. 14: OY
low
[k
/8x!1) (1) as an example, we have: ( (ww())
(X
(1) [a.) [i.)] (R.s) /Ox
//Wl and d
-x
w [J]] [
Then, for L xnor
[R,s]
A.i]
x(I and 5
58 (1)
w
[k. b. [kas]
[k [k,d] [6
A.3 BOOLEAN BACKPROPAGATION
This section presents how to apply the above principles of Boolean variation to define backpropagation for Boolean neural networks. The 1-th layer (Eq. 14), receives the backpropagation signal from the downstream layer + 1. Specifically, z [k.s]
denotes the variation of the loss function L [K.
w.r.t the output layer To optimize the Boolean weights, we need compute the corresponding
loss signal, denoted as O [tA
66 In addition, we also have to compute the loss signal for the
hal
upstream layer, defined P(D) concrete example.
Hereafter, we consider the logic gate L xnor 6 (..)
8C
[K.]]
First, using Proposition A.15, Proposition A.1 Proposition A.18 and extension the multivariate case by Proposition A.21 in the same manner as shown in Example A.22, we have:
YY
5xnor
(X
(1)
w!l))
[k.]] [(o)
X
(1) [s
(25)
Ww
(1)
8w())
(1)
[+j]
yY() [es]]
5xnor (X [ka]]
W!!)
[5]]
w
(
(26)
8X
(1)
6x (1)
[Jll
[K
k.i
Using the chain rules given by Proposition A.18, we have the following atomic variations:
5L
5L
[Bll
= xnor (Z (A.1)
la = xnor
(27)
(1 [.
(
sw!!)
(I)
p(U [ka
6x
8L
(l)
= xnor
(
SC
(x [k.4]
xnor (Zkk)) [i.51)
(!
(28)
[k.i]
[k.j]
The variations Q QR.11 and [k.i] can be then obtained by aggregating the above atomic variations
(
over the batch dimension and output dimension j. respectively. More specifically, denote 1(-) the indicator function. Additionally, for E B and variable æ,, define 1(x b) 1 if Mlgiic b
23

---

Published as a conference paper at ICLR 2026
and (xx b) = 0 otherwise. Then, we have:
(1)
8C
Σ
1(Q (1)
TRUE) (1)
(Hrkss) (Riis))
Σ II(Q Aiid))
FAISE) Q (1)
R.oalh (29)
ow
(1)
[6
PM
8C
1( [k. = TRUE) [k,e
Σ
1(P [k.a.]]
= -FALSE) P PRajgl
(30)
6x
[6,1]
A.4 BOOLEAN OPTIMIZER
Algorithm 1: Boolean learning process for linear layer.
Input Learning rate 17, number of iterations T;
Initialize: M (0.0 0; 3 1;
[i.]
for t O,...,T do
/a 1. Forward x/
Compute Y following Eq. 14;
/+ 2.
Backward +/
Receive Sc from downstream layer;
[R.]
/+ 2.1 Backpropagation */
Compute and backpropagate ((). to the upstream following Eq. 30;
/+ 2.2 Weight update process +/
Neoaal := 0, Nunchanged := 0;
foreach W do
Compute Q (0)++1 following Eq. 29;
[(J]
Update M (1),+1 B'M
(1)t
(1,4+1
[. 17 (5
Niotal + Neaal +
xxxoor(M (MMorM [i.j
(1)++1 (M.+)+1 ((, = TRUE then
(1,1
10
[..)]
/. -Flip weight */
Wl.)+++ -W
[i]]
[ij]]
Reset corresponding accumulator +/
M 01++1 0; [J]
13
else
/. Weight is unchanged //
W (1)++1
(l.t. (1).
[
Update statistics to update 3 */
N Munchanged Nunchanged + 1;
Update +1,,, gr+1
Nunchanged Niatl:
Given the above variations, the rule for updating the Boolean weight ( function C is as follows:
b to minimize the loss
(1(1) (W(() [i.]]
(Q (
if xnor
[i.]]
TRUE.
(31)
Based on this update rule, we can develop an optimizer that accumulates the signal Q
Q over
training iterations. Specifically, let (!).
accumulator, initialized as (().0 [i.j] denotes the weight at iteration t. and M ().t [J]]
represents its
[i.f] The update rule for the accumulator is then defined as: The
update rule for the accumulator is then defined as:
M 1 + 3'M (J B. [
(1)t 1 [e.il
(32)
where 77 is the accumulation factor acting as learning rate, and 8 is an rugulrrguuarizing factor that reflects the system's state at time f. In work, we use brain plasticity (Fuchs et al., 2014)
24

---

Published as a conference paper at ICLR 2026
and Hebbian theory (Hebb, 2005) adaptively set 8", that force the weights to adapt to their neighborhood during. For the chose weight's 's neighborhood, for instance, neuron, layer, or network level, 3 is set as:
G Number of unchanged weights at
(33)
Total number of weights
It temper the importance of weight variational according to how much neurons have changed. In our experiments, H is set to per-layer basis and initialized as ,00 1 'he learning process for a linear layer is described in Algorithm
B DISCUSSION ON HARDWARE CONSIDERATIONS
B. COMPUTATION PROPOSED IN $ 4.1
The Boolean framework supports both full and partial binary settings. The afforementioned Boolean variation calculus shows that:
xnor (real,
(Tral, Wlogic) Treal Whinary:
(34)
under the mapping TRUE - +1 and FALSE + -1. Consequently, matrix multiplication (matmul) between a real tensor X and logic tensor W can be implemented as follows:
Using binary weights +1]: Simply represent the logic weights in binary format. Then, matmul (Faeat Wlogie) is directly computed as matmul (Zraal,
(Zral, Weinary).
Using native logic (TRUE, FALSE]: The multiplication reduces to:
Treal
if Wlogic TRUE
matmul (Freal: Wlogic)
(35)
Treals
if Wlogic FAI.S.
Thus, sign flip of Freat conditioned Wogice followed by accumulation, suffices perform matmul (Xeal. W Wogic)
The first approach well-supported by modern hardware such as CPUs, GPUs, etc, where different bit-widths can be used to represent and simulate weight values in +1). Additionally, this approach can be implemented directly in Py Torch (Paszke et al., 2019). The second approach, in contrast, requires specialized Boolean accelerator. Such hardware massively accelerate the computation by directly leveraging logic operations instead of real-arithmetic.
B.2 MULTI-CORE COMPUTATION STRATEGY IN $ 4.2
Boolean design, as used the paper, employs Boolean weights and operates using logic operations. It is distinct from bit-level operations.
Boolean design: Weights are Boolean logic variables, taking values TRUE TRUE/FALSE or -1/ + 1. Operations are logic-based, such xnor, and or, See Eq. 35 for example.
Bit-level operations: These, such as bit-serial implementations in C/C++, operate bit-by-bit on multi-bit variables. For instance, bit-level AND between two n-bit variables produces an n-bit result, where each bit is the ADN of corresponding pair of bits from the inputs. Bit-level operations like bit-serial are inefficient terms of latency, whereas Boolean logic operations are significantly faster compared to real-arithmetic operations such as multiplication.
25

---

Published as a conference paper at ICLR 2026
C CODE SAMPLES OF CORE IMPLEMENTATION
C.I BOOLEAN LINEAR LAYER AND OPTIMIZER
In this section, we provide example Python code for implementing Boolean linear layer based on the xor logic gate. This implementation is based on the PyTorch framework (Paszke et al., 2019). As done in Nguyen al. (2024), the class definition for the Boolean linear layer is presented in Algorithm 2, and its backpropagation detailed in Algorithm 3. Each Boolean kernel primarily implemented using this Boolean linear layer.
-etaaniiim--csttiizeee via PyTorch's s autograd system-is
We consider both cases the incoming backpropagation signal: Boolean- (see Algorithm 4), and real-valued (see Algorithm 5). The latter is the main use case in this paper. An example implementation of the Boolean optimizer used to update the layer's parameters provided in Algorithm 6.
Algorithm 2: Python code of XOR linear layer
Import torch
from torch import Tensor, autograd from typing import Any, List, optional, Callable
class xoRLinear (nn. (in.Linea:):
def init (self, featuress: int, out out Features: int, bprop: bool, A*kwargs): super (KORLinear, self) init_ (d___duureeaeeee,, out_feacures, **xwarga] self. holff.ooo__pprog
10
beul beal_bprop
12
def reset_ cactt_parratttrrsseee (self):
14
self felf.weight = nn. Parameeee (torch, (C, 2, self welf.waight.shappp
15
16
self.bias not None:
17
solf,bie. = nn. Parameter (tttrch.randinn (0, 2,
18
19
def forward f, X):
return XORFunction applyy (X, self, weight, self.bias, sulf. gelf.bosl__preee
Algorithm 3: Python code of the backpropagation logic of XOR linear layer
class XORFunct SxFuuctiin Lon (aut ograd. . Funccion):
Butat setaticmehod
def forward X, www._aee__bacaaaaaeeeeaaaeaaaea. (X, etx.bool_bprog bool_bprep
bool bool_bprop: booll:
$ Elementwise XOR logic corch.logica)
(X[;, Nona, ;), [[None, 1, :])
10
11
$ Sum over the ippu dimansion
8. (dim=2)
13
0-centered for with BatchNorm when preferred W. shappeel11/2
15
16
17
return 5
18
Rstaticmethod
20
def backward (ctx, 2):
ctx. bool bprup: G_X, G_M, 5_3 backward (etx, Z)
22
23
else: x_x, G_W,
backward_red (etx,
25
6_X, G_M, G_B, None
26

---

Published as a conference paper at ICLR 2026
Algorithm 4: Backpropagation logic with Boolean received backpropagation
def backward_bool (ctx, 8)
enn
Variat ion of ingut:
/delta neg
dolta (Loss) (Lass)/delta (x) xnor (2, (2 neg w) xor (2, k]
Variation of weights:
delta (xor 1x, M) /delta
neg
delta (loss) /dalta = xnor nz x) xor (z,x)
Variation bias:
10
bias xnor (blas, Ess Variation of bias is driven in
the
basis that of waight with
logic and irput True.
12
Aggrægetionn
13
Count the number of TRUEs sum over the Boolean data
Aggr TRUEs FALSEs TRUEs
TRUEs) 2TRUES
15
where TOT is the size of the aggregated cimension
**.
17
X, H, B ctx. tax.savvd_tenao
18
19
Boolean variation insut
20
6_X = torch, logical (x[;,
K[None, :,)
22
$ Aggregate over the out out_features dimension
6_x
sux. (dim-1) [3)]
24
25
i Boolean variation of weights
torch.
Nonel, None, :])
27
$ Aqgregate
the batch dimension (.shape(0)
29
6_ = G_m.
30
Roal ean variation
32
if B is not None:
Aggregate over batch dimension
34
9_B 8.sum (din=0) s.shape(0)
36
Return
37
return 6_X, G_N, G_B
Algorithm 5: Backpropagation logic with real received backpropagation
def beckward_real (ctx, 2): X, H, etx. tensors
Boolean variation input proceased using torch avoiding loop:
-> xor (Z: Real, W: Boolean) -2 emb (X)
cmb (8) T->1, F->-1 emb (N) 20-1
delta (Ioss) /delta (X) 2* (1-20)
x_x = 2. (1-2+W)
11
Boolean variation of weights processed using torch avoiding loop: xor (2: Real, X: Boolean) -2 emb (X)
13
14
-> emb (X) T->1, F->-1 -> emb (X) 2X-1
15
deleaaaaaee (Laas) /delta (W) 2 (1-2x) ...
16
G 2.t 0) mmm (1-2
18
Boolcan variation bias if is not None:
20
21
23
return G_X, G_N, G_B
27

---

Published as a conference paper at ICLR 2026
Algorithm 6: Python code of Boolean optimizer
class BooleanOptimg imi ser (torch. (torch.oprim.c
def
_iiiiiitttte (self, perams, lr: Flust):
supex (BooleanOpt imizeI, self)
(params, diet (1r-11))
Faz param_ in self, self,param par.eeaaaammage
paa.....aa.ggrupss: param_groupi acccmms' [torch.zeros zeros_ data) For param_group
parama'
param (r self._nb flipa
[0 for in paran_group params') ]]
@property
11
def ni mii_llip (self);
12
==l f. ___n__llpp pl solf._nn_ffigge
14
return n
16
def step (selF):
17
for paran_group in self.peram_ for idx, p in enumerate (param_ group (paraapp"ppaam self.update param_group, idx)
18
19
21
def (self, param: Tensor, param_groms dict, idx: int);
gropmm [idx] param_group/ sccumm" [idx] param_groapi
1r' I * param.grad.
23
param_group accums' [idx] accum
param_to flip accum (2+param.cata-]
25
param. param.data dpt.apaaaantoofflll...
torch. (param. data (apaaaaaaaaaaattt_fff_..
param_group ('acoums") [idx] [param_te_flie
0.
27
param_group ( rat [idx] = param to ta_ flip. Flast () .)ean
28
fil..nno_fipp +- float farram_to_flip.Fee () smm(()
C.2 SUCCESSIVE SVID KERNEL EXTRACTION
Algorithm 7 illustrate the Python code of the SVID algortithm extract the optimal Boolean weights and scaling factors for one kernel. Based on this, Algorithm 8 illustrates the succesive SVID algorithm to extract all kernels.
Algorithm 7: Python code of SVID approximation of FP matrix.
def svid_approx intrroximmaiiiooo i (x) **. Approximate the input matrix 'w" by a boolean matrix and rank-1 matrix: w_bool (a_out a_in.T)
Arga:
* storch. :ensor): Input tensor shape ts, m, n).
Returns:
10
tuple:
w_bell (tarch, Teasor): Boolean matrix of the shape ww,
12
reses storch.Ienson. Residual matrix, w w_bool x (s_out in..))
13
s (torch. Tensori: scaled first left singular vector of 'w"
out (torch.Tensor) Scaled first right singular
15
torch. lirch..inall..dd (v.data. float()). ()), full_matrices-Falsee)
17
18
w_bool torch. sign (w)
19
s_in terch.sqrt (8[0)) Vh [0, :] reshape (1,
20
s_out turch.ngrt (S()]) Ul:, 0] reshape (-2, 1)
22
M_res bool m (s_out, s_in)
24
return w_bool, w_res, eii, a_out
28

---

Published as a conference paper at ICLR 2026
Algorithm 8: Python code of successively extracts kernels from FP matrix using SVID.
def successive_svidii_tt. n_kernels):
**.
Perform successive SVID on she input matrix to extract Boulean kernels.
Argu:
K_Ep (torch. Tensor): : raput weight matrix.
n_kernela (inti: Number of iterations to extract kernels.
Returns:
10
1ist: List of dictionaries containing 'n_kernels" kernels, each has: w_bool (torch. Tensor): Boolean matrix. isin (torch, Tensor]: Iaput scaling vector. s_out (torch. Output scal ng vactor.
12
13
15
boolean_kernee
I]
17
M =_fp The input to SVID at first iteration is the origiral weight
18
19
for range in_ kernela)
20
t Extract the Baolean weights, residual, and scal ing vecters k_bocl, w_res, s_in, s_out avid aprixiiaruiiiii
22
t Save the extracted kernel
24
boolean_xerness.ppee appendd wx_bord w_boor"; N_bool, ra_int: s_in, "a_oust: s_out
25
input to
the iteration is the current residual matrix
27
* k_xea
29
return bcol benlean__ aalleannkkenaa kernels
D PROOF OF PROPOSITIONS
For completeness, include the proofs of Propositions related to SVID approximation used in the main paper.
D.1 PROOF OF BOOLEAN LINEAR REFORMULATION USING SVID
Proposition D.1. (Xu et al., 2024) Given the weight matrix W vp and input X, the linear layer can be reformulated as the following using SVID approximation, FF = W bool O (SoutS]n) as follows:
XW FP
[X 2 (T))
W honl
Sout
T
(36)
Proof.
Due to the SVID approximation, we have WFP & W
Then, we have:
(WWP) FP
X (iy) WP
(37)
[i
Σ X WFP
(38)
(i,s) W Soun]
(39)
Σ (.,)SSI(())) W boolly
(40)
X T [id]]
W (.ul
(41)
(x
s W bool
Soutyj
(42)
[+
{[(x
I [i.]
o s
W bool
O Sout
(43)
Thus, the proposition is proved.
29

---

Published as a conference paper at ICLR 2026
D.2 PROOF OF PROPOSITION 4.1
Lemma D.2. (Xu et al., 2024) Denote ae (W) the i-th biggest singular value of matrix W. The
following inequality holds:
01 ( (W) 2 (W).
(44)
Proof. By the definition of induced norm, we have:
0 (W) || ||2 max |
(45)
x
01 ( |I| = max
||
(46)
y-1lv12=1
In addition, because Vx, ||||| = 1, we have:
| Σ
Σ
W Rosllxpa
(47)
2
Σ
W Bbl
(48)
Σ
Σ W (+,))*
(49)
Wx 12-
(50)
Therefore
max 1y
W||le 2
max
|x||2
(51)
x. x||x||=11
+
(
(52)
Thus, the lemma is proved.
Proposition D.3 (Restated from Xu al. (2024)). For W RM write W UΣV its
SVD. Let vo U1.1) and b Vo V..1. [.1..- Similarly, denote W-=UEV its SVD; Sia and Sout are given as: Si Vor V [..1] and Sout %on (oUU(.)) We decompose the matrix as
W Wbool ① W = Wbool O (Sont We then have:
W W bool
So 1
W ab T
(53)
Proof.
We denote the following error matrices:
E W ab
(54)
E² W
S
T
(55)
Multiplying W boot with both sides of Eq. 55, we have:
W bool ④ W W booo Sout Snn
T
W booo ④ Eg
(56)
W Woool Sout
W bool Eg.
(57)
30

---

Published as a conference paper at ICLR 2026
Thus, have:
W
W hool
@
Sout& l1p
bool O Ezll
(58)
W Whoolf. boollsul
+ E3i.
(59)
E
(60)
fd
E2 1|
(61)
For SVD decomposition, the norm of the above error matrices in the rank- approximation the um of squares of all singular values except the largest one. In particular, we have:
E |
(W)),
(62)
of ((W))
(63)
2=2
Since | = [|| we have:
Σ (w) = Σ of ((W)
(64)
==1
E 1 (w)
|| (l
(65)
Thus, according to Lemma D.2 and Eq. 61, we have:
Eg 1 < E 1
(66)
W W bool Sout
<
W ab
(67)
Thus, the proposition proved.
D.3 PROOF OF PROPOSITION 4.3
Proposition D.4. For W E Rm we denote [W|- = UEV its SVD. Sm and Sout are given as:
Sia Vo1 V 1. and Sout Vo.U We decompose the matrix as W W @ |W| z
W bool @ (S) We then have:
W bool @ cd I|. Vc g= AA Raxl
W- Wbool
Sont Sout s
w
(68)
Proof. Similar to the proof of Proposition 4.3, denote the following error matrices E, Sont I and E, | ed We have that
W
W boot
@ W Wbool bool Sout W bool Ej
(69)
W Wbool bool
Sout
W booo 2 ET.
(70)
Therefore,
W W hool So
T
| bool E Σ
W2 W bool(t.j EijjJ] Σ E [(.)]]
|
(71)
Similarly, we have that
C ab T/2 ||
W
W hool
(72)
31

---

Published as a conference paper at ICLR 2026
Thus, need to show that Additionally, we denote the rank-k approximation to | by SVD as Sy:
E
1 - E2l2
(73)
S = Σ o, b (.9)-
[::)-
(74)
With this notation, we have that S₁ Sout Sin is the rank- approximation of [W] by SVD.
From Eq. 73, we need to show that if there is an arbitrary rank-1 approximation [W]. Pi ed. we then have
< W ed
W
(75)
This can be done by using the Eckart- Yoo---Yonn---aa-t theorem (Eckart & Young, 1936). First, we have that
n
|| 11 W
Sout 11
Σ of U
of.
(76)
F
By the triangle inequality with the spectral norm, if W C+D then a ( S (C)+ (D). Suppose the C₃ and Dy denote the rank-k approximation to C and D by SVD method, respectively, Then, for any i.j 2 we have
(C) + (D) (C + (D-))-1)
(77)
2 [
(78)
2 ( -Sitj2) (since rank(C,-11 Sitj Si+1) 2-
(79)
-(W))-((W)).
(80)
Because 02 (P)) = 0, when C = |W| - Pi and D = Pi we have that for 2 1,j 2.
(:W P1) 0 ((W]) As a result,
W P P.l 11 Σ
I"
P
||W| - S i = 2"o:(
(81)
E21
N
E
(82)
W
W bool ① cd
2 W
W
bool Sout in
(83)
F
Hence the proposition is proved.
32

---

Published as a conference paper at ICLR 2026
E DETAILS ON KERNEL ALLOCATION
E. WEIGHT IMPORTANCE ESTIMATION
We assess the importance of linear weight in the original FP model by comparing the representations at its input and output. Let X E Rdxx and E Raaxm denote the input and output matrices of linear layer, respectively, where d is the number of samples, and n and m are the input and output feature dimensions. We hypothesize that a weight is important if it significantly transforms the input representations. For example, a weight matrix equivalent to the identity does not alter the representations and thus would be considered unimportant. To quantify this transformation, we use robust metric for comparing neural representations.
Various similarity measures can be used for this purpose, such cosine similarity, as done in (Gromov al., 2025). this work, we adopt PWCCA Morcos et al. (2018), which is particularly well-suited for essential property given that large language
setting: it is invariant to linear transformations- models (LLMS) are primarily composed of linear layers-and effectively captures shared structure while filtering out noise Morcos et al. (2018).
Specifically, we define the importance score as:
h=
PX, Y),
(84)
where e denotes the number of canonical vectors used in the comparison (typically, c= min m)). The matrices X and Y are obtained by simply forwarding set of data samples through the network. In our experiments, we use 128 random samples from the WikiText2 training set to estimate the importance score. Here, /PWCCA, represents the projection-weighte correlation along the i-th canonical direction. The following section describes in detail how this correlation is computed.
Algorithm 9: Kernel allocation.
Input
T
/ model expansien limit /
le(4 E TR Tg-WxKKum for E [1, Kmax), E [1, Nw]
/* residual approx error x/
[hi] KI
/ weight importance scores //
p = [pd] xw
/* weight size ratios
Initialize
= [1...., 1)" of length Nw
/* starting choice x/
f = k Kmax
/x feasible indicator x/
C= log 1
Ch@E; p
/N where @ broadcasted over E columns
10
While not all is False do 0. 1=1:=00 0;
for
Nw do
13
[[]] True then
14
9 = Cl1, k[[] C(1, + +]; /* gain by increasing kernel size by
Append to append to g: Sort g in decreasing order. and arrange accordingly; for (9.1) in (g. 1) do
16
17
k; k:
k kj[0] +
if k[] = k[[] +
7' then
21
21
break:
/+ escape the for 1oop +/
else
f[i] False;
25
f and(f,k ( < Kmax);
/+ element-wise legical and x/
26
return k
wrejection-weighted Canonical Correlation Analysis. Canonical Correlation Analysis (CCA)
finds bases for two matrices such that, when the original matrices projected onto these bases, the
33

---

Published as a conference paper at ICLR 2026
resulting projections maximally correlated. Without loss of generality, assume that S m. For S i I n, the i-th canonical correlation coefficient ps is given by:
Pj
corr Norr(wwix Yyw')
(85)
w subject to Xw 1 XW Vj <i
Ywy Yw Vj <.
The vectors w'x E R and w/y R that maximize P; are called the canonical weights. These weights transform the original data into the canonical variables Xw'x and Ywy.. The constraints in Eq. 85 enforce orthogonality among the canonical variables, ensuring that each successive pair captures a distinct mode of correlation.
The mean CCA correlation is then computed as:
Σi=
PCCA
(86)
where n is the number canonical correlation coefficients considered.
CCA is sensitive perturbation when the condition number of X and Y is large. To imporve robustness, Morcos et al. (2018) propose a strategy to reduce this sensitivity, which they term "projection-weightedd CCA" (PWCCA).
E i= OgPi
PPWCCA,
E.=1 O3
a
Σ
xy)1,
(87)
where xj is the j-th column of X, and h, = Xw> is the vector of canonical variables formed by projecting X to the i-th canonical cooridate frame.
E.2 KERNEL ALLOCATION ALGORITIIM
Algorithm 9 illustrates the details of our algorithm for kernel allocation.
F THEORETICAL ANALYSIS OF TRAINING COMPLEXITY
Consider linear layer without bias, defined as Y = XW where X E BBXL and W E NXM, Here, B is the mini-batch size, is the sequence length, N is the input dimension, and M is the output dimension. We analyze the number of multiplications (MULs) required.
Latent-weight approach (same cost as full-precision training):
Forward: B L N M (FP16-FP16 MULs)
Backward w.r.t. weights: B x L x N x M (FP16-FP16 MULs)
Backward w.r.t. inputs: X L x N x M (FP16-FP16 MULs)
Total: 3 X B
N M FP16-FP16 MULs
Boolean approach with K kernels: (assuming FP16 gradients for fair comparison). As shown in the main text, only the final Boolean kernel needs to be fine-tuned. The number of multiplications becomes:
Forward: K B L N M (BOOL-FP16 MULs, using all K kernels)
Backward w.r.t. weights: 1 B L N x M (FP16-FP16 MULs, for last kernel only)
Backward w.r.t. inputs: 1 x B x L x N x M (B00L-FP16 MULs, for last kernel only)
Total: +x+ N x M BOOL- 2016-FP16 MULs, and B x L x N x M (P16-FP16 MULs
Since K is typically small (c.g.. 2-4) while and L are large (thousands), most computation shifts from FP16-FP16 to the more efficient B001-FF11 operations. If we ignore the BOOL-FP16 MULss
34

---

Published as a conference paper at ICLR 2026
the FP16-FP16 operations reduced by factor of 2/3 (i.e., 66.7% reduction). Remarkably, this reduction is achieved while using more kemelss and attaining better performance, yet with significantly lower training complexity. According
BitNet (Wang et al., 2023) (Table 1), for = 512 and LLaaA-like 13B model on 7 nm hardware, 1Bit-FP16 operations yield an energy saving of approximately 56x compared to FP16-FP16. Hence, our method achieves substantial training efficiincy. Importantly, BitNet is a latent-weight approach, with efficiency gains realized primarily during inference, whereas our method provides significant benefits already during training and fine-tuning.
We note that the above analysis does not include optimizer cost. The latent-weight approach typically relies on Adam, which requires two full precciscnnioon momenta per parameter and a complex update rule involving multiple normalization statistics. By contrast, our Boolean approach employs a Boolean optimizer requiring only one full-precision momentum per parameter, coupled with much simpler update rule (see Eq. 3). This further underscores the reduction in overall training complexity offered by our method.
G ADDITIONAL EXPERIEMENTAL RESULTS
G.1 ADDITIONAL INFORMATION OF EXPERIEMENTAL SETTINGS
We use 12 Nvidia GPUs of Tesla V100 for our experiments. We follow exactly the experimental settings in Jo et al. (2024). The results of the baselines in Table are taken from Xu et al. (2024); Jo et al. (2024).
G.2 ON THE CHOICE OF KD Loss
logis (Forward KI.)
C
10
Danis
Wiki2
C4
Forward KL
31.39
28.50
Reverse KL
33.14
29.46
Symmetric KL.
32.67
29.26
10
JS Divergence
31.78
28.69
TV Distance
33.02
29.56
0
20000
40000
60000
40000
60000
Iteration
Iteration
Figure 12: The training convergence of Lis, and Llogits, measured by Forward KL, and the final results with respect the choice of D Diogits-
Fig. 12 illustrates the convergence and results of using different choices for Diogits Eq. 10. Despite its simplicity, forward KL. achieves the best performance. More complex measures, such as total variance (TV) distance (Wen al., 2023) and Jensen-Shannon (JS) divergence (Agarwal et al., 2024), offer no significant benefits in case. Furthermore, observe that the final perplexity is strongly correlated with Ligiit using forward KL, but not with Lis, as shown in Fig. 12 and Fig. 6. As result, employ the forward KI. in all experiments.
G.3 RESULTS OF DIFFERENT NUMBER OF KERNELS ON LLMS
To complement the Table 2, Table shows the benchmarking results LLMS using our MBOK method with varying numbers of kernels per weight. Consistent with the observations made smaller models in $ 6.1.1, we observe that increasing the number of kernels generally improves performance. However, the performance gains begin to diminish noticeably beyond three kernels.
35

---

Published as a conference paper at ICLR 2026
Table 6: Perplexity and zero-shot accuracy results our MBOK method with different number of kernels.
Perplexity (4)
Zero-shot Accuracy (f)
Model
Method
Wbits
Wiki2
C4
BoolQ
PIQA
Hclla.
WinoG,
ARC-e ARC-c
Average
MBOK (2 kernels)
2x
16.13
16.61
58.53
70.67
48. 11
56.75
48.19
27.90
$1.69
OPT-1.3B
MBOK kemels)
3x1
15.30
15.68
60.64
70.78
50.71
56.83
48.82
28.49
52.71
kemnes)
14.83
14.92
MBOK (2 kernels)
2x
6.83
8.53
69.20
74.32
64,80
60.30
49.05
34,90
58.76
LLaMA-7B
MBOK kemels)
3x1
6.20
7.76
67.89
76.15
68.91
63.30
48.94
37.62
60.47
kemnek)
4x
MROK (2 kernel))
2x
6.17
7.88
68. 10
76.33
69.88
64. 17
52.34
37.88
61.45
LLaMA-13B
MBOK kemels)
xx1
5.58
7.15
67.39
77.74
73.37
66.61
54.04
41.21
63.39
kernels)
5.38
G.4 ADDITIONAL RESULTS ON LLAMA-2
Table shows the results on LLaMA2-13B (Touvron al., 2023b). Similar the Table 2, the results of the baselines are taken from Xu et al. (2024) and Jo et al. (2024). It is clear that our method consistently outperforms baselines across different metrics and model sizes. This further emphasizes the robustness of our approach across various types of models.
Table 7: Perplexity and zero-shot accuracy results of Float16, quantized and binarized LLaMA2 models.
Perplexity ()) Wiki2
Zero-shot Accuracy (t) Hella.
Model
Method
Wbits
C4
BoolQ
PIQA
WinoG. ARC-e ARC-c Average
FP16
16
5.47
6.97
71.10
76.88
72.94
67.09
53.58
40.61
63.70
PB-LLM
76.75
85,92
62.17
$2.82
26.87
26.89
24.31
40.53
BiLLM
27.72
36.34
62.14
59.19
34.22
26.54
45.06
OneRit
70.40
54.24
LLaMA2-7B
71.55
OPTQ
7.7e3
NaN
42.97
49,46
26.19
50.28
26.77
28.58
37.38
LLM-QAT
1.1c3
6.6c2
59.14
50. 12
25.10
49.08
26.26
26.96
35,89
OmniQuant
31.21
64.34
58.69
56.53
33.87
51.22
33.63
24.32
43.12
MBOX [Ours]
2x1
6.87
8.74
66.94
74.97
65.59
61.72
44.82
34.21
58.04
MBOX [Ours]
3x1
6.12
7.81
65.46
75.79
69.59
62.04
49.11
37.80
$9.97
FP16
16
4.88
6.47
68.99
79.05
76.62
69.77
$7.95
44.20
66.10
37.82
27.19
62.51
38.05
27.73
OneBit
65.66
71.60
60.07
45.76
31.74
LLaMA2-13B
MoS
66.12
73.72
63.80
58.98
45.71
33.19
OPTQ
2.1c3
3.2c2
40.61
51.74
25.67
$1.85
25.46
27.30
37.11
LLM-QAT
5.1c2
1.1c3
39.85
49.08
24.37
$1.38
27.15
24.32
36.03
OmniQuant
16.88
27.02
62.05
62.24
50.34
$3,20
40.66
29.61
49.68
MBOX [Ours]
2x1
5.97
7.85
66.32
75.84
70.24
62.51
50.00
37.46
60.44
MBOK [Ours]
3x1
5.35
7.07
66.80
77.59
73.79
65.27
54.47
41.89
63.30
G.5 GENERATION QUALITY
Table 8 illustrates the generated text given zero-shot arbitrary questions from LLaMA-13B with FP16 weights and MBOK using three kernels. It is clearly that our model able to produce meaningful and coherennt text compared to the original FP16 model.
36

---

Published as a conference paper at ICLR 2026
Table 8: Comparison of generation quality (non-curated) on LLaMA-13B models with FPI6 and MBOK using three kernels.
Prompt: What are the capitals of China, US and Ruasia?
FP16:
The capital of China is Beijing. The capital of the United States is Washington D.C. and the capital of Russia is Moscow.
MBOK:
The capital of China is Beijing. The capital of the US is Washington D.C. and the capital of Russia is Moscow.
Prompt:
have friend who lives in London and I would like to would go there from Paris?
visit him.
FP16:
You can take the Eurostar from Gare du Nord in Paris to St. Pancras Station in London. The trip takes about 2 hours and 15 minutes.
MBOK:
The fastest way to get from Paris to London is to take the Eurostar train. The Burostar departs from the Gare du Nord station in Paris and arrives the st. Pancras International station in London. The trip takes about 2 hours and 15 mi minutes.
Prompt:
What is Fete Travail?
FP16:
It an annual holiday celebrated all over the world that resulted from the labour union movement, to celebrate the economic and social achievements of countries celebrate Labour Day
workers. The majority on May Day, or May 1, and it is popularly known as May Day and International Workers' Day
MBOK:
In France, May 1st is a public holiday. It is also known as Labour Day, International Workers_ Day, May Day. It a celebration of the social and economic achievements of the labouz movement. It originated from the eight-hour day movement, which advocated eight hours for work, eight for recreation, and eight for rest.
Prompt:
What are the main ingredients in pho, Vietnam_s iconic noodle soup?
FP16: Rice noodles, beef or chicken, onions, ginger, cilantro, bean sprouts, basil, lime, hoisin sauce, Sriracha, and fish sauce.
MBOK:
Bone broth, rice needles, fish sauce, herbs and spices, and beef, chicken, or seafood, depending on the type of pho yourre eating. Pho is usually served with bean sprouts, Thai basil, lime wedges, and hoisin and sriracha sauces.
G.6 ADDITIONAL BASELINES
G.6.1 COMPARIONS WITH QUIP AND SHIFTADDLMM
Both QuIP (Chee et al., 2023) and ShifiAddLLMM (You et al., 2024) are PTQ method for LLMS. QuIP a two-step process that leverages the insight that quantization performs better when weight and Hessian matrices are incoherent. It uses adaptive rounding procedure to minimize quadratic proxy objective, which measures the error between the original and quantized weights. Additionally, it applies pre- and post-processing steps using random orthogonal matrices
the weight and Hessian matrices are incoherent. Conversely, our method does not employ either these complicated pre- and post-processing steps costly Hessian matrices. Meanwhile, ShiftAddI I.M is post-training
37

---

Published as a conference paper at ICLR 2026
reparameterization process, which quantizes each weight matrix in the LLM into set of binary matrices and group-wise scaling factors. he original multiplication between activations and weights is then reparameterized into: (1) bitwise shifts for the activations, using the power-of-two quantized scaling factors, and (2) additions the results, guided by the binary weight matrices; this process can be implemented using look-up tables (LUTTs) on GPUs.
Table presents results on OPT models, with competitor results extracted from their respective original papers. Notably, ShiftAddLLM utilizes more computationally expensive group quantization, whereas our method does not. Our results clearly demonstrate that our approach consistently and significantly outperforms these baselines, particularly in the 2-bit scenario.
Table 9: Comparisons with QulP, ShiftAddL.LM using OPT models.
BIT-WIDTH
METHOD
OPT-125M
OPT-350M
OPT-1.3B
QuIP (Chee et al., 2023)
34.22
25.19
16.21
ShiftAddLLM (You et al., 2024)
31.29
24.24
21.53
MBOK [Ours]
29.10
23.12
15.03
QuiP (Chee et al., 2023)
347.40
672.30
41.64
3
ShiftAddLLMM (You et al., 2024)
51.15
40.24
29.03
MBOK [Our]
28.60
24.54
16.13
G.6.2 COMPARIONS WITH BITSTACK, DB-LLM AND AWQ
While BitStack (Wang et al., 2025) also decompose weights using SVD, its core method and goal fundamentally differ from method. BitStack is training-fiee method primarily aimed at saving storage for inference. In contrast, our method not only converts FP models into Boolean models but also includes further fine-tuning, with the goal of achieving low complexity in both training and inference. Furthermore, while BitStack packs the extracted binary matrix into GPU-supported data types to reduce inference memory, and its approach to loading residual blocks relies on their influence on perplexity, our approach to residual block management is distinct.
DB-LLMM (Chen al., 2024) is limited to fixed decomposition into two binary matrices, whereas MBOK method generalizes to an arbitrary number of Boolean kernels. In DB-LLM, the full-precision knowledge is preserved only through scaling factors and binary matrices derived implicitly via thresholding. There is no formal analysis proving the optimality of this formulation. In contrast, thanks the SVID our approach, each extracted kernel is accompanied by an optimal scaling vector and Boolean matrix. This allows us to only finetune the last kernel to calibrate the entire model. Like most existing binary LLMS, DB-LLM relies on full-precision latent weights during training and finetuning. Our method does not require this, as it directly operates in the Boolean domain. This distinction is particularly important in the LLM context, where training and finetuning can be computationally expensive.
Table 10 compares our method, MBOK (with 2 kernels), against BitStack, DB-LLM, and AWQ (Lin et al., 2024) on LLaMA2-7B. It is evident that our method consistently outperforms all baselines.
Table 10: Comparisons with AWQ, BitStack, DB-LIM using I.M.MA27BB with 2-bit setting.
METHOD
Wiki2 (L)
ARc- (1)
ARC-c (t)
PIQA (t)
Hella. (t)
WinoG. (t)
AWQ (Lin et al., 2024)
1.8c5
26.3
26.7
50.9
26.5
49,3
BitStack (Wang et al., 2025)
29.93
32.3
25.6
62.4
42.8
53.6
DB-LLM (Chen et al., 2024)
7.23
45.2
33.5
73.1
61.9
61.7
MBOK [Ours]
6.87
44.8
34.2
75.0
65.6
61.7
38

---

Published as a conference paper at ICLR 2026
G.7
EFFECTS OF KNOWLEDGE DISTILLATION
Diggis
Number of Flips
15000
6000
10000
Wiki2
C4
5000
31.47
28.62
2000
No
0-
25000
50000
25000
50000
25000
50000
lteration
lteration
lteration
Figure 13: Study on the effect of using knowledge distillation on OPT-125M with Boolean kernels.
Fig. 13 presents a comparison between training with and without Knowledge Distillation (KD). It evident that employing KD outperforms the baseline terms test perplexity on the Wiki2 and C4 datasets, behavior further, we visualize the convergence of Liggits and Lis- Aided by the informative guidance from the teacher, convergence with KD significantly faster. Furthermore, the model learns more effeetively-lleerriinng the additional signal from the teacher-as evidenced by the higher flipping rates compared to training without KD,
it provides more informative guidance during training. To investigate this
G.8 ANALYSIS OF SCALING VALUES
Kemel 1- KS: 0,005
Kernel 2 KS: 0.005
Kernel 3 KS: 0.005
Kerne 4 KS: 0.430
10.0
10
5
Initialization
Finetumed
KDE
KDE Finetuned
Figure 14: Histogram of output-scaling values for the first linear layer of OPT- 125M with four kernels, shown at initialization and after finetuning. The Volmogorov-Sminee (KS) statistic is also reported to quantify the difference between the scaling-valuc distributions before and after finetuning.
Fig. 14 illustrates how the output-scaling values of the four kernels in the first linear layer change from initialization to after finetuning. All output-scaling values are learnable; however, only those associated with the last kernel exhibit substantial shift during training. This is evident from both the histogram changes and the corresponding Kolmogorov-S -mimnov distance.
After finetuning, the scaling values of the last kemnel become significantly larger and more dominant, whereas the scaling values the other kernels change only minimally. This observation supports theoretical analysis: successive SVID extraction provides sufficiently strong initialization for the low-order kernels, and finetuning primarily the last kernel is already adequate.
G.9 CONVERGENCES OPT MODELS
Fig. 15 shows the training convergences of MBOK using 3 kernels with OPT models.
39

---

Published as a conference paper at ICLR 2026
10
Elogss
10
10
M We
10000 20000
30000
40000
10000
20000
30000
40000
lteration
Iteration
(a) OPP--15MM
Lagss
x 10
x 10
10
0
10000
20000
30000
40000
10000
20000
30000 40000
leraation
Iteration
OPT-350M
10a
Coola
10'
10
www we Wi www.
10000
20000 30000 40000
10000 20000 30000 40000
Iteration
Iteration
(c) OPT-1.3B
10a
+
101_
10000
20000
30000
40000
10000 20000 30000 40000
Itoration
Iseration
(d) OPT-2.7B
10.
Ciegis
10a
10'
2 10¹
10"
10000
20000
30000
40000
0
10000
20000 30000 40000
OPT-6.7B
10a
10a
10
10000
20000
30000
40000
10000
20000 sod00 40000
Itratiin
Ieration
(1) OPT-30B
Figure 15: The training convergences of MBOK using 3 kernels with OPT models.
40

---

Published as a conference paper at ICLR 2026
G.10
EFFECTS OF SUCCESSIVE SVID INITIALIZATION
Diegit
Cx
Number of Plips
15000
60000-
40000
5000
2-
20000
Sue. SVID
Rand. Init.
1.6E4 1.1E4
31.47
28.62
0
25000
50000
25000
50000
25000
50000
Figure 16: Study on the effect of using our successive SVID strategy and random initialization on OPT-125M with 2 Boolean kernels.
Fig. 16 compares our proposed successive SVID initialization with random initialization. is clear that our method delivers far better results, while the random initialization often fails to converge. Moreover, our initialization enables the model to learn efficiently, whereas the random initialization causes the model to struggle, as reflected by the large number of Boolean flips.
G.11 DISCUSSION ON LATENCY AND COMPARISON WITH VECTOR QUANTIZATION
Scalar and Vector Quantization. In the context LLMS, scalar quantization and vector quantiza- tion are two different approaches for compressing weights. Scalar quantization maps each weight or activation independently to a smaller set discrete levels (c.g.. 32-bit floating -oini to or 4-bit integers). It is simple, hardware-frient friendly, and widely used in practice, but it ignores correlations across dimensions, potentially discarding fine- fine-grained structure. Vector quantization (VQ) instead compresses entire vectors (e.g., weight groups) by replacing them with indices into learned code- book of representative vectors. By capturing cioss-dimensional correlations, VQ often achieves higher compression, particularly for large embedding tables. However, codebook training is more complex, and inference requires index lookups to reconstruct vectors. This adds significant overhead both quantization and dequantization, leading to much higher latency compared to scalar methods.
Our method is native 1-bit weight design, its nearest baselines are scalar weight quantization. As result, for fair comparison, in the main text we mainly consider state-of-the-arr scalar quantization like OmniQuant (Shao et al., 2024), OPTQ (Frantar et al., 2023), LLM-QAT (Liu et al., 2024c) as the main baselines. Nevertheless, for completeness, we also compare our approach against state-of-the-ar ultra low-bit vector quantization (VQ) methods for LLMs, including QTIP (Tseng et al., 2024b) and QUIP# (Tseng et al., 2024a) in 2-bit setting, specifically on LLaMA-7B with a sequence length of 2048 (results taken from the QTIP paper). The results summarized in Table 11. Remarkably, method's ' performance comparable to these state-of-the-ar (SOTA) VQ methods. This noteworthy given that approach directly utilizes native Boolean weights, eliminating the need for the very costly quantization and dequantization of high-dimensional vectors inherent in VQ.
Table 11: Perplexity comparison with SOTA vector quantization methods using LLaMA-7B.
METHOD
Wiki2 (4)
C4
QUIP# (Tseng et al., 2024a)
6.86
8.36
QTIP (Tseng et al., 2024b)
6.52
7.99
MBOK [Ours]
6.83
8.53
Empirical Evidence of Latency Gains. To demonstrate the practicality of our approach even modern hardware such as GPUs, we leverage the recently introduced BitBLAS library (Wang et al., 2024) for 1-bit matrix multiplications. Using FP16 activations with INT1 weights, latency of linear layers in LLaMA-7B (Table 12) and L LLaMA-13B (Table 13) under an inference batch size of 1, evaluating our method MBOK with two kernels. Our results show that MBOK achieves up to
measure the
Inttps:/ //g thub. com m crosoft/Ht EBLAS
41

---

Published as a conference paper at ICLR 2026
an 8.7x speedup over FP16 baselines, while substantially outperforming existing binarization and scalar quantization methods, as detailed in the main text. We also benchmark against 2-bit QUIP# and QTIP using the authors' official implementations"? All experiments conducted A100 GPU.
Remarkably, our method is not only much faster than these VQ baselines but also delivers comparable performance. This is expected, VQ-based methods incur significant overhead from the costly encoding and decoding steps required to realize their high compression ratios. Taken together, the results highlight that our native Boolean approach offers compelling and efficient alternative to state-of-the-art vector quantization methods. With dedicated Boolean hardware accelerators, the performance gains would be even more pronounced.
Table 12: Measured latency (ms) of linear layers in LLaMA-7B, with values in parentheses denoting speed-up relative the FP16 baseline.
WEIGHT SIZE
FP16
QUIP# (Tseng al., 2024a) QTIP (Tseng et al., 2024b)
MBOK (Ours)
4096 x 4096
0.10697
0,46196 (0.23x)
1.37137 (0.08x)
0.04989 (2.14x)
4096 x 11008
0.27935
0.55526 (0.50x)
3.13633 (0.09x)
0.05136 (5.44x)
11008 4096
0.27664
0.55988 (0.49x)
3. 3.16067 (0.09x)
0.05117 (5.41x)
Table 13: Measured latency (ms) of linear layers in LLaMA-13B, with values parentheses denoting speed-up relative to the FP16 baseline.
WEIGHT SIZE
FP16
QUIP# (Tseng al., 2024a)
QTIP (Tseng et al., 2024b)
MBOK (Ours)
5120 x 5120
0.16540
0.62260 (0.27
1.96368 (0.08x)
0.05074 (3.25
5120 x 13824
0.42830
0.62836 (0.68
5.23681 (0.09x)
0.05098 (8.40x
13824 x 5120
0.43411
0.62840 (0.69x)
5.21 (0.08x)
0.04987 (8. x0)
H ETHICS STATEMENT
This work makes a fundamental contribution to machine learning methodology. does not involve human subjects, sensitive data, applications with direct societal ethical risks. We do not foresee any immediate ethical concerns arising from this research.
I REPRODUCIBILITY STATEMENT
We provide detailed descriptions of all algorithms and illustrative code for the core components. Experiments are conducted on standard benchmarks using established testing procedures, and all experimental details and settings are fully declared to facilitate independent reproduction of our results.
THE USE OF LARGE LANGUAGE MODELS
We used large language models (LLMs) solely for non-substantive assistance, including grammar refinement and summarizing relevant literature. All research ideas, analyses, and conclusions are the authors' own.
Pnttps://g thub. thub.com/Cornel1 1
42


Reviews:
[{"review_id": "Yq2M1ZM4i0", "reviewer": "Reviewer_oMmC", "summary": "The manuscript proposes a new framework, MBOK, for efficiently compressing and fine-tuning LLMs entirely in the Boolean domain. Unlike prior binarization or quantization methods that rely on full-precision latent weights and gradient approximations, MBOK introduces a multi-kernel Boolean structure based on BOLD, and further integrates strategies such as adaptive bit-budget allocation and knowledge distillation. Experimental results show that the proposed method achieves competitive performance compared to other contemporary approaches.", "questions": "1. How are the benefits and stability of knowledge distillation demonstrated or quantified?\n\n2. Why does BitNet collapse with extremely high perplexity in the reported results, and are there comparisons on broader benchmarks such as ARC-Easy or MMLU?\n\n3. In Figure 9, how can the model allocate more than 20 kernels under an average bit budget of 3.5 bits?\n\n4. When fine-tuning only the last kernel, what are the approximate scaling values of each kernel at convergence? Does the last kernel ever become significantly more dominant than earlier ones?\n\n5. Can the proposed approach be extended to activation quantization, or is it limited to weight compression only?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The manuscript is the first to eliminate the dependency on latent weights in LLMs through the use of BOLD, and achieves strong compression and training results by combining multi-level weight decomposition, global bit allocation, and knowledge distillation strategies.\n\n2. The discussion of the technical approach is thorough and detailed, with extensive comparisons to similar compression methods such as BinaryMoS and BitStack.\n\n3. MBOK demonstrates excellent empirical performance, outperforming other methods in both efficiency and accuracy even under existing hardware conditions.", "weaknesses": "1. The innovation is somewhat ambiguous. Techniques such as BOLD, SVID, knowledge distillation, and bit allocation have all appeared in prior works. For example, approaches similar to SVID and bit allocation were already explored in BitStack.\n\n2. The paper’s structure and organization are somewhat confusing. It is unclear why bit allocation is presented at a separate hierarchical level compared to SVID and knowledge distillation. Conceptually, bit allocation and SVID seem to belong to the architectural design of MBOK, while knowledge distillation is more of a training strategy, which should logically be applied after bit allocation rather than before it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5UmsOZEMxv", "reviewer": "Reviewer_mF6v", "summary": "This paper proposes MBOK (Multiple Boolean Kernels), a novel framework for training and fine-tuning large language models entirely in the Boolean domain, without relying on floating-point latent weights. The key idea is to represent each weight matrix as a sum of multiple Boolean kernels, each with distinct binary weights and scaling factors, thereby improving representational capacity while keeping computation highly efficient. The authors further introduce a successive SVID-based extraction method to transfer knowledge from full-precision models and a dual-level knowledge distillation strategy for refinement. Extensive experiments on the OPT and LLaMA families show that MBOK achieves performance close to FP16 with only 1 to 2 bits per weight, surpassing state-of-the-art binarization and ultra-low bit quantization methods in both perplexity and zero-shot accuracy, while greatly reducing memory and computational costs.", "questions": "1. Could the authors evaluate the proposed MBOK framework on more recent and competitive open-source models, such as Qwen3, LLaMA3.2, to demonstrate its generalization across architectures?\n2. Can the authors provide additional analysis or experiments on the training stability and convergence characteristics of Boolean-domain optimization, especially for larger models (e.g., 70B parameters and beyond)?\n3. The authors have shown that fine-tuning only the last Boolean kernel achieves the best efficiency-performance trade-off. Could the paper include a deeper analysis explaining why joint optimization of multiple kernels leads to degraded performance, and whether this trend holds consistently across larger models or more complex tasks?\nThis is an interesting work, and if you can address my questions, I would be happy to raise my score.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a creative sound approach to Boolean-domain training for large language models. The proposed MBOK framework is conceptually clear and well-motivated, trying to address key limitations of existing binarization methods by removing reliance on floating-point latent weights. The methodology is thoughtfully designed, combining multiple Boolean kernels, knowledge distillation, and adaptive kernel allocation. Experiments are extensive and carefully executed, demonstrating consistent improvements over strong baselines in both efficiency and accuracy.", "weaknesses": "While the proposed Boolean-domain framework is novel, the evaluation is somewhat limited. The experiments mainly focus on older model families such as OPT and LLaMA1/2, leaving uncertainty about the method’s effectiveness on newer architectures (e.g., Qwen3). Moreover, the discussion on training stability, convergence behavior, and scalability to very large models (70B and above) is also relatively brief, making it difficult to assess the robustness of the approach under real deployment conditions. To achieve extremely high training efficiency, the authors adopted a strategy of fine-tuning only the last Boolean kernel (Section 6.1.2). While experiments show this is the most efficient strategy, this approach may limit the model's ultimate performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OtBPCTWfTN", "reviewer": "Reviewer_Mstc", "summary": "This work proposes a multi-boolean-kernel QAT method for LLMs. Extending BOLD’s single-kernel design, it integrates successive SVID and adaptive mixed-precision, outperforming prior binarized QAT methods. It also enables direct boolean domain LLM fine-tuning to save training resources.", "questions": "1. Is there any comparison of actual training time to demonstrate that MBOK is more efficient than other binarized QAT methods?\n2. For other questions, please refer to the weaknesses.\n\nI would be willing to reconsider my rating if the authors can address these concerns.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This work is supported by solid theoretical foundations, with a detailed explanation of the computational process of multi-boolean-kernels during training.\n2. The approach of training only the last kernel is insightful, and the authors have provided detailed experimental comparisons and explanations to validate it.\n3. Experimental results demonstrate that MBOK outperforms previous binarized QAT methods in both performance and resource efficiency.", "weaknesses": "1. The novelty of this work is limited, as its core contribution is merely a natural extension of BOLD, expanding from a single kernel to multiple kernels.\n2. The authors claim that MBOK lies on the Pareto frontier. However, the evidence provided in Table 3 only compares it with low-bit PTQ methods such as RTN and GPTQ. Given that MBOK is a QAT method, a fairer evaluation would involve comparisons with advanced low-bit QAT methods like SpinQuant and EfficientQAT.\n3. SpinQuant with W4A8 quantization can achieve better results (as shown in Table 1 of their paper) than MBOK with 4 kernels (i.e., W4A16). If compared under the same activation bit-width, SpinQuant may hold even greater advantages, which would undermine the scalability of multi-kernel designs (the core contribution of this work).\n4. Key ablation studies are missing. Specifically, analyses are needed on how initialization via successive SVID impacts final distillation performance, and on the effectiveness of the kernel allocation strategy (i.e., mixed precision) compared to uniform precision under the same kernel budget. Including such ablation results would strengthen the work’s contribution and novelty.\n5. Several typos and errors require correction. For instance, in Table 1, BiLLM also employs higher-bit salient weights. In Table 3, \"QPTQ\" should be corrected to \"OPTQ\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UkmDuJ4gAa", "reviewer": "Reviewer_xHhd", "summary": "This paper introduces MBOK, a binarization framework that trains LLMs directly in the Boolean domain with multiple Boolean kernels, avoiding latent weights. It achieves near-FP16 performance with lower memory and computation compared to existing binarization and quantization methods.", "questions": "Please refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper addresses an important problem in LLM binarization and makes a convincing case for reducing the reliance on latent full-precision weights.\n2. The proposed approach is technically sound and well-motivated, with clear formulations and ablation studies that support the design choices.\n3. Experimental evaluation is fairly comprehensive, showing competitive or even near-FP16 performance with much lower memory and computation cost.\n4. The presentation is clear, with tables and figures that make it easy to compare against strong baselines.", "weaknesses": "1. The paper claims Pareto-frontier results, but only compares against 2\\3-bit quantization without presenting a full Pareto curve, which makes the claim less convincing.\n2. Although the method is said to optimize directly in the Boolean domain, it is unclear whether training/finetuning actually reduces memory consumption, or if fake quantization is still used; concrete evidence of memory savings during optimization is missing.\n3. The comparison with BinaryMoS seems unfair: three Boolean kernels likely consume more memory than three MoS experts, yet the paper does not account for this discrepancy.\n4. No results are reported on real inference acceleration or end-to-end memory reduction; claims about efficiency remain theoretical.\n5. Large-scale validation is missing — while OPT and LLaMA-13B are evaluated, experiments on larger models (e.g., 65B) are absent, leaving scalability uncertain.\n6. The distinction between the proposed multi-Boolean kernels and existing multi-binary-base methods (e.g., BitStack, QBB, DB-LLM) is not sufficiently clarified, weakening the novelty claim.\n7. The initialization strategy (SVID) is largely borrowed from OneBit, which reduces the originality of that component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)