Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
Authors: Jinhee Kim
Jae Jun An
Kang Eun Jeon
Jong Hwan Ko
Venue: NeurIPS 2025 poster ()
Keywords: Multi-bit quantization network
once-for-all training
coreset sampling
Primary area: deep_learning
TL;DR: We propose a one-shot training framework that reduces the training cost for multi-bit quantization networks using weight-space bias correction and bit-wise coreset sampling.

Abstract: Multi-bit quantization networks enable flexible deployment of deep neural networks by supporting multiple precision levels within a single model. However, existing approaches suffer from significant training overhead as full-dataset updates are repeated for each supported bit-width, resulting in a cost that scales linearly with the number of precisions. Additionally, extra fine-tuning stages are often required to support additional or intermediate precision options, further compounding the overall training burden. To address this issue, we propose two techniques that greatly reduce the training overhead without compromising model utility: (i) Weight bias correction enables shared batch normalization and eliminates the need for fine-tuning by neutralizing quantization-induced bias across bit-widths and aligning activation distributions; and (ii) Bit-wise coreset sampling strategy allows each child model to train on a compact, informative subset selected via gradient-based importance scores by exploiting the implicit knowledge transfer phenomenon. Experiments on CIFAR-10/100, TinyImageNet, and ImageNet-1K with both ResNet and ViT architectures demonstrate that our method achieves competitive or superior accuracy while reducing training time up to 7.88×.

Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
Jinhee Kim 1.+ Jae Jun An
Kang Eun Jeon 133 Jong Hwan Ko
Department of Electrical and Computer Engineering, Sungkyunkwan University Department of Electrical and Computer Engineering, Duke University
3 Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST)
{a2j inhee, ajj8061, ke0061,,eeeooe jejeon, jhko jiko Hakku. edu
Abstract
Multi-bit quantization networks enable flexible deployment of deep neural networks by supporting multiple precision levels within single model. However, existing approaches suffer from significant training overhead as full-dataset updates are repeated for each supported bit-width, resulting in cost that scales linearly with the number of precisions. Additionally, extra fine-tuning stages are often required to support additional or intermediate precision options, further compounding the overall training burden. To address this issue, we propose two techniques that greatly reduce the training overhead without compromising model utility: (i) Weight bias correction enables shared batch normalization and eliminates the need for fine-tuning by neutralizing and aligning activation distributions; and (ii) Bit-wise coreset sampling strategy allows each child model to train compact, informative subset selected via gradient-basse importance scores by exploiting the implicit knowledge transfer phenomenon. Experiments with both ResNet and ViT architectures demonstrate that our method achieves competitive superior accuracy while reducing training time up to 7.88x. Our code is released at this link
inntiizationnnnnduee bias across bit-widths
1IFAR-10/100. TinylmageNet, and ImageNet- IK
Introduction
With the explosion of highly capable yet computationally demanding deep learning models, quantiza- tion has emerged as an effective strategy for balancing performance and efficiency [1] 2 3. Despite its advantages, most existing quantization methods are optimized for single fixed quantization preci- sion configuration, which limits their ability to adapt dynamically changing resource availability and deployment across various platforms of diverse memory, compute, and power specifications. This has led to line of recent work focused on training a single model capable of supporting multiple precisions [5] 6. S) thereby enabling instant adaptation to varying resource budgets at runtime without the need for further training. In such multi-bit quantization networks, henceforth multi-bit networks, a single full-precision parent model generates multiple reduced- -ereccddprecision child models, thereby neutralizing the overhead of maintaining separate models for inference. By supporting multiple quantization precisions- referred to as the model's switchable bit range, these networks enable adaptive deployment across wide range of compute-constraing devices 10 12).
Equal contribution.
Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Time
Dedicated
D
Any- Precision
Process
Triining
Bilowie cornert sampling
Bizs carrection
Adeptation
Model
5B
Tmined bits
Ours
Bb
Calibrased bits
Tb
2b
4b
6b
7b
5.33x~7.888 88x reduction
Figure 1: A conceptual diagram of i) Dedicated, ii) Any-Precision, and iii) our training pipelines. D and indicate the full training dataset and coreset, respectively.
Although the multi-bit networks provide flexibility during inference/deploymenn, this advantage comes at the cost of substantial training overhead, limiting their practical adoption. The prevalent multi-bit training approach, known as Any-Precision [13. jointly optimizes the model across a small subset of selected bit-widths, termed the trained bit range. While this approach is more efficient than training individual models across the switchable range (Dedicated), it still introduces considerable overhead due the additional calibration required to enable inference at bit-widths outside the trained range (calibrated bit range), as shown in Fig. T Specifically, calibration demands extensive computation using large amounts of training data preserve the accuracy of untrained bit models in the calibrated range, by aligning their mismatched activation distributions. We identify that these activation mismatches across different bit-width models stem from biases in the weight distribution induced by quantization. Based on this observation, we propose a novel bias correction technique that directly controls the shift and scaling biases in the quantized weights to align distributions across the entire switchable bit-widths. This alignment enables multiple bit-width sub-networks to share common set of batch normalization (BN) parameters, effectively eliminating the need for costly post-training calibration.
Another major source of the significant training overhead is the use of the entire dataset for updating models in the trained bit range. Although coreset selection methods have been introduced to reduce
the training overhead by identifying a subset of important data samples [14 15 16 17 these approaches have primarily targeted single-precision model training with fixed coresets. Extending this idea multi-bi bit quantization setting, observe that each bit-width child model can benefit from training on distinct and smaller data subsets due to implicit gradient alignment across bit-widths. Leveraging this insight, propose bit-wise coreset sampling method that dynamically selects informative samples individually for each child model, based on the gradients computed per bit-width. Furthermore, since sample importance changes throughout training, we periodically re-sample these coresets to reflect evolving model dynamics. The proposed sampling approach effectively reduces per-epoch computational costs while preserving strong performance through implicit cross-bit-wind knowledge transfer, phenomenon we discover for the first time.
To summarize, contributions as follows:
Weight bias correction for activation alignment: We correct irantization-indueee biases in the weight space instead of the activation space, enabling multiple child models to share normalization parameters, and in turn eliminating the need for extra training stage.
Bit-wise coreset sampling: We propose novel per-bit-width coreset sampling strategy that computes bit-wise importance scores using gradient-based methods, thereby reducing training redundancy in multi- bit quantization networks.
Extensive empirical validation:
We demonstrate that our method consistently improves or maintains accuracy while significantly reducing training cost across diverse datasets (e.g., CIFAR-10, CIFAR-100, TinylmageNet, and ImageNet-11 K) and architectures (e.g., ResNet, DeiT, Swin). Our method achieves up to 7.88x GPU hour reduction without sacrificing model utility, validating the scalability and generality of our approach.

---

Epoch 0
Training
Epoch T
Calibration
BNg
BN2
BNg
Any- Prec.
D
EN
Q CONV
BNN
D
D
BNg
BNN
BN
BN
BN+
weight
BN adaptation
shared
Ours
BN
Q CONY
BN
E[X] V
running
Epx] V(X]
running
re-sample
BN
Org, weight
Bizs-correcied
Figure 2: A summary of Any-Precision's 's and our training pipeline.
Backgrounds & Related Works
Multi-bit quantization networks. Unlike traditional quantized networks that are optimized for single reduced numerical precision, multi-bit quantization networks [13
7
I8
8 19] are capable
of supporting multiple quantization precisions, enabling adaptive and versatile inference/deploymmen across a wide spectrum of compute-constrained devices (9) [00 12). The mainstream approach to training these networks involves optimizing the model for multiple precisions simultaneously, typically by the sum of loss functions corresponding to each bit-width. Formally, this is stated as shown below:
min
Σ (x.y)ES BEB
Σ
L(x, y, Q(0,6)),
(I)
where 0 E Id denotes the learnable model parameter which is shared across multiple precisions; C(x, Q(0, b)) is the loss on training sample (x, y) in training set S; Q(0, b) E 2d is the quantized version of b-bit precision; and B, referred to the trained range, represents the set of all trained bit-widths. To perform this optimization in practice, the batch-wise training scheme, which interleaves parameter updates across child models corresponding to different bit-widths in batch-wiss manner (see Algorithm D is commonly adopted to promote generalization across the entire training range.
Training overhead of multi-bit networks. While multi-bit training is generally more efficient than networks indepen-
Algorithm Batch-wise training scheme
training multiple dently, it still incurs significant computational over- partiuaaticullrtlyy as the training range B expands. To mitigate this cost, recent approaches minimize the number of bit-widths included in the training range and instead introduce calibrated bit-widths/range to expand precision support. Specifically, the model first trained on a small set of bit- bit-widths (the trained range), after which large portion of its parameters are frozen. The remaining parameters are then cali- brated or lightly fine-tuned to support additional bit- widths (the calibrated range). The union of the trained and calibrated ranges defines the model's switchable range, R-i.e., the full set of bit-widths supported by the multi-bit network.
Input: Data X, label Y Output: Multi-bit network G 1: for epoch = 1, T do for batch from X, Y do
3:
for bit
do Set all layers in G to b-bit Compute forward pass of G
5:
Calculate gradients of G
7:
end for
Update parameters with EB Cb
9:
end for
10:
end for
Challenges in training multi-bit networks. One major challenge in multi-bit network training research is the accuracy degradation due activation distribution mismatch between different bit- widths. To address this mismatch, Any- Precision [33. along with CoQuant [18 and MBQuant (8). leverages the 'switchable batch normalization' approach first proposed by 20 While effective, assigning separate batch normalization layers to each bit- widtw incurs additional overhead: obtaining

---

Q-CONV Output
60000
2.08
Quantizution
Fosperes
1-bot
Varlance Ratio
3-bil
4-bi1
Activatiet Vilaz
5-bil
Norm
BatchNorm Output
6-hit
7-bit
90000
8-bit
Act.
50000
59k
Quant,
layort
layert.)
layert
byy22.0
laye2.11
layer22
layer2.3
layera
layer3.3
byyor3.
byya..
lapost.0
lapent.
loyen4.
20000
Quantized Weight Layer
Acticatic
(a)
(b)
Figure 3: (a) Mismatch in activation distributions between different bit-widths, and (b) variance ratio between quantized weights and original weights in ResNet-50.
these parameters for unseen bit-widths typically requires an extra training phase as shown Fig. 2 Some recent works [1 avoid this mechanism altogether, but often suffer from degraded perfor- mance at lower bit-widths or resort to computationally expensive strategies to reduce interference between the training objectives of different child models. In our proposed method, rather than relying multiple batch normalization layers or other costly techniques, we correct the weight distribution directly before and after quantization. Our key insight is that quantization introduces shifts and scaling in the weight distribution, and aligning these distributions helps reduce conflicts across child models during training.
Coreset selection. Coreset selection- also known as dataset pruning-aae aims to reduce model training cost by selecting small yet representative subset of the training data, while preserving model utility. The core challenge lies in accurately identifying the most informative samples. Feature- space based methods select subsets that preserve the geometry of the data distribution-for example, Herding [22) and Moderate [15] select data points with distance in feature space. Uncertainty-based methods prioritize ambiguous hard-to-classify samples; for example, Entropy (23 and Cal [24 select samples near decision boundaries. Gradient- methods leverage training loss gradients. GraND/EL2N [4] rank samples by their gradient magnitude (or prediction error). while Craig 25
-imis
and GradMatch 26 select subsets that best match/mimic the full dataset's s gradient signals. Training- dynamics based methods consider samples' behavior over many epochs. Forgetting [16] counts how often a sample is forgotten during training, and AUM [27] averages the confidence gap across all epochs. Finally, hybrid approaches fuse multiple criteria: TDDS [17] integrates gradient information with training dynamics by measuring each sample's variability in its epoch-wise contribution to the overall training gradient.
Shortcomings of existing coreset selection research. Despite this breadth of approaches, most coresct selection methods are investigated under the assumption of fixed, full-precision floating- point models, and their applicability to quantized neural networks remains largely unexplored. In particular, the integration of coreset selection into quantization aware training (QAT) has received little attention, let alone its extension to the more complex setting of multi-bit quantization, where cross-bit interactions can significantly affect saliency estimation and the underlying training dynamics.
Weight Bias Correction for Activation Alignment
Activation distribution mismatch in multi-bit networks. As discussed in Section 2 multi-bit networks often suffer from mismatched activation distributions across bit-widths. To isolate the source of this mismatch, we decompose the cost-convolutional activation into two components: quantized input activation and the quantized weight. simplify the analysis, we fix the input activation to a specific precision (e.g., 4-bit). Under this setup, any observed variation in the output can be attributed solely to the quantized weights, thereby reducing the problem to single source of quantization noise.

---

In Fig. Baa. we visualize the cost-convoll activations of ResNet-50 for bit-widthh b E B (2,3,5) 53 using a batch of ImageNet examples. Although the imput activation fixed, the output distributions vary noticeably across bit-widths. This indicates that the differences can be attributed to quantization- inducedd bias the weights. ti) supports this explanation by showing that the quantized weights exhibit clear scale distortions compared to the original weights. This observation is consistent with prior observations of 28 which highlight the presence of systematic bias introduced during quantization.
Many multi-bit networks [3] address this activation mismatch problem by training separate BN parameters for each bit-width to independently correct activation distributions. As illustrated in Fig. a), this approach successfully aligns BN outputs across different precisions. While ef- fective, aligning output activations typically requires access to the training data and additional forward/backward passes, which incurs additional training overhead.
Bias correction for quantized weights. Instead of rectifying the output activations, we address the bias its source by aligning the quantized weights prior to convolution. By doing this, we can match activation outputs across bit-widths just by correcting the weights during the initial training stage, without having to explicitly match the activations themselves. As result, BN layers can be shared across all bit-widths, as shown in Fig. - avoiding the additional overhead of calibrating separate BN layers. It important to note that this correction is performed under fixed activation bit-width (e.g.. 4 bits), meaning that aligning the weights directly translates to more consistent activation outputs across different bit-widths. Specifically, we adjust the quantized weight vector w, with respect to their full-precision counterpart w, and compute the corrected weights wa as follows:
W
W
[w (w. + (E[w)- - E(w,))))
where E[|] and V denote the expectation and variance, respectively. This weight alignment enables multiple child models to share a single set of BN parameters with minimal interference. To com- pensate for residual discrepancies not fully addressed by bias correction, we additionally apply BN adaptation [29]. While adjusting running statistics has been proven to be effective in fixed-quantized networks 30), its use in multi-bit networks SS] remains limited and often lacks clarity on when is applied (e.g.. applied at every epoch in [5), which is unnecessary). Applying BN adaptation once at the final training stage as shown in Fig. is sufficient to correct the running mean and variance for each bit-width achieving optimal performance without additional overhead.
4 Bit-Wise Coreset Sampling
To translate the benefits of coreset selection into multi-bit quantization networks, we propose two techniques tailored to this setting: (i) a coreset sampling strategy that accounts for variations in sample importance across bit-widths and training epochs; and (ii) bit-wise training scheme for accurate per- bit-width importance score evaluation. Together, these techniques enable more efficient and adaptive training across a range of quantization levels while maintaining strong model utility.
4.1 Coreset sampling strategy
The central idea behind our coreset sampling method to dynamically redraw training subsets along two axes: bit-width and training time. Rather than using static, global coreset, we select bit- coresets that evolve throughout training via sampling as shown in Fig. 2 This design motivated by two key observations: (i) gradient alignment across bit-widths, and (11) temporal drift in sample importance.
Observation 1- Gradient alignment across bit-widths. We find that gradients computed from different bit-widths using the same data sample are highly aligned. In Fig. [ we visualize the angles between the gradients of the 8-bit and 2-bit child models across several layers of ResNet-20 at various training epochs. It can be seen that the angle between the two gradients stays consistently below 28', with alignment improving in deeper layers. This implies that, without loss of generality, parameter updates based on 2-bit gradients positively influence 8-bit child model (and also that of other precisions in the trained range), and vice versa. We refer to this phenomenon as cross-iitwwidth implicit knowledge transfer, where shared parameters act as conduits for the transfer of learning signals between child models.
5

---

1.
20-40
1.00
0.62
0.62
0.61
(U6)
6.60
0,60 0.60
0.00
80
40-40-
EJ10
00
1.00
0.64
038
Angle (*)
80-1000
1,00
(UUS
0:68
0,65 0.69
0..5
20
00
Epoch 10
120-140 140
1,00
11,00
0.81
10
Epoch 15
Epoch 20
160-180
1.00
0.83
Epoch 25
IN0-200.
0.20
20-4H
40
6040
100 100.000
199-30
0
5
Layer Index
Epoch
Figure 4: Angle between the 8-bit and 2-bit gradients across layers.
Figure 5: Spearman correlation between ranks at different epochs.
This observation leads to a key insight: unnecessary to feed the same sample to all bit-widths during training. Accordingly, we construct bit-wise coresets separate subsets tailored for each bit-width. These bit-wise coresets not only reflect the variation in sample importance across bit- widths but also exploit the implicit gradient transfer phenomena to benefit from collective learning signal without accessing the full dataset. Therefore, this design significantly reduces per-epoch computational cost while preserving strong performance across the trained range.
Observation Temporal drift in sample importance. We also observe that a sample's importance evolves as training progresses. More specifically, samples influential in early epochs often become less relevant later stages, where the model nears convergence and the loss landscape flattens. In we visualize this effect using Spearman correlation of TDDS-bar based importance scores measured at different training stages on ResNet-18 18 trained with CIFAR- 100 dataset. Correlations between early and late-stage scores may drop as low as 0.54, indicating substantial shifts in sample influence over time.
Figure 5
To account for the temporal drift in sample importance, we periodically re-sample each bit-wise coreset throughout training. Although high-score samples may informative in the early stages, they often lose relevance as the model's learning dynamics evolve. Continually refreshing the coresets helps prevent overfitting to outdated importance estimates, especially critical when the true sample importance landscape is dynamic and only partially observable.
Sampling method. To construct the bit-wise coresets via sampling, first convert the sample importance scores into sampling probabilities by applying min-max normalization. We then further shape the sampling probabilities using temperature-base sampling 32 33), which simultaneously reduces overfitting to high-scoring samples and promotes diversity, effectively balancing exploitation (b)
and exploration. The sampling probability p; for sample at wit-width b is defined as:
(s (2())) 1/7
( p (T)
(x 1/7
where (b) denotes the min-max normalized importance score for sample at bit-width b; and > 0 denotes the temperature parameter. Note that importance scores are computed once prior to coreset sampling and remain fixed throughout training.
4.2 Bit-wise training scheme for score evaluation
Extracting accurate, bit-wise importance scores is particularly challenging in the context of training dynamics-bart based coreset selection methods. These mainstream approaches estimate sample importance over multiple training epochs to capture the intricate training dynamics and improve score reliability (see Section _ for an overview). A representative example is TDDS [I7. which accumulates intermediate gradient signals-rreerred referred as context vectors throouhuth training to capture the evolving contribution of each sample. While effective in single-precision settings, we find that applying such methods directly under the standard batch-wise training scheme (Algorithm [ fails to produce meaningful bit- importance estimates. The core issue lies in the interleaved update

---

Table 1: ResNet CIFAR-10 and 100. Pruning rate of Coreset Sampling is 80%.
Coreset Sampling
Test Accuracy
GPU hours
Dataset
Framework
Ibit
2bit
4bit
Shit
32bit
Avg.
(Speed up)
Dedicated
92.42
93.04
92.99
93.08
94,11
93.10
11.97 (1.00x)
Any-Pree.
92.85 -0.21
93.28 s0.17
93.61 #0.07
93.64 60.0
93.77 40.12
93.31
8.76 (1.36x)
CIFAR-10
11 a0.07
93.57
93.46
(1.59x
Ours
92.60 -0.44
93.01 10.10
93.03 10.00
93,00 +0.73
93.08 +0.12
92.97
1.52 (7.88*)
Dedicated
67.52
70.21
70.17
70,50
72.63
70.21
(1.000
Any-Prec.
70.54
71.54
71.60
72.23
71.47
8.27 (1.35x)
CIFAR-100
70.95 -0,09
71.
10.17
71.96
10.11
71.91
20.09
71,91
68
71.84
7.17 (1.56x)
Ours
69. 14 20.00
70.12 an
70.35
17
70.43 a0.11
70.41 an
70.26
1.47 (7.61
Table 2: ResNet on CIFAR-10. Comparison against previous methods at 80% pruning rate.
Test Accuracy
Method
Thit
2bit
4bit
Sbit
32bit
Avg.
Random
88.97 20.47
89.99 =0.22
90.21 20.40
90.12 20.28
89.68 =0.32
89.94
Entropy
85.62 20.04
86.20 =0.39
86.31 =9.24
86,25 =0.22
86.36 +0.
86.21
Forgetting
76.57 x7.10
78.18 =0.97
78.46 s0.99
78.36 +1.00
78.49 +0.90
78.14
EL2N
80.21 29.29
81,16 =6 /4
81.21 #0.20
81.15 a0.43
81.22 an
81.07
Moderate
87.63 40.22
88.18 =0.78
88.35 60.11
88.43 =0.21
88.27 #0.08
88.26
TDDS
87.67 20.35
88.35
88.72 a9
88.75 671
/9
88.54
Ours
92.60 20.74
93.01 =0 10
93.03 20 (M
93.00 =i
93.08 an /2
92.97
gradients from all bit-widths are aggregated before shared parameter update, resulting in a single unified context vector that masks the distinct training dynamics of each child model.
pattern:
To address this issue, we introduce bit-wise training scheme for score evaluation, as shown in Algorithm 2 In this setup, each child model corresponding to a trained bit- width is trained on the entire dataset be- fore proceeding to the next bit-width. This scheduling isolates the gradient updates for each precision, en- abling the computation of distinct context vectors and more accurate, bit-wise importance scores.
Algorithm 2
Bit-wise training scheme
Input: Data X, label Y Output: Multi-bit network G 1: for epoch 1.... T do for bit b B do
for batch from X, Y do
Set all layers in to b-bit Compute forward pass of G
5:
is important to note that the proposed bit-wise train- ing scheme is used exclusively for importance score extraction. Once the scores are computed, we revert to the standard batch-wise training scheme for actual multi-bit network training. This hybrid approach al- lows coreset construction benefit from accurate, bit-wise decoupled importance evaluation while pre- serving the generalization advantages of batch-wise training.
Calculate gradients of G
Update parameters with C.
end for
end for
10: end for
5 Experiments
5.1 Setup
Evaluation metries and baselines. We evaluate our method in terms of ber-bit-width accuracy and total GPU hours. Comparisons are made against: (1) the dedicated framework, (2) the standard multi- bit framework (e.g., Any-Precision [I3), and (3) our method which augments the standard framework with Bias Correction (Ours). Within Ours, we evaluate our proposed coreset sampling strategy, which uses bit-wise scores, against six baseline corset selection methods: Random, Entropy [23], Forgetting [16], EL.2N [1], Moderate [15]. and TDDS [17]. Since most existing coreset selection techniques are designed for dedicated training, we adapt each baseline to our multi-bit framework to ensure fair comparison.
7

---

70
Avg. Accuracy (%)
60
Full training
S0
Avg. Accuracy
30-
Readora
Entropy
40
Forgetting
60-
EL2N
30
TDDS
20
60
70
80
30
40
70
80
90
Pruning Rate (%)
Pruning Rate (%)
Figure 6: Accuracy comparison across different pruning rates. Left: CIFAR-IO, Right: CIFAR-100
Table 4: Accuracy comparison against previous methods 80% pruning.
Table 3: ResNet on ImageNet-IK, IK. Pruning rate is 80%.
Test Accuracy
Method
Framework
Coreset Sampling
Test Accuracy
GPU hours
32hir
Avg.
1bit
32hit
Avg.
(Speed up)
Random 66.13
70.68
12.07
72.05
72.97
71.36
Dedicated
57.93
68.74
74,12 74,96 75.95
72.04
39.91 (1.00x)
Entropy 65.32
69.28
70.66
70.77
71.68
70.03
Any-Pres
68.77 71.66
73,84
74.07
74.63
73.01
33.94 (1.18x)
Forgeting 55.79
60.70
64.87
65.48
66.78
63.73
68.12 72.34
73.97
74,20
74.32
73.22
27.21 (1.47+)
EL2N
63.19
68.17
69.64
69.75
70.72
68.90
Ours
67.22 71.93
73.29
73.40
73,92
72.51
6.94 (8.75x)
Moderate
64.57
68.88
70.39
70.63
71.58
69.74
TDDS
65.24
69.21
70.56
70.71
71.36
69.42
Ours
67.22
71.93
73.29
73.40
73.92
72.51
Datasets and networks. We evaluate our method on four canonical Citasets-CIFAR--IA 100, CIFAR- 100 (34). TinylmageNet [35). and ImageNet- IK with- diverse set of networks. These include three ResNet models: PreActResNNe-222 337. ResNet-18 18, and ResNet-50 [38] as well as three Vision
Transformers (ViTs): DeiT-T, DeiT-S 339 and Swin-T 40)
Implementation details. All experiments are conducted on a single NVIDIA A100 GPU. Each experimental setting is as follows: (i) Dedicated trains the model with single weight and activation bit-width. (ii) Precisson uses training range of B 2, 4, 8, 32 for ResNet models (and B = 2, 8 for ViTs). After training, independent parameters for the remaining bit-widths are calibrated for approximately one-third the training epochs to ensure convergence. (iii) Bias Correction adopts the same training range as (ii) but skips the calibration phase and instead performs BN adaptation. Specifically, we assign a separate BN layer for 1-bit and share BN layers for all other bit- bittw. Although inference for the calibrated range is supported and achieves accuracy comparable to the trained range in both (ii) and (iii,, those results are omitted here due to the page limit and are provided in the Appendix.
5.2 Results
the results on PreActResNet-20 for CIFAR- 10 and ResNet- for CIFAR-100, highlighting both performance and training time reduction achieved by our method. Compared existing baselines, Ours achieves competitive performance with reduced training time by eliminating the calibration phase, while coreset sampling further improves efficiency by reducing data usage without compromising accuracy. Additional results for CIFAR- 10 with PreActResNet-20 including comparisons between our coreset sampling method and six baselines Random selection has been shown to excel at high pruning rates in prior studies [17], and we observe the same trend in experimental setup. Our method shows consistent improvements in both accuracy and efficiency across bit-widths. trade-off between training cost and accuracy by plotting average accuracy against pruning rate. Our method consistently outperforms all baselines across the entire pruning spectrum and maintains high accuracy
ResNet on CIFAR-10 and 100.
(see Section 5.1), are presented Table 2
the
90% pruning rate.
ResNet on ImageNet-IK: We further demonstrate the effectiveness of our method for a bigger dataset like ImageNet- 1K. Table And Jableasiummarizz our results for ResNet-50 ImageNet- 1K with respect to the baseline and previous methods. For these experiments, we finetune from pretrained Any-Prece -ey-Precision model, where specific implementation details are provided in the Appendix.
8

---

Table 5: DeiT-T, DeiT-S, Swin-T ImageNet-1K for different pruning rates.
Pruning
Test Accuracy
GPU hours
Network Framework
2
3
7
8
Avg.
SSpeed up)
Any-Prec.
69.72
69.97 70.59 70.77
70.86
70.91
70.47
25.77 (1.00+)
DeiT-T
50%
69.30
69.42
69.93
70.15
70.22
70.16
69.86
10.00 (2.58*)
Ours
60%
69.03
70.00
7.55 (3.41x)
Any-Prec.
76.34
76.93
78.19
78.25
78.30
78.32
78.37
77.81
27.07 (1.00H)
DeiTT-
50%
76.05
76.45
78.04
78.21
78.16
78.18
78.15
77.61
13.33 (2.03+)
Ours
60%
76.22
76.59
78.05
78.25 78.36
78.27
78.31
77.72
11.67 (2.32
Any-Prec.
78.68
79.14
79.86
79.97
79,96
79.94
79.96
79.64
27.10 (1.00+)
Swin-T
50%
78.49
78.90
79.79
79.92
79.88
79.93
79.96
79.55
13.27 (2.04*)
Ours
60%
78.53
78.92
79.67
79.94 79.90 79.53
10.57 (2.56x)
Table 6: The effect of Bias Correction and BN Adaptation.
Bias
Test Accuracy
Datasct
Correction
Adaption
1bit
2bit
4bit
6bit
Sbit
Avg.
92.95
87.72
93.53
93.32
92.73
92.47
92.08
91.87
93.53
92.24
93.58
91.98
93.50
93.68
93.62
93.51
93.47
93.40
93.70
93.38
CIFAR-10
92.87
93.36
93.59
93.55
93.56
93.52
93.56
93.63
93.65
93.48
93.61
93.72
93.88
93.89
93.92
93.84
93.88
93.83
93,92
93.83
70.23
$3.18
70.48
70.88
69.52
68.19
67.25
66.88
70,21
67.42
69.03
71.58
72.03
71.63
71.39
71.23
71.10
71.83
71.22
CIFAR-100
70.36
70.95
71.53
71.56
71.40
71.46
71.52
71.47
71.45
71.30
71.37
72.10
72.31
72.37
72.27
72.34
72.38
72.33
72.26
72.19
Compared to the dedicated training setting, method substantially reduces training time by 5.75x with minimal impact on performance.
ViTs on ImageNet-IK. IK. We also evaluate our method on larger transformer-basee models to demon- strate the generality and scalability our method to other architectures. The results three different ViTs (i.e., DeiT-T. DeiT-S, and Swin-T) summarized in Table To the best of our knowledge, there not yet a standard multi-bit framework such as Any-Precision for vision transformers. To this end, we implement our own framework with similar configurations our method with two dataset pruning ratios- 50% and 60% respectively. With a bigger dataset, our method shows even significant reduction in training time (as large as 18.22 GPU hours reduction in DeiT-T), while showing consistent accuracy compared to the baselines.
Any-Precision Precisioo.. We compare
5.3 Ablation
Effect of Bias Correction and BN Adaptation. As shown in Table we conduct an ablation study to quantify the individual and combined effects of Bias Correction and BN adaptation in the final training stage. The results show that both components play distinct yet complementary roles in achieving stable alignment and high accuracy across bit widths. Bias Correction primarily compensates for systematic deviations introduced during quantization, restoring the representational balance in the weight space. However, since it does modify the batch normalization statisties used at inference. it alone cannot fully align the activation distributions. BN adaptation, applied at the final stage, addresses this limitation by recalibrating the running mean and variance through small number of forward passes, thereby aligning cost-quantization activations with their floating- point counterparts. Together, these two procedures act on different levels of the model weight and activation resulting in consistent improvements across all bit widths. Quantitatively. the combination yields the highest accuracy, confirming that the final BN adaptation provides additional activation alignment beyond what bias correction alone can achieve. These findings clarify the respective contributions of both components and highlight the importance of performing BN adaptation the last training stage for precise activation calibration in quantized models.

---

Table The effect of the bit-wise training scheme.
Pruning
CIFAR-10
CIFAR-100
Rate
Batch-wsic
Bit-wise
Batch-wisc
Bit-wise
70%
90.89
91.17 (0.291)
61.13
64.06 (2.931)
80%
88.55
88.91 (0.361)
54.23
59.51 (5.281)
90%
80.27
82,79 (2.521)
43.52
48.87 (5.351)
Effect of bit-wise schedule for extraction. To quantify the benefit of isolating per-bit- witth dynamics, we perform a fixed-coreset ablation under the multi-bir bit framework (i.e., Any-Precision (I3] with Bias Correction setting. We select the dataset once-oiteee
by (i) the conventional batch-wise TDDS scores [17] or by (ii) our bit-wise scores. We then train the full multi-bit schedule on these reduced sets. As reported in Table 7 bit-wise scoring yields higher accuracy every pruning ratio on both datasets. This improvement stems from our bit-wise extraction design which enables the collection of separate intermediate gradients per bit-width at each ppoch-allowing compute distinct context vectors for every sub-model.
6 Conclusion
In this work, we introduce two techniques to reduce the training overhead of multi-bit quantization networks. First, we correct quantization-induce bias in the weight space, removing the need for an additional training stage. Second, we design bit-wise coreset sampling strategy that leverages implicit knowledge transfer, allowing each child model to train compact, informative subset selected via gradient-based importance scores. Our approach preserves model utility while reducing training costs across various architectures such ResNets and ViTs, offering scalable solution for efficient multi-bit quantization training. By enabling more efficient multi- muuti-precision learning, method contributes the broader goal of sustainable and energy-efficient AI, helping make high nigh-performance models more accessible, affordable, and ubiquitous to everyone.
Despite strong empirical results, our evaluations are limited to computer vision tasks due to the high computational cost of training multi-bit networks a challenge shared by every prior work this space. By significantly reducing this overhead, our method paves the way for applying multi- bit quantization to more demanding applications such as generative AI and large-scale language tasks. Extending our approach to these domains will be the focus of our future work, advancing the broader applicability and impact of multi-bit quantization networks across diverse tasks.
Acknowledgment
We thank the anonymous reviewers for their constructive comments. This work was partly sup- ported by the National Research Foundation of Korea (NRF) grant (RS-2024-00345732, RS-2025- 02216217); the Institute for Information & communications Technology Planning & Evaluation (ITPP) grants PRS-2020-11201821, RS2019-11190421, RS-2021-11210022 02217613, 55-20255-1062296 RS-2025-25442569; the Technology Innovation Program (RS-2023-
RSS-2021-112122 2068, RS2025-
00235718, 23040-15FC) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea) grant
(1415187505).
10

---

References
[1] B. Jacob, Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, "Quantization and training of neural networks for efficient integer-arithmetic-only inference," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2704-2713, 2018.
[2] R. Banner, Y. Nahshan, and D. Soudry, "Post training 4-bit quantization of convolutional networks for rapid-deployment"" Advances Neural Information Processing Systems, vol. 32, 2019.
[3] M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort, "Up or down? adaptive rounding for
quantization," in International Conference on Machine Learning. pp. 7197-7206, PMLR, 2020.
[4] Y. Li, R. Gong, X. Tan, Y. Yang. Hu, Q. Zhang, F. Yu, W. Wang, and S. Gu, "Brecq: Pushing the limit of post-training quantization by block reconstruction," arXiv preprint arXiv:2102 05426, 2021.
[5] K. Xu, L. Han, Y. Tian, S. Yang, and X. Zhang, "Eq-net: Elastic quantization neural networks," " in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1505-1514, 2023.
[6] B. Chmiel, R. Banner, G. Shomron, Y. Nahshan, A. Bronstein, U. Weiser, et al., "Robust quantization: One model to rule them all," Advances in neural information processing systems, vol. 33, pp. 5308-5317, 2020.
[7] K. Xu, Q. Feng, X. Zhang, and D. Wang, "Multiquant: Training once for multi-bit quantization of neural networks.," in IJCAI, pp. (629-3635, 2022.
[8] Y. Zhong. Y. Zhou, F. Chao, and R. Ji, "Mbquant: A novel multi-branch topology method for arbitrary bit-width network quantization." arXiv preprint arXiv:2305.08117, 2023.
[9] F. Tahmasebi, Y. Wang, B. Y. Huang, and H. Kwon, "Flexibit: Fully flexible precision bit parallel accelerator architecture for arbitrary mixed precision ai," arXiv preprint arXiv:2411.18066 2024.
[10] S. Ryu, H. Kim, W. Yi, and J.-J. Kim, "Bitblade: Area and energy-efficient precision-scalable neural network accelerator with bitwise summation," Proceedings of the 56th Annual Design Automation Conference 2019, pp. 1-6, 2019.
[11) B. Moons, R. Uytterhoeven, W. Dehaene, and M. Verhelst, "14.5 envision: A 0.26-to-1 10tts/ subword- auballl-parallel
www.---lllaa-acuurcc----e-- -reauency-scallalle convolutional neural network processor in 28nm fdsoi," in 2017 IEEE International Solid-State Circuits Conference (ISSCC),
pp. 246-247, 2017.
[12] V. Camus, L. Mei, C. Enz, and M. Verhelst, "Review and benchmarking of precision-scalalle multiply-accumulate unit architectures for embedded nntrolk processing." IF:FE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 4, pp. 697-711, 2019.
[13] H. Yu, H. Li, Haoxiang opand Shi, T. S. Huang, and G. Hua, "Any-precision deep neural networks," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 10763-
10771, 2021.
[14] M. Paul, S. Ganguli, and G. K. Dziugaite, "Deep learning data diet: Finding important examples early in training." in Advances in Neural Information Processing Systems (NeurIPS), pp. 20596-20607, 2021.
[15] X. Xia, J. Liu, J. Yu, X. Shen, B. Han, and T. Liu, "Moderate coreset: A universal method of data selection for real-world data-efficient deep learning," " in International Conference on Learning Representations (ICLR), 2023.
[16] M. Toneva, A. Sordoni, R. Tachet des Combes, A. Trischler, Y. Bengio, and G. Gordon, "An empirical study of example forgetting during deep neural network learning," in International Conference Learning Representations (ICLR), 2019.
11

---

[17] x. Zhang, J. Du, Y. Li. W. Xie, and J. T. Zhou, "Spanning training progress: Temporal dual- depth scoring (tdds) for enhanced dataset pruning. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[18] X. Sun, R. Panda, C.-F. R. Chen, N. Wang, B. Pan, A. Oliva, R. Feris, and K. Saenko, "Improved techniques for quantizing deep networks with adaptive bit-widths," " in Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 957-967, 2024.
[19] Nair, P. Datta, Dean, P. Jain, and A. Kusupati, "Matryoshka quantization." arXiv preprint arXin:2502.06786, 2025.
[20] J. Yu, L. Yang. N. Xu, J. Yang, and T. Huang, "Slimmable neural networks." arXiv preprint 1812.088, 2018.
[21] C. Tang, Y. Meng, J. Jiang, S. Xie, R. Lu, X. Ma, Z. Wang, and W. Zhu, "Retraining-fre model quantization via one-shot weight-coupling learning," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15855-15865, 2024.
[22] M. Welling, "Herding dynamical weights to learn," in Proceedings of the 26th International Conference on Machine Learning (ICML), pp. 1121-1128, 2009.
[23] C. Coleman, C. Yeh, S. Mussmann, B. Mirzasoleiman, P. Bailis, P. Liang, J. Leskovee, and M. Zaharia, "Selection via proxy: Efficient data selection for deep learning," arXiv preprint arXiv: / 1829, 2019.
[24] K. Margatina, D. Tsipras, M. Sheehan, and N. Aletras, "Active learning by acquiring contrastive examples," in Conference on Empirical Methods in Natural Language Processing (EMNLP). 2021.
[25] B. Mirzasoleiman, J. Bilmes, and J. Leskovec, "Coresets for data-efficient training of machine learning models," Proceedings of the 37th International Conference on Machine Learning (ICML), pp. 6950-6960, 2020.
[26] K. Killamsetty, S. Durga, G. Ramakrishnan, A. De, and R. Iyer, "Grad-match: Gradient matching based data subset selection for efficient deep model training," in International Conference on Machine Learning, pp. 5464-5474, PMLR, 2021.
[27] G. Pleiss, T. Zhang, E. R. Elenberg, and K. Q. Weinberger, "Identifying mislabeled data using the area under the margin ranking," in Advances in Neural Information Processing Systems (NeurIPS), pp. 17044-17055 17056, 2020.
[28] M. Nagel, M. v. Baalen, T. Blankevoort, and M. Welling, "Data-free quantization through weight equalization and bias correction," in Proceedings of the IEEE/CVF international conference computer vision, pp. 1325-1334, 2019.
[29] B. Li, B. Wu, Su, and G. Wang. "Eagleeye: Fast sub-net evaluation for efficient neural network pruning." Computer Fision-ECCy 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part Il 16, pp. 639 6644, Springer, 2020.
[30] M. Nagel, M. Fournarakis, Y. Bondarenko, and T. Blankevoort, "Overcoming oscillations in quantization-aware training," in International Conference on Machine Learning, pp. 16318- 16330, PMLR, 2022.
[31] D. H. Ackley, G. E. Hinton, and T. Sejnowski. "A learning algorithm for boltzmann machines." Cognitive Science, vol. 9, no. 1, pp. 147-169, 1985.
[32] G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," " 2015.
[33] E. Jang, S. Gu, and B. Poole, "Categorical reparameterization with gumbel-softmax, 2017.
[34] A. Krizhevsky, G. Hinton, et al., "Learning multiple layers of features from tiny images," " 2009.
[35] Y. Le and X. Yang, "Tiny imagenet visual recognition challenge," CS 23/N, vol. 7, no. 7, p. 3, 2015.
12

---

[36] J. Deng, W. Dong, R. Socher, Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierar- chical image database," in 2009 IEEE conference on computer vision and pattern recognition. pp. 248-255, leee, 2009.
[37] K. He, X. Zhang. S. Ren, and Sun, "Identity mappings in deep residual networks." in Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14, pp. 630-645, Springer, 2016.
[38] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
[39] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, "Training data- efficient image transformers & distillation through attention," in International conference on machine learning. pp. 10347-10357, PMLR, 2021.
[40] Z. Liu, Lin, Y. Cao, H. Hu, Y. Wei, Zhang, Lin, and B. Guo, "Swin transformer: Hierarchical vision transformer using shifted windows," in Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
[41] S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, and Y. Zou, "Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients," arXiv preprint 1606.66 2016.
[42] S.-Y. Liu, Z. Liu, and K.-T. Cheng, "Oscillation-free quantization for low-bit vision transform- ers," in International conference on machine learning. pp. 21813-21824, PMLR, 2023.
[43] S. K. Esser, J. L. McKinstry, D. Bablani, Appuswamy, and D. S. Modha, "Learned step size quantization," arXiv prepriní arXiv:1902.000 1902.08/33 2019.
[44] Z. Yuan, C. Xue, Y. Chen, Q. Wu, and G. Sun, "Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization," in European conference on computer vision, pp. 191-207. Springer, 2022.
[45] M. Phuong and C. Lampert, "Towards understanding knowledge distillation," in International conference on machine learning, pp. 5142-5151, PMLR, 2019.
13

---

Appendix
Contents
Introduction
Backgrounds & Related Works
Weight Bias Correction for Activation Alignment
Bit-Wis Coreset Sampling
4.1
Coresct sampling strategy
4.2
Bit-wise training scheme score evaluation
5
Experiments
5.1
Setup
5.2
Results
5.3
Ablation
6
Conclusion
10
Acknowledgment
10
Implementation Details
14
Additional Experiments
15
Theoretical Analysis Cross-bit-width Implicit Knowledge Transfer
21
A Implementation Details
Quantization function. For CNN experiments, quantize the weights and activations following the baseline settings of Any-Precision [13, where the activation bit-width is fixed at 4-bits. Any- Precision employs the DoReFa [1 quantizer into its framework. The quantization process is as follows:
8 B([W))
(4)
tanh (W)
W 2 max( tanh (W)l) + 0.5
(5)
Q = [M, W'J
(6)
Q = s(2 /M,- -
(7)
where W is the normalized weight tensor; Qn is the quantized bin; and Qn is the quantized floating-poin weight. M,, = 2 is the maximum bin value with respect to the target bit-width n.
For VIT experiments, we adopt the StatsQ quantizer proposed in 42 for weight quantization. Activation quantization is performed using the LSQ quantizer [4]. fixed at 8-bits. StatsQ applies uniform quantization, using the channel-wise (or head-wise) mean of the absolute weight values as
14

---

the scaling factor. The quantization process is defined as follows:
(
(8)
W clip( W/s: 11
W
W
0.5
(10)
( (W")
Qn [M, W
(11)
where the remaining steps for mapping the quantized values back to the floating-pon point range follow the same procedure as DoReFa. Here, expectation in the scaling factor s taken over dimension(s) A, which depend on the shape of W: for 2D weights, A corresponds to the column dimension; for 3D weights, includes both the first and last dimensions.
Implementation of Any-Precision VITs. To the best of our knowledge, there is currently no standardized multi-bit framework like Any-Precision [13] available for vision transformers (ViTs). To address this, we develop our own framework, adopting configurations similar to those used in Any-Precision Our implementation is based on the QAT framework from the OFQ codebase, with minor modifications to the quantizer to match the quantization function described above and without the query-key reparameterization proposed in 42.
Configurations of baseline coreset selection methods. To evaluate the effectiveness of our method, compare it against several existing coreset selection methods. While prior approaches typically focus on selecting a single subset optimized for a single model or configuration, our method targets bit-wise unique subsets tailored for multi-bit training. Accordingly, we apply each baseline's scoring strategy independently to each sub-model, constructing distinct coresets, and train each sub-model on its respective subset. For CIFAR-I0 and CIFAR-I00, we train for 200 epochs; for ImageNet-IK, the number of epochs varies depending on the model. For ImageNet- 1K experiments, we initialize from pretrained models: ResNet models use the Any-Precisi weights [3], while ViTs are initialized from the 8-bit quantized checkpoints provided by PTQ4ViT [4] The specific configurations for each coreset selection method are as follows:
Entropy 23 EL2N 14 Scores are computed over 20 epochs for CIFAR-10/100 and 5 epochs ImageNet- KK Entropy identifies samples near decision boundaries, while EL.2N ranks samples based on the magnitude of their gradients.
Forgetting [6]
Scores are computed over 200 epochs for CIFAR-10/100 and 10 epochs for ImageNet-1 K. A forgetting
defined by the number of times sample is misclassified after being learned correctly, capturing how often a sample is "forgotten" during training.
Moderate [3]
Scores are computed using features extracted by a pre-trained model for 200 epochs on CIFAR-10/100 and for 70 epochs on ImageNet- IK, as importance proxy. Moderate quantifies importance by measuring distances between samples in the resulting feature space.
TDDS 17]: Scores are computed over 10 epochs for CIFAR-10. 20 epochs for CIFAR- 100, and epochs for ImageNet- K. TDDS combines gradient information with training dynamics to estimate sample importance.
Bit-wise Unique Scores (Ours): Our method computes per-bit-widht importance scores over 10 epochs for CIFAR- 10, 20 epochs for CIFAR- 100, and 5 epochs for ImageNet-IK. enabling dynamic coreset selection tailored to each sub-network.
B Additional Experiments
Comparison of score uniqueness in bit-wise vs. batch-wise training scheme. To analyze the effect of the proposed bit-wise training scheme, we examine the uniqueness of the importance scores produces for each bit-width sub-model, compared to those from the standard batch-wise training scheme. Figure 7 shows a heatmap of Spearman correlation values, which quantify the similarity between importance scores produced by different bit-width sub-models. On both CIFAR- 10 and CIFAR-100, the bit-wise scheme results in consistently lower correlation values, indicating that it yields more distinct and representative importance scores for each bit- This suggests that separating the training dynamics by bit-width is essential for accurately capturing per-sample
15

---

Batch-wise
Bit-wise
1.0
1.0
1.00
0.90
0.86
0.86
0.82
1.00
0.70
0.71
0.71
0.70
0.9
0.9
CIFAR-10
bit-width
bit-wldth
1.00
0.99
0.86
1.00
0.77
0.75
1.00
0.87
0.6
1.00
0.75
0.6
84
84
32
bit-wiith
bil-width
1.00
0.79
0.73
0.72
0.69
1.00
0.39
0.37
0.37
0.37
0.9
0.9
CIFAR 100
bit-andth
0.8
bit-wldth
0.33
1.00
0.98
0.81
1.00
0.39
0.37
1.00
0.81
0.6
1.00
0.57
0.6
0.5
0.5
53
85
32
bit-width
bil-width
Figure 7: Comparison of score similarity across sub-models under batch-wise and bit-wise training schemes, measured by Spearman correlation. The top row shows PreActResNet-20 on CIFAR- 10 results and the bottom row shows PreActResNet-188 on CIFAR-100; in each row, the left is batch-wise and the right is bit-wise.
importance in multi-bit settings, as avoids the gradient aggregation seen in batch-wise training and allows each sub-model to maintain distinct context vectors. This isolation is critical for deriving meaningful, bit-wise importance estimates, which are otherwise masked under the standard batch-wise update pattern.
Table 8: PreActResNett-0 on CIFAR-10 and PreActResNNet-18 on CIFAR-100, Breakdown of GPU hours.
Dataset
GPU hours
Total GPU
Framework
Training
Calibration
Adaptation
Scoring
hours (Speed up)
Dedicated
11.97
11.97 (1.00x)
Any-Prec.
7.51
1.25
8,76 (1.36*)
Blas Correction
7.51
0.004
(1.59x)
Bias Correction Coreset Sampling
1.52
0.004
0.37 (offline)
1.52 (7.88*)
Dedicated
11.19
11.19 (1.00x)
Any-Prec.
7.17
1.10
8.27 (1.36x)
CIFAR-100
Bias Correction
7.17
0.004
7.17 (1.56x)
Bias Correction Coreset Sampling
1.47
0.004
0.74 (oflline)
1.47 (7.61x)
Breakdown of GPU hours. We report GPU hours for all ResNet experiments, broken down by training stage, including: score evaluation, coreset training, calibration (if applicable), and adaptation. Table 8 also help clarify the computational cost structure of framework. Specifically, coreset sampling significantly reduces training GPU hours for multi-bit quantization models, and this efficiency gain increases with the pruning rate. However, coreset sampling alone does not eliminate the cost of the calibration phase, which is typically needed to support additional bit-widths. To address this, we apply Bias Correction and BN adaptation, which allow us remove the calibration step entirely without sacrificing accuracy.
Experiments on alignment achieved by Bias Correction. To demonstrate the effectiveness of our Bias Correction method, we conducted an analysis the output activations, comparing how well activations from different bit-widths align with one another. In all experiments, the activation
16

---

Table 9: MAE between different bit-widths and full precision models' activation with vs. without
Bias Correction.
Bias
MAE
Correction
2bit
3bit
4bit
5bit
6bit
7bit
8bit
Avg.
0.744
0.756
0.754
0.750
0.761
0.756
0.766
0.755
0.667
0.670
0.671
0.654
0.683
0.668
0.659
0.667
precision is fixed to 4 bits, and we measure the mean absolute error (MAE) between the activations of quantized and full- full-precision models, with and without applying Bias Correction. Table show that applying the correction consistently improves alignment, reducing the avg. MAE from 0.755 to 0.667. This improvement holds across all bit-widths, demonstrating our method's effectiveness. We will include these results in the revised version to highlight the impact of Bias Correction on activation alignment.
Table 10: PreActResNet-20 on CIFAR-10. BN sharing with and without 1-bit quantization.
Test Accuracy
Framework
1bit
2bit
3bit
4bit
5bit
6bit
7bit
8bit
32bit
Avg.
Share BN w/ Ibit
46.38
91.90
92.57
91,48
90.98
90.53
90.24
90.14
89.90
86.01
Share BN w/o 1bit
92.49
89.44
92.43
92.67
92.43
92.15
92.00
91.97
91.65
91.91
Experiments effectiveness of BN sharing for 1-bit quantization. In multi-bit settings, weight distributions tend to approximate a normal distribution, where our Bias Correction mechanism works effectively. However, in the 1-bit case, the weight distribution collapses into near-uniform or binary form, causing a significant distribution shift that Bias Correction alone struggles to adequately address. For that reason, prior works on multi-bit networks often omit the 1-bit setting from their analysis [5] 18 8] In contrast, our method explicitly incorporates this case and demonstrates that assigning a separate BN layer for 1-bit quantization is both a simple and effective solution.
To further support this, we conduct ablation studies comparing two configurations: one where BN shared across all bit-widths (including 1-bit), and another where the 1-bit case has its own BN, and the remaining bit-widths share a single BN. Table 10 show that including 1-bit in BN sharing significantly degrades 1-bit quantization performance. On the other hand, using separate BN for 1-bit achieves strong performance with negligible additional overhead, that is the additional parameters for the separate BN layer, which accounts for less than 0.01 % of the total number of parameters. This performance degradation stems from unstable BN statistics. The 1-bit weights produce activation distributions that are markedly different from those of higher bit-widths, leading to significant fluctuations in shared BN statistics. These fluctuations negatively impact the normalization of other bit-widths, ultimately harming overall model performance.
Table 11: PreActResNet-20 on CIFAR-10. Impact of only coreset sampling.
Coreset Sampling
Bias
Test Accuracy
GPU hours (Speed up)
(Pruning Ratio)
Correction
1bit
3bit
4bit
5bit
6bit
Shit
32hit
Avg.
(0.0)
92.95
87.72
93.53
93.32
92,73
92.47
92.08
91.87
93.53
92.24
7,51 (1.00x)
92.46
87.42
92.92
93.05
92.51
91.95
91.69
91.49
92.98
91.83
(0.7)
2.06 (3.65x)
92.62
92.98
93.08
93.05
93.10
93.15
93,02
92.29
87.61
92.84
92.27
91.51
90.99
90.68
90.46
92.84
91.28
(0.8)
1..44)))
92.60
92.96
93.03
93.02
92.99
93.01
93.00
93.08
91.47
87.07
91.61
91.22
90.42
89.62
89.23
89.00
91.62
90.14
(0.9)
0.83 (9.05x)
92.04
92.66
92.61
92.66
92.47
92.50
92.51
92.63
92.38
92.50
Experiments on impact of coreset sampling alone. To examine the standalone contribution of coreset sampling to both accuracy and training speedup, we provide results on a CIFAR-10 baseline where only coreset sampling is applied, with bit-widths except for 1-bit sharing BN layers. As shown in Table II while coreset sampling contributes most to the speedup, it is not sufficient on its
17

---

own to maintain strong accuracy at across every bit-width. In contrast, when Bias Correction and BN adaptation are applied alongside coreset sampling, we observe accuracy improvements everywhere. with pratically no additional GPU hours. The results indicate that although coreset sampling is the main driver of compute efficiency, Bias Correction and BN adaptation essential for best accuracy.
Table 12: PreActResNet-20 on CIFAR-10. Influence of varying fixed temperatures and scheduling methods.
Temperature
Test Accuracy
scheme
1bit
2bit
4bit
8bit
32bit
Avg
0.1
90.08
92,11
92.35
92.55
92.21
91.86
Fixed
0.5
90.61
92.61
92.78
92.85
92.61
92.29
1.0
90.33
92.41
92.51
92.62
91.87
91.95
Linear
90.40
92.29
92.07
92.22
91.55
91.71
Scheduling
Exp.
90.57
92.39
92.56
92.63
91.93
92.02
(0.1 1.0)
Log
90.77
92.64
92.65
92.89
92.06
92.20
Experiments with varying fixed temperature settings and scheduling methods. Table 12 presents the test accuracies of our method, evaluated in the Any-Precision setting, trained across bit-widths b E {1, 2, 4, 8, 32} using fixed sampling temperatures T E {0.1, 1.0} and three temperature scheduling strategies (e.g.. linear, exponential, logarithmic). We observe that among the fixed settings, moderate temperature of r = 0.5 consistently achieves the highest accuracy, outperforming both the low (T 0.1) and high (T 1.0) extremes. Dynamically increasing the temperature from 0.1 to 1.0 over training--rgardleess of the scheduling scheme-yiele performance that is comparable to or worse than using a fixed r = O.5. These results suggest that a single, well-chosen temperature is sufficient to balance the sampling distributionnfaavrinn informative samples while maintaining diversity. In contrast, dynamically adjusting the temperature throughout training introduces additional complexity without delivering clear performance benefits. Based on these observations, all coreset sampling experiments were conducted with the temperature fixed at r 0.5.
Table 13: PreActResNet-20 on CIFAR-10 10 and PreActResNet-18 on CIFAR-100 Performance of calibrated bit-wis when pruning rate for Coreset Sampling is 80%.
Coreset
Test Accuracy
Dataset
Framework Sampling
3bit
5bit
6bit
7bit
Any-Prec.
93.17 40.26 93.19 a0.18 93.16 40.25 93,10 a0.24
CIFAR-10
93.46 40.14
93.54
40.02
93.50 40.07
93.43 40.10
Ours
92,96 10.09 93.02
10.04 92.99 40.13 93.01 +0.13
Any-Pree.
71.28 20.26 71.43
a0.24
71.53
71.45 20.09
CIFAR-100
71.93 40.07
71.96
a0.11
71.96
40.12
71.95 40.M
Ours
70.41 40.15 70.42 40.10 70.47 40.08 70.57 40.13
Evaluation of calibrated bit-widths. In this section, we present the performance of calibrated bit-widths, which were omitted from the main experimental results in the main paper. Although we refer to these calibrated bil-widths, it is important to clarify that, in our method, these bit-widths are not explicitly trained or fine-tuned. Instead, we obtain their accuracy using bias correction and batch normalization (BN) adaptation, without additional training calibration stages. In contrast, Any-Precision [3] recovers calibrated bit-width performance by performing a separate post- ttairriining BN calibration procedure. As shown in Table 13 and Table 14 the calibrated bit-widths in our method achieve accuracy comparable to trained bidtws, confirming that our proposed weight bias correction effectively aligns activation distributions without the need for costly calibration.
Evaluation against baseline coreset selection methods on CIFAR- 100. presents additional experimental results CIFAR- 100 using the PreActResNNet- 18 architecture, comparing our bit bit-wise coreset sampling method against several baseline coreset selection strategies. Consistent with prior findings in TDDS [7 and our CIFAR-10 experiments reported in the main paper, we observe that
18

---

Table 14: PreActResNet-20 CIFAR-10 and PreActRReNNtt-18 18 on CIFAR-100. Performance of calibrated bit-widths compared to previous methods at an 80% pruning rate.
Test Accuracy
Dataset
Method
3bit
Sbit
6bit
7bit
Random
90.14 +0.25
90.15 -0.30
90.13 +0.32
90.10 +0.29
Entropy
86.29 10.37
86.30 =0.29
86.27 20.27
86.28 1..25
Forgetting
78.12 +0.95
78.43 =1.01
78.37 +1.00
78.24 +1.02
CIFAR-10
EL2N
81.15 =0. 22
81.21 =0. 38
81.20 =0.31
81.13 #0.43
Moderate
88.36 =0.37
88.35 =0.15
88.39 =0.22
88.40 +0.19
TDDS
88,72 =0. 13
88,72 =0.07
88,70 =0.05
88.71 +0.07
Ours
92,96 a0 09
93.02 =0.
92.99 =0.13
93.01 1..73
Random
63.01 +0.50
63.00 =0
63 =0.42
63.09 +0.34
Entropy
53.46 +0.31
53.45 -0.42
53.54 +0.51
53.53 +0.34
Forgetting
38.71 #0 46
39.40 =0.55
39.49 =0 60
39.27 +0.72
CIFAR-100
E1.2N
31.00 +0.59
31.44 -0.62
31.57 +0.62
31.49 +0.55
Moderate
58,54 +0.27
58.63
=0.79
58.67
=0.17
58.66 =0.13
TDDS
54.40 a0 27
54.40
=0.40
54.40
20.44
54.35
40.40
Ours 70.41 #0.15 70.42
70.47
=0.00
70.57
Table 15: PreActResNet- 18 on CIFAR-100. Comparison with previous methods at 80% pruning rate.
Test Accuracy
Method
1bit
2bit
4bit
8bit
32bit
Avg.
Random
60.32 10.62
62.45
63.11 10.49
62.87 10.53 61.39 10.60
62.47
Entropy
52.65 s0.37
53.28
+0.20
53.55 +0.37
53.53
+0.47
53.22 #0.62
53.36
Forgetting 35.21 20.37
38.00
20.34
39.18 10.50
39.45
un.53
37.94 10.68
38.52
EL2N
30.49 40.91
31.07
=0.57
31.38 #0.62
31.50 s0.37
29. 13.33 =0.29
31.04
Moderate
57.05 +0.40
58.24
=0.42 42
58.69
+0.16
58.67
+0.10
57.83 +0.37
58.33
TDDS
53.65 40.36
54.07
a0. 43
54,40
#0.40
54.36
#0.41
54.04 a0.55
54.23
Ours 69.14 +0.08
70. =0.11
70.35 #0.17
70.43
+0.11
70.41
#0.11
70.26
random coreset selection performs surprisingly well at high pruning rates. This trend persists in the CIFAR-100 setting, where random sampling remains a strong baseline under severe data reduction. Nonetheless, our proposed method consistently outperforms the baselines across different bit- bit-widths, demonstrating its effectiveness in selecting informative samples even under high pruning constraints.
Evaluation of DeîT-S on CIFAR-100 and TinylmageNet We further evaluate our method on a transformer-basee architecture, specifically DeiT-S, using smaller datasets such as CIFAR-100 and IinylmageNet. The results are presented in Table [6] To the best of our knowledge, there is currently no standardized multi-bit framework like Any Pre-Precn for vision transformers. To address this, we implement our own framework following configurations similar to Any-Precision, with slight modification the StatsQ quantizer- details of which are provided in Section A Our method demonstrates consistently strong performance compared to the basic Any-Precision setup, even when pruning the dataset by 60%, achieving up to an 8.41 x reduction in GPÚ hours on TinyImageNet.
Evaluation of stotage-onsttraiedd coreset sampling. Coreset-based approaches in multi-bit networks consistently reduce training time; however, due varying data importance across sub- models, it remains challenging to impose uniform constraint on the total number of training samples used by the entire model. To address this, we first discard samples that are consistently considered unimportant across all sub-models, and then apply our coreset sampling method with bias correction. To identify and remove consistently uninformative samples before applying coreset sampling, we first compute the importance of each training sample for every sub-model over single epoch, following our bit-wise training scheme. These importance values are then summed across sub-models,
and their variability is assessed -imilar to training dynamics approaches [17 16 to obtain
19

---

Table 16: DeiT-S on CIFAR-100 and TinylmageNet.
Pruning
Test Accuracy
GPU hours
Dataset
Framework
Rate
2bit
4bit
6bit
8bit
Avg.
(Speed up)
Dedicated
87.14 87.92 87.88 88.03
87.74
41.01 (1.00x)
Any-Prec.
87.52
88.30
88.20 88.21
88.08
10.47 (3.92x)
CIFAR-100
50%
87.83
88.56
88.68
88.59
88.43
6.05 (6.78x)
Ours
60%
87.61
88.45
88.54
88.65
88.31
5.20 (7.89x)
Dedicated
82.61
85.60
85.68
85.86
84.94
74.00 (1.00x)
Any-Prec.
82.10
84.61
84.47
84.70
84.07
19.32 (3.83x)
TinyImageNer
50%
82.54
84.95
85.33
85.17
84.59
10.50 (7.05*)
Ours
60%
82.89
84.39
84.95
84.86
84.26
8.80
(8.41x)
Table 17: 2r on CIFAR-10. Comparison across varying storage reduction rates.
Storage
Test Accuracy
Reduction
1bit
2bit
4bit
8bit
32bit
Avg.
0%
92.60
93.01
93.03
93.00
93.08
92.97
20%
92.39
93.17
93.26
93.32
93.32
93.20
30%
92.28
92.70
92.68
92.62
93.10
92.65
40%
92.09
92.61
92.75
92.77
92.55
92.64
50%
91.97
92.15
92.19
92.29
92.61
92.25
final importance score. As shown in Table I7 our coreset sampling method performs comparably existing approaches, even under 50% dataset storage constraint. Moreover, Table 18 shows that training performance can be further enhanced by tuning the pruning rate (i.e., training time), highlighting the adaptability of our method varying resource budgets in multi-be bit network training.
Experiments on Influence of coreset sampling frequency. In practice, the overhead of bit-wise coreset resampling is extremely small compared to the overall training cost. For example, even on an ImageNer-scale dataset, performing 100 resamplings takes only about 3.36 minutes. Given this negligible cost, resampling at every epoch is practical and effective choice.
To quantitatively demonstrate this, we conducted experiments with different resampling intervals and measured both validation accuracy and total sampling time. Table 19 show that resampling every epoch improves average accuracy by 1.33%p compared to resampling every 30 epochs, while adding just 53 seconds of overhead to a multi-hour training process. This demonstrates that frequent resampling can offer meaningful accuracy gains at virtually no additional cost.
Experiments on dynamic score re-evaluation. We conducted additional experiments where impor- tance scores dynamically re-evaluated every 10, 30, 50, 100 epochs, and coresets resampled accordingly. We evaluated how it impacts accuracy and GPU hours using PreActResNet-20 on CIFAR-IU 10 and PreAAtRReNNet-18 on CIFAR-I00 under both 80% and 90% data pruning. Table 20 and Table reveal a consistent pattern: while dynamic score re-evaluation leads to only marginal accuracy changes, it incurs substantial increase in computational cost. In many settings, our one-time scoring strategy already matches even outperforms more frequent re-evaluation in terms of final accuracy, while consuming significantly fewer GPU hours. This empirical finding validates our design choice, and shows that single, well-computed importance estimate-wihe paired with stochastic sampling-offer an effective and efficient balance, capturing most of the benefits of dynamic importance tracking without incurring its heavy cost. Looking ahead, with the observation that dynamic re-evaluation yields modest gains on the more challenging CIFAR-100, we believe dynamic sampling techniques could be the key to boosting performance complex, high-variability
20

---

Table 18: PreActResNet-20 on CIFAR-10 and PreActResNett-18 on CIFAR-100. Comparison across pruning rates when dataset storage is fixed at 30K out of 50K samples. Since retaining 30K out of 50K samples represents 40% reduction, pruning rate of 40% corresponds to the full-training scenario in this context.
Pruning
Test Accuracy
Dataset
Rate
1bit
2bit
4bit
8bit
32bit
Avg.
40% (full)
93.01
93.39
93.61
93.58
93.65
93.50
50%
92.81
93.07
93.11
93.15
93.39
93.13
CIFAR-10
60%
92.66
93.07
93.04
93.09
93.48
93.07
70%
92.52
92.91
93.07
93.11
93.17
92.97
80%
92.09
92.61
92.75
92.77
92.55
92.64
40% (full)
66.01
66.35
66.85
66.90
66.90
66.65
50%
66.32
66.98
67.60
67.93
67.49
67.52
CIFAR-100
60%
65.65
65.99
66.48
66.38
66.20
66.24
70%
66.10
66.86
66.69
66.90
67.03
66.78
80%
65.29
66.30
66.33
66.39
65.83
66.19
Table 19: PreActResNet-20 on CIFAR-10. Influence of sampling frequency.
Resampling
Test Accuracy
Total Sampling Time
Frequency
1bit
2bit
4bit
Sbit
32bit
Avg.
Total GPU time)
1 92.60
93.01
93.03
93.00
93.08
92.97
53.02s (0.96%)
10
91.96
92.40
92.65
92.66
92.75
92.55
5.09s (0.09%)
20
91.40
91.73
92.04
92.03
92.34
91.98
2.77% (0.05%)
30
91.29
91.53
91.63
91.73
91.68
91.64
1.40s (0.03%)
datasets where importance scores drift more drastically throughout training. The main hurdle the high cost of score re-evaluation during training, which currently limits the practicality of dynamic methods. In future work, we will explore lightweight techniques to reduce score-evaluation overhead while maintaining the quality of importance estimates.
C Theoretical Analysis of Cross-bit-width Implicit Knowledge Transfer
In this section, we use a simple linear classifier to examine how shared weights in multi-bit networks can implicitly transfer knowledge between sub-networks. We consider setting where an 8-bit model and 2-bit model share the same real-valued parameter vector w, with weights quantized using the DoReFa quantizer [41] Training alternates iteratively: the 8-bit model is trained on batch Xx with hard labels, followed by the 2-bit model trained on separate batch X2, also with hard labels. These batches are drawn independently from the data matrix X E Rdx= and do not overlap. The shared parameter w updated in-place using the cross-entropy loss and continuously modified by both models. The key question is: can we formally argue that, despite no explicit soft-label distillation and no shared data examples, the 2-bit model benefits from the 8-bit model's training- and vice versa- through the shared parameter?
Gradient update within combined data subspaces. When the 8-bit model observes batch B3 (Xs, Us), it performs gradient step using the cross-entropy loss. The model is blind to any component of the optimum that is orthogonal the plane that spans the ng dimeenniioo subspace of Xs [45]. That is, under an asymptotic assumption, the direction of the update is fully constrained to the subspace spanned by the input vectors the batch, as the gradient is linear combination of the each data point v;. Therefore, the gradient of the 8-bit model lies within the subspace spanned by its input batch, and thus the corresponding update step is bound follows: As E span( Xs). Similarly, the 2-bit model performs its update with batch X2. By induction, the net update to lies within sum
21

---

Table 20: PreActResNet-20 on CIFAR-10. Impact of score re-evaluation.
Pruning Ratio
Re-eval Frequency
Test Accuracy
Re-eval
Total GPU
1bit
2bit
4bit
8bit
32bit
Avg.
GPU hours
hours
92.60
93.01
93.03
93.00
93.08
92.97
1.52
100
92.08
92.64
92.80
92.82
92.88
92.75
0.38
5.31
0.8
50
92.45
92.83
92.96
93.04
92.96
92.97
1.13
2.65
92.64
92.90
93.03
92.88
92.69
92.84
2.27
3.79
10
92.21
92.56
92.73
92.86
92.75
92.73
7.33
8.85
92.04
92.66
92.66
92.63
92.38
92.38
0.84
100
91.43
92.09
92.09
92.09
91.89
92.09
0.37
1.21
0.9
50
91.37
91.98
92.22
92.29
92.32
92.09
1.15
1.99
30
91.94
92.13
92.12
92.11
92.38
92.10
2.31
3.15
10
91.24
91.64
91.94
92.03
91.77
91.82
7.33
8.17
Table 21: PreActResNet-18 18 on CIFAR-100. Impact of score re-evaluation.
Pruning Ratio
Re-eval Frequency
Test Accuracy
Re-eval
Total GPU
1bit
2bit
4bit
8bit
32bit
Avg.
GPU hours
hours
69.14
70.12
70.35
70.43
70.41
70.26
1.47
69.16
70.24
70.41
70.39
70.04
70.18
0.77
2.24
0.8
50
69.13
70.23
70.60
70.82
70.41
70.45
2.30
3.78
30
68.91
70.17
70.18
70.05
70.09
70.02
4.57
6.04
68.95
70.45
70.76
70.78
70.87
70.66
14.57
16.04
67.53
69.40
69.75
69.25
69.24
69.32
0.83
67.34
69.17
69.23
69.29
69.28
69.02
0.76
1.59
0.9
50
67.77
69.12
69.43
69.20
69.11
69.11
2.32
3.15
30
67.82
69.30
69.11
69.34
69.02
69.15
4.63
5.46
67.28
69.14
69.41
69.50
69.44
69.23
14.68
15.51
2==1 span(X,). where B {2, 8} our case. Thus, the shared weight vector evolves within the union of data subspaces: span( 3 U Uppa((X)) This shows that each sub-network updates its parameters based broader subspace that includes data from other sub-networks, thereby increasing its effective data exposure.
of subspaces as follows: Aust Σ5=
Gradient alignment between quantized sub-networks. Given that updates occur within shared subspace, we analyze whether the gradients from different quantized sub-networks are sufficiently aligned to enable mutual benefit. We assume that the optimum value for both quantized model similar [11. In our setting, when the 8-bit model receives batch Xg. this updates w towards minimizing Lg. Since Aws E span (X&), this update steers toward the optimum w" within span(Xs, (Xx). The 8-bit model moves w to a point where the 2-bit loss cannot be worse- and is often better (i.e., < 90). This is functionally equivalent capacity updates are immediately used by the 2-bit model, enabling generalization benefits without soft targets. This provides theoretical basis for implicit knowledge transfer the shared parameter acts as channel of indirect supervision.
soft distillation: the 8-bit model's higher-
Motivated by these observations, propose bit-wise coreset sampling method that trains each sub-network with its own compact, informative subset of the full dataset. As the multi-bit network implicitly accesses the collective data seen by all sub-networks, each can prioritize important ples by directly training them, while also benefiting from diverse data exposure through indirect supervision. This only reduces the overall training cost but also preserves model performance by ensuring sufficient coverage of the dataset across bit-widths.
22

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the
paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract clearly states the two primary contributions (the gradient- based bit-wise coreset sampling and the bias correction method), which align with the methods and results presented in the paper, accurately reflecting its contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Due to page constraints, we placed our discussion limitations in the Conclu- sion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [NA]
Justification: The methodology section simply outlines the observations that inspired our core idea.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
The paper thoroughly documents the experimental setup, including descriptions of the datasets used, model architecture details, training procedures. These details provide all the information needed to reproduce the key results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
24

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will release the code and necessary data along with documentation and instructions, enabling others reproduce the primary experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cc.
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
7 //nips. //nips.cc/pub.lee wwtpc//pppc.guueee..o.eppppp httpa///guuldes/odeeuuubiiieee SrbodeSubmissionool11..
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Our paper specifies all relevant training details, including dataset, and tuning ranges, model architectures. Information that could not be included due to length can be found in the code. This comprehensive reporting ensures that readers have all information needed to replicate the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: For the CIFAR-10 10 and CIFAR-100 experiments, report error bars rep- resenting the standard deviation over three independent runs, capturing variability due to initialization and data ordering.
Guidelines:
The answer NA means that the paper does not include experiments.
25

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our paper discusses computational costs by reporting GPU hours for training and highlights that our approach substantially reduces this cost compared baselines. also specifies the hardware environment (noting the GPUs used for training), giving clear sense of the resources required for reproduction.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported paper (e.g., preliminary or failed experiments that didn't ' make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Our work adheres ethical guidelines. Our work involves only widely-used public datasets and standard methodology with no human subjects or privacy issues. We confirm compliance with the NeurlPS Code of Ethics, there aspects of the research that conflict with ethical standards.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
26

---

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper's contribution is a general method for improving training efficiency, with no obvious negative societal impact. Thus, specific societal impacts are identified, beyond the positive aspect of reducing computational resource usage.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our research does not involve releasing any high-risk model. It introduces training framework, so no special safeguards are necessary or applicable.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
27

---

Justification: Our paper uses only established public datasets and tools, and it properly cites the sources. All assets are used in accordance with their licenses and terms of use, with appropriate references given to the original creators.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi theode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our work does not introduce new datasets or standalone model assets; focuses training algorithm and uses existing benchmark datasets. Therefore, this question is not applicable. aside from the planned code release which is documented and covered under reproducibility above.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No human subjects crowdworkers were involved in our research. Our study exclusively uses pre-existing machine learning datasets, so no participant instructions or compensation details are applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
28

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our study did not involve any human participants or personal data, so IRB approval was not required.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: LLMs were used only for sentence-level editing and grammar checking during manuscript preparation and did not influence or form part of the core technical methodology.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy pettps://neurjjg cttpiiipp.cc..ooooee cc/Conferencces/22255 /LLM) for what should or should be described.
29


Reviews:
[{"review_id": "p2PUCROJPp", "reviewer": "Reviewer_iWva", "summary": "This paper focuses on reducing the large training overhead of multi-bit quantization networks, caused by training separate batch-normalization parameters for each bit-width to correct activation distributions and full dataset updates for each bit-width. To that end, the authors propose: 1) weight bias correction, by correcting quantization-induced bias and aligning activation distributions, enabling shared BN across bit-widths, and 2) a bit-wise coreset sampling strategy that accounts for sample importance, obtained via a novel bit-wise training scheme, across bit-width and training epoch. The method is demonstrated using multiple datasets and models and achieves consistent GPU hours speedup while matching the performance of other works.", "questions": "1. How important is the BN adaptation performed at the last training stage? Why is bias correction for quantized weights not enough for activation alignment? Are most of the accuracy gains due to BN adaptation or bias correction?\n2. How often is coreset sampling performed? What is the impact of the frequency at which it is performed on accuracy and GPU hours? Also, from the results, it seems that most of the speedup is due to the coreset sampling method. What would the accuracy and speedup be if only coreset sampling is performed?\n3. In Section 3, how is the statement “achieving optimal performance without additional overhead” true? It is clear that BN adaptation will incur additional overhead that needs to be considered. \n4. For Table 3 and 4, it is mentioned that the results are obtained by fine tuning a pretrained Any-Precision model. Why are the GPU hours for Any-Precision model training then not included in the reported hours for the proposed method?\n5. Some results in Table 1, 3 and 5 are bolded while they are not the best. Why?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is well written and provides clear background information.\n2. The observations made throughout the paper are deeply insightful and clearly motivate the solution approach.\n3. The paper introduces a novel bit-wise training scheme for score evaluation that addresses the limitations of traditional methods aimed at single precision networks.\n4. The results are positive, showcasing a reduction in GPU hours while matching the performance of existing methods.\n\nWeaknesses:\n1. Activation alignment is showcased for the inefficient separate BN parameters method. However, the authors do not show the alignment achieved by the proposed bias correction for quantized weights method making the impact of the method unclear.\n2. The impact of BN adaptation at the final training stage is not clear. Ablation studies are needed to determine how much the alignment and final accuracy is impacted by bias correction for quantized weights vs. BN adaptation. \n3. Resampling is not addressed in the results. While it has been shown that sample importance changes over training epochs, the paper does not provide guidelines on how to choose the frequency of coreset sampling and its impact on accuracy and GPU hours.\n4. Some of the figures use symbols that are not explained anywhere, making it difficult to understand (ex. Figure 1 with D and S)", "was_revised": false, "final_justification": "", "rebuttal": "We deeply thank the reviewer for their constructive and detailed feedback, and also for recognizing the insights and empirical results of our work. Below, we provide point-by-point responses to the concerns and questions raised.\n\n**W1: Alignment achieved by Bias Correction**\n\nThank you for pointing this out. To demonstrate the effectiveness of our Bias Correction method, we conducted an analysis of the output activations, comparing how well activations from different bit-widths align with one another. In all experiments, the activation precision is fixed to 4 bits, and we measure the mean absolute error (MAE) between the activations of quantized and full-precision models, with and without applying Bias Correction. The results show that applying the correction consistently improves alignment, reducing the avg. MAE from 0.755 to 0.667. This improvement holds across all bit-widths, demonstrating our method’s effectiveness. We will include these results in the revised version to highlight the impact of Bias Correction on activation alignment.\n\n**[Table 1. MAE between different bit-widths and full precision models' activation with vs. without Bias Correction]**\n|Bit-width|With Bias Correction|Without Bias Correction|\n|-|:-:|:-:|\n|2bit|0.667|0.744|\n|3bit|0.670|0.756|\n|4bit|0.671|0.754|\n|5bit|0.654|0.750|\n|6bit|0.683|0.761|\n|7bit|0.668|0.756|\n|8bit|0.659|0.766|\n|**Avg.**|**0.667**|**0.755**|\n\n\n**W2+Q1: Clarify the role and relative impact of Bias Correction vs. BN adaptation**\n\nThank you for this point. As shown in Table 2, our ablation studies isolate each component to assess its individual contribution to performance. The results reveal that neither method alone is sufficient for optimal accuracy across all bit-widths. For example, using only Bias Correction leads to suboptimal performance at 2-bit, while using only BN adaptation underperforms at 1-bit. Using both techniques together consistently yields the best accuracy across all bit-widths.\n\nWhile Bias Correction offers a low-cost way to compensate for systematic errors in the weight space, it does not directly influence the BN running statistics that govern activation distributions at inference time. BN adaptation, on the other hand, recalibrates these statistics through forward passes on the training data, directly aligning the activation distributions. In this sense, BN adaptation complements Bias Correction by targeting residual discrepancies that remain after correcting the weights. Together, the two methods correct both weight distribution shifts and activation misalignments, enabling consistent activation behavior and improved accuracy, particularly for intermediate calibrated bit-widths (e.g., 3,5,6,7 bits). We plan to include these ablation studies in the revised version to more clearly demonstrate the distinct and complementary roles of each technique.\n\n**[Table 2. The impact of Bias Correction vs. BN adaptation]**\n|Dataset|Bias Correction|BN adaptation|1bit|2bit|3bit|4bit|5bit|6bit|7bit|8bit|32bit|Avg.|\n|-|:-:|:-:|-|-|-|-|-|-|-|-|-|-|\n|CIFAR-10|||92.95|87.72|93.53|93.32|92.73|92.47|92.08|91.87|93.53|92.24|\n||✔||93.58|91.98|93.50|93.68|93.62|93.51|93.47|93.40|93.70|93.38|\n|||✔|92.87|93.36|93.59|93.55|93.56|93.52|93.56|93.63|93.65|93.48|\n||✔|✔|93.61|93.72|93.88|93.89|93.92|93.84|93.88|93.83|93.92|93.83|\n|CIFAR-100|||70.23|53.18|70.48|70.88|69.52|68.19|67.25|66.88|70.21|67.42|\n||✔||71.12|69.03|71.58|72.03|71.63|71.39|71.23|71.10|71.83|71.22|\n|||✔|70.36|70.95|71.53|71.56|71.40|71.46|71.52|71.47|71.45|71.30|\n||✔|✔|71.37|72.10|72.31|72.37|72.27|72.34|72.38|72.33|72.26|72.19|\n\n**W3+Q2: Impact of resampling frequency**\n\nWe agree that sampling frequency is an important factor to examine. In practice, the overhead of bit-wise coreset resampling is extremely small compared to the overall training cost.  For example, even on an ImageNet-scale dataset, performing 100 resamplings takes only about 3.36 minutes. Given this negligible cost, resampling at every epoch is a practical and effective choice.\n\nTo quantitatively demonstrate this, we conducted experiments with different resampling intervals and measured both validation accuracy and total sampling time. The results show that resampling every epoch improves average accuracy by 1.33%p compared to resampling every 30 epochs, while adding just 53 seconds of overhead to a multi-hour training process. This demonstrates that frequent resampling can offer meaningful accuracy gains at virtually no additional cost.\n\n**[Table 3. Influence of sampling frequency on ResNet-20 / CIFAR-10]**\n|**Resampling Frequency**|1bit|2bit|4bit|8bit|32bit|**Avg.**|**Total Sampling Time (% of Total GPU time)**|\n|-|-|-|-|-|-|-|:-:|\n|1 **(Ours)**|92.60|93.01|93.03|93.00|93.08|**92.97**|53.02s **(0.96%)**|\n|10|91.96|92.40|92.65|92.66|92.75|**92.55**|5.09s **(0.09%)**|\n|20|91.40|91.73|92.04|92.03|92.34|**91.98**|2.77s **(0.05%)**|\n|30|91.29|91.53|91.63|91.73|91.68|**91.64**|1.40s **(0.03%)**|\n\n**Q2: Impact of coreset sampling**\n\nThank you for raising this point. To examine the standalone contribution of coreset sampling to both accuracy and training speedup, we provide results on a CIFAR-10 baseline where only coreset sampling is applied, with all bit-widths except for 1-bit sharing BN layers. \n\nIn the table below, we denote setups with coreset sampling as A, and those with Bias Correction and BN adaptation as B. The configuration A only (B not applied) represents coreset sampling alone, while A+B combines coreset sampling with Bias Correction and BN adaptation. As shown in the results, while coreset sampling (A) contributes most to the speedup, it is not sufficient on its own to maintain strong accuracy at across every bit-width. In contrast, when Bias Correction and BN adaptation are applied alongside coreset sampling (A+B), we observe accuracy improvements everywhere, with pratically no additional GPU hours. The results indicate that although coreset sampling is the main driver of compute efficiency, Bias Correction and BN adaptation is essential for best accuracy.\n\n**[Table 4. Impact of only coreset sampling on ResNet-20 / CIFAR-10 (A: Coreset Sampling, B: Bias Correction & BN adaption); Bold the highest performance for each pruning rate.]**\n|**A (Pruning Ratio)**|**B**|1bit|2bit|3bit|4bit|5bit|6bit|7bit|8bit|32bit|Avg.|**GPU hrs (Speed Up)**|\n|-|-|-|-|-|-|-|-|-|-|-|-|:-:|\n|X (0.0)|X|92.95|87.72|93.53|93.32|92.73|92.47|92.08|91.87|93.53|92.24|7.51 (1.00×)|\n|O (0.7)|X|92.46|87.42|92.92|93.05|92.51|91.95|91.69|91.49|92.98|91.83|2.06 (3.65×)|\n|O|O|**92.62**|**93.03**|**92.98**|**93.08**|**93.05**|**93.05**|**93.08**|**93.10**|**93.15**|**93.02**|2.06 (3.65×)|\n|O (0.8)|X|92.29|87.61|92.84|92.27|91.51|90.99|90.68|90.46|92.84|91.28|1.52 (4.94×)|\n|O|O|**92.60**|**93.01**|**92.96**|**93.03**|**93.02**|**92.99**|**93.01**|**93.00**|**93.08**|**92.97**|1.52 (4.94×)|\n|O (0.9)|X|91.47|87.07|91.61|91.22|90.42|89.62|89.23|89.00|91.62|90.14|0.83 (9.05×)|\n|O|O|**92.04**|**92.66**|**92.61**|**92.66**|**92.47**|**92.50**|**92.51**|**92.63**|**92.38**|**92.50**|0.83 (9.05×)|\n\n**W4: Miscellaneous figure errors**\n\nThank you for drawing our attention to this. For Figure 1, we will clarify that D and S denote the full dataset and the sampled coreset, respectively. For Figure 2, we will update the caption to explicitly define E[X] and V[X] as the running statistics of BN layers before adaptation, and E[X]’ and V[X]’ as those after the BN adaptation phase. These changes will be reflected in the revised version to improve clarity of our presentation.\n\n**Q3: Overhead of BN Adaptation**\n\nWhile BN adaptation does introduce additional computation, its overhead is negligible compared to the overall training cost since it involves only forward passes over a fixed number of samples and does not require backpropagation. For example, in the ResNet-20 with CIFAR-10 setting, BN adaptation takes only 0.004 GPU hours, accounting for less than 0.3% of the total training time in both the Bias Correction-only configuration (7.52 GPU hours) and the more efficient Bias Correction + Coreset Sampling configuration (1.52 GPU hours). Below, we include a detailed breakdown of GPU hours across all components, highlighting the negligeable overhead of BN adaptation step.\n\n**[Table 5. Breakdown of GPU hours for ResNet-20 / CIFAR-10 experiments with 80% pruning]**\n|Framework|Training|Calibration|Adaptation|Scoring|Total GPU hours (Speedup)|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Dedicated|11.97|-|-|-|11.97 (1.00×)|\n|Any-Prec.|7.51|1.25|-|-|8.76 (1.36×)|\n|Bias Correction|7.51|-|**0.004**|-|7.52 (1.59×)|\n|Bias Correction+Coreset Sampling|1.52|-|**0.004**|0.37 *(Offline)*|1.52 **(7.88×)**|\n\n\n**Q4: Pretraining settings for ImageNet-1K experiments**\n\nThank you for raising this important question. In our experiments, we fine-tune from publicly available Any-Precision ResNet-50 checkpoints [1], rather than training from scratch. This follows prior work, as full Any-Precision training on ImageNet-1K is extremely time-consuming, typically requiring around 300 GPU hours for a single run—making it infeasible to reproduce during the rebuttal period. To our knowledge, no existing multi-bit quantization work has explicitly reported this pretraining cost. Accordingly, the GPU hours reported in Tables 3 and 4 include only the additional fine-tuning and calibration time. However, we agree that including full pretraining costs would improve clarity and will report those figures in the revised manuscript.\n\n**Q5: Bolded results in tables**\n\nThe bolded values are meant to indicate results from our proposed method. We apologize that this was not clearly stated and will revise them accordingly. Additionally, we will consider using a different visual cue to distinguish our method from others more clearly.\n\n*Reference*\n[1] H.Yu et al., \"Any-precision DNN.\" AAAI 2021."}, {"review_id": "kU1trg9L5f", "reviewer": "Reviewer_Txhn", "summary": "This work proposes an efficient training framework for producing multi-bit neural networks. The authors introduce a weight bias correction technique to align activation distributions, enabling the use of shared batch normalization during training and shifting the calibration burden to a more simple post-training adaptation step. Additionally, they propose a novel bit-wise dynamic coreset sampling method to significantly reduce training time. Empirical validation and comparisons with existing methods demonstrate the effectiveness of the proposed approach.", "questions": "- Regarding weakness1, could the authors clarify this design choice and provide result of applying dynamic coreset sampling on spearman correlation?\n- Could the authors elaborate concerns on weakness 2 and 3?\n- Could the authors provide GPU hours with more detailed granularity— the time spent separately on training, calibration, and adaptation phases? This would help better assess the efficiency claims of the proposed method.", "limitations": "Limitations and potential societal impact are not addressed in the paper", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:  \n1. Building on observations, the authors propose a novel approach to handle activation misalignment and introduce coreset sampling dedicated for multi-bit training.\n2. The method demonstrates significant improvements in wall-clock training time while maintaining quality, validating its effectiveness.  \n\n  \nWeaknesses:\n1. If my understanding is correct, sample-wise importance changes throughout the training process. However, the proposed sampling strategy calculates importance only once, prior to coreset selection, without updating it over time. Even though temperature-based sampling is used, samples with initially high probability are more likely to be selected consistently, which seems misaligned with the motivation of dynamically capturing changing important samples.\n2. The proposed weight bias correction is claimed to enable shared batch normalization during training. However, a separate BN layer is still used for the 1-bit setting. Does this imply that the bias correction mechanism is less effective or insufficient for 1-bit quantization?\n3. In the ImageNet-1K experiments, the paper mentions that the Any-Precision model is used for fine-tuning. Does this mean that fine-tuning is performed starting from a pre-trained Any-Precision model after the calibration or training phase? Clarification on the training pipeline would be appreciated.", "was_revised": true, "final_justification": "Authors have addressed all of my concerns, and I raise my score", "rebuttal": "Thank you for your thoughtful feedback and insightful suggestions. In response, we have carefully followed your recommendations and provide additional results and clarifications below.\n\n**W1: Dynamic sample-wise importance**\n\nWe appreciate the reviewer’s insightful question. You are absolutely right that accounting for the dynamic nature of sample importance is a key consideration of our method. Our decision to compute importance scores only once at the beginning—rather than periodically throughout training—was primarily driven by the high computational cost associated with full-dataset scoring. To address the potential rigidity of this one-time evaluation, we incorporate temperature-based probabilistic sampling, which introduces stochasticity into the coreset selection process. This allows our method to be aware of potential importance variations, even without explicitly updating the scores over time.\n\nThat said, we acknowledge the validity of the reviewer’s concern that high-probability samples may still dominate selection, especially when importance is fixed. To directly examine this, we conducted additional experiments where importance scores are dynamically re-evaluated every 10, 30, 50, or 100 epochs, and coresets are resampled accordingly. We evaluated how it impacts accuracy and GPU hours using ResNet-20 on CIFAR-10 and ResNet-18 on CIFAR-100 under both 80% and 90% data pruning. \n\nThe results below reveal a consistent pattern: while dynamic score re-evaluation leads to only marginal accuracy changes, it incurs a substantial increase in computational cost. In many settings, our one-time scoring strategy already matches or even outperforms more frequent re-evaluation in terms of final accuracy, while consuming significantly fewer GPU hours. This empirical finding validates our design choice, and shows that a single, well-computed importance estimate—when paired with stochastic sampling—offers an effective and efficient balance, capturing most of the benefits of dynamic importance tracking without incurring its heavy cost.\n\nLooking ahead, with the observation that dynamic re-evaluation yields modest gains on the more challenging CIFAR-100, we believe dynamic sampling techniques could be the key to boosting performance on complex, high-variability datasets where importance scores drift more drastically throughout training. The main hurdle is the high cost of score re-evaluation during training, which currently limits the practicality of dynamic methods. In future work, we will explore lightweight techniques to reduce score-evaluation overhead while maintaining the quality of importance estimates. We will include a discussion of these future work directions in the revised manuscript.\n\n**[Table 1. Impact of score re-evaluation for ResNet-20 / CIFAR-10 & ResNet-18 / CIFAR-100 with 80% pruning]**\n|**Dataset**|**Re-eval Freq.**|1bit|2bit|4bit|8bit|32bit|**Avg.**|**Re-eval GPU hrs**|**Total GPU hrs**|\n|:-:|:-:|-|-|-|-|-|-|:-:|:-:|\n|CIFAR-10|**- (Ours)**|92.60|93.01|93.03|93.00|93.08|**92.97**|**-**|**1.52**|\n| |**100**|92.08|92.64|92.80|92.82|92.88|**92.75**|**0.38**|**5.31**|\n| |**50**|92.45|92.83|92.96|93.04|92.96|**92.97**|**1.13**|**2.65**|\n| |**30**|92.64|92.90|93.03|92.88|92.69|**92.84**|**2.27**|**3.79**|\n| |**10**|92.21|92.56|92.73|92.86|92.75|**92.73**|**7.33**|**8.85**|\n|CIFAR-100|**- (Ours)**|69.14|70.12|70.35|70.43|70.41|**70.26**|**-**|**1.47**|\n| |**100**|69.16|70.24|70.41|70.39|70.04|**70.18**|**0.77**|**2.24**|\n| |**50**|69.13|70.23|70.60|70.82|70.41|**70.45**|**2.30**|**3.78**|\n| |**30**|68.91|70.17|70.18|70.05|70.09|**70.02**|**4.57**|**6.04**|\n| |**10**|68.95|70.45|70.76|70.78|70.87|**70.66**|**14.57**|**16.04**|\n\n**[Table 2. Impact of score re-evaluation for ResNet-20 / CIFAR-10 & ResNet-18 / CIFAR-100 with 90% pruning]**\n|**Dataset**|**Re-eval Freq.**|1bit|2bit|4bit|8bit|32bit|**Avg.**|**Re-eval GPU hrs**|**Total GPU hrs**|\n|-|-|-|-|-|-|-|-|:-:|:-:|\n|CIFAR-10|**- (Ours)**|92.04|92.66|92.66|92.63|92.38|**92.38**|-|**0.84**|\n||100|91.43|92.09|92.09|92.09|91.89|**92.09**|**0.37**|**1.21**|\n||50|91.37|91.98|92.22|92.29|92.32|**92.09**|**1.15**|**1.99**|\n||30|91.94|92.13|92.12|92.11|92.38|**92.10**|**2.31**|**3.15**|\n||10|91.24|91.64|91.94|92.03|91.77|**91.82**|**7.33**|**8.17**|\n|CIFAR-100|**- (Ours)**|67.53|69.40|69.75|69.25|69.24|**69.32**|-|**0.83**|\n||100|67.34|69.17|69.23|69.29|69.28|**69.02**|**0.76**|**1.59**|\n||50|67.77|69.12|69.43|69.20|69.11|**69.11**|**2.32**|**3.15**|\n||30|67.82|69.30|69.11|69.34|69.02|**69.15**|**4.63**|**5.46**|\n||10|67.28|69.14|69.41|69.50|69.44|**69.23**|**14.68**|**15.51**|\n\n\n**W2: Effectiveness of BN sharing for 1-bit quantization**\n\nThis is an important point, and we appreciate this insightful observation. In multi-bit settings, weight distributions tend to approximate a normal distribution, where our Bias Correction mechanism works effectively. However, in the 1-bit case, the weight distribution collapses into a near-uniform or binary form, causing a significant distribution shift that Bias Correction alone struggles to adequately address. For that reason, prior works on multi-bit networks often omit the 1-bit setting from their analysis [1,2,3]. In contrast, our method explicitly incorporates this case and demonstrates that assigning a separate BN layer for 1-bit quantization is both a simple and effective solution. \n\nTo further support this, we conduct ablation studies comparing two configurations: one where BN is shared across all bit-widths (including 1-bit), and another where the 1-bit case has its own BN, and the remaining bit-widths share a single BN. The results clearly show that including 1-bit in BN sharing significantly degrades 1-bit quantization performance. On the other hand, using a separate BN for 1-bit achieves strong performance with negligible additional overhead, that is the additional parameters for the separate BN layer, which accounts for less than 0.5% of the total number of parameters. This performance degradation stems from unstable BN statistics. The 1-bit weights produce activation distributions that are markedly different from those of higher bit-widths, leading to significant fluctuations in shared BN statistics. These fluctuations negatively impact the normalization of other bit-widths, ultimately harming overall model performance.\n\n**[Table 3. BN sharing with and without 1-bit quantization on ResNet-20 / CIFAR-10]**\n|**Framework**|1bit|2bit|3bit|4bit|5bit|6bit|7bit|8bit|32bit|**Avg.**| \n|-|-|-|-|-|-|-|-|-|-|-|\n|Share BN w/ 1-bit|46.38|91.90|92.57|91.48|90.98|90.53|90.24|90.14|89.90|**86.01**|\n|Share BN w/o 1-bit|92.49|89.44|92.43|92.67|92.43|92.15|92.00|91.97|91.65|**91.91**|\n\n\n**W3+Q3(a): Clarification on the training pipeline**\n\nThank you for drawing our attention to this, as we fully agree that a more detailed explanation of the training pipeline is in need. For ImageNet-1K experiments, fine-tuning begins from the pretrained Any-Precision model before calibration. Specifically for our method, we use this non-calibrated model as the starting point and apply coreset sampling and Bias Correction during fine-tuning, followed by BN adaptation at the end. In contrast to the original Any-Precision [4], which requires an additional calibration stage after training to achieve comparable accuracy at calibrated bit-widths, our approach eliminates this step. Training concludes after fine-tuning and a lightweight BN adaptation phase.\n\n\n**W3+Q3(b): Provide GPU hours with more detailed granularity**\n\nWe truly appreciate the comment and fully agree that providing a more detailed breakdown improves transparency and clarity in evaluating computational efficiency. In response, we report GPU hours for all ResNet experiments, broken down by training stage, including:\n\n- Score evaluation\n- Coreset training\n- Calibration (if applicable)\n- Adaptation\n\nThese results also help clarify the computational cost structure of our framework. Specifically, coreset sampling significantly reduces training GPU hours for multi-bit quantization models, and this efficiency gain increases with the pruning rate. However, coreset sampling alone does not eliminate the cost of the calibration phase, which is typically needed to support additional bit-widths. To address this, we apply Bias Correction and BN adaptation, which allow us to remove the calibration step entirely without sacrificing accuracy.\n\n**[Table 4. Breakdown of GPU hours for ResNet-20 experiments with 80% pruning]**\n|Framework|Training|Calibration|Adaptation|Scoring|Total GPU hours (Speedup)|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Dedicated|11.97|-|-|-|11.97 (1.00×)|\n|Any-Prec.|7.51|1.25|-|-|8.76 (1.36×)|\n|Bias Correction (Ours)|7.51|-|0.004|-|7.52 (1.59×)|\n|Bias Correction+Coreset Sampling (Ours)|1.52|-|0.004|0.37 *(Offline)*|1.52 (7.88×)|\n\n**[Table 5. Breakdown of GPU hours for ResNet-18 experiments with 80% pruning]**\n|Framework|Training|Calibration|Adaptation|Scoring|Total GPU hours (Speedup)|\n|-|:-:|:-:|:-:|:-:|:-:|\n|Dedicated|11.19|-|-|-|11.19 (1.00×)| \n|Any-Prec.|7.17|1.10|-|-|8.27 (1.35×)| \n|Bias Correction (Ours)|7.17|-|0.004|-|7.17 (1.56×)| \n|Bias Correction+Coreset Sampling (Ours)|1.47|-|0.004|0.74 *(Offline)*|1.47 (7.61×)|\n\n*Reference*\n[1] K.Xu et al., \"Eq-net\" ICCV 2023.\n[2] X.Sun et al., \"CoQuant\" ICCV 2024.\n[3] Y.Zhong et al., \"MultiQuant\" CoRR 2023.\n[4] H.Yu et al., \"Any-precision DNN\" AAAI 2021."}, {"review_id": "BKsX8r9yKf", "reviewer": "Reviewer_8UF3", "summary": "This paper proposes a method to reduce the computation required to train multi-bit quantization models. The authors observed that the quantization error in the model comes from a bias between the different quantization widths and propose to address this bias by directly modifying the weights. This results in a lighter model that does not need a batch normalization layer per quantization width. Additionally, the authors also propose an efficient strategy to calibrate the model that leverages core set selection to reduce the number of samples that the model needs to visit, further reducing the computation required to train the model.", "questions": "NA", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is well written and easy to follow. The visualizations effectively support the claims in the text and help understand the proposed approach.\n- The approach is well designed: each element tackles a weakness of the task in an efficient manner. As a result, the proposed approach surpasses SOTA approaches in accuracy and computation efficiency.\n- The experimental section is comprehensive and well-designed.\n\n\nWeaknesses:\n- The novelty of the proposed approach is limited, and the design is somewhat simple. The contribution of the paper could be stronger with a more detailed discussion or a theoretical framework that explained the effect of the different elements of the model. This would bring valuable insights to the community.\n- The efficiency benefits of the coreset sampling strategy should be better highlighted. This is an important contribution, and there is no formal definition of the computation required in relation to a standard approach.", "was_revised": true, "final_justification": "The authors addressed most of the reviewers' questions.", "rebuttal": "We sincerely appreciate the reviewer’s thoughtful and detailed feedback, which has greatly enhanced the depth, clarity, and overall quality of our work.\n\n**W1: Strengthen the contribution with more detailed discussion or theoretical insights.**\n\nWe appreciate the suggestion and agree that a deeper theoretical discussion can greatly enhance the contribution. In response, we would like to highlight *Section C* of our appendix, which provides a detailed theoretical analysis of cross-bit-width implicit knowledge transfer—a core aspect of our method. Specifically, we offer a formal perspective on how sub-networks with different quantization levels (e.g., 2-bit and 8-bit) can implicitly benefit from each other by sharing weights, even without overlapping data or explicit soft-label supervision. We show that the gradient updates from each sub-network lie within the span of their respective input data subspaces, and that the shared parameter is effectively updated over the union of these subspaces. This results in an implicit data-sharing effect that broadens the effective training distribution for each sub-network (Lines 143–155). Furthermore, we theoretically analyze the alignment between gradients from differently quantized sub-networks and argue that this serves as a form of indirect supervision—akin to soft distillation—without requiring additional targets or training overhead (Lines 156–165). This insight is, to the best of our knowledge, unique to our work and has not been previously formalized in prior studies on multi-bit quantization.\n\nWe believe this framework not only strengthens the novelty of our method, but also provides valuable insights into the design principles behind multi-bit quantized networks trained with shared weights. We will make sure to better highlight these points in the main text for clarity.\n\n**W2: More clearly highlight the efficiency gains of coreset sampling with a formal comparison to standard approaches.**\n\nWe appreciate your helpful feedback. To more clearly highlight the efficiency gains of our proposed coreset sampling strategy, we have added a formal comparison to standard approches (i.e., Any-Precision) in the Table below. This breakdown presents the GPU hours required for training, adaption, calibration, and scoring steps under different frameworks using ResNet-20 on CIFAR-10 with pruning rate of 80%. We excluded the scoring time from GPU hours, as it is an offline process and can be reused across all experiments under the same or comparable experimental settings (e.g., dataset, model architecture). \n\n**[Table. Breakdown of GPU hours for ResNet-20 experiments with 80% pruning]**\n|Framework|Training (Speedup)|Calibration|Adaptation|Scoring|Total GPU hours (Speedup)|\n|-|-|-|-|-|-|\n|Any-Prec. (Standard)|7.51 (1.00×)|1.25|-|-|8.76 (1.00×)| \n|Bias Correction (Ours)|7.51 (1.00×)|-|0.004|-|7.52 (1.16×)| \n|Bias Correction+Coreset Sampling (Ours)|1.52 **(4.94×)**|-|0.004|0.37 *(Offline)*|1.52 **(5.76×)**|\n\nWith these results, we would like to briefly clarify our framework. First, coreset sampling significantly reduces the training GPU hours for multi-bit quantization models. This efficiency gain grows in proportion to the pruning rate. However, coreset sampling on its own does not remove the cost of the calibration phase, which is required to support additional bit-widths. To address this, we apply Bias Correction and BN adaptation, effectively eliminating the need for calibration. Therefore, if we formally define the computational cost of each approach with the pruning rate $p \\in [0,1)$, where $p$ is the fraction of samples removed from the full dataset, it can be expressed as:\n\nCost of the Standard Approach (Any-Precision):\n$$\n\\text{Full-training time per bit} \\times \\text{The number of trained bit-widths} + \\text{Calibration time per bit} \\times \\text{The number of calibrated bit-widths}\n$$\n\nCost of Our Method (Bias Correction + Coreset Sampling):\n$$\n(1 - p) \\times \\text{Full-training time per bit} \\times \\text{The number of trained bit-widths}\n$$\n\nHere, calibrated bit-widths refer to intermediate bit-widths that are not directly trained but require calibration, such as 3, 5, 6 and 7 bits. BN adaptation incurs only a negligible overhead of 0.004 GPU hours as shown in the table, so we omit it from the formula for clarity.\n\nWe hope this clarification, along with the updated comparison table, clearly demonstrates the efficiency benefits of our framework. By jointly applying coreset sampling, Bias Correction, and BN adaptation, our method achieves substantial reductions in computational cost while preserving accuracy across bit-widths."}, {"review_id": "PTp5aDYu2Y", "reviewer": "Reviewer_RkBj", "summary": "They proposes two techniques to reduce the training overhead of multi-bit quantization networks. The first technique is weight bias correction, which aligns the quantized weights to match the activation outputs across different bit-widths, enabling shared batch normalization and eliminating the need for an extra training stage. The second technique is bit-wise coreset sampling, which dynamically selects informative data subsets for each child model based on gradient-based importance scores, reducing training redundancy. Experiments on CIFAR-10/100, TinyImageNet, and ImageNet-1K with ResNet and ViT architectures demonstrate that the proposed methods achieve competitive or superior accuracy while significantly reducing training time.", "questions": "See Strengths And Weaknesses", "limitations": "See Strengths And Weaknesses", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper introduces two novel techniques, which are well-motivated and supported by both theoretical insights and empirical results. It is worth noting that this method is applicable to both ViT and CNNs and is not restricted by the BN layer.\n2. The proposed methods achieve up to 7.88× reduction in GPU hours, making multi-bit quantization networks more practical for deployment. \n3. The authors conduct extensive experiments across multiple datasets and architectures, demonstrating the effectiveness and generalizability of their approach.\n4. The paper is well-organized and clearly written. The authors provide detailed explanations of their methods, supported by intuitive figures and comprehensive tables, making it easy for readers to understand and reproduce the work.\n\nWeaknesses\n1. The evaluations are restricted to computer vision tasks. While this is understandable given the computational cost of training multi-bit networks, extending the approach to other domains such as generative LLMs would further validate the broad applicability of the methods. Of course, this is merely encouragement rather than a requirement.\n2. Eq. 2 was not properly introduced and was somewhat abrupt. The paper does not provide a formal theoretical analysis or explanation of the proposed techniques of Eq. 2. Why Eq.2 reduce the errors?\n3. Furthermore, does the [29] mentioned in line 165 have been utilized in the ViT network? This might need some ablation experiments.\n4. Implementing the bit-wise coreset sampling strategy might be complex and require significant modifications to existing training pipelines. The authors could provide more detailed guidance or tools to facilitate adoption by the research community.", "was_revised": true, "final_justification": "I appreciate author's detailed rebuttal to address my concerns. I have read the author response and decide to maintain my recommendation.", "rebuttal": "We would like to extend our heartfelt appreciation to the reviewer for your careful and constructive feedback, which has profoundly enriched the quality and clarity of our work.\n\n**W1: Broader validation on other domains would strengthen applicability.**\n\nWe fully agree that validating our approach on other domains such as generative LLMs would further strengthen its general applicability and impact. However, exploring multi-bit quantization in the context of LLMs is currently not technically feasible for us, primarily due to the substantial computational resources required to train such models from scratch. Moreover, this application domain remains relatively understudied, and additional foundational studies are needed to understand the behavior of multi-bit quantization in LLM settings. A further challenge is that existing sample-importance scoring methods are tailored to discriminative models, and adapting them for generative models remains an open problem. We therefore consider this an important direction for future work and will include a discussion of this in the revised manuscript.\n\n**W2: Provide the theoretical basis behind Eq. 2 and how it contributes to error reduction.**\n\nThank you for pointing this out. We agree that Eq. 2 was introduced somewhat abruptly, and hope to clarify the theoretical groundings and provide a more detailed explanation of how it contributes to reducing activation errors below.\n\nOur goal is to mitigate the mismatch in activation distributions across bit-widths by addressing the root cause: systematic bias introduced during weight quantization. Since we isolate the effect of quantized weights by fixing the input activation, we analyze the convolution output as follows. Let:\n- $x \\in \\mathbb{R}^d$: fixed input activation (same across all bit-widths),\n- $w, w_q \\in \\mathbb{R}^d$: full-precision and quantized weight vectors,\n- $y = x^\\top w$: full-precision output activation,\n- $y_q = x^\\top w_q$: quantized output activation.\n    \nThen the error induced by quantization is: $\\epsilon = y_q - y = x^\\top(w_q - w)$. This shows that any discrepancy in output activations is a direct result of the quantization error in the weight vector, scaled by the input activation. Since $x$ is fixed, the variation across bit-widths in $y_q$ comes solely from $w_q$. As shown in Fig. 3, quantized weights ($w_q$) tend to introduce systematic shifts in both mean and scale compared to the full-precision weights ($w$), leading to:\n- $\\mathbb{E}[w_q] \\neq \\mathbb{E}[w]$ (bias mismatch),\n- $\\mathbb{V}[w_q] \\neq \\mathbb{V}[w]$ (variance mismatch).\n\nThese mismatches result in distributional shifts in the output activation $y_q$, which become more severe under lower bit-widths. Our proposed correction aims to eliminate these mismatches at the source, i.e., the weight level. We seek a transformation of $w_q$ that makes it aligned with $w$ in terms of mean and variance. Let’s define a corrected quantized weight vector $w_q’$ such that:\n\t1.\t$\\mathbb{E}[w_q’] = \\mathbb{E}[w]$\n\t2.\t$\\mathbb{V}[w_q’] = \\mathbb{V}[w]$\n\nTo align the quantized weights with the full-precision weights, we apply an affine transformation of the form $w_q’ = \\alpha (w_q + \\beta)$, where $\\alpha \\in \\mathbb{R}$ controls scaling and $\\beta \\in \\mathbb{R}^d$ controls shifting. To match the mean, we require $\\mathbb{E}[w_q’] = \\alpha (\\mathbb{E}[w_q] + \\beta) = \\mathbb{E}[w]$, which gives $\\beta = \\mathbb{E}[w] - \\mathbb{E}[w_q]$. To match the variance, we require $\\mathbb{V}[w_q’] = \\alpha^2 \\mathbb{V}[w_q] = \\mathbb{V}[w]$, which yields $\\alpha = \\sqrt{ \\frac{\\mathbb{V}[w]}{\\mathbb{V}[w_q]} }$. Substitute $\\alpha$ and $\\beta$ into the affine form, and we get the Bias Correction equation:\n$$\nw_q’ = \\sqrt{ \\frac{\\mathbb{V}[w]}{\\mathbb{V}[w_q]} } \\left(w_q + \\left(\\mathbb{E}[w] - \\mathbb{E}[w_q]\\right)\\right).\n$$\n\nUsing this corrected $w_q’$, the new activation becomes:\n$$\ny_q’ = x^\\top \\left( \\sqrt{ \\frac{\\mathbb{V}[w]}{\\mathbb{V}[w_q]} } (w_q + \\mathbb{E}[w] - \\mathbb{E}[w_q]) \\right).\n$$\n\nThe expression above can be interpreted as an affine transformation of $y_q$ (i.e., rescaled and shifted), controlled by the quantized weight distribution’s discrepancy from the full-precision distribution. Since the input $x$ is fixed, correcting the weight vector’s mean and variance directly adjusts the distribution of the output $y_q’$, reducing systematic activation shifts and scale mismatches across bit-widths. In doing so, the challenge of aligning activation distributions is effectively reduced to a simpler and more stable problem of aligning weight statistics. As a result, different quantized sub-networks can share BN layers more reliably, as the activations are already well-aligned at the source (i.e., before normalization), eliminating the need for separate BN parameters or additional activation calibration. We will incorporate this detailed theoretical grounding into the main manuscript and provide the full derivation and extended discussion in an appendix of the revised paper.\n\n**W3: Clarify whether BN adaptation has been applied to ViTs.**\n\nViTs use Layer Normalization (LN) instead of Batch Normalization (BN), eliminating the need for BN-specific calibration, such as updating running mean and variance after training. Unlike BN, LN does not maintain such running statistics. As far as we are aware, there has been no prior investigation into how LN behaves in the context of multi-bit ViT networks. Our work is the first to experimentally demonstrate that LN parameters, once trained, remains relatively effective across all bit-widths without requiring additional calibration or adjustment. We will incorporate this discussion and the detailed results into the Appendix\n\n**W4: Implementation of bit-wise coreset sampling into existing pipelines seems complex, provide more guidance.**\n\nWe understand that ease of implementation is crucial for broader adoption, and we truly appreciate the feedback. In practice, our scoring and sampling code is already modularized, making it easy to plug into various network architectures and base models. To further support integration, we will provide a detailed guide covering key components such as pruning rate $p$, sampling frequency, and the temperature variable—three of the few hyperparameters closely tied to our method. We also plan to automate the full pipeline with a single bash script and include clear instructions for each stage (scoring, sampling, training, and if applicable, BN adaptation). Finally, we will publicly release all code and scripts to facilitate easy adoption and reproducibility."}]

Author rebuttal:

Meta-review: This paper introduces techniques to reduce the training overhead of multi-bit quantization networks, including weight bias correct enabling shared batch normalization and bit-wise coreset sampling which allows each child model to train on a compact subset. The methods are validated on CNN and Vision Transformer models, showing clear speedup while matching the accuracy of prior methods.

The paper is very well written, with observation, problem statements, and solutions clearly motivated, explained, and insightful. The proposed methods are reasonably novel. The results demonstrate significant improvement in training time over existing methods while preserving model quality. 

During the rebuttal, the authors provided additional data and clarification, such as the impact of resampling frequency, BN adaptation, the computation overhead for dynamic sampling. All reviewers gave positive evaluation. Therefore, this paper is recommended for acceptance.
Decision: Accept (poster)