Compute-Optimal Quantization-Aware Training
Authors: Aleksandr Dremov
David Grangier
Angelos Katharopoulos
Awni Hannun
Venue: ICLR 2026 Poster (2026)
Keywords: quantization-aware training
QAT
neural network quantization
compute optimization
scaling laws
large language models
LLMs
model compression
compute budget allocation
training efficiency
model optimization
quantized neural networks
efficient deep learning
Primary area: foundation or frontier models, including LLMs
TL;DR: The optimal fraction of quantization-aware training compute (vs. pretrain stage) increases with total compute budget. We derive scaling laws to predict optimal allocation and model loss, enabling higher-quality model training with the same compute.

Abstract: Quantization-aware training (QAT) is a leading technique for improving the accuracy of quantized neural networks. Previous work has shown that decomposing training into a full-precision (FP) phase followed by a QAT phase yields superior accuracy compared to QAT alone. However, the optimal allocation of compute between the FP and QAT phases remains unclear. We conduct extensive experiments with various compute budgets, QAT bit widths, and model sizes from 86.0M to 2.2B to investigate how different QAT durations impact final performance. We demonstrate that, contrary to previous findings, the loss-optimal ratio of QAT to FP training increases with the total amount of compute. Moreover, the optimal fraction can be accurately predicted for a wide range of model sizes and quantization widths using the tokens-per-parameter-byte statistic. From experimental data, we derive a loss scaling law that predicts both optimal QAT ratios and final model performance across different QAT/FP compute allocation strategies and QAT bit widths. We use the scaling law to make further predictions, which we verify experimentally, including which QAT bit width is optimal under a given memory constraint and how QAT accuracy with different bit widths compares to full-precision model accuracy. Additionally, we propose a novel cooldown and QAT fusion approach that performs learning rate decay jointly with quantization-aware training, eliminating redundant full-precision model updates and achieving significant compute savings. These findings provide practical insights into efficient QAT planning and enable the training of higher-quality quantized models with the same compute budget.

Published as a conference paper at ICLR 2026
COMPUTE-OPTIMAN QUANTIZATION-AWARE TRAINING
Aleksandr Dremov" David Grangier
Angelos Katharopoulos
Awni Hannun
Apple Inc.
(alexdremov, grangier, a kktharoppoulss, awni
ABSTRACT
Quantization-awee awarre training (QAT) is leading technique for improving the accu- racy of quantized neural networks. Previous work has shown that decomposing training into full-precision (FP) phase followed by QAT phase yields superior accuracy compared to QAT alone. However, the optimal allocation of compute be- tween the FP and QAT phases remains unclear. We conduct extensive experiments with various compute budgets, QAT bit widths, and model sizes from 86.0M to 2.2B investigate how different QAT durations impact final performance. We demonstrate that, contrary to previous findings, the loss- loss-optimal ratio of QAT to FP training increases with the total amount compute. Moreover, the optimal fraction can be accurately predicted for a wide range of model sizes and quan- tization widths using the tokens-per-pann tel---epppaameee--reeeee data, we derive loss scaling law that predicts both optimal QAT ratios and final model performance across different QAT/FP compute allocation strategies and QAT bit widths. We use the scaling law to make further predictions, which we verify experimentally, including which QAT bit width is optimal under a given memory constraint and how QAT accuracy with different bit widths compares full-precision model accuracy. Additionally, we propose a novel cooldown and QAT fusion approach that performs learning rate decay jointly with quantization- aware training, eliminating redundant full-precision model updates and achieving significant compute savings. These findings provide practical insights into efficient QAT planning and enable the training of higher-qualin quantized models with the same compute budget.
statistic. From experimental
INTRODUCTION
As Large Language Models (LLMs) grow in size and on-device applications gain traction (Xu al., 2024). significant attention has been devoted to reducing inference costs via model compression
(Frantar et al. 2022 Lin et al. 2024 Ma al., (023). One -fate-of-the-art method quantization-
aware training (QAT) (Chen et al. 2025a Lin et al. 2024 Liu et al.) [025 Jacob et al. 2018). To adapt the model the loss of numerical precision, QAT incorporates quantization directly into
the model training process. It has been shown that QAT outperforms post-training quantization al., 2019), where quantization is applied after training is completed. Moreover, Liu et al. (2025) demonstrated that the best accuracy is achieved when QAT phase follows full- precisionn (FP) training phase.
(PTQ) (Xiao et al.) 2023 Banner
For models designed for on-device use, the QAT stage is an important part of the training process and usually planned in advance, As model sizes grow and deployment constraints tighten. practitioners face a critical resource allocation problem: given fixed compute budget, how should training time be divided between full-precision pretraining and quantization-awwe directly impacts both model quality and deployment efficiency, yet existing guidelines assume fixed allocation ratios regardless of scale. As motivation, we note that Kumar et al. (2025) demonstrated that the error introduced by post-training quantization grows with the size of the pretraining data, which can actually make additional pretraining harmful. Intuitively, analogous to PTQ. having longer full-precision stage should make subsequent QAT more difficult. While Liu et al. (2025) showed that spending 10% of the training steps on QAT is optimal for some setups, the authors did not explore how this proportion varies across different training lengths and model sizes.
training? This decision
WWor done at Apple during internship while studying at Éeole Polytechnique Fédérale de Lausanne (EPFI.),
dleksandr.drreoo/ @epfl.ch

---

Published as a conference paper at ICLR 2026
In this work, show that previous conclusions about optimal QAT length do not hold with increased compute budget. Through a series experiments with different model sizes and token counts, we demonstrate that the optimal fraction of QAT compared to the total training length increases with the total compute budget. This optimum can be accurately predicted for a wide range of setups using the peeen--pe--prrmetee---
statistic. Additionally, law as a function of model parameter count (N), token count spent on full-precision training (Dip), token count spent on QAT (Dat), and QAT bit-width (B). The fitted law accurately captures the growth of the optimal QAT fraction with compute scale. The key contributions of this study are:
propose loss scaling
Unlike previously assumed, we find that the optimal fraction allocated for QAT in- creases with the growth of the higher-quality quantized models to be achieved with the same initial compute budget (fig- (T.eft).
pae-parramter--btte statistic. This finding allows
We propose loss scaling law that captures the optimal QAT fraction phenomenon and models the final expected loss of the FP and QAT pipeline (figure [ (Right)). We use the scaling law to make further predictions, including which QAT bit-width is optimal under a given memory constraint and how QAT accuracy with different bit- widtts compares to full-precision model accuracy.
We propose novel approach: QAT & Learning Rate Cooldown Fusion-a scheme where learning rate decay is performed jointly with quantization-awant training, eliminating redundant full-precision updates and achieving better accuracy for the same token count.
Optimal QAT Length Fit
Loss Scaling Law for Different QAT Fractions
1.0
1012
QAT fraction
0.8
FPT tokens, Dip
3.6
10t1
0.6
0.4
1
1010
0.2
2.8
109
0.0
102
10a
10*
109
10
10
10
Total tokens-per-byte
QAT tokens, qat
Best QAT fraction
Theory Experimental optim. Dip + Du const
Best
Figure 1: On the left, experimental and predicted optimal QAT fractions as a function of tokens- peeeetrramteer are shown. Different colors represent models of varying sizes, while point sizes indicate final perplexity normalized across experiments with identical total token counts for each model size. Results span multiple QAT bit-widths, and optimal QAT fraction values for endpoints are displayed. The plot demonstrates that the optimal QAT fraction increases with the full training petrts-pe--prattte---aee
statistic. On the right, loss scaling law predictions for 4-bit QAT 396M parameter model across varying QAT and FP training lengths. Both experimental and theoretical optima shown. The optimal QAT fraction predicted by the loss scaling law for each total token count closely matches the experimentally observed fraction.
RELATED WORK
Quantization of LLMs. Quantization is method for reducing both the memory footprint and the computational requirements of neural networks by lowering the precision used in the network. By reducing the bit-width of the weights, activations, both, quantization enables models to run faster, consume less power, and use less memory, which is particularly bene- ficial for deployment on resource-comstrainee devices. There are different quantization tech- niques, including post-training quantization (PTQ) (Xiao et al.) 2023, Banner et al.] 2019)
methods that transform model after training has been completed and usually require mini- mal additional computational usage. Another group of methods is quantization-awan training
(QAT) (Chen et al. 2025a Lin et al. 2024 Liu et al. 2025 Jacob et al. 2018), which quantizes
2

---

Published as a conference paper at ICLR 2026
model during training, allowing the model to better adapt to precision loss. As quantization opera- tions are non-differentiable, training relies on gradient approximations such as the straight-through estimator (Bengio et al. 2013). In contrast to PTQ. QAT requires more computation, as effectively the full model is trained with added quantization-rell as it is the method most commonly used in practice obtain high-quality quantized models.
operations. In this work, we focus on QAT,
Loss Scaling Laws.
Multiple works have previously addressed the problem of predicting final model loss (L) a function of parameter count (N) and consumed tokens (D) (Bi et al., [224]] Hoffmann et al. 2022 Kaplan et al., 2020). The Chinchilla (Hoffmann et al. 2022) loss model is one of the most commonly used: L(N, D) + AN + CDDP, where A, C, a, B. and E are fitted parameters. Bi et al. (2224) expand on this idea, fitting accuracy as a function of used non-embedding FLOPSs (FLOOP estimation of model inference without embedding layer caleulations), showing that such an approach works better across different model sizes. Additionally, they show that scaling laws are greatly influenced by data quality.
QAT Loss Scaling Laws. Chen al. (025b) proposed scaling law modeling specifically for QAT loss, adding a QAT-related penalty to the Chinchilla loss model:
C k DTD. (log, G)
L(N,D.G) = E+
No +
(1)
D
NYN
Chinchilla loss
QAT error
where G is the quantization granularity (number of elements in each quantization group), k, YD. YG. and yN are fitted parameters, and the Chinchilla loss parameters are fixed from the non-quantized model fit. Therefore, this approach effectively models QAT error relative to the full-precision model for the same token and parameter count. However, formulas are fitted exclusively for each quantization bit width, which complicates analysis of the relationship between different bit widths. Kumar et al. 2225)) propose precision-awan scaling laws for training and inference, predicting loss from wwwprrccision training and PTQ or QAT.
While Kumar et al. (2025): Chen al. (2025b) laws are useful for understanding the final accuracy of model trained with QAT from scratch, they overlook the fact that QAT is typically resumed from full-precision training to achieve better accuracy (Liu et al., 2025] Zhou et al. (025a)). Our work addresses this issue and presents novel loss scaling law that explicitly handles the case when QAT is started from a full- full-precision checkpoint and works across different bit widths.
3 OPTIMAL QAT COMPUTE ALLOCATION
To study how loss changes for different combinations of QAT/FP training length, train models of different sizes (N), different FP stage token counts (Drp), QAT token counts (Dqut), (, and different QAT bit widths (B). For the smallest model (86.0M parameters), we conduct experiments from 2.3B to 1.4T total tokens, while for the largest model (759.00 OM parameters). we conduct experiments from 8.5B to 669.2B total tokens. We focus primarily on 1-, 2-, 4-, and 6-bit quantization widths. Full description of our experimental setup is described in appendix A and token counts and QAT fractions used are reported in appendix(N] We also verify that the obtained post-QAT models maintain reasonable accuracy and present a comparison to full-precision models in appendix P Specifically, our 4- and 6-bit setups achieve quality close to that of the full-precision model for the same total token count, and the drop in quality for 1- and 2-bit is reasonable.
The main objective of this study is to determine the optimal QAT fraction fh fraction of the token count that should be dedicated to QAT for given total token count. This can be formalized as the following minimization problem:
L(N, Dip D B), f (N, potal, B) que (N, D B)
qui (N, D Protal B) arg min EN,
D
D
where L(N, Dip, Daat, B) is the final loss of the setup with Dip tokens dedicated to FP training and Dqat tokens dedicated to B-bit QAT. Intuitively, f" expresses trade-off. On one hand, too few QAT steps do not allow the quantized model to adapt to reduced precision. On the other hand, too many QAT steps (at the expense of fewer FP steps) should also lead to worse loss since QAT is trained with gradient approximations for quantization operators, which introduces biased and noisier gradients. Naturally, trade-off emerges, suggesting that such f" should be well-defined.
3

---

Published as a conference paper at ICLR 2026
3.1 PREDICTING THE OPTIMAL QAT FRACTION
Optima Plot (tokens)
In this section, we focus on fitting the opti- mal QAT fractions directly. To account for different QAT bit widths used, we introduce the tokens-pers peeeen--e-ppramete--be
QAT QAT tokens
10"
statistic. This
choice was made based on several observations: larger models are generally easier to quantize, models trained for longer are harder to quan- tize, and smaller QAT bit widths are harder to quantize well. While being intuitive from QAT accuracy perspective, it can also predict the optimal fraction with high precision. Fig- ure provides a comparison between the two approaches. It is clearly per-parameter-byy provides an interpretable ad- justment, facilitating better fit.
1010
10
10
10
Total tokens
Optima Plot (tokens-per-byte
that using tokens-
QAT tokenss-sppr ceens--pe-bby byte
10a
For optimal QAT fraction prediction, we fit function of the form
10a
exp (log Stotal log Sunal
F(Diotal N. B)
101
Swotl
total
10a
10a
Siotl
N
Total tokens-per-byte
where f is the predicted optimal QAT fraction for the total Swatl, and a fitted parameter. This function choice was made due to the observed almost lin- ear dependency between Stotal and optimal Sqir in log-log coordinates, but with the added con- straint that qat S D Droul. The optimal in our setup yields 6.7297.
On the top, QAT optima for 396M model plotted in token coordinates. Different op- tima for the same total token count and different QAT bit widths can be observed. On the bottom, QAT optima for 396M model plotted in tokens- per-parameter-byy coordinates. With byte ad- justment, different bit widths lie on the proposed fit line better.
Figure 2:
count
Optimal QAT Fraction Fit Results. We fit the proposed equation directly using Huber loss (Huber 1964) and gradient descent optimization. The approach achieves 0.091 MAE in fraction prediction across all model sizes and experiments. We also verify that the error remains low if we remove the largest tested model from the training and evaluate accuracy only it. The results are displayed in Optimal points lie close to the predicted optimal fraction. We can make the following high-level observations: the optimal QAT fraction grows faster with Diotal for lower bit widths, the optimal QAT fraction decreases with model size N increase and fixed Diotal One limitation of the fit is that is subject to the gramularity of the selected experiments -the set of QAT fractions being tested. Also, as we fit only optimal points, we do not use substantial amount of non-optimal data points, which also contain valuable information about loss behavior. One way to utilize all available data is to model the loss scaling law explicitly and infer optimal QAT fractions from it. We focus on this in section 3.2
Takeaway
Optimal QAT compute allocation fraction not stationary but grows with total training tokens- per-parameter-hyee (Swnal) and can be predicted from it.
3.2 Loss SCaLING LAW
As described in section Chen al. (2025)) were able to fit a loss scaling law for QAT started from scratch (Dip = 0). We extend this idea by making the loss scaling law dependent not only on D Diotal but also on D Dip, Dau. and B, essentially modeling loss for different QAT fractions and bit-widths. However, we do not follow the same functional form as that proposed by Chen et al. (2025b). This point with
is because in equation the QAT penalty overtakes the Chinchilla loss term

---

Published as a conference paper at ICLR 2026
the growth of token count, which causes the limit of the whole expression approach infinity as D total - co. This does not align with the expected loss decrease as token count grows and will hinder making any predictions from the scaling law in the future. We propose loss model in the form of
B D
6 N
L(N, D Dip, B) = a
+
+ 6(N, gatt Dip.
total
QAT fraction-aware penalty
Chinchilla like loss
2
1 2-
(2)
8(N, D Deat. Dip, B) =
0 0
N S
NY.
S 8o 1p
qat
Irreducible QAT error
Pure QAT penalty
FP/ /P/T interaction
where all lowercase Greek letters are fitted parameters and Squi Dag N. u. Sip N. are the corre- sponding parerr-rratteerbyytee This choice of ö(N, Dua. Dip. B) is motivated by the depen- dence of the optimal QAT fraction on tokens ceennssper-ppaae per-parameter-byee as discussed in section specific motivation for each term is described in equation 2
Dip
Loss Scaling Law Fit Results. We the proposed equation for 757 total QAT experiments directly using Huber loss (Huber) 1964) and gradient descent optimization setup consistent with that of Hoffmann et al. (022); Chen et al. (2025b). The results are highly dependent on initialization; therefore. select the best fit out of many random initializations. We achieve similar fit quality across different bit-widths: R² 0.982 for 1-bit QAT and R² = 0.991 for 6-bit QAT, where R is the presented in table T We present the fitted formula and its visualizations in appendix D and plot the 3D loss scaling law surface for fixed model size in figure 3 Also, in appendix G we fit formulas independently for each bit-width B as a baseline and verify that the unified formula achieves similar fit metrics. Additional scaling law fit notes are provided in appendix [ appendix K verifies that optimal QAT fraction prediction generalizes to larger model sizes (2.2B), and Oppennii/(Jconduue uncertainty analysis and parameters significance tests.
coefficient of determination. Full fit metrics
Table 1: Fit metrics for the loss scaling law. We report both the metrics of the loss fit and of the optimal QAT fraction prediction inferred from the loss scaling law. It is seen that the proposed formula in equation provides good fit of loss well as of the optimal QAT fraction.
MAE, loss fit
R², loss fit MAE, optimal QAT fraction fit
0.026
0.982
0.081
0.023
0.981
0.102
0.021
0.983
0.074
0.018
0.991
0.09
The fitted formulas are analytically sound: with an increase either Dip or D while the other is fixed, the total loss decreases. Additionally, the proposed form effectively captures the optimal QAT fraction. From experimental results and the fitted loss function, we observe that low-bit QAT is higher for I-bit than for 6-bit. Therefore, selecting the optimal QAT fraction is especially important in low-bit settings. To assess the generality of our formulation, reproduce our findings with different pre-training and QAT hyperparameters and the SlimPajama (Soboleva et al. 2023) dataset in appendix !
sensitive the QAT fraction being optimal. Loss increase for sub-optimal QAT is
Takeaway 2
Final loss after QAT can be accurately predicted from N, Duat- Dip and B for various settings using single formula. Moreover, the proposed loss scaling law effectively captures the phenomena optimal QAT fraction and can be used to infer it.
5

---

Published as a conference paper at ICLR 2026
4 Loss SCALING LAW IMPLICATIONS
The unified loss scaling law obtained in the previous section allows us to analyze practically important QAT properties. In this section, address previously unanswered questions such as: "How bad sub-optimal QAT compute allocation?", "When does QAT match FP accuracy?"" and "How should select QAT precision and parameter count?"
4.1 EVALUATING SUR-OPTIMAL QAT FRACTION
Using the loss scaling law, we compare the optimal QAT setup with a sub-optimal one. To do this, calculate wasted token count-the number of tokens effectively wasted by sub-optimal QAT fraction. Figure summarizes wasted token count for different bit widths and token counts; 10% QAT the reference for the sub-optimal setup. Formally, with fitted loss model L(N, qut. Dip. B), we can find such D and optimal D
qui (N, D B) that achieve the same loss
I as sub-optimal D subopt and D subopt Specifically: qai
total
= L(N, D suhopt
D total
suhopt
D subopt -B), qat
D total
arg min Dugg EN
[L(N, Dar Potal
Duu B) 1/.
P/ Mu-Dipl! ()m
D subopt D iotall
D wasted
total
and the reported percentage in figure is the fraction of total tokens:
Danated
Two factors influence the wasted tokens magnitude: closeness of 10% to the optimal QAT fraction and the overall flatness the loss scaling law for high token counts. If the predicted loss is generally flat for some token count, then even high deviation from optimality will yield minor wasted token count. In the extreme case, for 1-bit QAT, the same loss can be achieved with just around 50% of compute if the optimal QAT fraction is used. This effect is still present for 2 bit QAT but becomes relatively small for 6-bit.
Loss Scaling Law
Comparison: 10% QAT vs. Optimal QAT
B
Wasted tokens, %
2.8
20
Loss
2.7
2.6
10.0
10.0B
100.0B
1.0T
10.0T
10.5
Total tokens
10
12 12.0 10g (D
11.0
11.5
1og qur)
Figure 4: Comparison sub-optimal QAT setup with fixed 10% QAT fraction and optimal QAT setup for 1B parameter model. Wasted token count is the number of tokens effectively wasted by not utilizing an optimal QAT fraction setup. That is, if the wasted token count is n%, then the same loss can be achieved with (100-n)ft tokens and optimal QAT fraction. While results vary for different bit widths, the general relationship is similar, revealing high potential savings.
Figure 3: Visualization of the fitted loss scal- ing law for 759M model, 1-bit QAT, and dif- ferent D qat Dip- Orange lines represent constant Diotal D D levels, and stars represent loss minima for each such level. It is clearly seen that the loss structure yields an optimal QAT fraction for specific tostl- The overall phenomenon is consistent with what was discussed section

---

Published as a conference paper at ICLR 2026
Takeaway 3
Suboptimal QAT compute allocation significantly impacts final model performance, especially in low-bit settings. In the extreme case, for 1-bit QAT, the same loss can be achieved with just around 50% of compute if the optimal QAT fraction is used.
Note on Compute Budget-Token Budget Duality. So far, we have considered token count to be identical compute budget compute scales linearly with training token count. However, one may argue that since QAT employs additional operations, its complexity is higher than FP training. As QAT overhead depends only on model size, becomes negligible with sufficiently large batch size and sequence length (appendix D. Still, in setups where QAT overhead is substantial, one can obtain compute-based optimal QAT fraction from the token-based approach by making a substitution qut qar- where aware QAT token count. This will make QAT steps "more expensive" from loss minimization perspective. In this setup, Dip + D' = const represents not iso-token levels, but rather iso-FLOP levels. Therefore, the inferred optimal QAT fraction will be adjusted to account for the overhead.
the QAT overhead factor the specific setup and D the overhead-
4.2 WHEN DOEs QAT MATCH FP ACCURACY?
We plot the difference in perplexities between QAT and FP models for each total token count. Appendix [ describes how obtain the full-precision model loss scaling law. In summary, incorporate full-precision training results into the with 16, which not only allows us to predict full-precision model performance but also serves regularization for the fit. Figure presents such a plot for models of two different sizes.
FP Accuracy Reproduction. The practical observation that larger-parameter models tolerate lower-bit QAT is clearly observed. A second perspective from which to consider figure i is that of optimal QAT bit width. Specifically, for given total token count, there exists a minimum bit width that matches FP model loss. Therefore, there is no incentive to train higher-bit QAT, as this will not result better accuracy but only higher memory usage.
Takeaway 4
The proposed loss scaling law effectively captures the empirical observation that larger- parameee models can tolerate lower-bit QAT. Moreover, using the loss scaling law, one predict a range total token counts for which QAT accuracy will not differ significantly from that a model.
Model Size 500.0M
Model Size 16.0B
10.0
Perplexity diff, %
7.5
5
2
4
5.0
2.5
0.0
100.0B
1.TT
10.0T 100.0T
100.0B
1.TT
10.0T
100.0T
Tokens count
Tokens count
Figure 5: Difference in perplexity between FP loss scaling law and QAT loss scaling law for two model sizes. For QAT, the loss corresponding to the optimal QAT fraction used. Values below O correspond to QAT performing better than the FP model. It is clearly observed that the ability of QAT to match models are able to tolerate lower QAT precision for higher total token count budgets. Additional plot information is present in appendix
loss is greatly influenced by model size and token count. particular, larger
7

---

Published as a conference paper at ICLR 2026
4.3 PARAMETER PRECISIOO TRADE-OFF
An interesting question analyze is "for fixed model memory requirement, how should one select QAT precision and parameter count?".
Predicted Optimal QAT Bit- Width
8
Parameters memory
1TB
That is, to fit a specified memory constraint, choose high precision the expense
100GB
6
of a lower parameter count or vice versa. This
10GB
question is practically important LLM infer-
1GB
ence is largely bottlenecked by memory band- width (Davies al. 2025 Recasens al.) 2025)
100MB
3
-2
10MB
Dao et al. 2022). We can derive such optimal- ity from the fitted loss scaling law. The results are presented in figure [ It is clearly seen that for fixed memory budget, optimal QAT preci- sion decreases with training FLOP growth. This suggests that for achieving an optimal quantized model within some memory and training com- pute budget, one should select the parameter count in advance accordingly. We believe this finding to be important for practitioners trying to achieve the best-accuracy model within memory constraints. Figure 6 is verified experimentally in appendix
10
10
10
102a
10
10th
Training FLOPs
Figure 6: Optimal QAT bit width for different memory budgets and total training budgets. We use the loss corresponding to the optimal QAT fraction. For training FLOPs, we use the estima- tion C 6ND. The white area corresponds to < N, which is not practically important.
Takeaway 5
The proposed loss scaling law can predict what QAT bit width is optimal for fixed training compute budget and model memory footprint. It is revealed that for a fixed memory budget, optimal QAT precision decreases with training FLOP growth.
BEYOND QAT COMPUTE FRACTION: QAT & COOLDOWN FUSION
Section 3 revealed the importance of advance planning for QAT, accounting for the optimal fraction. This is possible only when one controls the entire pretraining process: both QAT and FP. this context, it may be worth adjusting the training procedure to make QAT more efficient. Specifically, in this section, we focus on modifications to learning rate scheduling techniques. Currently, classic way of training models is to perform full FP training with learning rate cooldown, and then start QAT with learning rate re-warmup. We used such a setup for the scaling law in section as is universally adopted. However, more optimal scheme may exist.
QAT Learning Rate Cooldown Fusion. Wen et al. (2024) show that re-initializing WSD from post-cooldown checkpoint rather than from constant stage yields better results. However, believe the behavior might be different when resuming training from a checkpoint with QAT. We propose a novel idea: QAT & Learning Rate Cooldown Fusion. Motivated by the idea that learning rate cooldown performs low-magnitude adjustments to weights, we speculate that a substantial part of updates during learning rate cooldown gets destroyed by QAT initialization, which, in essence, discards high-precision information. Therefore, we analyze a setup where QAT is started directly from the learning rate constant stage and learning rate cooldown is performed jointly with QAT. A schematic representation of the two schemes is presented in figure 7
We ran experiments with different model sizes and 4-bit QAT using the described "QAT Cooldown Fusion" scheme, taking experiments with the classic QAT scheme and optimal QAT fraction as baselines. The results are shown in table 2 In addition to perplexity, we report loss change in "wasted tokens" units. This is the total token distance between corresponding loss points in the scaling law for an optimal QAT fraction. Such a metric is reported for better impact understanding, small perplexity differences are harder to achieve with high overall token counts. We achieved improvements across all model sizes for 4- and 6-bit widths and all token counts. Results differed for 1- and 2-bit settings; believe this is due the large optimal QAT fraction, which makes the
8

---

Published as a conference paper at ICLR 2026
FP fraction small and, consequently, the impact of QAT & cooldown fusion lower. Full results available in appendix M While perplexity differences may seem small, judging the difference from the perspective of token count difference is significant. This implies substantial improvements in terms of training cost.
Takeaway
The proposed method of learning rate scheduling. "QAT & Learning Rate Cooldown Fusion," " further improves QAT efficiency by adjusting the training procedure beyond changing the QAT fraction. While being practically useful, this method also suggests that modifications to the universally accepted QAT pipeline can further improve QAT efficiency.
Classic QAT Scheme
Cooldown & QAT Fusion Scheme
Learning rate, max Ir %
100
75
50
25
FP training
QAT
25
25
Total training tokens, %
Total training tokens, %
Figure 7: Comparison between two different QAT schemes. In both setups, the QAT fraction is 40%. Red-shaded areas indicate zones with lowered learning rate, which we expect to correspond to minor weight updates that get effectively ignored by QAT initialization.
the left, classic QAT scheme visualization: QAT follows fully completed FP training that ends with 20% (of FP training length) learning rate decay. For QAT, the learning rate follows cosine shape with 5% re-warmup phase. On the right, the "QAT & Learning Rate Cooldown Fusion" scheme is displayed. QAT starts directly from the constant learning rate stage with small re- re-warmup, effectively resuming the FP learning rate scheduler training length). As QAT follows the classic FP learning rate recipe with usual cooldown, we call this approach QAT & Learning Rate Cooldown Fusion.
QAT was present all. QAT ends with 20% cooldown (of total
Table 2: Accuracy comparison between the classic QAT scheme and the "QAT Learning Rate Cooldown Fusion" training scheme. The loss difference is reported in "wasted tokens" differ- ence total token count between optimal QAT fraction loss points the loss scaling law (formally defined in appendix O). Substantial improvements noticeable across different model sizes and token counts.
Perplexity
Wasted tokens, Unfused total tokens, %
Unfused (baseline)
Fused (ours)
B
Model size, M
D Dotal
4
74
1.4T
16.26
16.25.o.00 0.06%
2.2%
163
901.3B
13.51
0.15%
9.2%
425
10.5B
16.3
16.02.1.72%
9.6%
31.8B
13.9
13.76.101%
10.4%
96.0B
12.62
12.54-0.63%
13.6%
816
281.9B
I1.07
11.02 0.45%
13.2%

---

Published as a conference paper at ICLR 2026
6 CONCLUSION
This work addresses resource allocation problem in quantization-aware training: how to opti- mally divide compute between full-precision pretraining and quantization-ane training. Through extensive experiments across model sizes, compute budgets, and quantization bit widths, we chal- lenge existing assumptions and provide practical guidelines for efficient QAT planning. Our key contributions are:
Discovery of Compute-Deper
Optimal QAT Fractions. Through extensive experi- ments across different model sizes, compute budgets, and QAT bit widths, we demonstrate that previous assumptions about optimal QAT allocation do not hold as compute budgets increase. Our findings reveal that the optimal QAT fraction is not fixed percentage but rather increases with the total compute budget, specifically with the tokens-per-paramee per-parameter- byte statistic. This challenges the previous conclusion that 10% is universally optimal for QAT length relative to total training length. We demonstrate that using suboptimal QAT fractions can result in substantial compute waste, with extreme cases showing that the same loss can be achieved with just around 50% of the compute when optimal QAT fractions are used, particularly for low-bit quantization scenarios.
Comprehensive Loss Scaling Law. We derive comprehensive loss scaling law that models the final expected loss of the full-precision and quantization-aware training pipeline as function of QAT bit width, model parameter count, and token counts for both training phases. This scaling law not only captures the optimal QAT fraction phenomenon but also enables prediction of final model loss across different QAT/FP allocation strategies. From the scaling law, we infer which QAT bit width is optimal under given memory constraint and how QAT accuracy compares to FP model accuracy.
Cooldown and QAT Fusion Technique. We introduce a novel approach that performs learning rate decay jointly with quantization-awan training, eliminating redundant full- precision updates and achieving significant compute savings. While being practically useful, this method also suggests that modifications to the universally accepted QAT pipeline can further improve QAT efficiency.
Limitations. While we checked the generality of our formulation by performing experiments with a different dataset and hyperparameters in appendix J our work still focuses on a specific LLM architecture, and exact results may differ for different model types. However, we expect the overall observed phenomena to be consistent across different architectures and datasets.
Future Work. We identify several research directions worth exploring. First, the relationship between the optimal QAT fraction and pretraining precision remains unknown. This direction is especially interesting with the emergence of 8-bit floating-point training (Peng al.) (023) and even 4-bit training (Zhou et al., 20256) Wang et al.) 2025). Second, we are interested in how the observed phenomena are preserved across different training stages. Specifically, how does the optimal QAT fraction change when the full-precision training stage incorporates additional stages such as Supervised Fine-Tuning (SFT) (Ouyang al. 2022). Reinforcement Learning (RL. (Rafailov et al.) 2023; Schulman et al. 2017), multimodal training? We speculate on these questions in
appendix R
Reproducibilityy Statement. We report exact hyperparameters and training approaches used in appendix A Additional experimental information that should facilitate reproduction summarized in appendix B C
and[
Ethics Statement: LLM Use Disclosure. LLMs such as Anthropic (2025): Mistral Al (025) were used in the preparation of this paper exclusively for improving grammar and wording.
REFERENCES
Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlícek, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von Werra, and
10

---

Published as a conference paper at ICLR 2026
Thomas Wolf. Smollm2: When smol goes big data-centric training of small language model. CoRR, abs/2502.02737, 2025. doi: 10.8850/ARXXV22500007777 URL org/ 10. 008550/arXiv... 2502. 02737
Anthropic. Claude [large language model], 2025. URL.atps://ww.anthoppi..
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele, Allen Hao Huang, Angelika Ro- manou, Antoni-Joan Solergibert, Barna Pasztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Durech, Ido Hakimi, Juan García Giraldo, Mete Ismayilzada, Negar Foroutan, Skander Moalla, Tiancheng Chen, Vinko Sabolčec, Yixuan Xu, Michael Aerni, Badr AlKhamissi, Inés Altemir Mariñas, Mohammad Hossein Amani, Matin Ansaripour, Ili Badanin, Harold Benoit, Emanuela Boros, Nicholas Browning, Fabian Bösch, Maximilian Böther, Niklas Canova, Camille Challier, Clement Charmillot, Jonathan Coles, Jan Deriu, Arnout Devos, Lukas Drescher, Daniil Dzen- haliou, Maud Ehrmann, Dongyang Fan, Simin Fan, Silin Gao, Miguel Gila, María Grandury, Diba Hashemi, Alexander Hoyle, Jiaming Jiang. Mark Klein, Andrei Kucharavy, Anastasiia Kucherenko, Frederike Lübeck, Roman Machacek, Theofilos Manitaras, Andreas Marfurt, Kyle Matoba, Simon Matrenok, Henrique Mendonça, Fawzi Roberto Mohamed, Syrielle Montariol, Luca Mouchel, Sven Najem-Meyer, Jingwei Ni, Gennaro Oliva, Matteo Pagliardini, Elia Palme, Andrei Panferov, Léo Paoletti, Marco Passerini, Ivan Pavlov, Auguste Poiroux, Kaustubh Ponkshe, Nathan Ranchin, Javi Rando, Mathieu Sauser, Jakhongir Saydaliev, Muhammad Ali Sayfiddinov, Marian Schneider, Stefano Schuppli, Marco Scialanga, Andrei Semenov, Kumar Shridhar, Raghav Singhal, Anna Sotnikova, Alexander Sternfeld, Ayush Kumar Tarun, Paul Teiletche, Jannis Vam- vas, Xiaozhe Yao, Hao Zhao, Alexander Ilie, Ana Klimovic, Andreas Krause, Caglar Gulcehre, David Rosenthal, Elliott Ash, Florian Tramèr, Joost VandeVondele, Livio Veraldi, Martin Raj- man, Thomas Schulthess, Torsten Hoefler, Antoine Bosselut, Martin Jaggi, and Imanol Schlag. Apertus: Democratizing open and compliant Ilms for global language environments, 2025, URL https://arxiv.org/ www.rgg/aas/////// 14233
Ron Banner, Yury Nahshan, and Daniel Soudry. Post training 4-bit quantization of convo- lutional networks for rapid-deployment. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion Processing Systems 2019. NeurIPS 2019, December 8-14, 2019, Vancouver, BC, neurips cc/paper/2019/ Abstract. html
Canada, pp. 7948 7956. 2019. URL haas:/ca5221388444ccc33bbbbbbb33
Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. CoRR, abs/1308.3432 2013, URL 3432
Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge. Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, Alex X. Liu, Bo Liu, Wen Liu, Xiaodong Liu, Xin Liu, Yiyuan Liu, Haoyu Lu, Shanghao Lu, Fuli Luo, Shirong Ma, Xiaotao Nie, Tian Pei, Yishi Piao, Junjie Qiu, Hui Qu, Tongzheng Ren, Zehui Ren, Chong Ruan, Zhangli Sha, Zhihong Shao, Junxiao Song, Xuecheng Su, Jingxiang Sun, Yaofeng Sun, Minghui Tang, Bingxuan Wang. Peiyi Wang, Shiyu Wang, Yaohui Wang, Yongji Wang. Tong Wu, Y. Wu, Xin Xie, Zhenda Xie, Ziwei Xie, Yiliang Xiong, Hanwei Xu, R. X. Xu, Yanhong Xu, Dejian Yang, Yuxiang You, Shuiping Yu, Xingkai Yu, B. Zhang, Haowei Zhang, Lecong Zhang, Liyue Zhang, Mingchuan Zhang, Minghua Zhang, Wentao Zhang, Yichao Zhang, Chenggang Zhao, Yao Zhao, Shangyan Zhou, Shunfeng Zhou, Qihao Zhu, and Yuheng Zou. Deepseek LLM: scaling open-source language models with longtermism. CoRR, abs/2401.00254, 2024. doi: 10.48550/ARXIV.240 202444444.. URL httht: org/ 10. 48550/ arxiv.iiivvvvv. 2401. 02954
Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang. Peng Gao, Kaipeng Zhang, and Ping Luo. Efficientqat: Efficient quantization-aware training for large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACL 2025, Vienna, Austria, July 27- August 1, 2025, pp. 10081-101000 Association for Computational Linguistics, 2025a. wtt.bbttt:///caathhttttt
acl long.

---

Published as a conference paper at ICLR 2026
Mengzhao Chen, Chaoyi Zhang, Jing Liu, Yutao Zeng, Zeyue Xue, Zhiheng Liu, Yunshui Li, Jin Ma. Jie Huang, Xun Zhou, and Ping Luo. Scaling law for quantization -wwee training. CoRR, abs/2505. 14302, 2025b. doi: 1044550/ARXIV.255544553 URL https://doi.org/ 10. 48550/arXivv 2505. 14302)
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Sanmi Koyejo, S. Mo- hamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neu- ral Information Processing Systems 35: Annual Conference on Neural Information Pro- cessing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 Decem- ftllss/paper/22///haaae ber 9, 2022, 2022. URL htwp://ooo--ssssttttt--ooeeeeeee .html
Michael Davies, Neal Clayton Crago, Karthikeyan Sankaralingam, and Christos Kozyrakis. Ef- ficient LLM inference: Bandwidth, compute, synchronization, and capacity are all you need. CoRR, abs/2507.14397, 2025. doi: 10.855W/ARXIV2500.4497 URL https://doi..rr 10. 48550//rXiv..5555 14397
DeepSeek-AI, Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongje Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen. Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li. Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge. Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, and Li. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. CoRR, abs/2501 12948. 2025. doi: 00-4550/ARXIV.2500..229
WtLLhttps:://oot orgg /010 48550/5500/0 /ar.vv/rrivvv./aa 2501. 12948
Aleksandr Dremov. Alexander Hägele, Atli Kosson, and Martin Jaggi. Training dynamics of the cooldown stage warmup-stable-deeat learning rate scheduler. Trans. Mach. Learn. Res., 2025, 2025. URL httphhtts://ppeeeeeee
net
Steven K. Esser, Jeffrey I.. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dhar- mendra S. Modha. Learned step size quantization. In International Conference on Learning Representations, 2020. URL htttp:://pperrevie... www.eewwweeeeeeeeeeeeeeeeeee www..n/eooooiidrroooeeee6.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: accurate post-training quantization for generative pre-trained transformers. CoRR, abs/2210.17323, 2022. doi: 10. 48550/ARXIV.2210. 17323. URL httphhtt::://ddl /rtp:::///oio.oo/110/0
org/10 48.5://1..85500000 http//aaxvvv....... 17323
Alexander Hägele, Elie Bakouch, Atli Kosson, Loubna Ben Allal, Leandro von Werra, and Mar- tin Jaggi. Scaling laws and compute-optimal training beyond fixed training durations. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Anmual Con- ference Neural Information Processing Systems 2024, NewrlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL htL nips. cc/psps.ccpppppp.pppp files paper/222// html
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Ruther- ford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbi, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute- optimal large language models. CoRR, abtp/203..15556 2022. doi: 10.48550/ARXIV.200.... htRl.https::/// //dtps:///di...o org.iiorrg/100 41w.11/.885553588 wtt.iv...22....
12

---

Published as a conference paper at ICLR 2026
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang. Weilin Zhao, Xinrong Zhang, Zhen Leng Thai, Kai Zhang, Chongyi Wang, Yuan Yao. Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm: Unveiling the potential of small language models with scalable training strategies. CoRR, abs/2404.06395, 2024. doi: 10.48550/ARXIV.
2404.06395. URL https://doi. org/10 48550/ 48550/arXiv wrwiv. 2404 06395
Peter J. Huber. Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35 (1):73-101, 1964. doi: 10.1214/ams/111770033
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew G. Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integerarrihmmeticonnf, inference. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 2704- 2713. Computer Vision Foundation IEEE Computer Society, 2018. doi: 10.1109/CVPR. 2018.00286. URL http://openaccesss thecvf. com/ content_ _018 m0018/hmm//acob. Quantization and WewdddTrriiinnnninn CVPR_ 2018 Lew1__aaee.e..._111e html html
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.08361. 2020. URL httts://arxiv.org/ab/////
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: //arxiv. .rgg abaaaass/111 111111112.6611 6980
Tanishq Kumar, Zachary Ankner, Benjamin Frederick Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan. Scaling laws for precision. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, 2pril24-28, 24- 2025. OpenReview.net, 2025. URLlattps://ppeeevvie.. wwwpereeieien.eeeefoooeee mer/forum? WdCWgIPC
Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Yitzhak Gadre, Hritik Bansal, Etash Kumar Guha, Sedrick Scott Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee F. Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng Yu Hsich, Dhruba Ghosh, Josh Gardner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah M. Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Raghavi Chandu, Thao Nguyen, Igor Vasiljevic, Sham M. Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo. Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alex Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, and Vaishaal Shankar. Datacomp-Ii Im: In search of the next generation of training sets for language models. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference Neural Information Processing Systems 2024, NeurIPS 2024, Van- couver, BC, Canada, December 10 15, 2024, 2024, http: httptt::pppppp...tttte cc/paper 1wtp:e5e412---bsracceee Datasets.
files tess/paper/2222//eee htte/////hhhhh/eeeeaaeeee and Benchmarks Track. html
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangx- uan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: activation-aware weight quan- tization for on-device LLM compression and acceleration. In Phillip B. Gibbons, Gennady Pekhimenko, and Christopher De Sa (eds.), Proceedings of the Seventh Annual Conference on Machine Learning and Systems, ML.Sss 2024, Santa Clara, CA, USA, May 13-16, 2024. ml- sys.org, 2024. URL https 22ae52/caaa0ddeebbbaaaaac22222-2 httpp//eeceecctaaaaaaatttttteeee
wwt.sys.org/pppppe files/ille/ebll hess//aper//22//aaaee .htll
Zechun Liu, Barlas Oguz, Aasish Pappu, Lin Xiao, Scott Yih, Meng Li, Raghuraman Kr- ishnamoorthi, and Yashar Mehdad. Bit: Robustly binarized multi-distilled transformer. In
13

---

Published as a conference paper at ICLR 2026
Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Ad- vances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- tion Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 De- cember 9, 2022, 2022. URL http: http://eabbbbbssaaaaa-o-ooeerrre
nip...iipsscc... hhmml
Zechun Liu, Changsheng Zhao, Hanxian Huang, Sijia Chen, Jing Zhang, Jiawei Zhao, Scott Roy, Lisa Jin, Yunyang Xiong, Yangyang Shi, Lin Xiao, Yuandong Tian, Bilge Soran, Raghuraman Krishnamoorthi, Tijmen Blankevoort, and Vikas Chandra. Paretoq: Scaling laws in extremely doi: 10.48500ARXIV.550200222 URL https:///obl org/10..1........... 2.....8550//raaa aatp...iii.... 2562. 02631
low-bit LLM quantization. CoRR, abs/2502.026311, 2025.
Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In Sth International Conference Learning Reprexentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
net/
forum?id-Sk989Scx
Xinyin Ma, Gongfan Fang, and Xinchao Wang- LIm-pruner: On the structural pruning of large language models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (cds.), Advances in Neural Information Processing Systems 36: Annual Con- ference on Neural Information Processing Systems 2023, NewrIPS 2023. New Orleans, LA, USA, December 10 16, 2023, 2023. URLhhttp://papers wwwperrs..iiiiiiiieeeeee...ee..e mips. wwtpcccppppppiieeeee /s1le//ppper/223// 0B55/44466551349055444444233333
Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginshurg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. Mixed precision training. In International Conference on Learning Representations, 2018. URLhttps: //openreview. wttpiarrvviieeettfforfoeeeee sete/foouumiid=r1g9ggggz
Mistral AI. Mistral small 3.1 [large language model], March 2025. URL httpht:/mmssttraall..
news/mistral 1ma11---mm11133111111
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo. Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsherg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannanch Hajishirzi. olmo doi: 10.4550/ARXIV225010000 //doi.orr//// 48550 www.ii.v2s..... 250100066660010
furious. CoRR, abs/2501.00656, 2025.
URL https:
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 27730-27744 Curran Associates, Inc., 2022. URL http:://///ppoeeeeennnneere.ppap www.:nnweiiii.o... cc/paperrccppppp..pcpemep laappeerrfllleesee wwtp:/flleeseepeeeeeeee fflle//paper//222/feee
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8035, Curran Associates, Inc., 2019. URL 9015 1115-pytorch-a an wawtoorc--an-mmerattiie imperative wwweiiivv-sttllee strereeeeeeeee----i-eooooorreeee www.pppwwffooeeeeeeeeee -eep learning- -aaaaiin--ibbrrry.. library. pdf
neuri heurips.cc/paperr cC/paper/
Houwen Peng, Kan Wu, Yixuan Wei, Guoshuai Zhao, Yuxiang Yang. Ze Liu, Yifan Xiong, Ziyue Yang, Bolin Ni, Jingcheng Hu, Ruihang Li, Miaosen Zhang, Chen Li, Jia Ning, Ruizhe Wang.
14

---

Published as a conference paper at ICLR 2026
Zheng Zhang, Shuguang Liu, Joe Chau, Han Hu, and Peng Cheng. FP8-LM: training FP8 large language models. CoRR, 1000//2310.18133,1 2023, doi: 10.4850/ARXX.:310.1881. URL
https://doi.org: org/ 10. 118550/arX10 2310. 18313
Hadi Pouransari, Chun-Liang Li, Jen-Hao Rick Chang, Pavan Kumar Anasosalu Vasu, Cem Koc, Vaishaal Shankar, and Oncel Tuzel. Dataset decomposition: Faster LLM training with variable sequence length curriculum. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Ad- vances Neural Information Processing Systems 38: Annual Conference on Neural In- formation Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. URL http::/papers..et../////// wwapper..niis.coom wwtpppppppppppppaappapppppppeppp saaeeer_illlepppppee httpe//eee/eap/////////aa hhtml
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. Al- ice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neu- ral Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA. USA, December 10 16. 2023, 2023. URL /attp://papeet wttppprrr.niss.cc///cpee httpc///ppppppplee/eeeeeeeeeeeee /ttps//pper/222/////// http://e6e77444eeeee0eeeeeeeeee-
.ttll
Pol G. Recasens, Ferran Agullo, Yue Zhu, Chen Wang, Eun Kyung Lee, Olivier Tardieu, Jordi Torres, and Josep Lluís Berral. Mind the memory gap: Unveiling GPU bottlenecks large- batch LLM inference. CoRR, foss/503.003311 2025. doi: 10.45500ARX122533303111 URL
https://doi. ortps: 48550/ http://XXi...... 08311
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL URLhttts::/ https://arxiv http://aaaii...r/////////00/. 06347
Noam Shazeer. GLU variants improve transformer. CoRR, abs/2002.05202, 2020. URL https:
//arxiv. orgg abs/ 2000/0002220222222
Daria Soboleva, Faisal Al-Khateeb, Robert Myers,
Jacob R Steeves,
Joel
Hestness, and Nolan Dey. Slimpajama: deduplicated version of redpajama. slimpa j ama- a- 627b tow.n-cleane-eeeeeeo-o------- and dwd.plicatedd versoon of redaaa
627b token cleaned and
https: https::/www.cerebras. www.:eeerreerebbb.ooo.o bet/blog/
June 2023. URL hwwp:http::/hbie /tt.:t/ts////gggggg.g...///rrara httpeeeeeeeeetatttteeeeeeraeaaae
627B
Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 668:127063, 2024. doi: 10. 101.1/..EEUCOM.2222.. 127063. URL
1016/ neucomn.e.mmmmmmmmm 2023.
127063
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jennmmy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux. Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.09228 2023. doi: 10.48550/RRIV.2307.. 10. 48550/aaXiv..... 2307 092888
URL https:///oi.nn.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus. S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates,
15

---

Published as a conference paper at ICLR 2026
Inc., 2017. URL http:://proceedinss.errrrip http://.w...........ppp////////p 3 Paper. Paper.pdf]
Ruizhe Wang. Yeyun Gong, Xiao Liu, Guoshuai Zhao, Ziyue Yang, Baining Guo, Zhengjun Zha, and Peng Cheng. Optimizing large language model training using FP4 quantization. CoRR, abs/2501. 17116, 2025. doi: 10.48500ARXXV.2550.771
URL https::/ddo..rgg///
38550/arXiv. 2501. 17116
Kaiyue Wen, Zhiyuan Li, Jason Wang, David Hall, Percy Liang, and Tengyu Ma. Under- standing sarmup-stable-decay learning rates: A river valley loss landscape perspective. CoRR, abs/2410.05192, 2024. doi: 1044850/ARXV:241000001222 URL https://dbi.org///
48550 arxiv 2410. .5192.
Guangxuan Xiao, Ji Lin, Mickaël Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learming Revearch, pp. 38087-38099, PMLR, 2023. URL https://prooeeeding mlr. Airess/v2022/iia00/
and Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang. Xin Gao. Qi Cai, and Ziyuan Ling. On-device language models: A comprehensive review, 2024. URL httpps://arxvv.orrg//.a 240//2499... 00088
Biao Zhang and Rico Sennrich. Root mean square layer normalization. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, wtt..eiiss....ppee/ee/e...... cc/paper/2019/
Canada, pp. 12360- 2019. URL https://proceecings.nt hess:/e88bD2222744884333cc1212.2
Hanzhi Zhou, Erik Hornberger, Pengsheng Guo, Xiyou Zhou, Saiwen Wang, Xin Wang, Yifei He, Xuankai Chang, Rene Rauch, Louis D D'hauwe, John Peebles, Alec Doane, Kohen Chia, Jenna Thibodeau, Zi-Yi Dou, Yuanyang Zhang, Ruoming Pang, Reed Li, Zhifeng Chen, Jeremy Warner, Zhaoyang Xu, Sophy Lee, David Mizrahi, Ramsey Tantawi, Chris Chaney, Kelsey Peterson, Jun Qin, Alex Dombrowski, Mira Chiang, Aiswarya Raghavan, Gerard Casamayor, Qibin Chen, Aonan Zhang, Nathalie Tran, Jianyu Wang, Hang Su, Thomas Voice, Alessandro Pappalardo, Brycen Wershing. Prasanth Yadla, Rui Li, Priyal Chhatrapati, Ismael Fernandez, Yusuf Goren, Xin Zheng, Forrest Huang, Tao Lei, Eray Yildiz, Alper Kokmen, Gokula Santhanam, Areeba Kamal, Kaan Elgin, Dian Ang Yap, Jeremy Liu, Peter Gray, Howard Xing, Kieran Liu, Matteo Ronchi, Moritz Schwarzer-Beckk
Yun Zhu, Mandana Saebi, Jeremy Snow, David Griffiths, Guillaume Tartavel, Erin Feldman, Simon Lehnerer, Fernando Bermúdez-Medina, Hans Han, Joe Zhou, Xiaoyi Ren, Sujeeth Reddy, Zirui Wang, Tom Gunter, Albert Antony, Yuanzhi Li, John Dennison, Tony Sun, Yena Han, Yi Qin, Sam Davarnia, Jeffrey P. Bigham, Wayne Shan, Hannah Gillis Coleman, Guillaume Klein, Peng Liu, Muyang Yu, Jack Cackler, Yuan Gao, Crystal Xiao, Binazir Karimzadeh, Zhengdong Zhang, Felix Bai, Albin Madappally Jose, Feng Nan, Nazir Kamaldin, Dong Yin, Hans Hao, Yanchao Sun, Yi Hua, and Charles Maalouf. Apple intelligence foundation language models: Tech report 2025. CoRR, abs/2500.113575 2025a. doi: 00.48500/RRXI..557..35
URL https://d// org/ 10. 085.44555///20 awr.vi..vvv.. 2507 13575
Jiecheng Zhou, Ding Tang, Rong Fu, Boni Hu, Haoran Xu, Yi Wang, Zhilin Pei, Zhongling Su, Liang Liu, Xingcheng Zhang, and Weiming Zhang. Towards efficient pre-training: Exploring FP4 precision in large language models. CoRR, abs/2502. 1458, 2025b, doi: 00.48550/ARXIV.2555 11458. URL https:://oi...... org/ 100 48550/ arxiv. 2502. 11458
A TRAINING SETUP
We use a decoder-only transformer /Vawaani et al. 2017) identical to Llama Touvron et al. 2023
The architecture incorporates SwiGLU activations (Shazeer 2020). RoPE (Su al. 2024). RM- SNorm (Zhang & Sennrich| 2019). alternating attention and feed forward layers, and tied embedding and londuag head weights. We use the Adam optimizer (B) 0.9, f2 0.99, 10 30-8)
16

---

Published as a conference paper at ICLR 2026
with decoupled weight decay of 0.01 (Loshchilov & Hutter 2017 Kingma & Ba 2015) for all pa- rameters outside the embedding and normalization layers. All experiments are trained with 66 automatic mixed precision (Micikevicius et al., 2018). Training is conducted on the DCLM dataset eL. 2024). tokenized with the Llama tokenizer with a 42,000-token vocabulary. We merge all tokenized documents into single sequence with appropriate delimiting tokens and take chunks of 1024 tokens (used sequence length) for the batch-an an approach also known as "concat-and -nda--unnnccuk" (Pouransari et al., 2024). The dataset is split into training and validation sets, and validation perplex- ity is used for evaluation. For QAT algorithms, we rely on ParetoQ (Liu et al. (025) for our setups, as this method achieves state-of-the-ee aceuracy across different bit widths by combining different approaches. The following subsections provide in-depth descriptions of each part of the training pipeline.
A. FULL-PREECSION TRAINING
The choice of learning-rate scheduler is an important aspect of our work. While cosine learning-rate scheduling is widely used, achieving optimal model loss for a specific token count requires matching the training duration to the cosine scheduler length (Hoffmann et al. (022). To obtain comparable experiments, would need to train models from scratch for each specific final token count, which is computationally wasteful. Therefore, we train full full-precision models with the warmup-stable- decay (WSD) learning-rate scheduler (Hägele al. 2024 Hu et al. 2024). The main advantage of WSD is the ability to obtain models for any desired total token count without needing to train from scratch; this can be achieved by resuming training from the constant-stage checkpoint and performing a learning-rate cooldown to achieve the needed token count. Hägele et al. (2024) showed that WSD accuracy closely follows that of cosine, making it an optimal choice for our setup, where many checkpoints for different token counts are needed. In our experiments, we follow the optimal setup from Hägele et al. 20rm))Dremov Dremov et al. (2025): we perform 1,000 steps of warmup and a 20% cooldown stage with
sqri learning-rate cooldown shape.
For different model parameter counts, we vary the number of layers and hidden dimensions, using Hoffmann et al.) (2022) as a reference. Our configurations and parameter counts are reported in appendix B For learning rate and batch size selection, we follow the scaling law proposed by Bi et al. (024). We choose the optimal batch size and learning rate corresponding to the average token count of the conducted experiments for each model size. Since the achieved loss is stable for wide ranges around the optimal batch size and learning rate (Bi et al. 2024 Zhou al. (025a), we remain close to the optimal learning hyperparameters for all our experiments. We report our settings for each model size in appendix C
A.2 QUANTIZATION
We rely on ParetoQ (Liu et al. 2025) for quantization setups, as this approach achieves state-of- the-art accuracy across bit widths. Specifically, we use different algorithms for different bit widths:
Elastic Binarization (Liu et al. (0022) for 1-bit quantization; LSQ (Esser et al. 2020) for 3-bit and higher quantization; and SEQ for 2-bit quantization (Liu et al. 2025). Additionally, this approach makes our results generalizable different QAT algorithms. Each setup employs per-output-fean quantization scales. While it is common not to quantize embeddings and the language modeling head (LM head), the ParetoQ approach shows negligible accuracy drop when quantizing embeddings and the LM head to 4 bits. Since embeddings constitute a substantial portion of parameters for small models, we quantize embeddings as well, but not to fewer than bits in all setups. That is, we quantize embeddings to 6 bits for 6-bit QAT, but to 4 bits for 4-, 2-, and 1-bit QAT experiments.
A.3 QAT
For QAT, we follow the same setup as for full-precision training, except for the learning rate schedule. At the start of QAT, we restore data readers from the full-precision checkpoint, which makes QAT and FP training data mutually exclusive for each experiment. Since we do not need QAT checkpoints at different token counts, we use cosine learning rate decay with 5% warmup and decay the learning rate to zero. The quantized model is initialized from appropriate post-cooldown full-precisio model, with quantization scale initialization as described by Esser et al. (2020); Liu et al. (2025) 2022) (appendix A.4). We disable weight decay for quantization scales.
17

---

Published as a conference paper at ICLR 2026
A.4 QAT ALGORITHMS
As described in appendix A.2 we use different quantization algorithms for different B. In this section, we summarize them for the reader's convenience.
Typically, QAT algorithms employ a version of the uniform quantization function:
Wi R
W
1.
w aW + B.
where WR is the original floating-poinn point-valudd weight, WR is the quantized integer-valued weight, We the quantization group. In our work, point-valued weight, and a, are parameters specific the i-th per-output-feature quantization groups. During training, We used to conduct calculations, and during inference, the model is stored as integer-valued weights WR. Below, we present details about the different algorithms we used.
Elastic Binarization (1-bit). Liu et al. 2022 2025 propose such a quantization scheme for WR taking values from 13:
WR Sign
W e aWR
where initially a [ li, and such straight-through Bengio et al. 2013) estimator gradient mw
estimations are used:
awi Q
aw
aw
Sign(Wi))
da
Stretched Elastic Quantization (2-bit). scheme for 2-bit WR:
Liu et al. 2025 propose the following quantization
Wi
1 1
WR = [Clip(
1, 1) x
wi Q (R i + 7
= a
where initially a max (wax) and such gradient estimations are used:
awi Q
ow
wf |<1
R
awi Q
Wi
δa
R
a
*
Learned Step Size Quantization (3-bit and Higher). Esser et al. (2020) propose the following quantization scheme for Wo. which is standard quantization scheme with B = 0:
Wi WR |Clip(
-2.11.2.22 1)7.
a
W Q
a WRR
where initially = max(( D. and such gradient estimations are used:
awl
e
ow -
w/
221-1-1
N
owi
Wi
Q
WR
da
a
22
wl 28-1-
<28
1
18

---

Published as a conference paper at ICLR 2026
B MODEL CONFIGURATIONS
Table summarizes the different transformer model configurations used. As noted, we use the number layers and hidden dimensions from the configurations table of Hoffmann et al. (2022).
Table Transformer hyperparameters used across experiments. Parameter counts are also reported.
model
ffinsize
kVsize
heads
llayers
N (M)
Noo emb (M)
640
2,560
64
10
10
86
65
768
3,072
64
12
18
194
169
1,280
5,120
128
10
18
396
355
1,536
6,144
128
12
25
759
709
2,176
8,704
128
17
28
2,191
2,121
C TRAINING HYPERPARAMETERS
As noted in appendix A. for learning rate and batch size selection, we follow the scaling law the chosen hyperparameters for each model size.
proposed by Bi et al. 2024
Table Main hyperparameters used during training. Learning rate and batch size selection follow
those of Bi al. 2024).
Model size (M)
Learning rate
Global batch size (tokens)
86
9.54e-04
1,097,728
194
8.93e-04
1,302,528
396
7.33e-04
1,572,864
759
7.29e-04
2,129,920
2,191
6.72c-04
2,490,368
19

---

Published as a conference paper at ICLR 2026
D FITTED Loss SCALING LAW FORMULA
In figure 8 we present the loss scaling law fitted to all our experiments. For simplicity, we substitute:
Squi
Sip
N Additionally, we plot experimental data and loss scaling law heatmaps in
Dip
N: &
figure 10 and optimal QAT fraction predictions in the same format figure (Left) in figure 9
2477.0
57.64
1091.0 2-1121
L(N, D Dun Dip. B) 1.598
+
0.0297
2 2-1.41-8
D0.4089
NO.2148'
NO.4004
50.076 qui
total
138.8
2-0.0833-B
NO.2135 lp
50.4819,
S0.1903 qui
Figure 8: Fitted loss scaling law formula. This is unified scaling law that predicts QAT loss for various N. D Dip, and B.
Optimal QAT Length Fit (B 1)
Optimal QAT Length
1.0
QAT fraction
0.8
70%
386k
0.6
3198
0.4
0.2
0.0
10S
10
10a Total tokene-per-byte
10
Total tokens-per-byte Optimal QAT Length Fit (B
Optimal QAT Length Fit
1.0
QAT fraction
0.8
0.6
28%
59%
0.4
2.2
0.2
0.0
10a
103
10
10a
10a Total tokens-per-bye
10
Total tokens-per-hye
Best QAT fraction
Best fit
Figure 9: Optimal QAT fraction predictions inferred from the loss scaling law (section that figure Juses formula from section[..] which less precise but allows a simple line prediction across all bit-widths and model sizes it depends only on Stotal-
3.2 Note
20

---

Published as a conference paper at ICLR 2026
Experiments for 1-bit QAT
Model 86M
Model 194M
10
10
FPT tokens
1010
10
109
10
108
10
10
109
10
10
10
109
10
10
Model 396M
Model 759M
10
10
FPI tokens
1010
10
109
109
109
10
10
109
10
10
QAT tokens
QAT tokens
(a) The loss fit metrics are: R² = 0.982, MAE 0.026, MAPE 0.895%. Inferred from loss QAT optimum
fraction prediction metries: MAE 0.081.
Experiments for 2-bit QAT
Model 86M
Model 194M
FPT tokens
1011
10
109
109
108
10
10
10
10
108
109
10
10
10
Model 396M
Model 759M
1012
10
FPT tokens
1011
10
10
109
109
109
1010
10
1012
100
10
10
QAT tokens
QAT tokens
(b) The loss fit metrics are:
R² 0.981, MAE 0.023, MAPE 0.817%. Inferred from loss QAT optimum
fraction prediction metries: MAE 0.102.
21

---

Published as a conference paper at ICLR 2026
Experiments for 4-bit QAT
Model 86M
Model 194M
FPT tokens
10
10"1
109
10
10a
109
10
10"
1012
108
10
10
10t
10
Model 396M
Model 759M
1012
1012
FPT tokens
10
10
1010
1010
10
109
10
10
10
10
10
10
10
10
QAT tokens
QAT tokens
(c) The loss metries are: R 0.983, MAE 0.021, MAPE 0.796%. Inferred from loss QAT optimum
fraction prediction metrics: MAE 0.074.
Model Experiments for 6-bit QAT
Model 194M
FPT tokens
10
10
10
109
108
109
10
10"
10
108
109
10
10
10
Model 396M
Model 759M
1012
1012
FPT tokens
10
10
10
10
109
109
10
10
10
10
109
10
10"I
10
QAT tokens
QAT tokens
(d) The loss metrics are: R² 0.991, fraction prediction metries: MAE 0.09.
MAE 0.018, MAPE 0.661%. Inferred from loss QAT optimum
Figure 10: Visualizations of the fitted loss scaling laws for different QAT bit-widths. Experimental data are plotted with point sizes corresponding to loss relative to the group of experiments with the same Duaal Orange stars correspond theoretical optima; purple stars represent experimental optima.
22

---

Published as a conference paper at ICLR 2026
E SCALING LAW PERFORMANCE FOR Low TOKEN COUNTS
One may notice that the scaling law optimal fraction prediction error is high for low token counts (ap- pendix D). Specifically, the optimal QAT fraction appears to be lower than the predicted one. In this section, we attempt to provide an explanation for this behavior. As low-token setups are not practically important, we do not include this discussion in the main text.
Intuitively, with low Sip the model is severely under-trained, and noise introduced by quantization does not significantly alter learned features. In the extreme case of Sip 0, simple QAT initialization already able to almost completely restore performance. We were able to capture such a drop in optimal fraction for low Sip using a more sophisticated form of the scaling law, but, as we noted previously, this has low practical value. Therefore, we prioritized a simpler scaling law form.
F SCALING Law FIT NOTES
In this section, we summarize methods implemented to achieve better loss scaling law fits. As noted in the main text, use Huber loss (Huber 1964) and gradient descent optimization. The Huber loss choice is consistent with the setup of Hoffmann et al. (2022);[Chen [Chnn et al. (2025b). Additionally, verified that simple MSE achieves worse generalization. We attribute this phenomenon to the presence of outliers in our experiments-this can be seen from the appendix (e figuures Specifically, one can notice both outliers for optimal experimental QAT fraction and disproportionate dot sizes. Also, to facilitate generalization over different bit bit-widths, we re-weight each sample loss contribution proportionally to the corresponding B inverse frequency.
Another important trick is the addition of full-precision loss regularization. This is done based on the expectation that for high B. the final loss should be indistinguishable from the full cell-precision model loss. Therefore, we add 374 full-precision model evaluation results to the fit, assigning B 16 to them, which brings the total fit data size to 1 1131 experiments. For Dip. yut assignment, we notice that only the FP/QAT interaction term of 5(N, D Daut, Dip. B) (equation 2 makes noticeable contribution with high B. Therefore, we assign such D D Daut Dip + Datt = Diotal that minimize the FP/QAT interaction term only. This way, the obtained QAT loss scaling law fit not only predicts QAT loss, but also predicts full-precision loss by using = 16. The fit achieves R² = 0.989, MAPE 0.8% MAE- 0.022 fit metrics for all obtained full-precision checkpoints.
G FITTED Loss SCALING LAw FORMULAS (SPECIFIC BIT-WIDTH)
11c and 11d we present loss scaling laws fitted to our experiments for each specific bit- width separately and the corresponding fit accuracies. For simplicity, we substitute: g. Additionally, table 5 showcases fit metrics of the unified scaling law (section 3.2 and wir-bht-width scaling laws (this section). The fit quality is overall comparable, with fits for each bit-width being slightly better. However, we prioritize the unified scaling law due its higher practical utility and as way to reduce fit variance.
In figures 1la
11b
Des N 2 Spp =
Di
Squi
E
2605.0
233.6
366.8
970.4
1.931
0.7155 total
N0.2921
NO.367
00.18
NO.2338
50.5702 1p qui
50.2388
(a) Fitted loss scaling 1 bits QAT bit-width. The loss metries are: R² 0.99,
MAE 0.02,
MAPE = 0.676%. Inferred from loss QAT optimum fraction prediction metrics: MAE = 0.06.
2321.0
368.2
33.01
115.9
1.885 + 0..428 tocal
N4.3434
NO.2426.
50.02699 qut
N761
$0.455 1p
0.2663 qat
(b) Fitted loss scaling law for 2 bits QAT bit-width. The loss fit metries are: R 0.989, MAE 0.019, MAPE 0.695%. Inferred from loss QAT optimum fraction prediction metries: MAE 0.061.
23

---

Published as a conference paper at ICLR 2026
2388.0
401.3
983.4
54.46
1.923 +
D0.3917
NO.3389
NO.6453
$0.1001 qui
NO.1323.
$0.7778 S0.2755 qui
total
(c) Fitted loss scaling law for bits QAT bit-width. The loss fit metrics are: R = 0.982, MAE = 0.02,
MAPE 0.735%. Inferred from loss QAT optimum fraction prediction metrics: MAE = 0.075.
1546.0
301.4
148.5
28.33
1.829
0.3826
NO.444
NO.2853
50.0004
NO.1381
50.5881
50.1595 qat
total
(d) Fitted loss scaling law for bits QAT bit-width. The loss fit metrics R 0.992, MAPE 0.604%. Inferred from QAT optimum fraction prediction metrics: MAE 0.049.
MAE 0.017,
Figure 11: Fitted loss scaling law formulas, fitted for each QAT bit-width separately.
Table 5: Comparison between unified QAT loss scaling law (section 3.2 and separate loss scaling laws for each bit-width. The fit quality is overall similar, with separate scaling laws achieving slightly better fits.
MAE, loss fit Separate
R². loss fit Separate
MAE, optimal QAT fraction fit Unified
B
Unified
Unified
Separate
0.026
0.02
0.982
0.99
0.081
0.06
0.023
0.019
0,981
0,989
0.102
0.061
4
0.021
0.02
0.983
0.982
0.074
0.075
0.018
0.017
0.991
0.992
0.09
0.049
H QAT AND FP Loss SCALING LAWS INTERPLAY
As discussed in appendix F. we fit the QAT scaling law such that 16 substitution approximates full precisioo model loss, so use this setup estimate full-precision model accuracy in the section 4.2 analysis.
Points of interest in figure are where lines cross y 0. Such point represents the maximum Drotal for which the corresponding QAT can reproduce FP loss. In tables and ? we show such values for models from figure 5] We consider 0.5% QAT/FP perplexity difference to be minor and calculate zero-crossing accounting for this margin. As expected, larger models can maintain FP quality for lower bit-widths and higher total token counts.
24

---

Published as a conference paper at ICLR 2026
Table 6: Token count for figure (Left) lines' zero-crossing. This represents the maximum to- tal token count for the 500.0M model when QAT of the corresponding bit-width can restore FP model quality. "N/A" means that for any token count, the bit-width cannot achieve accuracy sim- ilar to the full pree-icen model.
Table 7: Token count for figure 5 (Right) lines' zero-crossing. This represents the maximum to- token count for 16.0B model when QAT of the corresponding bit-width can restore FP model quality.
B
Max FP restore tokens count
Max FP restore tokens count
N/A
80.3B
2
N/A
2
212.1B
3
N/A
633.2B
4
83.6B
2.8T
5
1.IT
100 T 5
>
100 T
6
100 T
I OPTIMAL QAT BIT-WIDTH VERIFICATION
Section 4.3 analyzes which is optimal within specific memory and training compute budgets. We verify the presented plot in figure [2] To do so, we linearly interpolate information from conducted experiments. While such interpolation yields some artifacts, the general structure is consistent with the predicted one. Additionally, we plot loss levels of the optimal QAT selection in figure [3] Results reveal that loss levels closely follow the predicted ones.
Predicted Optimal QAT Bit-Width
Experimental Optimal QAT Bit Bit-Width
8
Parameters memory
Parameters memory
1GB
1GB
5
5
100MB
- B
100MB
4 B
3
33
10MB
-
10MB
10
1020
10
10
10
30
10
10
Training FLOPs
Training FLOPs
Figure 12: Comparison of predicted optimal QAT bit-width and experimental optima. On the left, we reproduce figure but with a reduced set of bit-wisths corresponding to the set of bit-withhs used in the conducted experiments (1, 2, 4, 6). On the right, we show optimal QAT bit-widths obtained from real experimental data. We take experiments with optimal QAT fraction and interpolate the grid into them. The white area represents the range of values where do not have experimental data. is clearly seen that the general structure predicted optima corresponds to the real experimental one.
25

---

Published as a conference paper at ICLR 2026
Predicted Optimal QAT Bit-Width
Experimental Optimal QAT Bit-Width
Loss Levels
Loss Levels
Parameters memory
39.8
Parameters memory
39.8
26.9
26.9
1GB
22.5
1GB
22.5
Perplexin dexes
19.8
19.8
17.7
17.7
100MB
15.9
100MB
15.9
14.2
14.2
10MB
-2.8
10MB
12.8
-1.4
11,4
1019
10a0
10at
10
9.2
10
10a00
10th
10ah
9.2
Training FLOPs
Training FLOPs
Figure 13: Comparison of predicted optimal QAT bit-width loss levels and experimental ones. The presented figures show loss levels of corresponding optimal QAT configurations from figure T2 We use the same color mapping and normalization for both plots. On the left, we show loss levels of figure [2 (Left).
the right, we show optimal QAT configuration loss levels obtained from real experimental data. The white area represents the range of values where we do not have experimental data. is clearly seen that predicted loss levels closely follow the true optimal loss levels. Note that the experimental plot incorporates experiments of different bit-widths as displayed in figure 12 (Right).
J DATASET AND HYPERPARAMETER IMPACT
To ensure that the observed phenomenon is not dataset- or hyperparrameter-induced, we conduct small-scale 4-bit QAT experiments, pretraining the model the SlimPajama (Soboleva al. 2023) dataset with different pretraining batch sizes and learning rate selections (table 8). The results are presented in figure [4] we plot the DCLM- based best fraction prediction fit that was used in the main text. It is clearly seen that the same optimal fraction growth phenomenon is observed, and except for several outliers, the fit is quite accurate. Even with dataset and hyperparameter substitution and no additional fitting, the optimal fraction fit achieves 0. 11 MAE. This shows that the conclusions made in the main text are minimally influenced by the exact hyperparameters and dataset choice we made. However, expect the loss scaling law fit to differ reported by Bi et al. (2024). The optimal QAT fraction inferred from the loss scaling law error is 0. 121 MAE.
due the dependence data quality as
Table 8: Hyperparameters used during the SlimPajama-based experiment reproduction. We pur- posefully changed hyperparameters to test how robust the observed phenomenon is.
Pretrain
QAT Learning rate
Model size, M
Batch size
Learning rate
Batch size
86
417,792
2.0e-04
208,896
1.0e-04
194
483,328
2.0e-04
245,760
1.0e-04
396
573,440
2.0e-04
204,800
1.0e-04
759
655,360
2.0c-04
262,144
1.0e-04
26

---

Published as a conference paper at ICLR 2026
Optimal QAT Length Fit (SlimPajama)
10
QAT tokens- tokens-per-bye per-byteeeeeee
48%
10a
10a
12%
10!
100
10
10a
101
Total tokene-per-byte
Best QAT
DCLM- best
fraction
Figure 14: Optimal QAT fraction for SlimPajama based experiment reproduction. is clearly seen that the optimal fractions also increase with the total Aakens-per-parametee the main text (DCLM-based fit) also plotted for reference. Even without additional re- re-fitting, the optimal fraction fit achieves 0. MAE. This indicates that the observed phenomenon is not dataset- or txyerpaarameterrnnduce..
statistic. The fit from
K 2.2B MODEL OPTIMAL QAT FRACTION PREDICTION
In this section, we verify the scalability of the obtained results. To do so, we train a 2.2B model with QAT using several different QAT fractions, including the predicted optimal QAT fraction. We verify that the predicted optimal QAT fraction from the loss scaling law generalizes to the 2.2B model, which is 2.9 times larger than the largest model in the loss scaling fit data. The results are presented in table 9
Table 9: Experiments for the 2.2B parameter model. We select the middle fraction to be close the predicted optimal one and two additional fractions: one smaller than optimal and one larger. We present the corresponding perplexities and the difference between the minimum perplexity and the perplexity corresponding to the predicted optimal QAT fraction (L.).. It is seen that in most cases the predicted QAT fraction is optimal, and in some cases it deviates from the optimum insignificantly we expect this to be noise.
Tested Fractions
Perplexities
Linin 1 Timin
%
B D
49.3B
10.0%, 38.3%, 53.3%
13.502, 13.017, 13.092
0.00%
109.5B
10.0%, 40.9%, 55.9%
12.563, 12.187, 12.25
0.00%
22.2B
10.0%, 39.2%, 54.2%
13.95, 13.828, 13.734
0.68%
49.3B
10.0%, 40.3%, 55.3%
12.335, 12.068, 12.084
0.00%
22.2B
10.0%, 26.5%, 41.5%
13.017, 13.049, 13.198
0.24%
49.3B
10.0%, 26.7%, 41.7%
11.515, 11.515, 11.545
0.00%
20.6B
2.9%, 17.9%, 32.9%
13.149, 13.114, 13.21
0.00%
27

---

Published as a conference paper at ICLR 2026
L QAT OVERHEAD
In this section, we show results of our benchmarks that measure the slowdown between QAT and FP training. In benchmarks, we select the maximum batch size that fits within GPU memory constraints and perform multiple measurements to reduce the variance of our results. We do not observe significant slowdown for all model sizes we have tested. Figure [3 summarizes our findings. is important to note that ensuring that PyTorch (Paszke et al. 2019) compile optimization processed quantization operators correctly and without slow fallbacks was crucial to achieving almost zero overhead.
QAT 1-bit
QAT 2-bit
1.04
1.02
Slowdown, times
1.00
0.98
0.96
QAT 3-bit
QAT 4-bit
1.04
1.02
Slowdown,
times
1.00
0.98
0.96
1000
2000
3000
4000
5000
0
1000
2000
3000
4000
5000
Model parameters,
Model parameters,
Figure 15: Measured overhead of QAT versus FP training. It is clearly seen that the slowdown fraction fluctuates around 1.0 and no significant slowdown noticeable.
28

---

Published as a conference paper at ICLR 2026
M QAT & LEARNING RATE COOLDOWN FUSION: EXTENDED RESULTS
In this section, we show results of "QAT & Learning Rate Cooldown Fusion" for all bit widths (table TOp. As discussed in section || the proposed approach shows consistent improvements for 4- and 6-bit QAT. For 1- and 2-bit experiments, improvements in some settings are present but less prominent than for 4- and 6-bit QAT. We explain this by the large optimal QAT fraction for lower bits, which minimizes the impact of QAT & Cooldown Fusion.
Table 10: Accuracy comparison between the classic QAT scheme and "QAT & Learning Rate Cooldown Fusion" scheme. The loss difference is reported in "wasted tokens" the difference in total token count between optimal QAT fraction loss points in the loss scaling law. Substantial improvements
noticeable across different model sizes and token counts for 4-bit and higher. For 1- and 2-bit experiments, improvements in some settings are present but less prominent. We explain this by the large optimal QAT fraction for lower bits, which minimizes the impact of QAT & Cooldown Fusion.
Perplexity
Wasted tokens, Unfused total tokens, %
Unfused (baseline)
Fused (ours)
B
Model size, M
D rotal
74
70.4B
23.82
24.14+1.44%
-1955%
163
17.0B
20.95
21.0 21.0640.53%
-5.3%
425
11.1B
18.53
18.43.0.54%
4.4%
33.5B
16.33
16.41
-8.1% %
305.8B
14.73
14.83+0.68%
-26.8%
816
22.2B
16.3
16.17.0.8055
7.3%
52.8B
14.83
14.940.47%
-6.9%
2
74
15.3B
21.36
21.32.0.19%
2.1%
70.4B
19.54
19.62.0.415
-7.1%
323.6B
18.66
18.7440.43%
-12.4%
163
17.0B
18.28
18.17.0.60%.
5.6%
65.0B
16.36
16.39+0.18%
-2.3%
425
11.1B
17.01
16.69-1.88%
13.3%
33.5B
14.59
14.51-
7.8%
101.3B
13.38
13.37-0.07%
2.1%
305.8B
12.65
12.6640.08%%
-5.9%
816
52.8B
13.35
13.27-0.60%
6.5%
297.5B
11.77
11.77.0.00%
2.0%
74
1.4T
16.26
16.25-0.06%
2.2%
163
901.3B
13.51
13.49.0.15%,
9.2%
425
10.5B
16.3
16.02-1.72%
9.6%
31.8B
13.9
13.76.1011%
10.4%
96.0B
12.62
12.54.0.63%
13.6%
816
281.9B
11.07
11.02.0.5%
13.2%
6
74
306.6B
16.45
16.41 0.24%
9.1%
1.4T
15.85
15.82.0.19%
14.3%
163
61.6B
14.92
14.83.0.60%
9.5%
901.3B
13.21
13.18 0.23%
9.23%
27.9%
425
3L.8B
13.72
13.59.
10.4%
96.0B
12.44
12.36.0.6495
15.5%
289.7B
11.63
11.58.0.43%
38.8%
816
118.7B
11.59
11.51.0.9%%
11.4%
281.9B
10.92
10.85.064%
16.6%
29

---

Published as a conference paper at ICLR 2026
N EXPERIMENT TOKEN COUNTS
Table
11 summarizes the total token counts used throughout the experiments. For each token count, Pp D qut ratios were tested. Selected ratios for different setups are displayed tables [2]
several
15 In addition to the reported structured experiments, we conducted experiments with extreme QAT fractions (close to 1% and close to 100%) to improve loss scaling law fitting across the range of different values.
13
and
Table 11:
List of total token counts analyzed for different model sizes.
Model Size (M)
Total Tokens
86
2.3B, 2.4B, 2.6B,
10.5B, 13.2B, 13.9B, 14.5B, 14.8B,
15.3B, 27.0B, 41.8B, 60.6B, 61.7B, 64.0B, 66.7B, 70.4B, 123,9B, 171.3B,
274.4B, 278.7B, 294.2B, 306.6B, 323.6B, 569.3B, 1.2T, 1.3T, 1.4T
194
3.2B, 3.3B, 4.0B, 4.2B, 4.4B, 6.5B, 9.5B, 14.6B, 15.5B, 16.1B, 17.0B, 24.9B,
36.3B, 56.0B, 59.1B, 61.6B, 65.0B, 95.2B, 138.8B, 182.6B, 214.2B, 226.1B,
235.6B, 248.7B, 364.2B, 530.8B, 698.3B, 819.48, 901.3B
396
4.3B, 8.2B, 9.6B, 9.7B, 10.5B, 11.IB, 12.8B, 24.6B, 28.9B, 30.5B, 31.8B,
33,5B, 56.5B, 84,4B, 87.2B, 92.1B, 96.0B, 10133,, 170.6B, 263.4B, 289.7B,
305.8B, 515.2B, 874.8B
759
8,5B, 21.IB, 22.2B, 48.0B, 50.0B, 52.8B, 113.9B, 118.7B, 125.3B, 281.8,,
297.5B, 536.7B, 669.2B
Table 12: List of different QAT fractions analyzed for the 86M parameter model and different total token counts.
D total
D qut/ total
Model Size (M)
B
86
1
3.3B
10.0%, 20.0%, 30.6%
15.3B
10.0%, 20.0%, 37.3%, 41.7%, 49.7%, 63.3%, 85.0%
1
70.4B
10.0%, 20.0%, 41.7%, 42.9%, 63.3%, 66.6%, 85.0%
323.6B
10.0%, 20.0%, 41.7%, 47.6%, 63.3%, 85.0%
2
3.1B
4.6%, 10.0%, 10.0%, 20.0%, 23.3%, 26.3%, 40.0%
14.5B 2
5.0%, 6.3%, 10.0%, 20.0%, 23.3%, 33.5%, 41.7%, 45.7%,
60.0%, 63.3%, 78.0% 85.0%
2
66.7B
5.0%, 8.5%, 10.0%, 20.0%, 23.3%, 39.6%, 41.7%, 56.9%,
60.0%, 63.3%, 78.0%, 85.0%
2
306.6B
5.0%, 10.0%, 11.4%, 20.0%, 23.3%, 41.7%, 44.7%, 60.0%,
31.3%, 78.0%, 85.0%
2
1.4T
5.0%, 10.0%, 15.0%, 23.3%, 41.7%, 60.0%, 78.0%, 83.2%
4
3.0B
1. 1.0%, 4.6%, 10.0%, 23.3%, 26.7%
4
13.9B
1.0%% 6.3%, 10.0%, 23.3%, 29.7%, 41.7%, 60.0%
4
64.0B
1.0%, 5.0%, 8.5%, 10.0%, 23.3%, 36.8%, 41.7%, 60.0%
4
123,9B
1.0%, 50.0%, 90.8%
4
274.4B
1.0%, 50.0%, 79.7%
4
294.2B
1.0%, 10.0%, 11.4%, 23.3%, 41.7%, 49.2%, 60.0%
4
569.3B
1.0%, 50.0%, 57.9%
4
1.2T
1.0%, 10.5%
4
1.4T
5.0%, 10.0%, 15.0%, 23.3%, 41.7%, 60.0%, 65.0%
6
3.IB
4.6%. 5.0%, 10.0%. 17.0%. 20.0%, 23.3%, 23.4%
6
14.5B
5.0%, 6,3%, 10.0%. 20.8% 23.3%, 28.6%, 41.7%, 60.0%
66.7B
5.0%, 8.5%, 10.0%, 23.0%, 23.3%, 38.0%, 41.7%, 60.0%
6
171.3B
1.0%, 50.0%, 87.3%
306.6B
5.0%, 10.0%, 11.4%, 23.3%, 29.0%, 41.7%, 45.3%, 60.0%
6
1.4T
5.0%. 10.0%. 15.0%, 23.3%, 40.7%, 41.7% 60.0%
30

---

Published as a conference paper at ICLR 2026
Table 13: List of different QAT fractions analyzed for the 194M parameter model and different total token counts,
D total
D qut total
Model Size (M)
194
4.4B
10.0%, 20.0%, 28.1% 1%
I
17.0B
10.0%, 20.0%, 34.3%, 41.7%, 43.9%, 63.3%
65.0B
10.0%, 20.0%, 39.7%, 41.7%, 56.4%, 63.3%, 85.0%
1
248.7B
10.0%, 20.0%, 41.7%, 44.4%, 63.3%, 85.0%
2
4.2B
5.0%, 10.0%, 20.0%, 23.3%, 23.6%, 38.8%
16.1B 2
5.0%, 7.7%, 10.0%, 20.0%, 23.3%, 30.3%, 41.7%, 42.5%,
60.0%, 63.3%, 78.0%
2
61.6B
5.0%, 10.0%, 20.0%, 23.3%, 36.1% 1%, 41.7%, 49.7%, 60.0%,
63.3%, 78.0%, 85.0%
2
235.6B
5.0%, 10.0%, 12.8% 20.0% 23.3%, 41.1%, 41.7%, 60.0%,
63.3%, 78.0%, 85.0%
2 901.3B
5.0%, 10.0%, 16.4%, 23.3%, 41.7%, 78.0%
4
4.0B
1.0%, 5.0%, 10.0%, 23.3%
4
15.5B
1.0%, 5.0%, 7.7%, 10.0%, 23.3%, 27.9%, 41.7%, 60.0%
4
56.0B
1.0%, 50.0%, 93.5%
4
59.1B
1.0%, 5.0%, 10.0%, 23.3%, 32.0%, 41.7%, 60.0%
4
95.2B
1.0%, 50.0%, 89.0%
4
138.8B
1.0%, 84.0%
4
182.6B
1.0%, 50.0%, 78.9%
4
214.2B
1.0%, 50.0%, 75.2%
4
226.1B
1.0%, 5.0%, 10.0%, 12.8% 23.3%, 39.9%, 41.7%, 60.0%
4
364.2B
1.0%, 50.0%, 57.9%
4
530.8B
1.0%, 38.6%
4
901.3B
5.0%, 10.0%, 16.4%, 23.3%, 41.7%, 51.9%, 60.0%
4.2B
5.0%, 10.0%, 12.6%, 23.3%
6
16.1B
5.0%. 7.7%, 10.0%, 20.3%, 23.3%, 41.7%, 60.0%
61.6B 6
5.0%, 10.0%, 21.6%, 23.3%, 32.8%, 41.7%, 60.0%
6
235.6B
5.0%. 10.0%, 12.8%, 23.3%, 24.4%, 40.3%, 41.7%, 60.0%
901.3B 6
5.0%, 10.0%, 16.4% 23.3%, 41.7%, 60.0%
31

---

Published as a conference paper at ICLR 2026
Table 14: List of different QAT fractions analyzed for the 396M parameter model and different total token counts,
D total
D qut total
Model Size (M)
396
11.1B
10.0%, 20.0%, 29. 1%, 37.9%, 41.7%, 63.3%
1
33.5B
10.0%, 20.0%, 34.2%, 41.7%, 43.6%, 63.3%, 85.0%
101.3B
10.0%, 20.0%, 38.7%, 41.7%, 53.5%, 63.3%, 85.0%
1
305.8B
10.0%, 20.0%, 41.7%, 42.7%, 63.3%, 65,9%, 85.0%
10.5B 2
5.0%, 8.5%, 10.0%, 20.0%, 23.3%, 24.7%, 39.3%, 41.7%,
60.0%, 63.3%
2
31.8B
5.0%, 10.0%, 20.0%, 23.3%, 30.1%, 41.7%, 42.3%, 60.0%,
63.3%, 78.0%, 85.0%
2
96.0B
5.0%. 10.0%, 12.9%, 20.0%, 23.3%, 35.0%, 41.7%, 47.9%,
60.0%, 63.3%, 78.0%, 85.0%
2
289.7B
5.0%, 10.0%, 15.8%, 20.0%, 23.3%, 39.3%, 41.7%, 56.4%,
60.0%, 63.3%, 78.0%, 85.0%
2 874.8B
5.0%, 10.0%, 19.2%, 41.7%, 60.0%
4
10.5B
5.0%, 8.5%, 10.0%, 23.3%, 26.3%, 41.7%, 60.0%
4
31.8B
5.0%, 10.0%, 23.3%, 27.9%, 41.7%, 60.0%
4
96.0B
5.0%, 10.0%, 12.9%, 23.3%, 31.0%, 41.7%, 60.0%
4
170.6B
1.0%, 50.0%, 79.7%
4
263.4B
1.0%, 50.0%, 68.6%
4
289.7B
5.0%, 10.0%, 15.8%, 23.3%, 36.3%, 41.7%, 60.0%
4
515.2B
1.0%, 38.6%
4
874.8B
5.0%, 10.0%, 19.2%, 23.3%, 41.7%, 60.0%
10.5B 6
5.0%, 8.5%, 10.0%. 14.3%. 19.8%, 23.3%, 41.7%, 60.0%
31.8B 6
5.0%, 10.0%, 20.3%, 23.2%, 23.3%, 41.7%, 60.0%
96.0B 6
5.0%, 10.0%, 12.9%, 21.1%, 23.3%, 31.0%, 41.7%, 60.0%
289.7B
5.0%, 10.0%, 15.8%, 22.9%, 23.3%, 41.7%, 60.0%
874.8B 6
5.0%, 10.0%, 19.2%, 23.3%, 41.7%
32

---

Published as a conference paper at ICLR 2026
Table 15: List of different QAT fractions analyzed for the 759M parameter model and different total token counts,
D total
D qut / total
Model Size (M)
759
22.2B
10.0%, 20.0%, 29.3%, 38.0%, 41.7%, 63.3%
1
52.8B
10.0%, 20.0%, 33.3%, 41.7%, 42.3%, 63.3%, 85.0%
125.3B
10.0%, 20.0%, 37.0%, 41.7%, 63.3%, 85.0%
1
297.5B
10.0%, 20.0%, 40.3%, 41.7%, 58.1%, 63.3%, 85.0%
21.1B 2
5.0%, 10.0%, 11.1%, 20.0%, 23.3%, 24.9%, 39.4%, 41.7%,
63.3%, 78.0%
2
50.0B
5.0%, 10.0%, 13.0%, 20.0%, 23.3%, 29.2%, 41.6%, 41.7%,
60.0%, 63.3%, 78.0%. 85.0%
2
118.7B
5.0%. 10.0%, 20.0%, 33.2%, 41.7%, 45.4%, 63.3%, 85.0%
2
297.5B
10.0%, 20.0%, 36.8%, 41.7%, 50.9%, 63.3%, 85.0%
4
21.1B
5.0%, 10.0%, 11.1%, 23.3%, 26.3%, 41.7%, 60.0%
4
50.0B
5.0%, 10.0%, 13.0%, 23.3%, 27.6%, 41.7%, 60.0%
4
118.7B
5.0%, 10.0%, 15.2%, 23.3%, 29.5%, 41.7%, 60.0%
4
281.8B
5.0%, 10.0%, 17.7%, 23.3%, 32.7%, 41.7%, 60.0%
4
536.7B
1.0%, 16.4%
669.2B 4
5.0%, 10.0%, 20.6%, 23.3%, 37.6%, 41.7%, 60.0%
6
21.1B
5.0%, 10.0%, 11.1%, 19.8%, 23.3%, 41.7%, 60.0%
6
50.0B
5.0%, 10.0%, 13.0%, 20.2%, 23.3%, 41.7%, 60.0%
6
118.7B
5.0%, 10.0%, 15.2%, 20.6%, 23.3%, 41.7%, 60.0%
6
281.8B
5.0%, 10.0%, 17.7%, 21.6%, 23.3%, 41.7%, 60.0%
6
669.2B
5.0%, 10.0%, 20.6%, 23.3%, 41.7%
WASTED TOKENS COUNT FORMULATION (SECTION 5
In this section, we formalize how wasted tokens are calculated for table 2 Let have loss of fused and unfused experiments for some Dotal: Liused and Lunfised- Then, similarly to wasted tokens formulation from section 3.2 we can calculate token-distance between Tusee and Lunfused on QAT optimality curve:
D Donal (N, B,1)
arg min D Ding EN D' DD; qu Diaap
|LN, D'ga- D'otal - D'ga- B) 1/.
D wasted
1)" total (N, B, Liused) D iotal (N, B, Lunfused),
and the reported percentage is the fraction of unfused total tokens: D (N B Lautreal)
Dusnted
33

---

Published as a conference paper at ICLR 2026
P QAT ACCURACY
In figures 16a 16b
plot how optimal QAT fraction experiments compare the full-precision model with the same total token count. Results reveal that the optimal QAT fraction in 4-bit and 6-bit settings achieves loss close the full-precision counterpart.
16c and T6d
Experiments for 1-bit QAT
Model 86M
Model 194M
Perplexity
30
25
20
20
15
Model 396M
Model 759M
Cooldown
20
20
Perplexity
QAT
QAT predicted
15
10
10.0B
100.0B
1.OT
10.0B
100.0B
1.0T
Total tokens
Total tokens
(a) Final loss plots for 1-bit QAT.
Experiments for 2-bit QAT
Model 86M
Model 194M
30
Perplexity
30
20
20
15
Model 396M
Model 759M
Cooldown
20
20
Perplexity
QAT
QAT predicted
15
10
10.0B
100.0B
1.0T
10.0B
100.0B
1.0T
Total tokens
Total tokens
(b) Final loss plots for 2-bit QAT.
34

---

Published as a conference paper at ICLR 2026
Experiments for 4-bit QAT
Model 86M
Model 194M
30
Perplexity
30
25
20
20
Model 396M
Model 759M
20
Cooldown
20
Perplexity
QAT
QAT predicted
15
15
10
10
10.0B
100.0B
1.0T
10.0B
100.0B
1.0T
Total tokens
Total tokens
(c) Final loss plots for 4-bit QAT.
Experiments for 6-bit QAT
Model 86M
Model 194M
30
Perplexity
30
20
20
Model 396M
Model 759M
20
Cooldown
20
Perplexity
QAT
QAT predicted
15
15
10
10.0B
100.0B
1.0T
10.0B
100.0B
1.0T
Total tokens
Total tokens
(d) Final loss plots for 6-bit QAT.
Figure 16: Final loss of QAT compared to the full-precision post-cooldown model for the same total token count. For QAT, we plot the best loss for the total token count (optimal QAT fraction experiments). Additionally, we plot the loss predicted for the optimal QAT fraction from the appropriate loss scaling law, and confidence bands correspond to the predicted range of QAT loss for the 5-95% range of QAT fraction.
35

---

Published as conference paper at ICLR 2026
Q UNCERTAINTY ANALYSIS
We analyze fit uncertainty and parameter significance from the perspective of their influence on loss model fit metrics. Formally, we can formulate the problem as follows:
Ho S(m(O 0)) Sm O)), H1 S(m(O 0)) S(m(0 0)),
where S' a fit metric of interest for some model, m 0) represents fitted model with parameter forced to zero, and m(0 0) represents fitted model with parameter 0 allowed vary. We will analyze two metrics: R p--a general goodness-of fit metric, and QAT optimal fraction fit MAE (the inequality H1 reversed). Together, these two metries capture two important properties of scaling law: the ability to predict final model accuracy and the ability to predict the optimal QAT fraction accurately.
To estimate the distribution of the fit metric, use bootstrapping. We employ the following scheme:
1. Generate a bootstrapped dataset by sampling with replacement from the original dataset.
2. Fit both models m(0 = 0), m # 0) to the bootstrapped dataset. several model initialization seeds, and the best fit is selected.
This step is repeated for
3. Calculate the fit metric for both models.
4. Repeat steps 1-3 B 100 times for each model parameter.
In the end, for each parameter, we obtain two metrics for each bootstrapped dataset corresponding to m0 = 0), m(0 # 0). Then, calculate the difference of metrics and calculate one-sided 95% quantile confidence interval of the difference. We conclude that the parameter significant if is not covered by the interval, which means that the model with the parameter is significantly better than the model without it.
Results are presented in figure 17 Combining results for both metries, all parameters except those corresponding to constant shifts (ar, and #) are significant. This result is expected for QAT fraction MAE, as constant shifts affect the absolute loss value but the relative position of the optimal QAT fraction (the argmin over curve). However, this is not expected for R² Nonetheless, we retain those parameters they have clear conceptual meaning: a comes from the Chinchilla scaling law, and k, / model irreducible QAT error. What is more important is that two other added terms in the equation 2 ("pure QAT penalty" and "FP QAT interaction") are significant.
36

---

Published as a conference paper at ICLR 2026
5
B
71
Density
1000
0.0
2.5
5,0
0
5
0.0
2.5
5.0
0.0
0.5
1.0
x10
x10-3
x10
x10
0
A
Density
1000
0
5
2
4
0.0
2.5
5.0
x10
x10
x10
X10-2
E
V
Density
1000
0.0
0.5
1.0
2
0.0
0.5
1.0
4
x10P
x10
x10
x10
V
X
Density
1000
0.0
2.5
5.0
0
5
0.0
0.5
1.0 0.0
2.5
5.0
x10
x10
x10
x10
Diff.: R²
Diff. R²
Diff.: R² 1
Diff.: R²
(a) Significance of the parameters for the fit metric R².
a
B
17
Density
50
-4
-2
0 x10
-2
2
-5.0
-2.5
0.0
2
x10
x10
x10
y
0
A
Density
50
-5.0
-2.5
0,0
-4
-2
0
-4
-2
-5.0
-2.5
0,0
x10-1
X10-2
x10
x10-1
Density
-5.0
-2.5
0.0
-5.0
-2
-2
x10
x10
x10
x10
X
Density
50
-1
X10
x10
x10
x10
Diff:: MAEL
Diff.: MAEJ
Diff:: MAE]
Diff.: MAEL
(b) Significance of the parameters for the QAT optimal fraction fit metric MAE.
Figure 17: Shaded area represents one-sided 95% quantile confidence interval of the difference in
metrics corresponding to constrained and unconstrained models m(# = 0) and m(0 # 0) (where the parameter of interest, indicated by the subplot title). Green color means that not covered by the interval, meaning that the parameter is significant. Red color means that 0 is covered by the interval, meaning that the parameter's 's insignificance not rejected.
37

---

Published as a conference paper at ICLR 2026
R
FUTURE WORK
In this section, we speculate on possible results for the future work directions proposed in the paper (section 6).
R.1 PRETRAIN PRECISION & QAT PRECISION INTERACTION
The question of interest is "How do QAT scaling laws change when pretrain precision is re- duced?" Specifically, a practically important question is how optimal QAT compute allocation changes. Kumar et al. (0225) analyze this question in the context of post-training quantization. While QAT and PTQ yield significant differences in accuracy (especially for lower bits (Liu et al. expect general trends to be similar.
2025
Kumar et al. 2025) report that "overall, models trained lower precision are more robust to post- training quantization in the sense of incurring lower degradation." We expect the same phenomenon in the context of QAT. Therefore, one may expect the optimal QAT fraction to be smaller when a model is pretrained in lower Moating-point precisions (fp4, fp8) than in high precision (fp16, bf16, fp32). Still, we expect the optimal QAT fraction to grow with increasing total compute.
R.2
QAT SCALING LAw FOR MULTI-STAGE PRETRAINING
Current state-of-the-art chat models commonly incorporate multiple training stages. Commonly, after general cross-entropy pretraining, additional supervised fine-tuning (SFT) and reinforcement learning stages are performed (DeepSeek-A)
al.. 2025 OLMo et al. 2025| Apertus et al. 2025
Ouyang et al. 2022 Allal | 2025 Rafailov al.) 2023 Schuuman al. [017]] Zhou et al.,
2025a). This raises not only the question of how much compute to allocate for QAT but also how to distribute this compute among different stages.
possible solution is to conduct all post-pretraining stages over the QAT model. Usually, post- training constitutes minor percentage of compute when compared to pretraining (DeepSeek-A] jtt al., 2025] Allal et al.) 2025 OLMo et al. 2025 Apertus et al.) (025). Therefore, it is natural to expect the optimal QAT fraction be larger than the entire post-pretraining stage. This means that it is possible to start QAT during pretraining and finish QAT with post-pretraining tuning.
Such methodology also motivated by the fact that QAT ineurs representation changes, especially in the case of small QAT bit-widths (Liu et al.) (2025). Therefore, we believe it is beneficial not to postpone this process of representation change until after post-pretraining stages.
38


Reviews:
[{"review_id": "Nnwj5ZvNZJ", "reviewer": "Reviewer_Hsq3", "summary": "In this paper, the authors talk about the scaling law for quantization aware training as well as the impact of the fraction for quantization aware training compared to the full-precision pretraining. The topic seems to be interesting and the authors provide some experiments to demonstrate the results. However, the paper is very badly written and results presentation is very bad (e.g., the figures are very difficult to read and the conclusions made from the figures are difficult to follow or should be arguable).", "questions": "In Figure 1 left, the y-axis is QAT tokens-per-byte, and x-axis is Total tokens-per-byte. The author would like to demonstrate that the optimal QAT fraction increases with the full training tokens-per-parameter-byte (line 092 in the original paper). To demonstrate this, the y-axis should be the ratio between QAT tokens-per-byte to the total tokens-per-byte, instead of only list the two end ratios in the figure. Could the author provide the plot with ratio as the y-axis to prove the conclusion?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The topic and problem proposed seems to be interesting, and the final conclusion or results, if correct as the author claimed, might be useful. Especially, the conclusion about how to determine the optimal fraction for quantization aware training, and also the possibility to eliminate redundant full-precision training via novel learning rate scheduling, might be useful.\n- Regarding the techinical contribution, the authors first study the optimal fraction allocated for QAT and its relationship with the tokens-per-parameter-byte. Afterwards, they propose the loss scaling law for optimal QAT fraction and model the final expected loss of the FP and QAT pipeline. They finally propose a novel approach of cooldown and QAT fusion, where learning rate cooldown in FP training and learning rate warmup in QAT are fused, and thus redundant full-precision updates can be eliminated.", "weaknesses": "- The writing and results presentation are not good. Figure1 is difficult to read.\n- Improvement in Table 2 might not be strong enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4jjmkGxOjl", "reviewer": "Reviewer_cjQW", "summary": "This work is focused on QAT fine tuning which is done after full-precision (FP) phase pre-training.\nQAT bit widths, and model sizes from 86.0M to2.2B to investigate how different QAT durations impact final performance. \nThey show that contrary to previous findings, the loss-optimal ratio of QAT to FP training increases with the total amount of compute.\nThis work derive a loss scaling law that predicts both optimal QAT ratios and final\nmodel performance across different QAT/FP compute allocation strategies and\nQAT bit widths.  Additionally, they propose a novel cooldown and QAT fusion approach that performs learning rate decay jointly with quantization aware training, eliminating redundant full-precision model updates and achieving significant compute savings.", "questions": "Do you reset or keep optimizer state, from pre-trained model, when apply QAT fine tuning?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Authors run multiple experiments for fitting generating scaling law into proposed function and demonstrated small prediction error. They showed that the optimal QAT fraction is not a fixed percentage but rather increases with the total compute budget, specifically with the tokens-per-parameterbyte statistic. This challenges the previous conclusion that 10% is universally optimal for QAT length relative to total training length.\n\nThey proposed a loss scaling law that captures the optimal QAT fraction phenomenon and\nmodels the final expected loss of the FP and QAT.\n\nProposed a novel approach: cooldown & QAT fusion—a scheme where learning rate decay is performed jointly with quantization-aware training, eliminating redundant fullprecision updates and achieving better accuracy for the same token count.", "weaknesses": "This work is focused on a specific LLM architecture and data sets, and exact results may differ for different model types different QAT methods and data.\nOnly int quantization explored, it would be interesting to see fp4 and fp8 results.\nLimited size LLM are explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TJTlQ1ad2l", "reviewer": "Reviewer_w7Wt", "summary": "This paper investigates the optimal resource allocation ratio between Full-Precision (FP) and Quantization-Aware Training (QAT) phases. Through extensive LLM experiments, the authors found that the optimal QAT fraction increases with the total compute budget and introduced tokens-per-parameter-byte as a key metric to accurately predict this optimum. Loss Scaling Law is proposed to predict the performance of QTA.", "questions": "How sensitive are the fitted parameters to the range of model sizes and token counts used in the experiments? Can the authors provide an uncertainty analysis? While fine-tuning the FP model, how would the optimal QAT fraction for the final QAT phase shift? There might be a distribution difference between pre-training phase and fine-tuning phase.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper establishes a novel loss scaling law capable of predicting both the optimal QAT fraction and final model performance across varying bit widths and allocation strategies, demonstrating high practical utility. The proposed method integrates learning rate decay with QAT, effectively eliminating redundant model updates and improving both computational efficiency and accuracy.", "weaknesses": "Experiments are conducted on a specific LLM architecture（llama2）, and the scaling law parameters may require refitting for other model types. Different types of models may exhibit varying degrees of tolerance to quantization, particularly those with inherent redundancy.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jSBOjyC7RY", "reviewer": "Reviewer_1gxP", "summary": "This paper extends scaling-law analysis to the two-stage full-precision & QAT pipeline, asking how to split a fixed training budget between full-precision pretraining and quantisation-aware training. Through hundreds of runs across model sizes (86M–2.2B) and bit-widths, the authors show that the loss-optimal QAT fraction increases with scale, and that this dependency is well captured by a simple _tokens-per-parameter-byte_ statistic. They fit a unified loss scaling law (over ($N$, $D_{fp}$, $D_{qat}$, $B$) that includes pure-QAT and FP/QAT interaction terms, and use it to predict optimal QAT ratios as well as when low-bit QAT can match FP loss. The study also proposes a practical “cooldown & QAT fusion” training schedule (merging LR decay with QAT) and reports small but consistent savings in “wasted tokens” at moderate/high bit-widths. Overall, the paper moves the literature from “QAT should be ~10%” heuristics to compute-aware prescriptions grounded in scaling behaviour, while relying on a strong, modern QAT baseline (ParetoQ)", "questions": "* Figure 1 (right), add $D_{fq}$ and $D_{qat}$ along the axes and include a loss heatmap in the legend\n* Equation 2: more discussion and motivation for the forms chosen related to QAT.. What is pure QAT penalty, and why does it take form? * * Why does FP/QAT interaction take this form? Are these grounded in theory or chosen for fitting reasons? Did you do any ablation studies on the form?\n* Appendix D. Figure 9: I did not see any discussion or explanation on why the theoretical and experimental optima diverge so much at the lowest $D_{total}$ iso curves for 2,4, and 6 bits for the smallest model of 86M.\n* Line 277: wasted token count: I had to read the definition multiple times to understand it. Could you add an equation? \n* Line 322: iso-flop. Capitalise flop because it’s hard to understand it when it’s in lower case.\n* Figure 5: Make the colour of the bitwidths 5 and 1 more distinct. It’s very hard to tell which curve corresponds to each bitiwidthth\n* Section 7: wasted unfused total tokens: could you maybe add an equation? I don’t fully understand how you reach that number.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "All in all, this is a solid experimental paper that answers fundamental questions in LLM deployment with a sound methodology. I enjoyed how the authors used their loss scaling curves to answer interesting questions in their discussion. For example, the question of what the optimal QAT bitwidth per model and training budget is has been a topic in the community for years. The main strengths are:\n\n* The paper addresses a very significant and unanswered question in LLM inference and of increasing importance for LLM deployment. What budget should be allocated to QAT vs FP training? No previous work has answered this important question in experimentally sound ways, and this work paves the way for significant future work.\n* The loss scaling laws are solid, based on hundreds of experiments, and provide strong transparency on the fit of the curve. The introduction of the token-per-byte statistics is also a very useful metric capturing the idiosyncrasies of QAT. The vast corpus of QAT experiments makes for a solid experimental methodology.\n* A very extensive appendix, including many results, experimental set-ups, and curves.", "weaknesses": "* The paper is somewhat too long given the level of novelty and contribution, and as a result, the central narrative becomes hard to follow. While each section presents interesting findings, the overall flow sometimes feels like a set of independent conclusions about QAT rather than a single, coherent story. I found myself re-reading parts to connect the results to the main scaling-law argument. \n\n* Section 7 reads as an addendum: it explores LR-scheduler fusion, which is interesting but tangential to the main scaling-law contribution, and the reported gains are relatively small and bit-width dependent. I would think this should be part of another paper or appendix.\n\n* The full-precision loss regularisation used to fit the unified law is crucial to interpreting Fig. 5 and the FP-vs-QAT comparisons, but it’s only described in Appendix E. Similarly, the “wasted token count” metric is under-explained and would benefit from a formal definition and more apparent motivation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)