Scaling Laws for Precision

## Abstract

Abstract Low precision training and inference affect both the quality and cost of language models, but current scaling laws do not account for this. In this work, we devise “precision-aware” scaling laws for both training and inference. We propose that training in lower precision reduces the model’s effective parameter count , allowing us to predict the additional loss incurred from training in low precision and post-train quantization. For inference, we find that the degradation introduced by post-training quantization increases as models are trained on more data, eventually making additional pretraining data actively harmful. For training, our scaling laws allow us to predict the loss of a model with different parts in different precisions, and suggest that training larger models in lower precision may be compute optimal. We unify the scaling laws for post and pretraining quantization to arrive at a single functional form that predicts degradation from training and inference in varied precisions. We fit on over 465 pretraining runs and validate our predictions on model sizes up to 1.7B parameters trained on up to 26B tokens.

### 1 Introduction

Scale has emerged as a central driver of progress in deep learning (Brown, ). Key work on scaling (Kaplan et al., ; Hoffmann et al., ) studied tradeoffs between model/dataset size to balance performance and compute. However, the precision in which models are trained and served is an important third factor that contributes to both cost and performance. Deep learning is trending towards lower precision: current frontier models like the Llama-3 series are trained in BF16 (Dubey et al., ), and there is widespread effort to move the pretraining paradigm to FP8 (Micikevicius et al., ). The next generation of hardware will support FP4, and advances in weight-only quantization have led to training in binary and ternary at scale (Ma et al., ; Wang et al., ). How far will these paradigms go? Specifically, we ask:

What are the tradeoffs between precision, parameters, and data?
How do they compare for pretraining and inference?

Studying scaling in precision is challenging because work on scaling laws generally aims to drop fine-grained implementation details in pursuit of universal functional forms while work on quantization generally does the opposite, focuses on the details: how quantization is done, with what type, to what part of the model. In seeking a balance, we consider a variety of plausible functional forms, and choose one that abstracts implementation details of quantization away from loss scaling, allowing us to predict loss scaling in many situations of practical interest. This functional form that posits bit precision and parameter count interchangeably contribute to a model’s “effective parameter count,” $N_{\text{eff}}$, and implementation details like which parts of a model are quantized to what precision, interact with loss scaling only through their effect on this quantity.

Figure: Figure 1: Schematic of key findings. (Left) Training a fixed model size to various data budgets in BF16 and quantizing weights at the end. We find that degradation due to post-train quantization increases with tokens seen during pretraining, so that eventually additional pretraining data can be harmful. (Right) Our scaling suggests training larger models in lower precision can be compute-optimal according to the cost model in Section 4.3. Weights, activations, attention quantized, all models trained on the same data budget, details in Appendix H.
Refer to caption: x1.png

Overall, we study the scaling of the effects of precision on loss as we vary data and parameters, both during and after training. We first study how the degradation induced by post-train quantization scales with parameters and data. We find that the degradation increases with data, so that for a fixed model, training on additional data after a certain point can be actively harmful if the model will be quantized after training. We then shift our focus to quantized training, examining both the quantization-aware-training (weights only) and low-precision training (weights, activations, attention all quantized) settings. Our scaling laws for pretraining suggest that the compute-optimal pretraining precision is in general independent of compute budget. Surprisingly, however, this independence ceases to be true if model size is constrained, in which case the compute-optimal precision grows slowly in compute.

In all, we pretrain a suite of 465 language models in 3 to 16 bit precisions, as well as post-train quantize each to multiple precisions. For a language model with $N$ parameters, trained on $D$ tokens with training precision $P_{\text{train}}$, and post-train weight precision $P_{\text{post}}$, we ultimately find a unified scaling law that takes the following form:

$$ $L(N,D,P_{\text{train}},P_{\text{post}})=\underbrace{\underbrace{AN_{\text{eff} }^{-\alpha}}_{\text{Training-time Effects}}+BD^{-\beta}+E}_{\text{Usual Chinchilla form}}+\underbrace{\delta_{\text{PTQ}}(N_{\text{eff}},D,P_{\text{ train}},P_{\text{post}})}_{\text{Post-Training Effects}}$ (1) $$

where $A,B,E,\alpha,\beta$ are positive fitted constants, and $\delta_{\text{PTQ}}$ refers to the loss degradation induced by post-training quantization before inference. Altogether, our results for post-train quantization illustrate how more pretraining FLOPs do not always lead to better models at inference-time, and our results for low-precision pretraining suggest that both the standard practice of training models in 16-bit, and the race to extremely low (sub 4-bit) pretraining precision, may be suboptimal.

### 2 Background, Related Work, and Setup

Notation.
Throughout, $D$ denotes dataset size in tokens and $N$ denotes model size in parameters. $P_{\text{w}},P_{\text{a}},P_{\text{kv}}$ refer to the bit precision, in integer-type, of the weights, activations, and key-value cache (“attention”)(^1^11We study KV, rather than QKV, because understanding scaling in the KV cache alone is important for many inference settings. For pretraining claims in Section 4.3, we quantize the entire attention computation, including queries, finding additionally quantizing the query vectors makes a negligible difference to scaling.) during training, and $P_{\text{post}}$ refers to the precision we post-train quantize (PTQ) weights to at the end for model inference. When $P$ or $P_{\text{train}}$ is used without reference to a part of the model, all three model parts are tied to the same precision. The inference-time loss degradation induced by post-train quantization will be denoted $\delta_{\text{PTQ}}(N,D,P_{\text{train}},P_{\text{post}})$, and it is defined as the change in loss from performing post-training quantization compared to the end of pretraining. We use “high precision” to mean 16-bit or above.

#### 2.1 Quantization Fundamentals: How, What, When

The Problem: Compute vs Memory-Bound Workloads. Most deep learning workloads are bottlenecked by either compute, in the form of matrix multiplications, or memory bandwidth, in the form of data movement between different parts of the GPU. Different types of workloads have different bottlenecks: most time is spent doing large matrix multiplications during pretraining, so it is compute-bound; in contrast, small-batch inference is bandwidth-bound by model weights; long-sequence decoding is bandwidth-bound by KV cache, etc. This motivates studying scaling in the training precision of the (weights, activations, KV cache) both in isolation and in combination.

Quantization: How. Quantization of an operation typically refers to rounding of values in matrices involved in some computation on the forward/backward pass, with accumulation of gradients in high/full precision. Quantization is usually done to integer or floating-point type.

Quantization: What. Only weights. “Quantization-aware training” Quantizing only weights during training does not offer any compute savings because matrix multiplications are still done in high precision. However, this is commonly done to allow weights to adapt to low precision so they can be served at very low precision at inference-time, thereby alleviating memory bottlenecks (Ma et al., ; Wang et al., ). We will refer to this as “quantization-aware-training.”

Weights, activations, attention. “Low-precision training” Quantizing and activations and attention in addition to weights allows for compute gains because matrix multiplications can be done in low precision (if the hardware supports it) since everything is in the same precision. FP8 training on the Hopper line of GPUs is an example (Micikevicius et al., ). We will refer to this setting as “low-precision training” to distinguish it from quantization-aware training.

Quantization: When. Quantization can be done during or after training. In practice, when seeking to reduce inference-time memory costs, one first attempts post-train quantization. If that degrades the model too much, quantization-aware-training is used. Post-train quantization is typically only applied to model weights (Frantar et al., ; Dettmers et al., ; Lin et al., ; Xiao et al., ). To reduce pretraining costs, low-precision-training is needed. We will study scaling laws for post-training quantization in Section 3, for quantized training in Section 4 (examining both quantization-aware training and low precision training) and unify the two in Section 5. The numerical values of all our fitted constants can be found in Appendix I.

#### 2.2 Scaling Laws and Parametric Fits

Scaling Laws. Hoffmann et al. () model loss scaling using the functional form $L(N,D)=AN^{-\alpha}+BD^{-\beta}+E$ where $A,B,\alpha,\beta,E$ are positive fitted constants, finding that data and parameters should be scaled in roughly equal proportion as more compute becomes available. We will refer to the scaling of (Hoffmann et al., ) as “Chinchilla-optimal” or just “Chinchilla” and note this is often used colloquially as $D/N\approx 20$ being pretraining compute-optimal. On the theoretical front, work on scaling laws (Bahri et al., ; Bordelon et al., ; Lin et al., ) finds that noise to various parts of model or data affects loss in a predictable way. While previous works have explored the scaling behavior of post-training quantization in terms of total model bits (Dettmers and Zettlemoyer, ) and knowledge capacity (Allen-Zhu and Li, ), we focus instead on data scaling. We note that in general the exact fitted values of all coefficients and exponents can vary drastically based on small implementation differences: Besiroglu et al. () find different constants when attempting to replicate (Hoffmann et al., ), Sardana and Frankle () fit coefficients $A,B$ of different orders of magnitude. For this reason, we emphasize our contribution is not the numerical values we fit, but the trends and functional forms we identify.

Overtraining. In practice, accounting for inference costs means training smaller models for substantially longer than Chinchilla-optimal (Sardana and Frankle, ; Gadre et al., ). For instance, Llama-3-8B is trained to $D/N\approx 2000$ (Dubey et al., ) and the Gemma-2 series up to $D/N>1000$ (Team et al., ). We refer to such models as “overtrained” in this paper, with the token/parameter ratio $D/N$ being a key quantity throughout. Work on inference-time compute (Snell et al., ; Brown et al., ) and on synthetic and multimodal data (Yang et al., ; Fan et al., ; Bauer et al., ) suggests future models may be even more overtrained. Therefore, modern work on scale must consider ratios much larger than Chinchilla-optimal, and in this work we perform experiments up to $D/N\approx 10^{3}$ and analyze the predictions found by our scaling law for up to $D/N\approx 10^{5}$. See Appendix B for additional related work.

#### 2.3 Setup

We train and evaluate a suite of OLMo-style models on the Dolma V1.7 dataset (Groeneveld et al., ; Soldaini et al., ), using a standard Transformer++ implementation; see Appendix A for hyperparameters and ablations. Our experiments consist of a sweep of language model pretraining runs over $N\in[30,60,110,220]$ million parameters (non-embedding) and $D\in[1.5,3,6,13,26]$ billion tokens. Our model sizes are relatively small because we train up to a very high $D/N\approx 10^{3}$ to study data scaling and set off over 20 runs at every $(N,D)$: we sweep 8 values of precision for each of the (weights, activations, attention).

### 3 Scaling Laws for Post-Train Quantization

The easiest and most common quantization technique is post-train quantizing a model off-the-shelf (Chee et al., ; Huang et al., ; Dettmers et al., ; Lin et al., ; Xiao et al., ). In this section, we consider models trained in BF16 and use GPTQ (Frantar et al., ) to post-train quantize them, replicating our findings with two other methods in Appendix F. We quantify the resulting loss degradation $\delta_{\text{PTQ}}$, finding that post-train quantization scales poorly in data.

#### 3.1 Overtrained Models Degrade more when Post-Train Quantized

Figure: Figure 2: Loss degradation from PTQ increases with data. Top row is loss after PTQ, bottom row is loss degradation compared to end of training, before PTQ. The top row is thus the gray line in each plot plus the corresponding value in the bottom row. We can see that degradation grows with data, bottom row is fitted with Equation 2. For $D/N$ sufficiently large (left), loss can increase in data. Even at lower $D/N$, where post-quant loss continues to decrease with data, the value of data is reduced compare to the baseline. $R^{2}=0.97$ over all fitted points (bottom row).
Refer to caption: x2.png

We consider different model sizes (columns) trained on various data budgets (x-axis of each plot) and plot in Figure 2 both the loss after post-train quantization (top row) and the degradation incurred relative to end of training (bottom row). We find that the degradation $\delta_{\text{PTQ}}$ increases in training data size across all model sizes, but that for a fixed dataset size larger models incur a smaller degradation. We additionally observe that $\delta_{\text{PTQ}}$ increases exponentially as we decrease the precision we quantize to.
Based on these observations we model $\delta_{\text{PTQ}}$ as taking the form:

$$ $\delta_{\text{PTQ}}(N,D,P_{\text{post}})=C_{T}\left(\frac{D^{\gamma_{D}}}{N^{ \gamma_{N}}}\right)e^{-P_{\text{post}}/\gamma_{\text{post}}}$ (2) $$

where $C_{T},\gamma_{D},\gamma_{N},\gamma_{\text{post}}$ are positive fitted constants. As we find the fitted values of $\gamma_{D}$ and $\gamma_{N}$ to be similar (see Appendix I for numerical values), we can think of this as an approximate power law in the token/parameter ratio $D/N$. The intuition for this poor data scaling might be that as models train on more data, they compress more information into their weights, so that perturbations to weights in the form of quantization are more harmful to loss, all else equal. We discuss formal theoretical interpretations in Appendix G.

This finding implies that for models that will be post-train quantized, there exists an amount of pretraining data beyond which additional data is actively harmful to performance at inference-time (see top-left, Figure 2). This can be defined as the point where additional data increases post-train degradation more than it decreases loss during pretraining. We solve analytically for this critical data size in Appendix E. We thus summarize our first scaling finding as follows.

### 4 Scaling Laws for Quantized Training

In this section we study pretraining with weights, activations, and KV cache in various precisions. Importantly, only training precision, not test-time precision, is varied in this section; we discuss the interaction between train and test-time precision in Section 5. We sweep the training precisions of the weights, activations, and KV cache $P_{\text{w}},P_{\text{a}},P_{\text{kv}}\in[3,12]$ individually, as well as training BF16 baselines. We also pretrain models with arbitrary combinations of $P_{\text{w}},P_{\text{a}},P_{\text{kv}}$ to validate our scaling laws. To perform quantization during training, we follow the standard specification of Micikevicius et al. () unless otherwise noted (see Appendix D for implementation details).

#### 4.1 Quantization-Aware-Training: Quantizing Weights During Training has a Consistent and Predictable Effect

We first examine the trade-off between weight precision $P_{\text{w}}$ and parameters $N$ while holding $P_{\text{a}}=P_{\text{kv}}$ fixed at high precision. We fix $D=13$B tokens and perform a grid sweep over combinations of $N$ and $P_{\text{w}}$. We plot the resulting IsoLoss contours where we linearly interpolate the final loss values in Figure 3.
We observe that the bit precision of the weights can be traded off for the number of parameters, i.e., a model with smaller $N$ but larger $P_{\text{w}}$ can achieve the same loss as a model with larger $N$ but smaller $P_{\text{w}}$.
Additionally, we find that the gains from increasing the bit precision of the weights are large at lower precisions but saturate at higher precisions (typically around 6-7 bits per weight).

Figure: Figure 3: (Left) $N_{\text{eff}}/N$ from our final scaling law. Our fit of $N_{\text{eff}}(N,P_{\text{w}})$ in this section is the first step towards this (blue). Empirical (center) and predicted (right) IsoLoss contours illustrating the precision-parameter tradeoff. Y-axis is weight precision during quantized training. All runs plotted trained on $D=13$B tokens. Predictions from a fitted version of Equation 3, darker lines correspond to lower loss.
Refer to caption: x3.png

In line with the empirical trends in Figure 3, we find the best fit for the tradeoff between weight precision and parameters is $N_{\text{eff}}(N,P_{\text{w}})=N(1-e^{-P_{\text{w}}/\gamma_{\text{w}}})$, where $\gamma_{\text{w}}$ is a fitted constant measuring the sensitivity of model weights (alternative fits explored in Appendix I). We therefore modify Chinchilla scaling to account for $N_{\text{eff}}$ by making the substitution $N\mapsto N_{\text{eff}}(N,P_{\text{w}})$, giving the modified form:

$$ $L(N,D)=A[N(1-e^{-P_{\text{w}}/\gamma_{\text{w}}})]^{-\alpha}+BD^{-\beta}+E$ (3) $$

where we recall that $A,B,E,\alpha,\beta$ are fitted positive constants in the usual Chinchilla scaling form, and $\gamma_{\text{w}}$ is a fitted constant we introduce. We plot the predictions of our fit compared to observed values in Figure 4 for a range of $(N,D)$.

Figure: Figure 4: Predicting final validation losses $L(N,D,P_{\text{w}})$ for various $N,D,P_{\text{w}}$ to test our proposed functional form. Points are experimental values, lines are predictions of a single parametric fit of the form in Equation 3. We train only two model sizes at 26B due to compute constraints.
Refer to caption: x4.png

#### 4.2 Low-Precision-Training: The Effects of Quantizing Weights, Activations, and Attention are Compositional and Multiplicative

Quantization-aware training does not change the cost of pretraining. This is because modern GPUs require inputs to a matrix multiplication to have the same precision, i.e. $P_{\text{w}}=P_{\text{a}}=P_{\text{kv}}$ (Micikevicius et al., ). To understand the interplay between precision and pretraining compute we must now analyze the scaling behavior of $P_{\text{a}}$ and $P_{\text{kv}}$ as well.

Precision of activations and KV cache affect loss in a similar way. We first verify in Appendix Figure 15 that varying $P_{\text{a}}$ and $P_{\text{kv}}$ in isolation give rise to scaling behavior that is best fit by a functional form analogous to the form for $P_{\text{w}}$ (Equation 3, Figure 5, left).

We refer to the scaling coefficients computed by varying the precision of just one part of the model at a time as marginally fitted constants, and those found by fitting on runs that include multiple model components in low precision at the same time as jointly fitted constants.

Constants fitted marginally and jointly make similarly good predictions.
We now turn our attention to understanding the interactions between weights, activations, and attention. If the effects of quantizing weights, activations, and attention are independent, then a factorized, multiplicative interaction of the following form is a natural proposal.

$$ $N_{\text{eff}}(P)=N(1-e^{-P_{\text{w}}/\gamma_{\text{w}}})(1-e^{-P_{\text{a}}/ \gamma_{\text{a}}})(1-e^{-P_{\text{kv}}/\gamma_{\text{kv}}})$ (4) $$

We test whether this independence approximately holds by comparing the predictive power of a model with marginally fitted constants and a model with jointly fitted constants.
We show the predictive power of both models in Figure 5(b, c), finding that both methods for fitting constants have approximately the same predictive power.
These results suggest that the independence assumption is reasonable.
We both present further evidence that this “factorized” functional form is a strong fit to the data as well as discuss alternative factorization schemes in Appendix K.

Figure: Figure 5: (Left) Predicted loss based on fitted values with Equation 4. (center) Fitting $\gamma$ parameters jointly on sweeps with combinations of precisions vs (right) fitting them on “marginal” sweeps where only one model part is in low precision at a time. Outliers are those at extremely low precision whose training runs are sometimes unstable.
Refer to caption: x5.png

#### 4.3 Implications For Pretraining

When training in a precision $P$, meaning $P_{\text{w}}=P_{\text{a}}=P_{\text{kv}}=P$, compute cost scales linearly in $P$ (Abdelkhalik et al., ; Peng et al., )(^2^22In practice, the gains are less than linear due to systems overhead.).
Hoffmann et al. () performed all experiments in 16-bit precision and use a cost model of $C=6ND$ FLOPs. We generalize this to $C=\frac{6}{16}NDP$ to account for the linear relation between compute and precision, which reduces to the Chinchilla cost function for $P=16$. We now examine three practically relevant variants of the following optimization problem.

$$ $\min_{N,D,P}L(N,D,P)=A[N(1-e^{-P/\gamma})^{3}]^{-\alpha}+BD^{-\beta}+E\text{ subject to }C=\frac{6}{16}NDP$ (5) $$

Since derivations are algebraically involved, we will work up to proportionality and verify proposed solutions numerically. See Appendix E for mathematical details. We note that the implications of our functional form are true no matter the scale at which future experiments are done, but the numerical values we predict depend on our fitted constants which are fitted on smaller-scale, integer-type experiments.

Figure: Figure 6: Scaling law predictions (left, fitted on integer type) vs empirical values (right, floating-point type). Precision of weights, activations, attention fixed to $P_{\text{train}}$. Predictions closely match the empirical trend, but are shifted up by a small amount since floating-point is a more expressive type and will incur lower loss at the same precision. (Right) When $N$ is held fixed, compute-optimal precision increases approximately logarithmically with data. Markers correspond to predicted compute-optimal precision for Llama-3 (8b, 70b, 405b), denoted by (circle, triangle, star) at each IsoFLOP (lines), illustrating how compute-optimal precision increases in data when model size is held fixed.
Refer to caption: x6.png

##### 4.3.1 If You Must Train in Low Precision, Increase Parameters Before Data

Minimizing $L(N,D)$ with $P$ fixed, subject to $C\propto NDP$. We get with some algebra that at precision $P$ and compute budget $C$, the optimal allocations $N^{*},D^{*}$ of parameters and data relative to Chinchilla-optimal $N_{\text{Ch}},D_{\text{Ch}}$ will be given by

$$ $\frac{N^{*}(P,C)}{N_{\text{Ch}}(C)}\propto\left[1-e^{-P/\bar{\gamma}}\right]^{ -\frac{3\alpha}{\alpha+\beta}}P^{-\frac{\beta}{\alpha+\beta}}\text{ and }\frac {D^{*}(P,C)}{D_{\text{Ch}}(C)}\propto\left[1-e^{-P/\bar{\gamma}}\right]^{\frac {3\alpha}{\alpha+\beta}}P^{\frac{\beta}{\alpha+\beta}}$ (6) $$

which suggests as precision of training decreases at fixed compute, we should increase parameters and decrease data. The interpretation of this is that at very low precisions, our effective parameter count vanishes so that increasing parameter count is compute-optimal since data egregiously outstrips effective parameters.

##### 4.3.2 Compute-Optimal Pretraining Precision is in General Independent of Compute

Jointly minimizing $L(N,D,P)$ with $C\propto NDP$. This is the setting of pretraining without constraints on $N,D,P$ except for a fixed compute budget. Solving this joint minimization problem gives an implicit equation for $P^{*}(C)$. Denoting $u(P)=[1-e^{-P/\bar{\gamma}}]^{-3\alpha}$, we find (see Appendix E) that this equation takes the form

$$ $\displaystyle\frac{3\alpha}{\bar{\gamma}}\ u(P)^{\frac{3\alpha+1}{3\alpha}}e^{ -P/\bar{\gamma}}=P^{-1}u(P)$ (7) $$

which reveals that in general the optimal pretraining precision is independent of compute budget. This suggests that compute-optimal precision should be held fixed to $P^{*}$ while $N,D$ are scaled according to Equation 6. We find this $P^{*}\approx 7$ bits when fitting our scaling law on runs with quantization done to integer type. This has two consequences: first, this means the de-facto practice of training models in BF16 may be suboptimal. Second, the race to low-precision training will likely have to stop before going below 4-bits, since this would force model sizes to become disproportionately (more than 4x) larger to maintain loss scaling (see Figure 3, left).

We test our predictions in Figure 6 at a larger scale. We train compute-matched models at various parameter count and precision ranging from FP4 to FP32 and 220M to 1.6B parameters. We train in floating-point type since that is standard in pretraining (Groeneveld et al., ; Deitke et al., ), though our scaling laws are fitted on integer type. We plot our predicted trend in Figure 6 (left) and the empirical values in the middle. We find that scaling fits on integer type are a strong fit until 4-bit precision, at which points the difference between the two types becomes more apparent. Integer fits assume all bits contribute in the same way, but the split of floating-point bits into exponent and mantissa means that each likely has its own scaling behavior. The matching of qualitative trends throughout, with the optimum being close to the predicted optimum of $P^{*}\approx 7$ bits suggests that similar scaling laws may exist across types.

##### 4.3.3 But Compute-Optimal Pretraining Precision Can Increase in Compute if Model Size N 𝑁 N italic_N is Constrained

Minimizing $L(D,P)$ with $N$ fixed, subject to $C\propto NDP$. A common use case in practice is to train a suite of models of various sizes on similar data. The Llama-3 and Gemma-2 series (Dubey et al., ; Team et al., ) are examples. In this setting, $N$ is fixed in advance and only $D,P$ are jointly optimized. Surprisingly, our scaling laws predict that models of differing sizes should not necessarily be trained in the same precision, and that compute-optimal precision scales as $P^{*}(C)\propto\log C$. Since $N$ is held constant and we show in Appendix E that $\log C\approx\log D$ in proportion, we can write $P^{*}(C)\propto\log(D/N)$. The intuition for this is that, for a fixed $N$, precision acts as a new lever to bring highly overtrained models closer to pretraining optimality(^3^33An important subtlety here is that since models are overtrained for inference, we want to keep the cost of a forward pass—which is proportional to $NP$—fixed, not just $N$. While $NP$ is the same for both a model of $N_{0}$ parameters in 16-bit and one with $2N_{0}$ parameters in 8-bit, the latter has higher $N_{\text{eff}}$ with our $\bar{\gamma}$, so will reach a lower pretraining loss on the same data with the same training/inference costs.) by reducing $D/N_{\text{eff}}$.

### 5 A Unified Scaling Law for Precision

In this section, we combine the two scaling laws presented into a unified functional form that predicts both training/post-training effects, including interactions between the two. We now treat $\delta_{\text{PTQ}}$ as a function $\delta_{\text{PTQ}}(N,D,P_{\text{train}},P_{\text{post}})$ rather than just $\delta_{\text{PTQ}}(N,D,P_{\text{post}})$ as we did earlier in Section 3. We find two competing effects at play when predicting $\delta_{\text{PTQ}}$, but overall, models trained in lower precision are more robust to post-train quantization in the sense of incurring lower degradation.

Two competing effects at play during post-train quantization. Intuitively, training any of $P_{\text{w}},P_{\text{a}},P_{\text{kv}}$ in low precision forces the model to learn weights that are robust to “quantization noise,” so they degrade less under PTQ. However, the reduced $N\mapsto N_{\text{eff}}$ effective parameter count of a model trained in low precision suggests that models trained in low precision will degrade more because $\delta_{\text{PTQ}}$ increases with $N^{-\gamma_{N}}$ as we found in Section 3. We call this second effect the “overtraining” effect. In practice, the first “robustification” effect wins out, so that models trained in lower precision overall degrade less when post-train quantized compared to models trained in high precision. We confirm using $N_{\text{eff}}$ rather than $N$ to predict degradation given various training precisions leads to a substantially stronger fit in Figure 16(top left, top center), to verify the competing overtraining effect is real.

Figure: Figure 7: Combined plots for predicting degradation. (Left) demonstrates the quality of our fit on all our runs, including all combinations of pre and post-training precisions. (Center, right) illustrate visually that our unified degradation form can predict degradation when training and serving in any precision. Plots (center, right) vary $P_{\text{w}}$ only, but fits in (left) include runs where $P_{\text{a}},P_{\text{kv}}$ are also jointly varied.
Refer to caption: x7.png

Modifying $\delta_{\text{PTQ}}$ to account for training precision. We assume training precision is strictly greater than inference precision, and define degradation as identically zero if they are equal. We begin by studying how degradation scales with just weight-precision during training, $P_{\text{w}}$.

Consider Figure 7(center). We fix $(N,D)$ and each cell of the heatmap represents the empirical degradation $\delta_{\text{PTQ}}(P_{\text{w}},P_{\text{post}})$. We observe that degradation very quickly increases to its exponentially large value from Section 3 if there is any gap between training and inference-time precision. This motivates modifying our initial functional form fitted in Section 3 to

$$ $\delta_{\text{PTQ}}(N,D,P_{\text{w}},P_{\text{post}})=C_{T}e^{-P_{\text{post}} /\gamma_{\text{post}}}\underbrace{\left(\frac{D^{\gamma_{D}}}{N_{\text{eff}}^{ \gamma_{N}}}\right)}_{\text{Overtraining effect}}\underbrace{[1-e^{-C_{\text{w }}(P_{\text{w}}-P_{\text{post}})}]}_{\text{Robustification effect}}$ (8) $$

where $C_{\text{w}}$ is the only new fitted value. Then, we can extend this to include the precision effects of activations/attention in the natural way:

$$ $\delta_{\text{PTQ}}(N,D,P_{\text{w}},P_{\text{a}},P_{\text{kv}},P_{\text{post} })=C_{T}e^{-P_{\text{post}}/\gamma_{\text{post}}}\left(\frac{D^{\gamma_{D}}}{N _{\text{eff}}^{\gamma_{N}}}\right)\prod_{\text{x}\in\{\text{w},\text{a},\text{ kv}\}}[1-e^{-C_{\text{x}}(P_{\text{x}}-P_{\text{post}})}]$ (9) $$

We measure the fit to the data of such a functional form in Figure 7, and find a strong fit with $R^{2}=0.90$ on over 1000 data points (each of 465 pretraining runs post-train quantized to multiple precisions).

An interpretable, unified functional form. Now we simplify and interpret the resulting functional form. Consider training with only weights in low precision and take $C_{\text{w}}=1$ for illustrative purposes so we can simplify Equation 9. Denote $\sigma^{2}_{\text{tr}}:=e^{-P_{\text{w}}/\gamma_{\text{w}}}$ as “training noise” reflecting the decrease in effective parameter count due to training weights in lower precision. Then, Equation 9 simplifies to

$$ $\delta_{\text{PTQ}}(N,D,P_{\text{train}},P_{\text{post}})=C_{T}\underbrace{( \sigma^{2}_{\text{PTQ}}-\sigma^{2}_{\text{tr}})}_{\text{Robustification effect }}\cdot\underbrace{\left(\frac{D^{\gamma_{D}}}{N_{\text{eff}}^{\gamma_{N}}} \right)}_{\text{Overtraining effect}}$ (10) $$

which we note is the intuitive modification one might make to the form of the initial post-training quantization degradation we fitted in Section 3, in Finding 3.1, with a small competing effects factor from $N_{\text{eff}}$ pushing in the opposite direction. It cleanly reflects the intuition that models are robustified to PTQ noise to the extent they were trained with similar noise.

### 6 Conclusion and Limitations

We find that the common inference-time technique of post-train quantization can incur large degradation at very high data budgets, demonstrating a striking example of how more pretraining compute does not always imply stronger models at inference-time. Seeking better data scaling, we study quantization-aware and low precision training. We find that parameters and bit precision are well modeled as interchangeably controlling an “effective parameter count” of the model allows us to predict finite-precision loss effects accurately during both training and inference. The resulting scaling law makes surprising predictions that we qualitatively validate on language models pretrained from scratch with up to 1.7B parameters.

There are limitations to our analysis. First, we use a fixed architecture throughout to examine the effects of precision, parameters, and tokens in a controlled manner. In contrast, low precision training often involves architectural tweaks (Ma et al., ; Zhu et al., ) that can close much of the gap from a vanilla full precision model. Second, while compute costs do scale linearly with precision, the gains from halving precision are usually less than 2x due to systems overhead. Third, we only consider loss scaling without downstream model evaluations. We emphasize that the trends we find aim to be suggestive rather than prescriptive, and hope future work can more comprehensively examine these effects at larger model scale. In all, we find that the effects of precision on loss are predictable and consistent, with important and surprising implications.

### 7 Acknowledgements

Tanishq Kumar thanks Tim Dettmers, Chris De Sa, Neil Band and Luke Bailey for helpful comments and discussion. Blake Bordelon is supported by a Google PhD Fellowship. Cengiz Pehlevan is supported by NSF grant DMS-2134157, NSF CAREER Award IIS-2239780, and a Sloan Research Fellowship. This work has been made possible in part by a gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial Intelligence. Aditi Raghunathan acknowledges support from AI2050 program by Schmidt Sciences (Grant G2264481), Google Research Scholar, Apple, NSF, Cisco.

We gratefully acknowledge the support of NIH under No. U54EB020405 (Mobilize), NSF under Nos. CCF2247015 (Hardware-Aware), CCF1763315 (Beyond Sparsity), CCF1563078 (Volume to Velocity), and 1937301 (RTML); US DEVCOM ARL under Nos. W911NF-23-2-0184 (Long-context) and W911NF-21-2-0251 (Interactive Human-AI Teaming); ONR under Nos. N000142312633 (Deep Signal Processing); Stanford HAI under No. 247183; NXP, Xilinx, LETI-CEA, Intel, IBM, Microsoft, NEC, Toshiba, TSMC, ARM, Hitachi, BASF, Accenture, Ericsson, Qualcomm, Analog Devices, Google Cloud, Salesforce, Total, the HAI-GCP Cloud Credits for Research program, the Stanford Data Science Initiative (SDSI). Benjamin F. Spector is supported by a Hertz Fellowship. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views, policies, or endorsements, either expressed or implied, of NIH, ONR, or the U.S. Government.

## Appendix

### Appendix A Hyperparameter Details and Ablations

We launch over 20 runs for each $(N,D)$ combination to study scaling in precision, trained and validated on the common crawl split of the Dolma dataset [Soldaini et al., ]. We use a Transformer++ implementation: SwiGLU activations [Shazeer, ], RoPE embeddings [Su et al., ], RMSLayerNorm, Adam $\beta$ values of $(0.9,0.95)$. We adopt a cosine learning rate schedule with 10% warmup period and peak learning rate of 6e-4 for the smallest model and learning rates scaled with width and depth according to depth-$\mu$P for the larger models [Yang et al., , Bordelon et al., ]. We use a sequence length of 1024 and batch size of 256 throughout, with Adam $\epsilon$ 1e-15, following [Wortsman et al., ]. We use weight decay of $0.1$, as [Ahmadian et al., ] find some results in the quantization literature may be artifacts of insufficient weight decay. We follow [Ma et al., ] in including a LayerNorm before projections because they find it is important for low precision training to be stable. These are the hyperparameters and settings used for the main scaling law experiments.

To check robustness, we then ablate these hyperparameter choices, with results in Figure 8. In our ablation we use a sequence length of 512 with batch size 128, weight decay of 1e-3, Adam $\epsilon$ of 1e-10, a peak learning rate of 1e-4 and a warmup period of duration 3%. We train models with these alternative hyperparameters at various weight, activation, and KV cache precisions. We train and val on C4 [Raffel et al., , Dodge et al., ] instead. Though these ablations are at rather smaller scale due to compute constraints, the loss curves follow the same trends – rapid decrease in final loss with an initial increase in precision from 4 bits, then diminishing returns as we approach higher precision – as in the main text, suggesting the trends are robust to hyperparameter choices.

Figure: Figure 8: $L(P_{\text{w}}),L(P_{\text{a}}),L(P_{\text{kv}})$ for ablated hyperparameters, $N=30$M, $D=1.5$B. We can see the trends persist, where the first few bits reduce final val loss significantly, with diminishing/saturating returns quickly setting in at higher precision. We do not fit constants on these ablated runs.
Refer to caption: x8.png

### Appendix B Additional Related Work

###### Efficient training and inference

Low precision has been key to improving the efficiency of training and using LLMs [Micikevicius et al., , Shoeybi et al., , Wortsman et al., , Zhu et al., ]. Prior works generally study either precision during training [Courbariaux et al., , Dettmers et al., , , Sun et al., , Liu et al., ] or the effects of changing the precision after training (post-training quantization) [Frantar et al., , Lin et al., , Dettmers et al., , Xiao et al., , Sheng et al., , Dettmers et al., ]. In this work we study both, the precision during training and after, and unify them from a scaling perspective.

###### Large language models and scaling

By scaling up the transformer architecture [Vaswani et al., ] a variety of large language models have been proposed [Brown, , Rae et al., , Touvron et al., , , Dubey et al., , Le Scao et al., , Muennighoff et al., , , Groeneveld et al., , Jiang et al., , Zhang et al., , Allal et al., , Li et al., , Lozhkov et al., , Luukkonen et al., , Bai et al., , Chowdhery et al., , Team et al., , Üstün et al., , Deitke et al., ]. To improve our understanding of these models, various works have investigated their scaling properties [Ruan et al., , Allen-Zhu and Li, , Hägele et al., ]. Many aspects are relevant to scaling including the architecture [Tay et al., , Krajewski et al., , Tao et al., , Clark et al., , Tay et al., , Scao et al., , Peng et al., ], the modalities considered [Aghajanyan et al., , Alabdulmohsin et al., , Cherti et al., ], the performance metrics [Wei et al., , Srivastava et al., , Isik et al., ], the data composition [Li et al., , Liu et al., , Albalak et al., ] and data repetitions [Muennighoff et al., ]. Our work analyzes one such aspect, which is key to better scaling: the numeric precision during and after training.

### Appendix C Alternative Functional Forms

There are several plausible functional forms to try a priori. The key junctions are whether a form is 1) additive or multiplicative and 2) interacts with parameters/data or is independent, 3) a power law or exponential. We try a variety of combinations of these three and find the formulation in the main text one of the best fits, notably with the fewest fitted parameters. We emphasize that several fitted forms are likely to be reasonable fits to the data, and an important desiderata for choosing a functional fit is interpretability. Several scaling law papers find multiple fits plausible in terms of predictive power [Muennighoff et al., , Kaplan et al., ], and ultimately make a decision based on interpretability.

We make these fit choices on sweeps of the form $L(N,D,P_{\text{W}})$ and discuss alternatives to the decomposition/factorization to account for activations and KV cache in Appendix Section K, which assumes an effective parameter count formulation. In this section, a power law refers to a term of the form $C_{\text{w}}\cdot P^{-\alpha_{\text{w}}}$ where $C_{\text{w}},\alpha_{\text{w}}$ are fitted. In general, we find modeling precision effects with power law fits on their own causes the fitted constants $A,B$ to blow up, whereas this does not happen with exponential fits, suggesting the power law does not change sharply enough to match the change in loss induced by precision. We note that while fitting parameters using a double notion of effective parameters and effective data leads to a slightly better fit, it requires more fitted parameters so we stick with the $N_{\text{eff}}$ formulation for simplicity and interpretability. When choosing between fits we validate on held-out data and the $R^{2}$ values below reflect the fit on the held out data. This is in contrast to our plots in the main text, where we have chosen a functional form and we fit and plot on the same data, as is standard in scaling laws [Muennighoff et al., ].

**Table 1: Comparison of Functional Forms with $R^{2}$,and Number of Fitted Parameters**
| Functional Form | Val R^2 | Number of Fitted Parameters |
| --- | --- | --- |
| $N_{\text{eff}}$ | 0.82 | 3 |
| Additive/independent power law | 0.71 | 2 |
| $D_{\text{eff}}$ | 0.74 | 3 |
| $N_{\text{eff}}$ and $D_{\text{eff}}$ (tied) | 0.79 | 3 |
| $N_{\text{eff}}$ and $D_{\text{eff}}$ (not tied) | 0.84 | 4 |
| Multiplicative power law, $N,P$ | 0.75 | 2 |

### Appendix D Quantization Implementation Details and Types

Two canonical types for neural network quantization are floating-point (FP) and integer (INT) quantization. Despite their differences in representation, we hypothesize the scaling behavior between floating-point and integer quantization can be described by similar functional forms, where 1(b) provides preliminary evidence for this.

#### D.1 Integer Quantization

In integer quantization, continuous values are mapped to discrete integer values. Typically, this is done by scaling the original values according to a fixed scale factor. Mathematically, for a real number $x$, the quantized integer value $x_{\text{int}}$ is computed as:

$$ $x_{\text{int}}=\left\lfloor\frac{x}{s}\right\rceil$ $$

where $s$ is the scaling factor, and $\left\lfloor\cdot\right\rceil$ denotes rounding to the nearest integer specified by the number of bits. The value can then be dequantized back to an approximate real value by multiplying by $s$:

$$ $x_{\text{dequant}}=s\cdot x_{\text{int}}$ $$

This process introduces quantization error, defined as the difference between the original value $x$ and the dequantized value $x_{\text{dequant}}$. The goal of quantization is to minimize this error while still reducing the precision. One can think of this as rounding to the nearest point on a uniform lattice. More complicated quantization schemes involve selecting the lattice points in a data or model-dependent manner. Integer quantization, as implemented, uses a fixed-point scaling based on the maximum absolute value of the tensor, and then scales the values within the range $[Q_{n},Q_{p}]$, where $Q_{n}=-2^{(b-1)}$ and $Q_{p}=2^{(b-1)}-1$, with $b$ being the number of bits.

Integer quantization first rescales the inputs into the range specified by the number of bits by

$$ $s=\frac{Q_{p}}{\max(|x|)}$ $$

for tensor-based scaling, or

$$ $s=\frac{Q_{p}}{\max(|x|,\text{dim}=k)}$ $$

for channel-based scaling. After scaling, the result is rounded to the nearest integer and then clamped to the range $[Q_{n},Q_{p}]$. After matrix multiplication, the result is rescaled back into the original range. We use integer quantization throughout to fit our scaling laws for simplicity. However, when making pretraining predictions, we test our scaling laws using floating-point quantization since that is used for pretraining in practice.

#### D.2 Floating-Point Quantization

Floating-point quantization is slightly more sophisticated, aiming to make a non-uniform lattice roughly matching the distribution of the weights, which are assumed to be Gaussian. A floating-point number is in general represented as:

$$ $x_{\text{fp}}=(-1)^{s}\cdot m\cdot 2^{e}$ $$

where $s$ is the sign bit, $m$ is the mantissa, and $e$ is the exponent. In floating-point quantization, both the mantissa and exponent are quantized to reduce the bit width. For exponent-mantissa allocations of bits and details of exponent bias, we follow the guidelines from [Micikevicius et al., ] in terms of details, since that reflects how, for instance, FP8 training is done as is common in production settings. Further in line with [Micikevicius et al., ], we quantize weights per channel and activations per-tensor.

Making a full scaling law for floating-point quantization is more involved than our integer treatment, because the effects of scaling mantissa vs exponent bits are not the same. In contrast, in integer quantization, each additional bit simply causes us to round into a finer-grained lattice after rescaling, thereby reducing quantization error by a predictable amount. In floating-point quantization, altering the exponent affects the dynamic range, while altering the mantissa changes the precision within that range. This flexibility at once makes floating-point quantization more suitable for model training, but harder to analyze. We leave a commensurately detailed analysis of mantissa vs exponent scaling to future work.

#### D.3 Hardware Details

Weight-only quantization can accelerate inference because software can be written to accommodate moving data between GPU parts (HBM-SRAM) in smaller units (types), so that a given bandwidth can move more data per second. This reduces memory (IO) bottlenecks that often dominate during inference, even with high-batch workloads. However, we emphasize that the type and therefore speed at which the GPU can do matrix multiplications in natively is determined by the hardware provider, so that even when $P_{\text{w}}=P_{\text{a}}=P_{\text{qkv}}$ (including queries), compute savings are only achieved when these correspond with both a bit-width and type that the GPU supports. For instance, many GPUs support FP32, BF16, INT8, INT4, and the Hopper line of GPUs from NVidia support FP8 matrix multiplication, and the next generation at the time of writing (Blackwell) will support FP4 matrix multiplication. We aim to study scaling in a fairly hardware-agnostic manner so that our work may be useful in the future. We train all our models with fake (simulated) quantization on NVidia H100 GPUs.

### Appendix E Derivations

#### E.1 Critical Dataset Size for PTQ

We seek a $D_{\text{crit}}$ that satisfies $\frac{\partial L(D_{\text{crit}})}{\partial D}=\frac{\partial\delta_{\text{PTQ
}}(D_{\text{crit}})}{\partial D}$. Taking both derivatives for the functional forms presented in the main text and equating their opposing effects, we get the equation

$$ $BD_{\text{crit}}^{-\beta-1}=\gamma_{D}C_{T}N^{-\gamma_{N}}e^{-P_{\text{post}}/ \gamma_{\text{post}}}D_{\text{crit}}^{\gamma_{D}-1}$ (12) $$

which implies

$$ $D_{\text{crit}}=\left(\frac{\beta BN^{\gamma_{N}}e^{P_{\text{post}}/\gamma_{ \text{post}}}}{\gamma_{D}C_{T}}\right)^{\frac{1}{\gamma_{D}+\beta}}$ (13) $$

is the predicted point after which pretraining on more data can increase loss of a model that is post-train quantized. Note that this quantity explodes in $P$, so that a truly unreasonable amount of data is required for longer pretraining to be harmful at commonly used precisions (eg. 8-bit). However, we find that on overtrained models $D/N\gg 10^{3}$, these overtraining-degradation effects become nontrivial around 5-bits, and dominant below that.

#### E.2 Compute-optimality calculations

We set a constraint $C\propto NDP$ throughout. Working up to proportionality is essentially rescaling the compute constraint, so it doesn’t affect the scaling trends we identify, which is our focus.

##### E.2.1 Fixed Precision Compute Optimal Scaling

Under fixed precision, the loss takes the form

$$ $\displaystyle L=u(P)AN^{-\alpha}+BD^{-\beta}$ (14) $$

where $u(P)=[1-e^{-P/\gamma}]^{-3\alpha}$ is a fixed constant. The compute optimal scaling when minimizing the loss over $N,D$ gives

$$ $\displaystyle L=u(P)AN^{-\alpha}+BC^{-\beta}N^{\beta}P^{\beta}$ (15) $$

by replacing $D=\frac{C}{NP}$. Optimizing over $N$, we see that this is equivalent to the original chinchilla optimization problem but with $A\to Au(P)$ and $B\to BP^{\beta}$. Performing this optimization, we find

$$ $\displaystyle N^{*}(P,C)=\left(\frac{u(P)A\alpha}{BP^{\beta}\beta}\right)^{ \frac{1}{\alpha+\beta}}C^{\frac{\beta}{\alpha+\beta}}\quad,\ D^{*}(P,C)=\left( \frac{u(P)A\alpha}{BP^{\beta}\beta}\right)^{-\frac{1}{\alpha+\beta}}C^{\frac{ \alpha}{\alpha+\beta}}$ (16) $$

We can relate the above expressions to the original Chinchilla-optimal $N,D$ at full precision $N_{\text{Ch}}(C),D_{\text{Ch}}(C)$.

$$ $\frac{N^{*}(P,C)}{N_{\text{Ch}}(C)}\propto\left[1-e^{-P/\bar{\gamma}}\right]^{ -\frac{3\alpha}{\alpha+\beta}}P^{-\frac{\beta}{\alpha+\beta}}\text{ and }\frac {D^{*}(P,C)}{D_{\text{Ch}}(C)}\propto\left[1-e^{-P/\bar{\gamma}}\right]^{\frac {3\alpha}{\alpha+\beta}}P^{\frac{\beta}{\alpha+\beta}}$ (17) $$

##### E.2.2 Fixed model size N 𝑁 N italic_N

Now, we investigate the case where model size $N$ is fixed but precision and data are jointly optimized at fixed compute $C=NDP$. This optimization problem takes the form

$$ $\displaystyle L=u(P)AN^{-\alpha}+BD^{-\beta}$ (18) $$

Under fixed compute, we have $D=\frac{C}{NP}$ so replacing the second term, we have

$$ $\displaystyle L=u(P)AN^{-\alpha}+BC^{-\beta}N^{\beta}P^{\beta}$ (19) $$

where $N$ is a constant. We therefore have a single variable $P$ to minimize the above formula over

$$ $\displaystyle\frac{\partial L}{\partial P}=u^{\prime}(P)AN^{-\alpha}+BC^{- \beta}N^{\beta}\ \beta\ P^{\beta-1}=0$ (20) $$

First, we note that $u^{\prime}(P)$ has the following form

$$ $\displaystyle u^{\prime}(P)=-3\alpha[1-e^{-P/\gamma}]^{-3\alpha-1}\times\frac{ 1}{\gamma}e^{-P/\gamma}=-\frac{3\alpha}{\gamma}e^{-P/\gamma}\times u(P)^{\frac {3\alpha+1}{3\alpha}}$ (21) $$

We thus desire a solution to the implicit equation

$$ $\displaystyle\frac{3\alpha}{\gamma}e^{-P/\gamma}\times u(P)^{\frac{3\alpha+1}{ 3\alpha}}AN^{-\alpha}=BC^{-\beta}N^{\beta}\ \beta\ P^{\beta-1}$ (22) $$

We now aim to find an approximate asymptotic relationship between $P$ and $C$ as $C\to\infty$. Taking a logarithm of both sides, we find (neglecting additive constants that are independent of $C,P$)

$$ $\displaystyle-(3\alpha+1)\ln(1-e^{-P/\gamma})-\frac{1}{\gamma}P\approx-\beta\ln C$ (23) $$

The correct dominant balance at large $C$ is to take $P^{\star}\sim\beta\gamma\ln C$, as can be verified numerically. With the constraint that $C=NPD$ we have that $D^{\star}\approx\frac{C}{N\beta\gamma\ln C}$.

##### E.2.3 Minimization over N 𝑁 N italic_N , D 𝐷 D italic_D , P 𝑃 P italic_P with Fixed Compute

Recall our three-way loss function is given as below. We separate $N_{\text{eff}}$ into terms involving $(N,P)$ explicitly here as it makes the math easier to follow.

$$ $\displaystyle L(N,D,P)=AN^{-\alpha}u(P)+BD^{-\beta}\ ,\ u(P)=[1-e^{-P/\gamma}] ^{-3\alpha}$ (24) $$

Under the constraint $C\propto NDP$, we can replace $D$ in terms of $C,N,P$ giving the loss expression

$$ $\displaystyle L=AN^{-\alpha}u(P)+BN^{\beta}P^{\beta}C^{-\beta}$ (25) $\displaystyle\frac{\partial L}{\partial N}=-\alpha AN^{-\alpha-1}u(P)+\beta BN ^{\beta-1}P^{\beta}C^{-\beta}=0$ (26) $\displaystyle\frac{\partial L}{\partial P}=-3\alpha/\gamma AN^{-\alpha}u(P)^{ \frac{3\alpha+1}{3\alpha}}e^{-P/\gamma}+\beta BN^{\beta}P^{\beta-1}C^{-\beta}=0$ (27) $$

Multiplying the first equation by $N$ and dividing the second equation by it reveals that the optimal $P$ satisfies a compute-independent implicit equation

$$ $\displaystyle\frac{3}{\bar{\gamma}}\ u(P)^{\frac{1}{3\alpha}}e^{-P/\bar{\gamma }}=P^{-1}u(P)$ (28) $$

This exercise reveals that the compute optimal strategy when allowed to jointly optimize $N,D,P$ is to choose a fixed precision that satisfies the above equation and then to scale up $N,D$ with the prescription in Appendix I.1.1.

### Appendix F Replicating PTQ Scaling with other Quantization Methods

Here we replicate the finding that post-train degradation due to post-train quantization increases with token/parameter ratio as $D^{\gamma_{D}}/N^{\gamma_{N}}$. We fit the same functional form as in the main text, but get slightly different values of fitted constants, as expect. We replicate on AWQ [Lin et al., ] and round-to-nearest quantization. The former is a modern and sophisticated technique, and the latter a simple and naïve approach to quantization. The fact they, as well as GPTQ in the main text, share the same failure modes suggests that poor post-training quantization data scaling should be the default expectation for any newly proposed PTQ technique.

Figure: Figure 9: Replicating Section 3 results with AWQ.
Refer to caption: x9.png

Figure: Figure 10: Replicating Section 3 results with RTN.
Refer to caption: x10.png

### Appendix G Why do language models get more sensitive with overtraining?

This section is speculative.

Sharpness. A canonical line of work in optimization demonstrates that model sharpness increases during learning until it hovers at a maximal value (the “edge of stability”) [Cohen et al., , Gilmer et al., ], so that movement along the top Hessian eigenvector degrades loss by more throughout training. Though sharpness is formally a worst-case sensitivity, we conjecture similar results hold for average case, such as loss degradation induced by isotropic noise. It may be possible that sharpness during language model pretraining does not reach its maximal value for a long time, which is why sensitivity to noise monotonically seems to increase as $D/N\to\infty$ on realistic data budgets. Closely related is the largest eigenvalue of the neural tangent kernel (NTK) which captures the magnitude of the variance of the predictor under parameter noise. This quantity is known to empirically increase during training in a variety of settings, and is closely related to generalization guarantees [Nguyen et al., , Atanasov et al., ].

Hierarchical learning strategies become more sensitive throughout training. Our expectation that overtrained language models may degrade more when quantized at inference-time is motivated in part by the following results. The hierarchical nature of learning is by now well understood in some toy settings: in [Abbe et al., ], it is shown that “staircase” polynomials of increasing degree are learned faster than high-degree monomials since neural networks combine existing features to learn new ones. In [Abbe et al., ] this result was strengthened to show that such hierarchical structure is both necessary and sufficient to learn sparse functions with SGD in two layer neural networks. In this setting, damage to features encoding lower-order polynomials affects all higher-order ones, so that such networks are increasingly sensitive to fixed feature noise throughout learning. Another result of a similar flavor is that of [Barak et al., ], who explicitly require high-precision gradients for sparse parity to be learned, since sparse parity is learned by the amplification of a small initial signal. If language models learn hierarchically, it is possible that the features that are learned late into overtraining as $D/N\to\infty$ are reliant on base features, so that noise harms the base features and therefore significantly damages higher-order features.

### Appendix H Main Figure Details

The model on the left is $N=30$M parameters, chosen because we could train it to the highest token/parameter ratio given our compute budget. On the right we train a suite of models with $NP$ kept constants on 16B tokens (so that $C=\frac{6}{16}NDP$ is matched throughout under our cost model). We plot val loss on Dolma, as throughout the main text, and use floating-point (rather than integer) to make the pretraining claims as realistic as possible.

### Appendix I Numerical Fits

Following [Muennighoff et al., ], we tie $\alpha=\beta$ so they do not become very different, though this is not required. Distinct $\alpha,\beta$ only add expressivity to the model and we have verified the plots look similar without tying. We also only use the full scaling law when specified in the text, since the law is developed piecewise through the text. For instance, Figures 3 and 4 solely fit Chinchilla with a substitution $N\mapsto N_{\text{eff}}(P_{\text{w}})$ because at that point $P_{\text{a}},P_{\text{kv}}$ have not been introduced. Figures 5, 6, and 7 use our full scaling law, for instance to make predictions. We emphasize our numerical constants are unlikely to be useful because as [Hoffmann et al., , Sardana and Frankle, ] show, fitted constant depend heavily on the architecture and dataset used, which differs from setup to setup. Rather, the trends we identify are the key findings. With that said, our fitted constants are as follows.

**Table 2: Fitted constants and their values**
| Constant | Value |
| --- | --- |
| $A$ | 4.299e3 |
| $\alpha$ | 0.4965 |
| $B$ | 1.806e4 |
| $E$ | 2.7648 |
| $\gamma_{\text{w}}$ | 2.6745 |
| $n_{\text{w}}$ | 0.3037 |
| $\gamma_{\text{i}}$ | 2.2102 |
| $n_{\text{i}}$ | 1.4072 |
| $\gamma_{\text{kv}}$ | 0.9578 |
| $n_{\text{kv}}$ | 2.4185 |
| $C_{T}$ | 0.0598 |
| $\delta_{D}$ | 0.5068 |
| $\delta_{N}$ | 0.3439 |
| $\gamma$ | 0.5907 |
| $b$ | 1.1277 |

Note that we include biases in our exponent fits, for instance when modelling $N_{\text{eff}}$ as a saturating exponential, we find that the different parts of a model cause numerical instability at different values of low precisions, so even if they are the same functional form, they may be translated (left/right shifted versions) of eah other. For instance a fit of the form $e^{x/\gamma_{x}}$ in the main text is really computed with offset $e^{x/\gamma_{x}+n}$, but including biases everywhere clutters notation and obscures mathematical insight.

### Appendix J Are Weights, Activations, and KV Cache Equally Sensitive?

We find that training runs with $P_{a}\leq 3$ or $P_{\text{kv}}\leq 3$ are not numerically stable, and often diverge, while $P_{\text{w}}=3$ is still well behaved. In particular, we find activations are more sensitive, though this could be because we quantize activations per-tensor and weights-per channel following [Micikevicius et al., ], rather than activations being inherently more sensitive. Consequently, we do not fit or validate on runs with activations or attention bits equal to 3. We leave a more detailed analysis of fine-grained sensitivity across layers and types of parameters to future work. The Figure below illustrates the empirical sensitivity by plotting $L(P)$ for the three quantities for various runs $(N,D)$.

Figure: Figure 11: Sweeping $L(P)$ for the three model parts at various $N,D$.
Refer to caption: x11.png

### Appendix K Empirical N eff subscript 𝑁 eff N_{\text{eff}} italic_N start_POSTSUBSCRIPT eff end_POSTSUBSCRIPT

Figure: Figure 12: Plotting what $N_{\text{eff}}$ looks like empirically. Each black point is a pretraining run, mathematical details of what is plotted here in Appendix E. Blue lines are parametric fits of a saturating exponential.
Refer to caption: x12.png

Consider a model trained with some arbitrary $(N,D,P_{\text{w}})$. Assuming a Chinchilla function form with $N\mapsto N_{\text{eff}}(P_{\text{w}})$, we can write the difference between its loss and the loss of a full precision model as

$$ $L(N,D,P_{\text{w}})-L(N,D,\infty)=A[N_{\text{eff}}^{-\alpha}-N^{-\alpha}]$ $$

as the terms involving $B,D,E$ cancel. Note that $N_{\text{eff}}(P_{\text{w}}=\infty)=N$ by construction. In practice, we use a BF16 model as the “infinite-precision” model, finding no real difference if we use an FP32 model or even a functional fit estimating $P_{\text{w}}\to\infty$ based on our integer quantization loss results. Our goal is to plot what $f(P)$ looks like where $N_{\text{eff}}=N\cdot f(P)$. Therefore, we can rearrange the above equation as follows

$$ $f(P):=\frac{N_{\text{eff}}}{N}=\frac{1}{N}\left[\frac{L(N,D,P_{\text{w}})-L(N, D,P_{\text{w}}=\infty)}{A}+N^{-\alpha}\right]^{-1/\alpha}$ (29) $$

Then plotting this quantity using our fitted numerical values (See Appendix I) gives us the empirical tradeoff between precision and parameters. We can see that the tradeoff is quickly saturating in $P$ to a value near 1. While the functional form is the same for the three model parts, the fitted constants are different. For instance, runs with $P_{\text{a}}\leq 3$ or $P_{\text{kv}}\leq 3$ often diverged, and this was not the case with weight precision. Further, we can see that the KV cache is not sensitive to quantization at higher bit value, but very quickly becomes sensitive around 4-5 bit precision.

Then as far as the joint functional form for $N_{\text{eff}}(P_{\text{w}},P_{\text{a}},P_{\text{kv}})$ is concerned, we acknowledge that alternative factorizations that do not decompose the model into weights, activations, and KV cache, may have an equally good fit. For instance, decomposing the weights term into a product of layer-wise effects has a reasonable fit though introduces more parameters, and a more coarse-grained version may not decompose the model into parts at all, but only consider tied precisions. We choose this factorized form because QAT considers weights only, and activations and attentions are the two other things that must then be kept in low precision to see compute gains. Since practitioners often care about KV cache on its own, we chose to decompose “activations and attention” as “activations and KV cache.” We emphasize that our main point is not that this factorization is objectively correct, but in observing that such a factorization that assumes approximate independence is possible in the first place.

### Appendix L Additional Plots

Figure: Figure 13: Illustration of what finite-precision effects during training and inference look like on learning curves.
Refer to caption: x13.png

Figure: Figure 14: Predicted vs actual $\delta_{\text{PTQ}}$ for several $N,D$.
Refer to caption: x14.png

Figure: Figure 15: Marginal sweeps for precision of activations and KV cache, along with predictions from an $N_{\text{eff}}$ functional form analogous to Equation 3 fitted from scratch.
Refer to caption: x15.png

Figure: Figure 16: Combined plots for predicting degradation. (a) and (b) illustrate different fitting approaches to model degradation, demonstrating a stronger fit when $N\mapsto N_{\text{eff}}$ is used. (c), (d) (e) illustrate our unified degradation form can predict degradation when training and serving in any precision. Plots (c-e) made for varied $P_{\text{w}}$, but fits in (a) and (b) include runs where $P_{\text{a}},P_{\text{kv}}$ are also jointly varied.
Refer to caption: x16.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: potential typos: 

1. row303: P_a and =P_{kv} as well
soundness: 3
strengths: Strengths:

1. The paper tackles an important issue with the introduction of a bit precision scaling law. While this topic has been explored before, the theoretical scaling law presented in this work offers valuable guidance for the efficient deployment of models in real-world applications. The implications of this work could be transformative for the field.

2. The authors have provided a wealth of experimental results that not only validate the existing scaling laws across different model sizes but also demonstrate the generalizability of previously unseen scenarios. This thorough experimental section strengthens the paper's contributions and is persuasive.

3. The manuscript is particularly strong in its methodological rigor, with a clear articulation of the scaling laws and their implications for precision in deep learning models.
summary: This manuscript provides a thorough investigation into the impact of bit precision on inference performance and introduces a scaling law that correlates performance with precision. The paper is commendable for its extensive experimental validation. The study addresses a significant problem in the field of deep learning optimizations and offers practical insights for efficient model deployment. The manuscript is well-structured and the arguments are clearly presented
weaknesses: no clear weakness.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: I respect the amount of experiments to support that the proposed scaling law works well. However, the counterintuitive findings are more attractive to me. The paper summarizes the findings in Fig 1. Could the author further explain the underline reasons/mechanism of such counterintuitive phenomenons?
soundness: 3
strengths: (1) The paper studies a meaningful topic, the scaling laws of precision, which is a new topic following the scaling law of data and parameters.

(2) The paper gives a good presentation. I especially appreciate the introduction to quantization. I'm not familiar with how quantization works in detail, so it helps a lot.

(3) The paper shows interesting findings in Sec. 3.1 Fig. 2: more pretraining tokens result in lower performance for post-train quantization with a high quantization rate.

(4) The paper shows interesting findings in Sec. 4.1 Fig. 3: KV cache is more sensitive to the change of precision when precision is low, but when precision is high, KV cache is more robust to the change of precision compared with weights and activations.

(5) The paper shows interesting findings in Sec. 4.3 Fig. 6: there would be cases where training in low precision leads to better evaluation loss.

(6) The paper generally shows that the proposed scaling law works well in the experimental setting of the paper.
summary: The paper studies the scaling law for precision, including exploring the #parameters, #tokens, pretraining precision, and inference precision.
The paper first introduces the background via (1) giving a decent introduction to quantization, (2) presenting the existing scaling laws on #parameters and #tokens, and (3) experimental setup.
Then the paper introduces the scaling laws for post-train quantization, and quantized training, sharing interesting findings.
Finally, a unified scaling law is introduced.
weaknesses: (1) The paper uses the dataset Dolma for experiments. Though it's hard, it would be interesting to see how pretraining data affects this law.

(2) The paper uses the OLMo-style models for experiments. It would be great to give a general introduction to OLMo-style. Are they transformer-based model? While the abstract states the scaling law for language models, there would be other types of language models other than OLMo-style models, such as SSM.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: I already specified some of them above, but the questions are particularly as in the following:

- While the paper primarily focuses on integer-type quantization, you mention that floating-point quantization  is commonly used in practice, especially in pretraining. Can you elaborate on how your scaling laws might differ when applied to floating-point quantization? 

- You mention in the paper that activations and KV cache are more sensitive to low precision than weights, particularly when precision drops below 4 bits. Could you provide more detailed insights into why activations and KV cache are more sensitive? Is this primarily due to the per-tensor vs per-channel quantization method, or are there other factors at play?

- Your experiments are conducted using specific hardware such as Nvidia H100 GPUs. How do you expect the scaling laws to generalize across different hardware architectures, especially those that may handle precision differently, for example future GPUs with native support for FP4 or binary/ternary quantization?

- Given that your largest model size is 1.7B parameters, do you anticipate any limitations or deviations from your scaling laws when applied to much larger models with hundreds of billions or trillions of parameters?
soundness: 3
strengths: - The paper introduces a new dimension to the well-established scaling laws by incorporating precision as a critical factor. This is an important contribution because most prior work focused on model size and dataset size without considering precision, which is increasingly relevant due to hardware advancements supporting lower-precision computations. By doing so, the authors offer a more comprehensive framework for understanding and optimizing model performance under different training and inference conditions.

- The authors fit on over 465 pretraining runs across different precisions (3-bit to 16-bit) and sizes (up to 1.7 billion parameters), providing a robust dataset to validate their proposed scaling laws. The empirical results are consistent with the theoretical predictions, achieving high R^2 values (e.g., R^2 = 0.97 for post-training quantization degradation). 

- The paper offers actionable insights into how low-precision training can be compute-optimal, particularly in scenarios where hardware constraints or cost considerations are paramount. For example, it shows that training larger models at lower precision can sometimes be more efficient than using higher precision, which is a valuable insight for practitioners looking to optimize both performance and computational costs.
summary: This paper explores how precision -- specifically, low-precision training and inference -- affects the performance and compute cost of large language models. The authors propose new "precision-aware" scaling laws to predict the degradation in model performance when trained or quantized at different precision levels. Their work is motivated by the increasing trend toward low-precision training, driven by the need to reduce computational costs while maintaining model quality. While previous research has focused on scaling laws that balance model size and dataset size (for example Hoffmann et al. Chinchilla scaling laws), these do not account for the role of precision. The authors argue that precision is a critical factor that influences both compute efficiency and model performance, especially as hardware evolves to support lower precisions. They aim to fill this gap by developing scaling laws that incorporate precision as a third variable alongside model size and dataset size.
weaknesses: - While the paper focuses extensively on integer-type precisions (e.g., 3-bit, 8-bit), it does not explore floating-point types like FP8 or BF16 in as much depth. Given that floating-point formats are widely used in modern hardwares, this omission limits the generalizability of the findings to real-world applications where floating-point precision is common. This could limit the applicability of the scaling laws in environments where floating-point precision dominates, potentially requiring further research to adapt these findings.

- The experiments are conducted on specific hardware setups that support low-precision computations, such as GPUs optimized for integer-type operations. The fitted constants and trends may not generalize well across different hardware architectures or future technologies that handle precision differently. This may reduce the long-term relevance of the paper’s findings as hardware evolves.

- Maybe I'm missing this, but the paper suggests that compute-optimal precision is around 8 bits but does not deeply explore scenarios where precision drops below 4 bits (e.g., binary or ternary quantization). Given that future hardware may support even lower precisions, this limits the scope of the findings.

- While pretraining cost optimization is thoroughly explored, inference-time costs -- especially in real-time or latency-sensitive applications -- are not given as much attention. In many practical deployments, inference-time efficiency is more critical than pretraining cost savings. This imbalance might limit the practical applicability of some of the findings in scenarios where inference-time efficiency is more important than pretraining considerations.

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: As shown in above.
soundness: 4
strengths: 1.	The proposed scaling law unify the post train quantization and quantized training into a single functional form.
2.	The finding in the section 4.3 is inspired and the conclusions are consistent with usual experience and give a theoretical explanation.
3.	The experiment is adequate and reasonable and the paper is well written.
summary: This paper propose the scaling laws for precision through replacing the N in the original Chinchilla with the effective parameter count $N_{eff}$ and adding the post-training effects.
weaknesses: 1.	The paper use the $N(1-e^{P_{w}/\gamma_{w}})$ to fit the left in the figure 3. But I think the power law is the most commonly used in all kinds of scaling law form. I suggest the author could compare the exponential with power law like $N(1- A*P_{w}^{\alpha})$.

Meta-review: This paper investigates the scaling laws for precision, focusing on factors such as the number of parameters, tokens, pretraining precision, and inference precision. It provides a comprehensive background of an overview of quantization, a review of existing scaling laws for parameters and tokens, and a description of the experimental setup. The study then introduces scaling laws for post-training quantization and quantized training, highlighting notable findings. Lastly, it proposes a unified scaling law that integrates these insights.

The reviewers concur that this paper makes a significant contribution to the field. It excels in methodological rigor, offering a clear and well-articulated analysis of scaling laws and their implications for precision in deep learning models. The introduction of a bit-precision scaling law addresses an important issue, providing a theoretical framework that builds upon prior work. This novel scaling law offers practical insights for the efficient deployment of models in real-world scenarios, with the potential to drive transformative advancements in the field. The experimental results are convincing which not only validate the existing scaling laws across different model sizes but also demonstrate the generalizability of previously unseen scenarios.

There are also several weaknesses pointed out by the reviewers, which have been addressed during the discussion phase. Therefore, I would recommend acceptance of this work. I encourage the authors to incorporate the discussions in their next version.

Decision: Accept (Oral)