FP8 Quantization: The Power of the Exponent
Author(s)
Abstract
When quantizing neural networks for efﬁcient inference, low-bit integers are thego-to format for efﬁciency.
However, low-bit ﬂoating point numbers have an extradegree of freedom, assigning some bits to work on an exponential scale instead.
This paper in-depth investigates this beneﬁt of the ﬂoating point format for neuralnetwork inference.
We detail the choices that can be made for the FP8 format,including the important choice of the number of bits for the mantissa and exponent,and show analytically in which settings these choices give better performance.
Then we show how these ﬁndings translate to real networks, provide an efﬁcientimplementation for FP8 simulation, and a new algorithm that enables the learningof both the scale parameters and number of exponent bits in the FP8 format.
Ourchief conclusion is that when doing post-training quantization for a wide range ofnetworks, the FP8 format is better than INT8 in terms of accuracy, and the choiceof the number of exponent bits is driven by the severity of outliers in the network.
We also conduct experiments with quantization-aware training where the differencein formats disappears as the network is trained to reduce the effect of outliers.
Introduction
Neural network quantization is one of the most effective ways to improve the efﬁciency of neuralnetworks.
Quantization allows weights and activations to be represented in low bit-width formats,e.g. 8 bit integers (INT8).
When executing networks on any device, this leads to a reduction indata movement and enables the use of low bit-width computations, resulting in signiﬁcantly fasterinference and lower energy consumption.
Roughly speaking, values in neural networks are represented in either integer (INT) or ﬂoating-pointformats (FP).
Most quantization research has gone into low-bit integer formats such as INT8 and
INT4, as the corresponding hardware for this is widely available.
However, some research suggeststhere might be beneﬁts to using low bit-width ﬂoating-point formats for neural network computations.[35]
In this paper, we set out to thoroughly investigate the potential beneﬁts of the ﬂoating point formatfor neural network inference.
We show that ﬂoating point numbers add an extra dimension on topof the INT8 format that makes outliers in the quantized distribution less harmful to the overallperformance.
We investigate the effect of the quantization formats on neural network quantizationon three levels: 1) Analytically for several common data and weight distributions, 2) practically in
INT8 and FP8 post-training quantization (PTQ) settings, and 3) in quantization-aware training (QAT)settings with both INT8 and different FP8 formats.
We will show there is a strong agreement betweenour theoretical results and our practical results on real networks.
In order to compare the two formats, we introduce a novel ﬂoating-point quantization simulationimplementation that enables us to quickly run PTQ and QAT experiments with many ﬂoating pointformats.
Furthermore, this quantizer enables us to learn the FP8 exponent bias value, as well as the best trade-off between the number of bits used for the exponent and the mantissa, making good useof the ﬂexibility the ﬂoating-point format provides and providing a way to learn this automaticallywithout manual intervention.
Our conclusion from the study presented in this paper is that ﬂoatingpoint quantization can be better than integer quantization for inference, but it needs a carefully tunedbias term and correct bit-settings between the exponent and mantissa.
Background2.1 Integer quantization
Quantization is often applied to enable efﬁcient integer matrix multiplications instead of expensive32-bit ﬂoating point (FP32) multiplications.
A matrix X 2 R m ⇥ n is quantized to an integermatrix X ( int ) with an associated scale s as follows: X ( int ) =
clip
[Equation]  , where b · e 47 indicates the round-to-nearest operation, and clip ( · , · , · ) indicates the element-wise clipping operation,which ensures that X ( int ) can be represented in the chosen bit-width [29].
The dequantizationoperation then produces a matrix X ( q ) which is a quantized approximation of the input matrix X :
X ( q ) =
s X ( int ) ⇡ X .
This allows the use of efﬁcient integer matrix multiplication [29].
For amatrix Y 2 R n ⇥ k ,
[Equation]
2.2 Floating point number system
A ﬂoating point number set F ⇢ R is a set whose elements are deﬁned as follows:
[Equation]
where s 2 { 0 , 1 } is the sign bit, d i 2 { 0 , 1 } is the m -bit mantissa, p 2 Z ; 0  p < 2 e is the e -bit 55 exponent, and b is an integer exponent bias , commonly deﬁned to be 2 e  1 .
Floating point numbers can be seen as a uniform m -bit grid between two consecutive (integer) powersof two 2 a , 2 a +1 .
The distance between grid points in the range [2 a , 2 a +1 ] is 2 a  m .
Increasing thenumber of mantissa bits thus increases the number of grid points in each range [2 a , 2 a +1 ] .
In thedeﬁnition provided earlier in this section, the number of ranges [2 a , 2 a +1 ] that can be representedby a ﬂoating point number system, is determined by the number of exponent bits e .
Increasing thenumber of exponent bits thus increases the dynamic range (i.e. ratio between largest and smallestnon-zero value) of values that can be represented.
Any ﬁxed bit-width ﬂoating point number systemmust make a trade-off between the dynamic range of representable values ( e ), and the precision( m ).
For example, IEEE-754 32-bit ‘full-precision’ ﬂoating point numbers (FP32) use 1 sign bit, 23mantissa bits, and 8 exponent bits.
The resulting effect is that, compared to integer formats, ﬂoatingpoint formats have more precision close to zero, as the ranges [2 a , 2 a +1 ] will be smaller for lower
We consider three versions of the format: ﬁxed format with the bias 2 e  1 , integer bias, FP32 scale.
values of a , and less precision away from 0.
This is visualized in the left plot in Fig.
Intuitively,they are a better match for peaked distributions like Gaussians that have more density around 0, and abetter ﬁt for distributions with large tails and outliers like the Student’s t-distribution.
The right plotin Fig.
1 shows the different distributions of values for various FP8 formats.
Note that this deﬁnition does not allow for a representation of 0.
To allow 0 to be represented,the exponent value p = 0 is reserved to indicate subnormal numbers.
In this case, the exponentvalue is implicitly set to 1 , and f = (  1) s 2 1  b
[Equation]
.
Besides allowing the exactrepresentation of 0, subnormal numbers also allow a graceful representation of values close to 0.
See
Fig.
1 for an intuition behind this.
In Section 4 we show (de)quantization operations analogous tothose of INT8.
2.3 Assumptions and extensions
In this work we make a number of assumptions and consider several extensions of the standardﬂoating point format.
We assume that the exponent bias b is not restricted to the value 2 e  1 .
Instead,we introduce a (per-tensor or per-channel) quantization scale  , similar to the quantization scale usedin integer quantization [29, 23].
Since there is no standard allocation of mantissa and exponent bitsfor 8 bit ﬂoating point formats, we consider various allocations of mantissa and exponent bits, as weassume that the choice of trade-off between precision and dynamic range will have more impact forlower ﬂoating point bit-widths.
We use the notation xMyE to denote a format with x mantissa bitsand y exponent bits.
Speciﬁcally, we consider the formats 5M2E, 4M3E, 3M4E and 2M5E.
Expected quantization error
In this section we perform an analytical analysis of the FP8 and INT8 formats and show that formatswith more exponent bits perform better when outliers are a factor.
This theoretical result lays afoundation for the comparison between the formats that is predictive of when to use the FP8 formatand how many exponent bits to use.
We investigate the error induced by quantizing values drawn from several insightful distributions,considering the expected MSE of these values as it has been shown to be indicative of the ﬁnal loss ina neural network [28].
3.1 Expected quantization error
Given a quantization grid ↵ = { ↵ 1 , ↵ 2 , . . .
↵ k } , we can deﬁne the quantization operation Q ↵ ( w ) and 96 the corresponding quantization error R ↵
[Equation]
w ) :
[Equation]
Examples of the rounding error function for the 3M4E, 4M3E, and INT8 formats are shown inﬁg. 2(left).
We model neural network weights as a random variable W ⇠ p w ( w ) .
The expected value of the quantization MSE can be expressed as follows:
[Equation]
where ↵ min = min
i ↵ i and ↵ max = max
i ↵ i are the limits of the quantization range.
The ﬁrst 101 term corresponds to the rounding error, while the remaining two terms correspond to the clippingerror.
We will use this formulation to analytically compute the quantization errors for differentdistributions.
This is done by splitting the integration into sub-intervals corresponding to each pointof the quantization grid.
We present further details of this procedure the appendix A.1.
the rounding by the probability density is given in
3.2 Scalar product quantization error
We can now model the expected output error for a quantized scalar product, the essential buildingblock for both fully-connected and convolutional layers.
In order to model the activation values, we introduce a random variable X ⇠ p x ( x ) and the corresponding quantization grid  = {  1 ,  2 , . . .
k } , 110  i 2 R .
While computing the scalar product of X and W , we only consider the error incurred by 111 quantization of the inputs, as the error incurred by quantization of the output Y is identical to theinput quantization error as described in the previous section.
We denote the output tensor Y =
WX , and its quantized version as Y Q .
The quantization error
Ythen can be expressed as
[Equation]
The sample mean value  Y 2 corresponds to the mean error of the scalar product between two vectors 116 W 2 R n and X 2 R n , where W i ⇠ p w ( w ) , X i ⇠ p x ( x ) are weights and activations samples:
[Equation]
Computing E (  Y 2 ) for different distributions and quantization grids allows us to analyze how theywould work in practical scenarios.
In many cases, the expected output MSE can be approximated bythe following weighted sum of the rounded errors on W and X :
[Equation]
We see that the output error mainly depends on the quantization error of the inputs and the spread ofthe distributions of the inputs.
We describe the full procedure for computing E (  Y 2 ) and furtheranalysis in appendix A.2.
Discussion We use these formulations to analytically compute the quantization errors for severaldistributions.
We consider the uniform distribution, Gaussian distribution and the student-t distribution.
The latter is essentially a more heavy-tailed Gaussian, allowing us to model what happens whenoutliers are added into the picture.
It is important to choose the bias term in the FP8 format properly.
We see in ﬁg. 2(right) that thestandard ﬁxed format of setting the bias to 2 e  1 can fail.
This happens when values are either clippedtoo much, or not enough grid-points are used as the representable range is too big.
We also seethat having an integer bias performs worse than having a ﬂoating-point bias.
This is similar to whathappens in INT8 quantization, where setting the scale parameter correctly can have a large impacton the network’s performance [29].
As the bias shifts the dynamic range, this effect is less strongfor the FP8 formats with a high amount of exponent bits that have a high dynamic range.
For lowerexponent-bits, one would preferably use a per-channel bias/scale parameter, similar to what is donein per-channel quantization [23].
Further justiﬁcation for this can be found in section 5.2.
We also analyze the effect of different settings of the number of mantissa and exponent bits.
Theresults of this are presented in ﬁg. 3.
We observe a clear pattern.
For a uniform distribution, the INT8format performs the best.
For the Gaussian distribution, the format with 2 exponent bits performs best.
Then, when increasing the relative outliers by decreasing the degrees of freedom in the Student’st-distribution, the best format tends towards having more exponent bits.
In neural networks, most weights and activations can be modeled relatively well by a Gaussiandistribution.
So based on this analysis, we would expect the 5M2E format to work the best for mostwell-behaved layers with little outliers.
Speciﬁcally in the weights outliers are infrequent, as theweights are often explicitly, and otherwise implicitly regularized [40].
For example, we considera layer of Resnet18 model pre-trained on ImageNet.
We ﬁt Gaussian distributions in the weightsand activations sample, and compute the expected MSE analytically in ﬁg. 3 (right).
We see that the5M2E format works the best in this case.
A more detailed per-layer study is given in appendix B.1.
However, the stronger the outliers are in the distribution, the more the FP8 format with a highernumber of bits will help with its quantized representation.
We would thus predict that for networkswith severe outliers, formats like M4E3 would perform better.
We will show in section 5 that thesepredictions hold for real networks, and that networks that are known to have larger activation outlierssuch as transformers beneﬁt the most from more exponent bits.
FP8 quantization simulation
In the previous section our analysis showed that FP8 quantization can theoretically yield better tensorreconstruction MSE than INT8 quantization, but careful selection of the division between exponentand mantissa bits, as well as the value of the exponent bias are crucial.
We investigate whether theseﬁndings can be extended to FP8 quantization of neural networks.
In order to do so, we need anefﬁcient simulation of FP8 quantization.
Methods commonly used for FP8 quantization are eithertoo slow, e.g. nearest grid point search or bit-masking [20], or too complicated, e.g. custom CUDAkernels [35], for quick experimentation.
In this section we introduce a novel method of FP8 quantization simulation.
This method can easilybe implemented in common deep learning frameworks.
An added beneﬁt of this method is that itexposes the parameters of the FP8 quantizer (i.e., the number of mantissa/exponent bits and theexponent bias), thus allowing the parameters to be learned by back-propagation.
4.1 FP8 simulated quantization formulation
In our FP8 quantizer we exploit the fact that FP8 quantization can be seen as the union of m -bituniform quantization grids between consecutive integer powers of two [2 p  b , 2 p +1  b ] .
This meanswe can simulate FP8 quantization of an input vector 1 x using the same method as for simulation ofuniform quantization as described in Section 2.1, with the distinction that each element x i in x hasits own associated scale s i :
[Equation]
where x ( q ) i denotes x i quantized to ﬂoating point.
The scale s i depends on the number of mantissa bitsm and the range [2 p  b , 2 p +1  b ) in which x i falls.
This is given by log 2 s i = p
i
=
b log 2 | x i |
c
m .
To ensure that x ( q ) i can be represented given m , e and b , both x ( q ) i and s i need to be clipped.
Valuesof x ( q ) i greater than maximum value c or smaller than  c are clipped c and  c respectively, wherec = (2  2  m )2 2 e  b  1 is the largest representable value for a given ﬂoating point format.
Since
1  b  m is the smallest representable value, values of p i smaller than 1  b  m are clipped to 177 b  m .
Note that this approach is identical to the quantization operation deﬁned in Eq.
3, provided that therounding mode in Eq.
8 matches the tie-breaking procedure in Eq.
3 for numbers equidistant to twonumbers in F .
See Fig.
4 for an intuition.
Source sample text
Full algorithm =
CD = max min =
C , A , −A ) = 8 − ( −<* = 2 # − log ' A − log ' 2 − 2 * " +
H C =
detach_grad max log ' = CD + <* , 1 − ( − <*
E C = 2 E " =
CF =
E C = CD
E C
Input: =, (, A
Output: = (F) , input tensor = quantized to the grid defined by ( and A .
Notes: • The straight-through estimator is used to define gradients to =, ( and I " #
J " • The function detach_grad(⋅) indicates that its argument should be treated as a constant.
This prevents exploding gradients of the loss w.r.t.
= .
[Table]
[Equation]
In case the scaling factor  6 = 1 this needs to be reﬂected in p i .
In order to accommodate the scalingfactor, we ﬁrst fold it into a reparameterized bias value b
[Equation]
.
We then compute p i as 183 follows:
[Equation]
4.2 Quantization-aware training with FP8
To enable QAT using this quantizer, we need to make a few changes.
First, to allow gradients toﬂow through each step of the quantizer, we use the straight-through estimator (STE) for gradientsof non-differentiable rounding operations.
Second, we ﬁnd that learning the maximum clippingvalue c instead of b b improves training stability.
b b can be found from c as follows:
[Equation]
log
[Equation]
[Equation]
)  1 .
Lastly, we treat
[Equation]
as a constant that receives no gradient.
Thisprevents the (sometimes extremely large) gradients of this operation w.r.t.
x i to propagate backwards.
The result of this is that x receives the ‘straight-through’ gradient for the full quantization procedure,i.e. @@ x i F ( x i , m, c ) = 1 , where F ( · , · , · ) denotes the FP8 quantizer.
4.3 Toy experiment: Learning minimal MSE on common distributions
To investigate whether our quantizer can indeed learn the maximum value c and number of mantissabits m , we run a toy experiment.
We sample 10 5 values from N (0 , 1) , and initialize an FP8 quantizerwith 3M4E and a bias of 8, which corresponds to maximum value of c = 240 .
We then use
SGD to learn the values of c and m that minimize MSE on the reconstruction loss: L ( M, c )
=1 N
P i ( x i  F ( x i , m, c )) 2 .
After 500 iterations, c has converged to 4.35 and m oscillates around 199 5.5.
The oscillation behavior can be explained by the fact that MSE can be further minimized byincreasing precision through higher m , however increasing m to 6 yields a uniform quantizer, whichresults in higher MSE.
Similar behavior was observed in uniform quantization by [30].
A line searchshows that indeed m = 5 and c = 4 .
37 minimize the MSE on the target data, meaning our algorithmfound the optimal values in this example.
See Fig.
5 for details.
Experiments
In this section, we aim to empirically validate the analytical ﬁndings from section 3 on full neuralnetworks.
We use our analytical results to investigate various FP8 formats for weight and activationtensors in neural networks, and show that Finally, we perform quantization-aware training (QAT) on
FP8 quantized models.
5.1 Baselines and experimental setup
We run our experiments on a variety of different tasks, datasets, and models: We experiment on
ResNet18 [18], MobileNetV2 [33], and ViT [13] for ImageNet classiﬁcation [32]; BERT-base [11]for language understanding on the GLUE benchmark [38]; HRNet [34] for semantic segmentation onthe Cityscapes dataset [9]; DeepLabV3 [6] for semantic segmentation on the Pascal VOC dataset [ SalsaNext [10] for LIDAR point cloud segmentation on the SemanticKITTI dataset [2].
For eachmodel, we report the 32-bit ﬂoating point (FP32) results on the same model instance (trained weightsand code base) used in our other experiments.
As baselines we compare against models quantized using INT8 quantization.
Following [30] we donot apply batch normalization folding, and re-estimate the batch normalization statistics (runningmean and variance) before ﬁnal validation, as this improved results for every model we considered.
For each model, we consider several methods for range estimation separately for weights andactivations, per-channel and per-tensor range estimation for weights.
Results reported are those forthe best combination of range estimation setting and per-channel or per-tensor range estimation.
Ourcode is written in PyTorch and all our experiments are performed using NVIDIA Tesla V100 and
A100 GPUs.
5.2 Post-training quantization results
We compare post-training quantization using INT8 and FP8 quantization.
To allow an apples-toapples comparison, we do not use any methods to improve post-training quantization that have beendeveloped in the past years (e.g. those mentioned in [29]), other than the methods described in
Section 5.1.
For our FP8 results, we report the best fully ﬁxed format, i.e. the same combination of m, e and bused for the full network, the best ﬂexible bias format, i.e. m and e ﬁxed for the full network, but b b 232 applied per channel or tensor, whichever yields best results, and a fully ﬂexible format where m ande are set for each tensor, and b b is set for each channel, to minimize MSE.
The full procedure for thisapproach is detailed in Section C. For an example setting, see Figures 8 and 9 for an example of FP8formats that minimize MSE for each tensor in ResNet18 and BERT, respectively.
Discussion PTQ results can be found in Table 1.
The full set of results for our PTQ experiments isdetailed in Section D. In this table we can see that networks with large activation outliers (ViT, Bert,
SalsaNext and HRNet) show better results for formats with more exponent bits, and thus a largerdynamic range.
Furthermore, we see that, as predicted in Section A.1, formats with more mantissa bitsare better able to represent weight and activation tensors in convolutional neural networks.
However,since increasing the number of mantissa bits implies reducing dynamic range, ﬁnding the right valuefor the bias for each channel is important.
Lastly, we see that fully ﬂexible formats only sometimes outperform ﬁxed m/e formats, with slimimprovements when they do.
This is surprising, as a more ﬂexible format should be able to better ﬁta network’s tensors.
We attribute this discrepancy to our local greedy method for assigning m, e andb b , which may not ﬁnd settings that are optimal globally.
[Table] [Table]
5.3 Quantization-aware training
We investigate whether we can improve on the FP8 PTQ results by performing training with FP8quantization in mind (quantization-aware training, QAT).
We compare against INT8 QAT, which haspreviously been shown to result in INT8 models with accuracy near the original FP32 models.
We consider three different initializations based on our PTQ results from Table 1: best ﬁxed format,best ﬂexible bias format and best fully ﬂexible format.
For each of these initializations we perform
QAT with the format ﬁxed at initialization, with learnable maximum value c (cf.
range learningin INT8 QAT), and with learning both c and the number of mantissa bits m .
We run experimentswith various learning rates for model and quantization parameters, as well as per-tensor and perchannel quantization, and report results for the best learning setup.
We train our models for 20epochs and use Adam for the model parameters and SGD for the quantization parameters.
Resultsof these experiments can be found in Table 2.
Full experimental details and results can be found in
Appendix E.
Discussion From these results, we see that QAT can improve performance of FP8 quantized models.
We also see that learning both maximum value c and number of exponent bits m improves resultscompared to just learning weights, for fully ﬁxed and ﬂexible bias formats.
See also Figure 6 for anexample of how initialized c and m differ from learned c and m after 20 epochs of training.
However,this difference disappears for fully ﬂexible PTQ initialization.
Generally, we see that QAT reduces the accuracy gap between different formats, as the weights of thenetwork can adapt to the distributions the quantizers represent.
Related work
Integer quantization Integer quantization, sometimes also called ﬁxed-point quantization, hasbeen an active ﬁeld of research since neural networks became more popular and the need for efﬁcientinference became important.
On a high level there are two lines of work, post-training quantization(PTQ) [26, 31, 12, 1, 8, 5, 28, 25] and quantization-aware training (QAT) [17, 21, 41, 7, 27, 14, 22,
For more details we refer the interested read to the surveys of [24, 29, 16].
Our approachto learning the FP8 conﬁguration is inspired by [14] which assume the straight through estimator(STE)
[3] on the rounding operation in order to derive a gradient for the scaling factor of the ﬁxedpoint format.
[37] extends this to jointly learn the bit width and scaling factor, which we explore forlearning the trade-off between the number mantissa and exponent bits.
Floating point formats The accuracy and stability of ﬂoating point formats and algorithms are wellstudied in computer science and electrical engineering, we refer the interested read to [19].
However,these studies usually focus on high bit with ﬂoating point formats (e.g. FP32 or FP64) and do notconsider the impact on neural networks which are signiﬁcantly more robust to noise.
Early work by [39] showed that convolutional networks can be trained using FP16 and FP8 using the2M5E format.
The work of [35] introduces a hybrid FP8 format and ﬁrst discovered that 3M4E hasbetter inference performance than 2M5E. While they use a different FP8 format for the backwardspath, they keep the bias and mantissa and exponent division ﬁxed for all layers in the forward path.[20] introduces a fully ﬂexible FP8 format similar to the one we consider.
They perform a layer-wiseoptimization to ﬁnd the best conﬁguration (mantissa, exponent, sign bit, bias).
Their work doesnot go in-depth in the analysis of why some formats are better, and it has a fairly limited resultssection with easy to quantize models like ResNet18 and VGG [24].
[36] uses a custom FP4 formatfor the backwards path and INT4 for the forward path to enable full 4 bit training with a small dropin performance.
To the best of our knowledge we are the ﬁrst work with an extensive study of the different FP8 formatsbased on both analytical insights and empirical results across several tasks and data modalities.
Asopposed to our new FP8 simulation method, other works rely on dedicated FP8 simulations that donot allow for efﬁcient gradient based learning of the bias or mantissa/exponent bit conﬁguration.
Conclusion
In our analysis of the FP8 format and its beneﬁts for quantized neural network inference we havetouched on many points.
We have shown that analytically the FP8 format can improve on the INT8format for Gaussian distributions that are common in neural networks, and that higher exponent bitswork well when outliers occur.
This paper introduced a new way to simulate FP8 quantization in
FP32 hardware that speeds up FP8 quantization simulation and makes it possible to learn the biasand mantissa-exponent bit-width trade-off.
We validated the FP8 format for many networks in apost-training quantization setting, showing that generally for neural networks the 5M2E and 4M3E
FP8 format works the best, and that for networks with more outliers like transformers increasingthe number of exponent bits works best.
We have also shown that when doing quantization-awaretraining, many of these beneﬁts of the format disappear, as the network learns to perform well for the
INT8 quantization grid as well.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes](c)
Did you discuss any potential negative societal impacts of your work?
[N/A]
To the best of our knowledge, efﬁcient representations for neural network do not have anynegative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A] No theoreticalclaims.(b)
Did you include complete proofs of all theoretical results?
[N/A] No theoretical claims.3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] All models anddata is publicly available, the code of our FP8 simulation is proprietary.(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] See section 5.1 and supplementary material.(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] Most results are deterministic and only depend on thepre-trained model (with exception of QAT results in Table 2.(d)
Did you include the total amount of compute and the type of resources
(a) If your work uses existing assets, did you cite the creators?
[Yes]
[Table]
(b) Did you mention the license of the assets?
[N/A]
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
you include the full text of instructions given to participants and screenshots, if [Table] applicable?
[N/A]
[Table]
you potential participant risks, links to Institutional Review
[Table]
Board
approvals [Table] you the estimated
wage paid
participants the amount
[Table]
spent on participant compensation
[N/A] [Table]