FlexRound: Learnable Rounding by Element-wise Division for Post-Training Quantization
Authors: Jung Hyun Lee
Jeonghoon Kim
Se Jung Kwon
Dongsoo Lee
Venue: Submitted to ICLR 2023 (2023)
Keywords: Efficient Inference
Quantization
Post-Training Quantization

Abstract: Post-training Quantization (PTQ) has been gaining popularity for the deployment of deep neural networks on resource-limited devices since unlike quantization-aware training, neither a full training dataset nor end-to-end training is required at all. As PTQ schemes based on reconstructing each layer or block output turn out to be effective to enhance quantized model performance, recent works have developed algorithms to devise and learn a new weight-rounding scheme so as to better reconstruct each layer or block output. We notice that, however, such new rounding schemes are established on element-wise addition. In this work, we propose a simple yet effective new rounding mechanism for PTQ, coined FlexRound, via element-wise division to learn not only a common quantization grid size but also a different scale for each pre-trained weight. Thanks to the reciprocal rule of derivatives induced by element-wise division, FlexRound is inherently able to exploit the importance of a pre-trained weight when updating its corresponding scale, and thus, flexibly quantize a pre-trained weight depending on its own importance. We empirically validate the efficacy of FlexRound on a wide range of models and tasks. To the best of our knowledge, our work is the first to carry out comprehensive experiments on image classification, natural language understanding, and natural language generation in the per-tensor uniform PTQ setting. Our code will be open-sourced soon.

Under review as a conference paper at ICLR 2023
FLEXROUND: LEARNABLE ROUNDING BY ELEMENT- WISE DIVISION FOR POST-TRAINING QUANTIZATION
A Anonymous authors Paper under double-blind review
ABSTRACT
Post-training Quantization (PTQ) has been gaining popularity for the deployment of deep neural networks on resource-limited devices since unlike quantization- aware training, neither full training dataset end-to-end training is required at all. As PTQ schemes based on reconstructing each layer or block output turn out to be effective to enhance quantized model performance, recent works have developed algorithms to devise and learn new weight-rounding scheme so as to better reconstruct each layer or block output. We notice that, however, such new rounding schemes are established on element-wise addition. In this work, we propose a simple yet effective new rounding mechanism for post-training weight quantization, coined FlexRound, via element-wise division to learn not only a common quantization grid size but also different seale for each pre-trained weight. Thanks to the reciprocal rule of derivatives induced by element-wise division, FlexRound is inherently able to exploit the importance of pre-traiie pre-trained weight when updating its corresponding scale, and thus, flexibly quantize pre-trained weight depending on its own importance. We empirically validate the efficacy of FlexRound on a wide range of models and tasks. To the best of our knowledge, our work is the first to carry out comprehensive experiments on not only image classification and natural language understanding but natural language generation in the per-tensor uniform PTQ setting. Our code will be open-sourced soon.
INTRODUCTION
Recent years have witnessed the unprecedented success of deep neural networks in wide variety of domains including computer vision, natural language processing, automatic speech recognition, and so on. Although state-of-the-art deep neural networks surpass human-leve performance, these neural networks cannot help requiring more and more computation cost and memory usage as networks become deeper and wider. In order to reduce the model size and accelerate inference operations, many researchers have attempted diverse compression techniques such as network quantization (Courbariaux al. 2016) and network pruning (Han et al.) (016). this paper, we concentrate on network quantization due the advantage that INT4 or INT8 quantization allows to accelerate quantized neural networks using off-the-shelf accelerators such as the NVIDIA A100 Tensor Core
GPU (wu et al. 2020) ARM Cortex MCUs (Kim al. 2021).
Network quantization techniques can be generally divided into two categories: quantization-awee aware training (QAT) and post-training quantization (PTQ). When quantizing neural networks via QAT (Jung et al. 2019 Jain et al., 2019: Zhao et al., 2020) Esser et al., 2020 Lee et al.] 202T), the performance gap between a full full- preciseoo neural network and its quantized counterpart can be marginal. Yet, QAT requires end-to-end retraining or fine-tuning on full training dataset, which often causes an enormous amount of time and resources obtain a quantized neural network with competitive performance. Furthermore, a whole training dataset may be available due to data privacy issues or demands to utilize legacy models. Such drawbacks of QAT the reasons why

---

Under review as a conference paper at ICLR 2023
researchers recently pay more attention to PTQ (Zhao et al. 2019; Wang et al., 2020 Nahshan et al. 2021 that needs neither a full training dataset nor end-to-end learning at all.
PTQ had been initially performed via rounding-to-neares scheme by minimizing the quantization error in the parameter space. Unfortunately, this approach suffers from severe performance degrada- tion. Since it is reported that the loss degradation resulting from quantization can be approximated as the second-order error in Taylor Expansion by viewing quantized weights as perturbed weights, Nagel et al. (2020) and Li al. (2021) substantiate that reconstructing each output of layer or block equivalent minimizing the approximation of loss degradation resulting from quantization under some assumptions. Accordingly, recent works (Nagel et al. 2020 Li et al. 2021 Hubara al.
2021 Wei et al. 2022) have suggested reconstruct each output of layer or block by devising and learning new weight-rounding scheme, deviating from sounding-to-neareee as an effort to preserve the performance of a full-precision model. However, all those new rounding schemes designed in existing studies either round or quantize pre-trained weights adaptively via element-wise addition.
Changing the perspective of a new rounding policy from element-wise addition to element-wise divi- sion, we propose a simple yet effective post-training weight quantization method called FlexRound, which flexibly quantizes pre-trained weights by learning how much each pre-trained weight should be divided by. Interestingly, thanks the reciprocal rule of derivatives induced by element-wise division, FlexRound can inherently leverage pre- trained weights when updating individual scale for every pre- pre-trained weight. Specifically, we corroborate that relatively wider range of discrete values needs to be explored when quantizing pre-trained weights of large magnitude. The rationale behind such an approach is that the magnitude of weight can be considered as its importance. Given that it crucial to retain the knowledge of important weights even after quantization so as to maintain the performance of a pre-trained model, the constraints associated with quantizing weights of large absolute value should relaxed compared to those of small absolute value (i.e., those important weights can be quantized to one of not only its two nearest discrete values but also discrete ones far from it). Accordingly, FlexRound quantizes pre-trained weights flexibly depending on each their own importance, thereby leading better performance.
Our contributions are threefold:
We propose FlexRound as a new rounding scheme for post-training weight quantization based the principle of element-wise division to enable learning separate scales for all pre-trained weights as well common quantization grid size across group (e.g., a channel or layer).
We demonstrate that such a new rounding scheme via element-wise division takes into con- sideration the importance of pre-trained weights when updating their corresponding scales so that FlexRound can quantize pre-trained weights of large magnitude (i.e., important pre-trained weights) more flexibly.
To the best of our knowledge, we are the first to conduct extensive experiments in the form of per-tensor uniform PTQ reconstruction on natural language generation as well as image classification and natural language understanding. We verify the effectiveness of FlexRound using numerous models such as ResNet, MobileNetV2, BERT, GPT-Neo, and OPT.
RELATED WORK
Recently, many researchers have attempted to quantize wide range of models for various tasks such as vision and language understanding/generatitn without any (re)training. OCS (Zhao et al. 2019) replicates channels entailing outliers, and then, halves outliers of those channels. Unfortunately. even though OCS explicitly addresses outliers, it still suffers from severe accuracy degradation when both weights and activations are quantized into low-bit. As an alternative solution, Wang et al. (2020) proposed Bit-Split that splits an integer into several bits and optimizes them separately. Although Wang al. (220)) showed that the performance of Bit-Split is close to that of a full-precision model in the low-bit setting, Bit-Split may not be effective for certain architectures including MohileNerV2.
2

---

Under review as a conference paper at ICLR 2023
To overcome the limitations discussed above, Nagel et al. (2020) and Hubara et al. (2021) minimize the mean squared error (in a layer-by-layer fashion) between the full-precision layer's output and its quantized layer's output by inventing and learning new weight- troggiig mechanism dubbed as AdaRound and AdaQuant, respectively. As such layer-wise reconstruction error minimization opens the door to 4-bit PTQ regime, Li et al. 22221) proposed block-wise reconstruction, titled BRECQ. to consider cross-layer dependency along with the possibility of fully quantizing MobileNetV2 into 4-bit. In addition block-wise reconstruction, Wei et al. (022) proposed Qrrop that drops the quantization of activations at random during reconstruction to induce activation quantization to be synchronized with weight quantization. Both BRECQ and Qrrop, however, based on AdaRound, which cannot learn quantization grid size while quantizing weights allows for rounding either up down only at most. AdaQuant quantizes weights adaptively. AdaQuant, however, does not consider the magnitude of weights for quantization that turns out to be important as we discuss later.
As another line of post-training quantization (PTQ) research, some PTQ techniques are specialized in quantizing language models such as BERT and GPT-like models. [Bondarenko et al. (202T) first applied PTQ to BERT by introducing per-embedding- activation quantization scheme to deal with highly dynamic activation ranges. [Bai al. (202T) studied the PTQ reconstruction parallel for BERT. Yao al. (022)) proposed ZeroQuant that quantizes BERT and GPT-3 in group-wise weight quantization manner driven by token-wise activation quantization via layer-by-layer knowledge distillation. Dettmers et al. 2022) quantizes large language models like OPT with vector-wise weight quantization and mixed- mixed-precision precision decomposition with FP16 activation. All those methods do not consider per-tensor weight quantization which can enable integer matrix-to-maite multiplication API/function calls (Migacz 2017).
Most the aforementioned PTQ studies are targeted to either vision models or language models only, but not to both. Most experimental results the above PTQ works are conducted via channel- vise//rroupwwiseveecttrrvv wise/vector-wiin weight quantization at the expense of reduced parallelism. To the best of our knowledge, our work is the first carry out extensive experiments on diverse tasks ranging from image classification to natural language generation assuming per-tensor uniform PTQ setting.
METHODOLOGY
In this section, we first present the notations used in the paper, describe the concept and design of FlexRound for per-tensor uniform post-training quantization (PTQ) reconstruction, and then, scrutinize how FlexRound can leverage the importance of a pre-trained weight.
3.1 PRELIMINARIES
Notations. A scalar, vector, and matrix (or tensor) expressed non-bold letter, small bold letter and a capital bold letter (e.g. 8, 8 and S) respectively. W indicates the quantized counterpart of W. The input to convolutional or fully-connected layer is denoted as X if all previous layers are intact or as X all previous layers are quantized. The (i.j) element of matrix W is represented as W(..))- We let @ and / indicate element-wise product and element-wist division, respectively, similar to the broadcasting process in Python Numpy. and [-] express the rounding represents the Frobenius norm.
function and the floor function.
The conventional uniform PTQ approach is to quantize pre-trained weights W with respect to the quantization grid size 31, but the minimization of quantization error in the parameter space is not equivalent to that of the final task loss. On the grounds that Li al. (202T) proves that the loss degradation resulting from quantization can be approximated the quadratic form of the network output and its Hessian matrix, several existing studies have strove to minimize W X W X || layer-by-layer or block-by-bloch with respect to continuous variables V with only small amount
PTQ Background. W to be W = $1 via rounding- to-nearest and to minimize || |
3

---

Under review as a conference paper at ICLR 2023
I
Mipe
Backward
Common Scoling Factor
Ricu
10 =
wx
Stua
Pa.
Aqui
Wioe
(a) A new rounding scheme via element-wiss division. Both $1 and S are updated toward minimizing the reconstruction error, L.
(b) Rounding functions with learned parameters and S as shown in (a).
Figure 1: Illustration of FlexRound in the per-tensor uniform PTQ reconstruction. As seen in (b), FlexRound flexibly quantizes pre-trained weights by observing (22.4) < ((3,2) but W(2.)) > (3.2))
of data, where W either 81(
+ h(V)) with certain function I(-) (Nagel et al. 2020
81 W+V
Hubara et al. 2021 However, all these aforementioned rounding mechanisms are
founded element-wise addition.
3.2 FLEXROUND
Unlike prior works based on element-wise addition, exploit element-wise division for quantizing pre-trained weights. We can formulate our proposed weight-rounding scheme via element-wise division as follows:
W
51
S
where the shape of S is equal to that of W while all entries of as well as the quantization grid size 81 are positive and learnable, Similarly to preceding studies, both 81 and are updated as an attempt to minimize ||WX- P
Eq. D implies that the basic formula FlexRound supports per-tensor uniform PTQ. Notice that although FlexRound can adopt per-channel weight quantization scheme simply by replacing scalar 81 with vector $1, since we show later that per-tensor uniform PTQ (using FlexRound) is enough to provide the accuracy of full-precision model, we single quantization grid size 8 for each layer (Per-tensor quantization schemes might enable integer matrix-to-matrix multiplication API/function calls that can facilitate efficient inference of quantized models. (Migacz) 2017 From on, thus, we study only the per-tensor uniform PTQ reconstruction. The overall procedure of FlexRound is described in Figure !
Now let us discuss how to design S. Let W E RoCs Cin in the case of a fully-conmected layer and W E XC.n
in the case of con- volutional layer. We first start formulating S as S si 3 S2 where Sa E R RC CouttCin in the case of fully-connected layer and S₂ E RCouxx X HxW in the case of a convolutional layer while all elements of S2 are learnable. Then, motivated by wide acknowledgement that the statistics of output chan- nels can vary greatly (Nagel al.) 2019 Lou al.) 2222)), account for the variation of output channel's statistics by complementing S with additional learnable tensor $3, where $3 &RCoue x in the case of a fully-connected layer and 83 E RR xIxxx1 in the case convolutional layer. For convolutional layer, additionally complemented by another learnable tensor 84, where 8, IRIx x1x1 Consequently, S is formulated 81 @ S2 83 for fully-connected layer as displayed in Figure and 81 ④ S2 83 84 for convolutional layer.
Figure 2: Formation of S for linear layer.
4

---

Under review as a conference paper at ICLR 2023
Accordingly, quantization process for FlexRound can be expressed
81
if W a fully-connected layer
W
81
81
W
if W a convolutional layer
#1
where all entries of 22, $3, and 84 are initialized to ones order to enable learning S2, S3, and $1 from rounding-to- to-nearest, $1 subject to the constraint that all elements of 81, S2, $3, and 84 are positive.
W $1, S2. 83. and 84 are updated to minimize [WX X W
Since S2, and 8, are all learnable and FlexRound does not need any explicit regularization terms, no additional hyper-parameter necessary, and thus, FlexRound would be convenient for practitioners. Moreover, as all entries of 81. Sy, 83, and 84 are positive and FlexRound is based on element- wise division, FlexRound encourages W to employ the same sign as W. Hence, FlexRound prevents extreme changes of weights through quantization process unlike some clement-wise addition rounding scheme such as AdaQuant (Hubara et al. 20211
4 EXPERIMENTS
In this section, present experimental results for benchmark datasets and network models in computer vision and natural language processing tasks. We first empirically confirm that additional tensors 83 and 84 introduced in Section 3.2 implement distinct contributions in the per-tensor uniform post-training quantization (PTQ) setting. Then, we compare the performance of FlexRound with that of some state-of-the-art PTQ approaches in the following cases: image classification on the ImageNet (Russakovsky et al. 2015) dataset with the ResNet (He et al.) 2016) and MobileNetV2 Sandler et al. (0188) architectures (Section 4.3). natural language understanding (NLU) on the GLUE (Wang et al. 2018) benchmark with the BERT (Devlin et al., 2018) and GPT-Neo (Black et al. 2021 architectures (Section 4.4). and natural language generation (NLG) on WikiText2 (Merity al. 2016) and Penn Treebank (PTB) (Mareus al.) 1993) with the GPT-Neo and OPT (Zhang al. 2022) architectures (Section 4.4).. For brevity, let "B + X" and "Q+ X" indicate that certain rounding scheme 'X' is performed in the experimental setup described in BRECQ (Li et al. 2021 or QDrop (Wei et al.) (022), respectively (an experimental setup includes the definition of block unit for reconstruction error minimization or how much the probability of dropping the quantization of activations is). As introduced in BRECQ and QDrop, we also utilize the LSQ technique (Esser et al. 2020) when updating an activation step size for activation quantization. Throughout our comprehensive experiments, we verify that FlexRound can achieve competitive performance with a full-precision model for the above tasks even in the per-tensor uniform PTQ reconstruction, which has been introduced previously. All experimental results in this section are conducted by our own implementation based open-source codes.
4.1 LEVERAGING THE IMPORTANCE OF A PRE-TRAINED WEIGITT
As we discussed previously, either element-wise addition or element-wise division effective to produce better rounding scheme than a rounding to the nearest scheme. order to investigate the difference between element-wise addition and element- division, it would be instructive to analyze the gradient of the reconstruction error L | X W X || with respect to S' (where S' is S2 83 for fully-connected layer and S₂ 83 @ for convolutional layer). Through analysis, unlike clement-wiss addition, we show that element-wise division enables 05 leverage the importance of pre-trained weights W. as follows
Using the straight-through estimator (Bengio al. 2013), for every i and j.
ac (S
directly
proportional to W (8.d) aWa..
ac
which implies that St (6.)) is (partially) affected by W(.))- As result,
For simplicity, we take into account the of fully-connected layer.
5

---

Under review as a conference paper at ICLR 2023
W|
W
Amount of grid shifts from RTN grid
150
AWs
Agpresstve tenabled by FlacRound)
10
AW
AW
W
(a) MobileNetV2
[W| <
W >1
Amount grid shiftss from RTN grid
count
27011
count
AW
Aggressive rounding
(onabled RexRound)
Round down
-2
AW
AW
W
(b) ResNet-18
Figure 3: Weight updates through FlexRound of the first convolutional layer in the first block of (a) MobileNetV2 and (b) ResNet-18 18, after quantizing pre-trained weights into 4-bit (by FlexRound) while activations are kept in full-precision.
W (1J)
WEL SIG (v.3)
can also be updated and influenced by W(()) as well. In other words, as the
magnitude of pre-trained weight W(s.) is larger, the chance of W (t.j) receiving larger update becomes higher during the PTQ reconstruction. In light of the fact that the magnitude of a weight can be regarded as a metric measure importance during compressing a neural network (Han et al.) 2013 Zhu & Guptal 2017). if the goal is to enhance model accuracy after quantization, it would be reasonable to have less important (that is, smaller magnitude) weights rounded either up or down only while allowing more important (i.e., exhibiting larger magnitude) weights to be quantized to one of the two closest quantization grids or more.
Figure 3 presents the amount of weight updates through FlexRound for MobileNetV2 and ResNet-18 18. On the left side and the center side of Figure 3 histograms describe the change of W (i.j) grouped for small pre-trained weights (WII < 1, left) and large pre-trained weights ([W) > 1, center). On the right side, scatter plots show the amount of grid shifts from the grids obtainable by the rounding-to- nearest (RTN) scheme. We note that MobileNetV2 and ResNet-18 18 are quantized distinctively due to FlexRound. For example, in the W (K.)) attained by minimizing L is more aggressive (i.e., rounding can be deviated by more than one-step up or one-step down) when the absolute value of W(()) is larger than one, which means that FlexRound more flexibly quantizes pre- trained weights of large magnitude as illustrated in red dotted squares in Figure ((a) The amount of aggressively rounded weights in the first convolutional layer of the first block of MobileNetV2 is around 12.8% of the total. For ResNet- 18, however, there are no pre-trained weights whose magnitudes are larger than one. Thus, most pre-trained weights are rounded either up or down as shown Figure 33b) (e.g., only about 1.5% weights are rounded aggressively in the first convolutional layer of the first block of ResNet-18). Different rounding results by FlexRound, AdaRound, and AdaQuant are visually compared in Appendix A
of MobileNetV2 illustrated in Figure (3)a the change of
4.2 ABLATION STUDY
To justify the introduction of sy and 84 on FlexRound in the per-tensor uniform PTQ setting, we investigate the impact 83 and 8₁ on the performance of FlexRound using the ImageNet dataset with pre-trained weights quantized into 2-bit (activations are not quantized). As shown in the last two rows in Table the presence of 83 and 8, enhances the accuracy for all models. Interestingly, FlexRound outperforms both AdaQuant and AdaRound even without 83 and 84, which would support

---

Under review as a conference paper at ICLR 2023
Table I: Top-1/Top-5 accuracy (%) on ImageNet by R8sNet-18, ResNet-50,, and MobileNetV2 with only weights quantized into 2-bit. "B + X" denotes the implementation in the setting of BRECQ. We employ pre-trained models available from the official PyTorch repository.
Method
ResNet-18
ResNet-50
MobileNetV2
B AdaQuant
1.13/4.10
0.12/0.60
0.10/0.50
AdaRound
63.01 03.01/85.1 88.20
68.31 08.31/88.90
88899
33.10/60.58
B FlexRound without $3, 84
63.19/85.
70.00/ 89.82
34.75/62.51
B FlexRound with 83, 84
63.73/85.41
70.57 90.077
38.09/66 64.90
Table 2: Top-1/Top-5 accuracy (%) for ResNet-18, 18, ResNet-50, and MobileNetV2 on ImageNet when only weights are quantized. "B + X" expresses the implementation of X in the BRECQ's setting. We employ pre-trained models available from the BRECQ github repository
Method
Bits (WJA.)
ResNet-18
ResNet-50
MobileNetV2
Full-precision
32/32
71.00/89.99
76.63/93 /93004
72.62/90.66
B + AdaQuant
4/32
67.50/87.1
72.79/90.77
15.17/32.8
B + AdaRound
4/32
70.18/89, /89.38
75.86/92.00
69.46/ 446/88.88 88.85
B + FlexRound (Ours)
4/32
70.28
10.28/89.44 89.44
75.95/99 29.95/92.68
70.82 89.67
B + AdaQuant
3/32
57.09/800 88..2
52. 52.13/75.22
0.20/0.79
B AdaRound
3/32
68.79/88.69 62
74.31/91.81
62.51/84.52
B FlexRound (Ours)
3/32
68.65/88.54
74.38 74.38/91.81
/91.81
66.87 66.87/87.56 87.56
B AdaQuant
2/32
0.23/0.92
0.10/0.50
0.10/0.50
B AdaRound
2/32
61.99/84.8
48.47/77.09
39.57/66.15
B FlexRound (Ours)
2/32
62.57 84.84
63.67/85.72
46.04/72.48
our claim that new rounding scheme, shifted from element-wise addition to element-wise division, is the key to improving quantization quality significantly.
4.3 RESNET- 18, RESNET-50, AND MOBILENETV2 ON IMAGENET
In this subsection, quantize ResNet- 18, ResNet-50, and MobileNetV2 in the low-bit PTQ recon- struction with 1024 randomly sampled images. Linear symmetric per-tensor quantization format is assumed to quantize weights and/or activations. For FlexRound, the output of each layer or block is reconstructed during 5k iterations while all learnable parameters (i.e., 51. S2, 8g, and 84) are updated by using one learning rate (e.g., 4e-4 for ResNet models quantized by 3-bit or 4-bit, or 1e-3 for the ResNet models quantized by 2-bit and MobileNetv2). The first and last layers are quantized into 8-bit and the batch normalization layer is folded into convolution, as done Li al. (021)) Our experiments performed based on full- full-precision pre-trained models available from the BRECQ [ and we report the median over five random trials.
al. 2021 github repository
Assuming the quantization of weights only, we compare FlexRound with AdaRound and AdaQuant that utilize the principle of element-wise addition to decide rounding operations. Table 2 shows that FlexRound consistently outperforms those two addition-basee rounding policies. Note that the performance of AdaQuant is inferior to that of AdaRound in Table Correspondingly, FlexRound would be compared to AdaRound only to save space hereafter. Table provides model accuacy when AdaRound and FlexRound (to quantize both weights and activations) are associated with the should be noted that FlexRound is particularly successful
settings of BRECQ or QDrop. In Table for MobileNetV2 incorporating weights of large magnitude, for the reason that we explained in Section 4.1 It also interesting to see that even when both weights and activations of the ResNet
https://github/ www.hhhh....m.hhhhhh.c.. B/RRCQ
7

---

Under review as a conference paper at ICLR 2023
Table 3: Top-1/Top- accuracy (%) for ResNet-18, ResNet-50, and MobileNerV2 on ImageNet when both weights and activations are quantized. "B + X" and "Q + Y" represent the implementation of the BRECQ's setting and that of Y the QDrop's setting, respectively, We employ pre-trained models available from the BRECQ github repository.
Method
Bits (W.A.))
ResNet-18
ResNet-50
MobileNetV2
Full Full-precision
32/32
71.00/ 89.97
76.63/93 /93.04
72.62/90.66
AdaRound
4/4
69.18/88. 1488888885 8.85
74.44/91.80
61.05/83.30 /8..00
B FlexRound (Ours)
4/
69.32/ 88 83
74.56/91. /91.
63.74/85
Q AdaRound
4/4
69.20/ 88.96
74.90/92.15
65.42
65.42/86.23 /6.223
Q FlexRound (Ours)
4/4
69.26/ 88.81
75.08/92 20
66.66 (666/88.22 21
B AdaRound
3/3
64.83/86 1..12
67.01/87.28
3.74/11.54
B FlexRound (Ours)
3/3
64.99 88.93
68.29/87.88
25.43 48.28
Q AdaRound
3/3
65.71 86.96
70.49/80 99.993
39.86/66 /6.00
Q + FlexRound (Ours)
3/3
65. 43.43/ 86.60
70.74/89.78
51.49/76. 96.90
Table 4: Performance of BERT Base. BERT Large. on the GLUE benchmark. For evaluation metrics, matched and mismatched accuracies are reported for MNLI, Fl score and accuracy are reported for QQP, Mathews correlation is reported for CoLA, Pearson and Spearman correlations are reported for STS-B, and accuracy is reported for the others. "Q + X" indicates the implementation of X in the QDrop's s setting.
Dataset
Method
BERT
BERTLARGE
GPT-Neo1zMM
GPT-Neo; JB8
GPT-Neog 78
Full-precision
84.49/85.20
66.05/85.98
79.11/79.63
85.12/86.04
86.36 87.02
MNLI
QoAdaRound
83.69/88 /84.61
85.75/85.86
72.67 /7.111
84.90/85.82
86.33/ 886
Q+FlexRound (Ours)
84.53 /84.88
85. 95.93/ 19/88.99
72.94/74.24
85.5 85.56/86
86.41 //6.89
Full-precision
88.06/2 /91.08
88.66/91 /9.59
85,20/88.99
29,12/9
88.62/ 88.62/91..0 29.19
QQP
Q+AdaRound
87,65/ /90.58
87.45/90.62
77.35
87.98/91.00 /91.0
88.38 991.27
++FlexRonnd (Ours)
87. 81/ //00.83
88 /91.31
73.75/88066
88.27 88.27/91.J3 /91.18
88 /91.999
Full-precision
91.25
92.15
85.
91.36
92. 46
QNLJ
QoAdaRound
91.16
92.24
80.87
91.40
92.04
Q4FlexRound (Ours)
91.16
92.04
80.52
91. 54
92 50
Full-precision
93.00
93.35
94.50
SST-2
Q+AdaRound
92
00.00
92.55
81
Q+FlexRound (Ours)
92.43
93.
83.03
93.
94.04
Full-precision
58.55
63.57
37
57.42
58 88
CoLA
Q+AdaRound
56.79
54.30
20.15
58.93
57.14
Q+FlexRound (Ours)
57.53
60.57
21.59
59.
57.37
Full-precision
88.52 /88.20
88.98
88.98/88.89 /88.88
79.87/80.1
88.94/88.90
89.75, /89.82
STS-B
QtAdaRound
88,00 /87.53
86.87/ 11.577886.60 //6.6G
68, 55, 28..22
88 97 007/88.7 77
89.03/ 88888888 91
RoFlexRound (Ours)
88 29 88.991
88 82/82/888 32//88.. 76
67. 66//6.33
88 88.82/88.55
89.06/88 (8/888.. 69
Full-precision
85.05
85.5 55.54
80.15
85. 05
87.99
MRPC
Q+AdaRound
81.62
82.35
75.25
84.80
85.78
Roule (Ours)
84.07
84.31
75.49
85.05
86.76
Full-precision
64.62
71.19
64.98
76. 17
80.87
RTE
Q4AdaRound
63.54
66.79
62.82
75.09
80.51
Q+FleRRound (Ours)
64.62
68.95
62.82
76.17
81.23
models are quantized into 4-bit under the per-tensor uniform PTQ setting, the performance degradation (compared to a full-precision pre-trained model) negligible (less than 1.5%) in Table 3
4.4 LANGUAGE MODELS
All language models we consider this paper are based on the structure of Transformers (Vaswani al. 2017). To quantize Transformers into 8-bit, apply linear asymmetric per-tensor quantization scheme for both weights and activations, while reconstruction (for PTQ) is considered for each
8

---

Under review as a conference paper at ICLR 2023
Table 5: Performance of GPI-Ne0125M. GPT-Neo .3, GPT-Neo2.77. 12T
OPT and OPT on the WikiText2 and PTB datasets. The perplexity (PPL) is employed as a performance metric, The lower PPL, the better. "Q X" means the implementation of X in the QDrop's 's setting.
Dataset
Method
2P7-Ne0125M
GPT-Neoj,
GPT-Neo2 78
125M
OPT 1,38
OPT2.78
Full-precision
31.54
15.40
13.35
56.08
29.76
26.13
WikiTex12
Q+AdaRound
35.60
15.75
13.95
226.48
40.40
47.48
Q+FlexRound (Ours)
33.44
15.68
13.80
66.07
40.01
40.38
Full-precision
64.63
31.51
27.22
129.90
76.06
68.81
PTB
AdAaRounnd
70.16
31.97
28.24
220.01
103.15
120.37
Q+FlexRound (Ours)
66.62
31.74
27.68
145.45
101.81
106.88
Transformer layer that includes attention sublayers and feedforward sublayers. All weights are quantized into 8-bit except the last randomly initialized layer. As for activation quantization, on- the-fly fly (static) quantization is conducted before every fully-connected layer except the inputs of the softmax layer and the normalization layer that remain to of full cell-precision in Zafrir et al. (2019) and Zhang et
2220
BERT and GPT-Neo on GLUE We evaluate the natural language understanding (NLU) per- formance of FlexRound using various models including BERT Base. BERT GPT-Ne0125M: GPT-Neo and GPT-Neo2.78
the GLUE benchmark. The learning rate applied all learnable parameters (s1, S2, and 83) is selected to be 2e-4 for BERT and to be 3e-4 for GPT-Neo. Reconstruc- tion process is performed by using 1024 random samples for 20K iterations. For all experiments, the batch size is 64 and maximum sequence length of all experiments is 128. We utilize pre-trained language models (PLMs) and datasets available from the HuggingFace Woll et al. 2020) repository Further experimental details are referred to Appendix G In Table we report the performance of 'Q + AdaRound" and 'Q FlexRound' that are potentially promising as shown in Table 3 We can notice that 'Q FlexRound' yields better NLU scores than 'Q AdaRound' for most NLU tasks. In particular, for the MNLI and QQP datasets, 'Q FlexRound" can achieve comparable even superior performance full-precision model in the per-tensor uniform PTQ setting except GPT-Ne0125M-
GPT-Neo and OPT WikiText2 and PTB We test the natural language generation (NLG) performance of FlexRound on the WikiText2 and PTB datasets. PLMs (for NLG) are quantized by FlexRound (in per-tensor quantization manner) while small amount of data of downstream tasks are used for reconstruction and evaluation. Specifically, PLMs include GPT-Ne0125M. GPT-Neo,.at GPT-Ne02,.TB OPT OPT1.38 and 2PT,,7B while 256 downstream task data samples are chosen at random for reconstruction. More details on the experimental setup are provided in Appendix T the results of GPT-Neo and OPT on NLG tasks and is clear that 'Q + FlexRound' superior to 'Q + AdaRound' for all models and NLG tasks. Note that for GPT-Neo, 'Q + FlexRound' can achieve the similar performance
full-precision PLM even in the per-tensor uniform PTQ setting, while some previous attempts rely on group-wise vector-wise quantization (Yao al.
2022 Dettmers et al. 2022
CONCLUSION
We propose a new rounding scheme, named FlexRound, for post-training quantization under the the principle of element-wise division, to enable learning both a common quantization grid size and an individual scale for each pre-trained weight. We validate that FlexRound can flexibly quantizes pre-trained weights by exploiting their magnitude as a metric to measure importance. Consequently, Flex can achieve comparable performance to full-precision model even in the per-tensor uniform PTQ setting. As future work, we plan to quantize large language models beyond 6.7B parameters in the per-tensor uniform PTQ setting.
tttps://gitthb.cco httpe//////tttooooo/ggiiffffffee

---

Under review as a conference paper at ICLR 2023
REFERENCES
Haoli Bai, Lu Hou, Lifeng Shang, Xin Jiang, Irwin King, and Michael R Lyu. Towards efficient post-training quantization of pre-trained language models. arXiv preprint MxXiv:21099/0822 2021.
Yoshua Bengio, Nicholas Leonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 1exXiv:330000 /308.3432, 2013.
Sid Black, Leo Gao, Phil Wang. Connor Leahy, and Stella Biderman. GPT-Neo: Large Scale Autore- gressive Language Modeling with Besh--Tensrfflow March 2021. URL https: org/ 10. 1..1122enzzoooo. 5297715 If you use this software, please cite it using these metadata.
Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Understanding and overcom- ing the challenges of efficient transformer quantization. In Proceedings of the 2021 Confer- ence Empirical Methods in Natural Language Processing, pp. 7947-7969. Association for Computational Linguistics, November 2021. doi: 10.18653/v1/001 10.18663//1//021eemmmm/ 1et/2/022..mmmmmmmi..221111 URL
wwtp:h/oooo.oo...oooo/o emn ma maiin. me.ii...22. 627
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El- El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ or-1. arXiv preprint arXiv:1602.0228 02830, 2016.
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Llm. int8 O:: 8-bit matrix multiplication for transformers at scale, arXiv preprint arXiv:2208.0733) 2022.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800. 2018.
Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dhar-
mendra S. Modha. Learned step size quantization. In International Conference Learning Representations, 2020. URL htttstts::// httttss:://ppeeeviieeee. nwt. http:/wwww..eefooomm/////rrooooe
Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for
efficient neural network. Advances in neural information processing systems, 28, 2015.
Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In International Conference on Learning Representations, 2016. URL https: arxiv. org/ paf 1110..1111.. 00149. pdf
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry. Accurate post training quantization with small calibration sets. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 4466 4475. PMLR, 2021. URL httpt:ttt///mm pateps::/proceedingg...... www.mm...p.essse.3.. http:///////uubbaaaa.a. /tt.//ubbaaaaa.. html
Sambhav R Jain, Albert Gural, Michael Wu, and Chris H Dick. Trained quantization thresholds for ac- curate and efficient fixed-point inference of deep neural networks. arXiv preprint 10X0::9003.0066, 2019.
Sangil Jung, Changyong Son, Seohyung Lee, Jinwoo Son, Jae-Joon Han, Youngjun Kwak, Sung Ju Hwang, and Changkyu Choi. Learning to quantize deep networks by optimizing quantization intervals with task loss. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4359 2019.
Sumin Kim, Gunju Park, and Youngmin Yi. Performance evaluation of int8 quantized inference on mobile gpus. IEEE Access, 9: 9:14455--642255, 2021.
10

---

Under review as a conference paper at ICLR 2023
Jung Hyun Lee, Jihun Yun, Sung Ju Hwang, and Eunho Yang. Cluster- prommtr-nomoiingg quantization with bit-drop for minimizing network quantization loss. In 2021 IEEE/CVE International Conference IEEE Computer Society. 2021. URL https
on Computer Vision (ICCV), pp. wtt.:////..111111/1../222222222 200.C4448222.221.000555. 00532
//doi.
Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. BRECQ: Pushing the limit of post-training quantization by block reconstruction. In International Conference on Learning Representations, 2021. URL /ttppst:t/////eeeee///
net net/foruu httt/forum:id=P0Wv55htt
Qian Lou, Feng Guo, Minje Kim, Lantao Liu, and Lei Jiang. Autoq: Automated kernel-wise neural network quantization. In International Conference on Learning Representations, 2020. URL
https://openzeviiet net/ oow.::/oooormminngggnnnnnn
Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated
corpus English: The Penn Treebank. Computational Linguistics, 19(2):11-3330 1993. URL
https: www. www.wwww.bbb......... wttpoooorr.atoooggggooooooo -90020044
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture
models, 2016.
Szymon Migacz. 8-bit inference with tensorrt. In GPU technology conference, volume 2, pp. 5, 2017.
Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling, Data-free quantization through weight equalization and bias correction. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 1325-1334, 2019.
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? Adaptive rounding for post-tree post-training quantization. In Proceedings of the 37th International Conference on Machine Learning, volume 119 Proceedings of Machine Learning Research, pp. 7197-7206. PMLR, 2020. URL httpttttt////cceeeeeeee ngs ml wwwps//esssssseee html
Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Banner, Alex Bronstein, and Avi Mendelson. Loss aware post-training quantization. Machine Learning. 11011).3245-3322. 2021.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ Questions for Machine Comprehension of Text. arXiv e-prints, art. arXiv:1606.05555 000.00000000.0... 2016.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision. 115(3):211-22) 2015.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chies Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv: 1804.07404 2018.
11

---

Under review as a conference paper at ICLR 2023
Peisong Wang, Qiang Chen, Xiangyu He, and Jian Cheng. Towards accurate post-training network quantization via bit-split and stitching. In International Conference on Machine Learning, pp- 9847-9856. PMLR, 2020.
Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu. Qrrop: Randomly dropping quantization for extremely low-bit post-training quantization. In International Conference on Learn- ing Representations, 2022. URL /tLL:ttt///ppeereiiiiee
net/ Forum StaerrummiidyyS00000yyy
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- ger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 38-45, Online, October 2020. Association for Computational Linguistics. doi: 10. 100116653/111/022mmmml/p// http::///aaaaattlooooooooo//////
URL
Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius. Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv preprint arXiv:2004.06602
2020.
Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. arXiv preprint arXiv:2206.01861, 2022.
Ofir Zafrir. Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. Q8bert: Quantized 8bit bert. In 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing-Neurlly Edition (EMC2-NIPS), pp. 36-39. IEEE, 2019.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01066 2022.
Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, and Qun Liu. Ternarybert: Distillation-aware ultra-low bit bert. arXiv preprint arXiv:2009. 12200009.221.... 2020.
Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Chris De Sa, and Zhiru Zhang. Improving neural network quantization without retraining using outlier channel splitting. In International conference on machine learning, pp. 7543-7552, PMLR, 2019.
Xiandong Zhao, Ying Wang, Xuyi Cai, Cheng Liu, and Lei Zhang. Linear symmetric quantization of neural networks for low-precision integer hardware. In International Conference on Learning Representations, 2020.
Michael Zhu and Suyog Gupta. To prune, not prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01888, 2017.
12

---

Under review as a conference paper at ICLR 2023
A COMPARISON OF FLEXROUND TO ADAROUND AND ADAQUANT
Figure shows that the comparison of FlexRound to AdaRound and AdaQaunt, As seen in Figure 4(a)
FlexRound can quantize pre-trained weights more flexibly than AdaRound and AdaQuant, As
weights of large magnitude are not quantized aggressively in the middle of Figure 4(a) compared to the right of Figure 4(a) AdaQuant quantizes weights of large importance marginally, which seems to make it difficult for AdaQuant to quantize MobileNetV2 into 4-bit.
AdaRound
AdaQuant
FlexRoand (Ours)
AW
AWs
-2-
-3
-3
-6-
-5
-15
-10
-5
10
-15
-10
-5
10
-15
-10
20
W
W
W
(a) MobileNetV2
AdaRound
AdaQuant
FlexRound (Ours)
W
W
W
(b) ResNet-18
Figure 4: Scatter plot of the amount of grid shifts from oounding-to-nearee gird in the first layer of the first block in MobileNetV2 and ResNet-18 18 when only weights are quantized into 4-bit.
13

---

Under review as a conference paper at ICLR 2023
B DERIVATION OF SECTION 4.1
Let = [W - x and S' be S2 O 83 for fully-connected layer and S2 + O 84 for convolutional layer. In the case of fully-connected layer,
2C
(
Oc
(8((5)
aS (1.))
aW
aW(i.)
(i.5)
0
W(n
OC
as (v5)
(1.))
(W()))
0
(
dC
= DS (s,)
aW(s
0
W(w.))
0C
81 0S (1.))
(1
(:: Straight-Through Estimator)
(b.)
OW (45)
W(i
OC
$1
as
(1.5)
S (1,)
(W()))
(3.5)
1
OL
W(as))
(w.)
W
OL
((.)) (W()))
The derivation in the case of convolutional layer can be done by just replacing W (1,)) with W (.JAD and S/s,)) with Sy (1..A.11)
14

---

Under review as a conference paper at ICLR 2023
C RESNET-18, RESNET-50, AND MOBILENETV2 ON IMAGENET WITH PRE-TRAINED MODELS FROM THE OFFICIAL PYTORCH REPOSITORY
Table 6: Top- 1/Top-5 accuracy (%) for ResNet-18 18, ResNet-50, and MobileNetV2 on ImageNet when only weights are quantized. "B + X" expresses the implementation of X in the BRECQ's setting. We employ pre-traing pre-trained models available from the official Py repository.
Method
Bits (W/A.)
ResNet-18
ResNet-50
MobileNetV2
Full-precision
32/32
09.76/89.08
76.15/92.87
71.88/90.29
B + AdaQuant
4/32
67.55/87.77
74.09/91.77
0.48/0.53
B + AdaRound
4/32
09.15/88..
75.51/92.75
67.76/88
B FlexRound (Ours)
4/32
69.21/88. 88.76
75.59/92.. /92663
69.56 89.02
B AdaQuant
3/32
60.75/83. 83.41
66.19/87.08
0.10/0.52
B AdaRound
3/32
07.98/88.11
74.51/92.20
60.18/83 81.88
B FlexRound (Ours)
3/32
68. 68.02/88. 000088800
74.61 74.61/92.11
64.85/ 86.
B AdaQuant
2/32
L.13/4.10
0.12/0.60
0.10/0.50
B AdaRound
2/32
63.01/85: /85.00
68.31 08.31/88.98
33.10/60.58
B FlexRound (Ours)
2/32
63.73 885.41
70.57 /9007
38.09/64.90
Table 7: Top-l/Top 1/Top-5 accuracy (%) for ResNet-18, ResNet-50, and MobileNetV2 on ImageNet when both weights and activations are quantized. "B + X" and "Q + Y" represent the implementation of X in the BRECQ's setting and that of Y the QDrop's s setting, respectively. We employ pre-trained models available from the official PyTorch repository.
Method
Bits (WJA.
ResNet-18
ResNet-50
MobileNetV2
Full-precision
32/32
69.76/89.00
76.15/92.87
71.88/90.
B AdaRound
4/4
68.32/ /0/./88.. 13
74.28/92.. /9200
28.46/52.
FlexRound (Ours)
4/4
68.34/ 88. 19
74.42/92.. /92.04
35.25/78.61
Q AdaRound
4/4
68.19/88.18 88.18
74.68/
74.68/92.02 /92.02
56.68/80 80.95
Q FlexRound (Ours)
4/4
68.23/88.0 22
74.83/ /92.11
61.56 84.18
B AdaRound
3/3
64.44/85.73 73
68.80/88.. 00/88..99 79
2.11/7.24
B FlexRound (Ours)
3/3
64.61 (4.61/85.c /85.8
19.62/89.11
8.80/21.79
Q AdaRound
3/3
65.33 86.60
71.80/90.7 /900
62.41/59.22
Q FlexRound (Ours)
3/3
65.28 86.49
71.84 /90.48
71.84/90.48
41.51 1..51/68.07 /68.02
15

---

Under review as a conference paper at ICLR 2023
D IMPORTANCE OF JOINTLY LEARNING THE QUANTIZATION GRID SIZE $1 WITH ROUNDING
Table 8: Top- 1/Top-S accuracy (%) on ImageNet by ResNet-18 18, ResNet-50, and MobileNetV2 with only weights quantized into 4-bit. "B + X" denotes the implementation We employ pre-trained models available from the official Py PyToch repository.
the setting of BRECQ.
Method
ResNet-18 18
ResNet-50
MobileNetV2
B AdaQuant
67.55/87.74 73
74.09/91.77
0.48/0.53
B AdaRound
69.15/88.70 70
75.51/92.73
67.76/88 7.77/818.12
FlexRound with 81 fixed
09.11/88.64
75.52/92..0
68.19/88 /88.45
FlexRound (Ours)
69.21/88 78 76
75.59/ /22.63
69.56 89.02
To demonstrate the importance of jointly learning 81 with the rounding, did additional study with $1 fixed. When fixing $1, for ResNet models the performance of FlexRound is almost comparable to that of AdaRound, while for MobileNetV2 FlexRound is somewhat superior to AdaRound. When
jointly learning with the rounding, however, FlexRound outperforms AdaRound for all models. It is therefore critical to learn jointly with the rounding.
16

---

Under review as a conference paper at ICLR 2023
E
ABLATION STUDY ON SAMPLE SIZE
70
B FlexRouudd AdaRound
69
68
Accuracy
67
66
65
64
2048
1024
512
256
128
64
32
Calibration set size
Figure 5: Ablation study on sample size when quantizing MobileNetV2 into 4-bit. Only weights are quantized, with activations kept full- full-precision We employ pre-trained models available from the official PyTorch repository.
No matter how much data is used, B+FlexRound always outperforms B+AdaRound. When the sample size decreases from 64 to 32, the accuracy of B+FlexRound declines by almost percent. Correspondingly, a sample size of 32 would be breakthrough point.
17

---

Under review as a conference paper at ICLR 2023
F COMBINING ELEMENT-WISE ADDITION AND ELEMENT-WISE DIVISION
Table 9: Top- 1/Top-5 accuracy (%) for ResNet- 18, ResNet-50, and MobileNetV2 on ImageNet when only weights are quantized. "B + X" expresses the implementation of X in the BRECQ's setting, We employ pre-trained models available from the official PyTorch repository.
Method
Bits (W.A.))
ResNet-18
ResNet-50
MobileNetV2
Full-precisie
32 /2
69.76/89
/8.08
76.15/92.87
71.88/90.29
B + AdaQuant
4/32
67.55/87 /87.73
74.09/91.77
0.48/0.53
B AdaQuant + FlexRound
4/32
68.75/80 48.45
75.14/92.45
68.36 0866888.9 49
B + FlexRound (Ours)
4/32
69.21/88. 76
75.59/92.c0
69.56 89.02
B + AdaQuant
3/32
60.75/83.41
66. 66.19/87.08
() 0.10/0.52
B + AdaQuant + FlexRound
3/32
67.36/87.71
74.05/91.87
61.64/84. /8.4/884..
+ FlexRound (Ours)
3/32
68.02 22.02/88.05
74.61/92.1
64.85/ 86.38
+ AdaQuant
2/32
1.13/4.10
0.12/0.60
0.10/0.50
+ AdaQuant + FlexRound
2/32
62.23/ 88.77
69.39/89.38
34.11/61.66
FlexRound (Ours)
2/32
63.73 85.41
70.57/90.00
38.09/64.90
To identify whether there comes any benefit from both addition and division, we combine AdaQuant with FlexRound. AdaQuant FlexRound superior AdaQuant but inferior FlexRound. This might be due to the naive combination of AdaQuant with FlexRound. Considering both addition and division would be interesting future work.
18

---

Under review as a conference paper at ICLR 2023
G BERT AND GPT-NEO ON GLUE
The experimental setting of 'Q AdaRound" follows Wei al. (2022) To investigate the natural language understanding performance of FlexRound from BERT to GPT-Ned we directly fine-tune pre-trained models on the GLUE dataset. For BERT, we use uncased models. Hyper-parameter selection for fine-tuning a pre-trained model is given in Table I0 We use ADAM optimizer as default for all methods and models. the QDrop's ' setting, the probability of dropping activation quantization is set to 0.5. We utilize the Huggingface repository for the evaluation method without any modification.
Table 10: Hyper-parameter selection for fine-tuning BERT BERT Large GPT-Ne0125M. GPT-Neo1.3n and GPT-Nco2.77 on the GLUE benchmark.
Configuration
BERT Base
BERT Large
GPT-Neo 1P75Ne0125M
GPT-Ne01,3B
GPT-Ne02.78
Learning Rale
2e-5
2c-5
2c-5
2e-5
le-5
Batch Size
32
32
32
32
16
Epoch
3
Max Sequence Length
128
Weight Decay
0.01
Ditorchhieettcllassficctooo
19

---

Under review as a conference paper at ICLR 2023
H BERT ON SQUAD
Table 11 additionally shows the performace of FlexRound on the SQuADv (Rajuurka et al. 2016 dataset for the BERT models. For experimental details, Both BERT and BERT are uncased models. For 'Q + FlexRound', the learning rate is set to le-4 for both models. For both "Q + AdaRound" and 'Q+ FlexRound', the batch size and the number of iterations for reconstruction are 64 and 20k respectively. We use ADAM optimizer as default for all methods and models. The other experimental setting of 'Q AdaRound' follows Wei et al. (2022). Table 12 shows the hyper- parameter selection for fine-tuning. Both BERTBase and BERTLarge are using the same configuration. The other setting for fine-tumi fine-tuning and the evaluation method are the same as HuggingFace repository"
Table 11: score for BERT Base and BERT on SQuADy1 dataset when both weights and activations are quantized into 8-bit. "Q + X" represent the implementation of in the QDrop's setting.
Method
# Bits (W.A..)
BERT Base
BERT Large
Full-precision
32/32
87.05
89.31
Q + AdaRound
8/8 8/8
86.90
88.89
Q + FlexRound (Ours)
87.25
89.25
Table 12: Hyper-parameter selection for fine-tuning BERT Base and BERT Large SQuADv1 dataset.
Learning rate
Batch size
Epoch
Maximum sequence length Document stride
le-4
32
384
128
20

---

Under review as a conference paper at ICLR 2023
I GPT-NEO AND OPT ON WIKITEXT2 AND PTB
To evaluate FlexRound for natural language generation tasks, utilize the WikiTex 10 and PTB datasets. Table 13 reports the learning rate, the batch size, and the number of iterations for 'Q + FlexRound'. The experimental setting of *Q + AdaRound' follows Wei et al. (2022) except the number of iterations; we employ 15k iterations for GPT-Neo and 20k iterations for OPT The batch size for 'Q + AdaRound' is same as that for 'Q + FlexRound'. We use ADAM optimizer as default for all methods and models. The probability of dropping activation quantization is set to 0.5 in the QDrop's setting. We use the Huggingface repository for the evaluation method without any modification.
Table 13: Hyper-parament selection for 'Q + FlexRound" in Table 5
Dataset
Configuration
GPT-Ne0125M
GPT-Neo1.st
GPT-Ne02.78
OРT
OPT1.s
OPT2.78
WikiTex12
I Learning rate
2e-3
Ge-4
4e-4
le-3
9e-5
8e-5
Batch size Iteration
32
16
8
32
16
15k
15k
PTB
I Learning rate
4e-3
4e-3
2c-3
8e-4
Te-3
5e-3
Batch size
32
16
8
32
16
8
Iteration
15k
15k
httess//ongginneeeecce//ssssssss only
Dytoreh//arggaaemmdelnggg
21

---

Under review as a conference paper at ICLR 2023
J FINETUNED GPT-NEO AND OPT ON WIKITEXT2 AND PTB
As for the evaluation of quantized pre-trained language models, the performance (i.e., accuracy) of quantized OPT (by Q+AdaRound or Q++FleXRundd is not close to that of full-precision OPT, while GPT-Neo can be quantized without noticeable accuracy degradation. To investigate whether such an observation is also valid for finetuned OPT or not, we conduct additional experiments on finetuned OPT and GPT-Neo with Wikitext2 and PTB dataset. As shown in the table 14] quantized model's performance of finctuned OPT turns out to be close to full-precision performance. Considering that the model was finetuned with each downstream dataset, We utilize smaller dataset and lesser iteration for reconstruction. We use 128 samples for calibrations set and the iteration is fixed 500 for all experiments. Learning rate and batch size for the experiments shown in Table 15 the same Appendix !
Other settings
Table 14: Performance of GPT-Neo, 25M, GPT-Neo, GPT-Nee2..7, OPT OPT 125M- OPT and O.T2.78 Finetuned on the WikiTex12 and PTB datasets. The perplexity (PPL) is employed as performance metric. The lower PPL, the better. "Q + X" means the implementation of X in the QDrop's s setting.
Dataset
Method
GPT-Ne0125M
GPT-Neo1.st
GPT-Neoa.1
(PT
OPT
OPT2.78
Full-precision
21.96
12.09
10.78
11.32
10.27
WikiTex12
Q+AdaRound
30.52
12.47
14.09
27.96
12.66
10.97
Q+FlexRound (Ours)
24.30
12.37
12.43
21.43
12.02
10.63
Full Full-precision
24.20
16.09
14.70
16.50
11.62
10.80
PTB
AdARooudd
31.40
16.63
19.80
20.28
13.00
12.02
O+FlexRound (Ours)
26,03
16.32
16.87
17.68
12.22
11.29
Table 15: Hyper-parameter selection for 'Q FlexRound' in Table 14 iteration is 500.
Sample size is 128 and
Dataset
Configuration
GPT-Neo125M
GPT-Neo1.38
GPT-Neu2.78
OPT)2sM
OPT
OPT2.70
WikiText2
Learning rate
5e-3
4e-4
4e-3
3e-5
7e-6
1e-5
Batch size
32
16
16
PTB
Learning rate
5e-3
7e-3
7e-3
5e-5
3e-5
8e-6
Batch size
32
16
8
32
16
8
22


Reviews:
[{"review_id": "oXiAIRPQHI", "reviewer": "Reviewer_vGda", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This submission proposed to conduct post-training quantization by learning element-wise scaling parameters $\\boldsymbol{S}$. Pre-quantized parameters are divided by the scaling parameters and then rounded to integer. Basically, the proposed method is a subtle re-training for pre-trained parameters for quantization.", "summary_of_the_review": "Basically, the core method can be considered as a subtle re-training for pre-trained parameters for quantization. Besides, the author missed some important points for precise evaluation. ", "strength_and_weaknesses": "Weaknesses:\n1. As the author mention that the previous works used element-wsie addition in parameters refine before quantization while the proposed method used division, why addition is not considered in the submission? Since both division and addition can refine parameters, would it be better if both are used?\n2. It seems author miss mentioning how much data is necessary to learn $\\boldsymbol{S}$.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ec-0JL27Rcm", "reviewer": "Reviewer_oCmz", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a new post-training quantization algorithm called FlexRound. Unlike prior algorithms such as AdaRound, the learnable rounding factors are division parameters instead of an addition parameters. It has the advantage that it can learn the integer scale and rounding jointly, and also take the weights ‘importance’ into account according to the authors. The proposed approach is well evaluated on CV and NLP tasks and shows good empirical performance compared to their baselines.\n", "summary_of_the_review": "The paper proposes a simple adaptation to AdaRound/BRECQ/AdaQuant which has some novelty as the learned parameter is relative instead of additive. In general the paper has a good empirical evaluation except a few questions/inconsistencies with respect to prior work. A few of the papers claims are a bit strong or can be misleading and should be adapted accordingly. Overall the paper is borderline. ", "strength_and_weaknesses": "\nStrength:\n* The paper is well written and easy to follow. The visualizations are nice and helpful and the work is put well in context of existing literature.\n* FlexRound allows learning the scale and ‘rounding’ at the same time which can potentially be an advantage over SOTA current methods (AdaRound/BRECQ).\n* The proposed method is extensively evaluated on image classification and NLP tasks and shows good empirical performance (except some missing comparisons, more on that later).\n* Has ablation study on empirical choices such as introducing the factors s3 and s4 which should be in theory not needed. \n\nWeaknesses:\n* A proper comparison to literature and prior work is missing. While table 2 should in theory be comparable to the papers, their numbers do not match. To my understanding,’ B+AdaRound’ should be exactly what BRECQ is, however, the stated numbers are significantly below the results in the original BRECQ paper. Where does the difference come from? Also the 4-bit MobileNet v2 results are below the AdaRound paper while actually BRECQ+AdaRound improves over vanilla AdaRound (and the original AdaRound paper uses per-tensor quantization and the first/last layer are in 4 bits).\n* Claim 3 states that they are the first that do extensive per-tensor study on image classification and NLP. This is not fully true. The white paper of Nagel et al. 2021 (which the authors also cite) has in its PTQ chapter (table 6) a similarly extensive study which also includes per-tensor quantization. On the per-tensor vs per-channel point later, the original AdaRound paper is also with per-tensor quantization (not sure about BRECQ).\n* Claim 2 says they demonstrate that element-wise division includes the importance of the pre-trained weight. This is a fairly strong claim and IMO they only show this partly. They show that the gradient is proportional to the magnitude of the weights but then the link to importance is a bit soft/vague.\n* The authors argue that addition schemes may change the sign of a weight. However, for the most compared schemes, AdaRound and BRECQ, this can not happened based on how it is defined. Only in the case of AdaQuant, which they show performs poorly, this could theoretically happen.\n* Arguing that FlexRound has no extra hyper-parameters (compared to BRECQ/AdaRound) is only somewhat a benefit as the AdaRound paper keeps al hyper-parameter constant inter experimentation and is therefore de-facto also hyper-parameter free.\n\nQuestion: \n* It seems s1 (the general scaling factor) is learned jointly with the ‘rounding’. Could this be also a reason why FelxRound is empirically better than AdaRound/BRECQ? Due to their formulation, AdaRound/BRECQ can not learn the scales jointly with the rounding which is a clear drawback. An additional ablation (in table 1) with a fixed s1 could potentially give some interesting insights into this.\n* Did the author explore combining the additive approach (AdaRound/BRECQ) with the devision based approach? Given that empirically they need 3 new learnable scaling factors (s2, s3, s4,  cf table 1), it might be interesting to see if there comes benefits from both and additive and multiplicative term.\n\n\nEditorial:\n* Would suggest to use $\\lfloor \\cdot \\rceil$ for rounding such as in most prior literature (BRECQ, AdaRound etc).\n* Section 3.2: “per-tensor quantization schemes facilitate higher parallelism for implementation compared to per-channel quantization schemes (Nagel et al., 2021)”. This seems a not widely known/acknowledged statement and I could not find such a statement in the referred work. Could the authors please point me to the sections where this is discussed or other resources?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "KyrdPPazPc", "reviewer": "Reviewer_y3vc", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a post-training quantization (PTQ) weight/activation quantization method.\nThe gist is to simultaneously optimize a multiplicative element-wise scale factor applied to the target tensor\nand the quantization step size, under the reconstruction loss. The method can be seen as closely related in spirit\nto AdaQuant and AdaRound, with the critical difference that instead of adding optimal noise to the tensor before\nrounding, multiplicative noise is used instead. If can also be seen to be related to the weight scaling method of\n(Nagel et al, 2019), in the sense that weight tensors are scaled prior to rounding, but with the critical difference\nthat DFQ does this prior to quantization, while FlexQuant does it during the quantization step.  \n\nExperiments carried out on image classification (ImageNet), natural language understanding (GLUE) and natural language\ngeneration (WikiText2 and PTB) show superior performance to AdaRound and AdaQuant.", "summary_of_the_review": "I think this work provides a simple yet effective weight scaling approach within the PTQ framework, and,\nup to my knowledge, the problem formulation is qualitatively different from previous approaches. While evaluation\nis broad in terms of the benchmarks used, I have two concerns. First, the results reported for the \nBRECQ + AdaRound baselines are different from the ones in the original paper. Second, I would have liked to also see a\ncomparison/combination with the weight equalization approach of (Nagel et al., 2019). The clarity of the presentation\ncould be improved.\n", "strength_and_weaknesses": "* Strengths:\n\n  * The main idea is simple and attractive from both a theoretical and practical standpoint, extending neatly the\n  existing algorithmic frameworks (e.g. BRECQ and QDrop) \n  * Evaluation scope covers a wide variety of domains (language, vision) and architectures (ResNet, MobileNet, ViT).\n\n* Weaknesses:\n\n  * There are points where the paper is unclear. Please see comments on clarity below.\n  * The experimental results reported for BRECQ in table 3 are different from those reported in the original papers.\n  I presume this is because a per-tensor quantization comparison is being used (but the authors should confirm this).\n  * No study in conjunction with the related weight range equalization method introduced in\n  (Nagel et al., 2019) is shown, nor is a comparison carried out. Please see comments below on quality.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3G3w7s6cvB", "reviewer": "Reviewer_mQzD", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new approach named FlexRound to improve the post-training quantization of deep neural networks. Previous methods primarily rely on element-wise addition for the rounding function. The proposed FlexRound, instead, uses element-wise division, which can exploit the parameter magnitude to update the quantization scale. The authors validate the efficacy of FlexRound over various CV and NLP tasks, together with a variety of network architectures.", "summary_of_the_review": "Despite the methodological change is minor, the authors provide a reasonable explanation why FlexRound can exploit the importance of pre-trained weights to better calibrate the quantized model (Figure 3 and Figure 4). Experimentally, I appreciate the authors provide comprehensive studies across computer vision, natural language understanding and natural language generation tasks, and results demonstrate the success. Nonetheless, there are still issues with the experimental designs and comparisons, as mentioned above. There are also concerns w.r.t. the methodology, e.g., if FlexRound can be applied to activation quantization; and the necessity in introducing multiple scaling factors $\\mathbf{s}3$, $\\mathbf{s}4$, e.t.c..", "strength_and_weaknesses": "Strengths:\n\n- The authors propose FlexRound, a new approach that applies element-wise division to exploit the parameter magnitude to update the quantization scale. Figure 3 and Figure 4 provide good explanations on the advantages of FlexRound. \n\n- The authors conduct comprehensive empirical studies over the image classification, natural language understanding and natural language generation tasks. Moreover, the authors testify various network architectures across CV and NLP, and especially, GPT-Neo and OPT, for post-training quantization. \n\n- The writing is clean.\n\nWeakness:\n\n- Lack of comparisons with existing approaches, especially BRECQ and Q-Drop. According to the published results of BRECQ and Q-Drop, the results in this paper seem to be worse than these two baselines (e.g., W3-A3 Q-Drop for MobileNet-v2 is 57.98%, but only 41.51% in this paper), even under the same configuration. More comparisons and explanations should be presented.\n\n- While BRECQ and AdaRound rely on element-wise addition for quantization, it is still unclear what is their disadvantage according to the paper.\n\n- The proposed approach seems to be only applicable to weight quantization, yet this is hardly mentioned. For Table-3, it should be made clear that what type of activation quantization is adopted. \n\n\n\nDetailed comments:\n\n- Despite ablated, it is still not clear the benefits of introducing $\\mathbf{s}_3$ and $\\mathbf{s}_4$ in Equation 2. Intuitively, both factors can be absorbed into $\\mathbf{S}_2$. \n\n- What is the initialization method for $\\mathbf{S}$? This should be important as PTQ does not allow intensive training like those in LSQ.\n\n- Provide more derivations (in appendix) for the gradient w.r.t. the scaling factor $\\mathbf{S}^{'}$.\n\n- As a minor suggestion, a loss curve or accuracy curve to compare FlexRound with BRECQ during the PTQ iterations could help illustrate FlexRound. This is based on the intuition that parameters with larger magnitude tend to move faster and thus converge better given the limited training iterations and samples.\n\n- Some recent research on PTQ for your reference.\n\n  - Liu Z, Wang Y, Han K, et al. Post-training quantization for vision transformer. NeurIPS 2021.\n\n  - Bai H, Hou L, Shang L, et al. Towards efficient post-training quantization of pre-trained language models. NeurIPS 2022.\n\n  - Park G, Park B, Kwon S J, et al. nuQmm: Quantized MatMul for Efficient Inference of Large-Scale Generative Language Models. arXiv preprint arXiv:2206.09557, 2022.\n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject