PowerQuant: Automorphism Search for Non-Uniform Quantization
Authors: Edouard YVINEC
Arnaud Dapogny
Matthieu Cord
Kevin Bailly
Venue: ICLR 2023 poster (2023)
Keywords: deep learning
quantization
compression
acceleration
data-free

Abstract: Deep neural networks (DNNs) are nowadays ubiquitous in many domains such as computer vision. However, due to their high latency, the deployment of DNNs hinges on the development of compression techniques such as quantization which consists in lowering the number of bits used to encode the weights and activations. Growing concerns for privacy and security have motivated the development of data-free techniques, at the expanse of accuracy. In this paper, we identity the uniformity of the quantization operator as a limitation of existing approaches, and propose a data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation, non-uniform quantization shall not change the nature of the mathematical operations performed by the DNN. This leads to search among the continuous automorphisms of $(\mathbb{R}_+^*,\times)$, which boils down to the power functions defined by their exponent. To find this parameter, we propose to optimize the reconstruction error of each layer: in particular, we show that this procedure is locally convex and admits a unique solution. At inference time, we show that our approach, dubbed PowerQuant, only require simple modifications in the quantized DNN activation functions. As such, with only negligible overhead, it significantly outperforms existing methods in a variety of configurations.

Published as a conference paper at ICLR 2023
POWERQUANT: AUTOMORPHISM SEARCH FOR NON- UNIFORM QUANTIZATION
Edouard Yvinect.2 Arnaud Dapogny² MMathhieu Cord Kevin Bailly Sorbonne Université*, CNRS, ISIR, f-75005, 4 Place Jussieu 75005 Paris, France Datakalab², 114 boulevard Malesherbes, 75017 Paris, France eyēdataka l com com
ABSTRACT
Deep neural networks (DNNs) are nowadays ubiquitous in many domains such as computer vision. However, due their high latency, the deployment of DNNs hinges on the development of compression techniques such as quantization which consists in lowering the number of bits used encode the weights and acti- vations. Growing concerns for privacy and security have motivated the devel- opment of data-free techniques, the expanse of accuracy. In this paper, we identity the uniformity of the quantization operator as a limitation of existing ap- proaches, and propose data-free non-uniform method. More specifically, we argue that to be readily usable without dedicated hardware and implementation. non non-uniform quantization shall not change the nature of the mathematical oper- ations performed by the DNN. This leads to search among the continuous auto- morphisms of (R: x). which boils down to the power functions defined by their exponent. To find this parameter, of each layer: in particular, we show that this procedure is locally convex and admits unique solution. At inference time, show that our approach, dubbed PowerQuant, only require simple modifications in the quantized DNN activation functions. As such, with only negligible overhead, it significantly outperforms existing methods in variety of configurations.
propose optimize the reconstruction error
INTRODUCTION
Deep neural networks (DNNs) tremendously improved algorithmic solutions for wide range of tasks. particular, in computer vision, these achievements come at a consequent price, as DNNs deployment bares great energetic price. Consequently, the generalization of their usage hinges on the development of compression strategies. Quantization is one of the most promising such technique, that consists in reducing the number of bits needed to encode the DNN weights and/or activations, thus limiting the cost of data processing on a computing device.
Existing DNN quantization techniques, for computer vision tasks, numerous and can be distin- guished by their constraints. One such constraint is data usage, as introduced in Nagel et al.) (2019), and is based upon the importance of data privacy and security concerns. Data-frce approaches such as Banner et al. (2019); Cai et al. (2020); Choukroun et al. (2019); |Fng et al. (2020); Garg et al. (021)) Zhao al. (2019) Nagel et (2019), exploit heuristics and weight properties in order to perform the most efficient weight quantization without having access to the training data. As com- pared data-driven methods, the aforementioned techniques are more convenient use but usually come with higher accuracy loss at equivalent compression rates. Data-driven methods performance offer an upper bound on what can be expected from data-free approaches and in this work, we aim at further narrowing the gap between these methods.
To achieve this goal, we propose to leverage a second aspect quantization: uniformity. For sim- plicity reasons, most quantization techniques such as Nagel et al. (2019): Zhao et al. (2019): Cong et al. 2022) perform uniform quantization, i.e. they consist in mapping floating point values to an evenly spread, discrete space. However, non-uniform quantization closer to the network weight distributions, thus better preserving the network accuracy. Previous work non-uniform quantization either focused on the search of binary codes (Banner al. 2019
theoretically provide

---

Published as a conference paper at ICLR 2023
80
OCS
DFQ
ImageNet top1, Acauracy
70
SQuant
SPIQ
PowerQuant
WB/NB
WSVN5
W4/A4
Figure 1: Comparison of the proposed method to other data-free quantization schemes on DenseNet 121 pre- trained on ImageNet. The proposed method (right bin in blue) drastically improves upon the existing data-free methods especially in the challenging W4/A4 quantization.
2016; Zhang et al. 2018) or leverage logarithmic Hubara et al., 2016 Jeon et al. 2020 Wu et al.
distribution (Miyashita al. 2016, /hou al. 2017). However, these approaches map floating point multiplications operations to other operations that are hard to leverage current hardware (e.g. bit-shift) as opposed to uniform quantization which maps floating point multiplications to in- teger multiplications (Gholami et al. 2021: Zhou al. 2016). To circumvent this limitation and reach a tighter fit between the quantized and original weight distributions, in this work, we propose to search for the best possible quantization operator that preserves the nature of the mathematical operations. We show that this search boils down to the space defined by the continuous automor- phisms of (Ri, x), which is limited to power functions defined by their exponent. We optimize the value this parameter by minimizing the error introduced by quantization. This allows us to reach superior accuracy, as illustrated in Fig T
To sum it up, our contributions are:
We search for the best quantization operator that do not change the nature of the mathemat-
ical operations performed by the DNN, i.e. the automorphisms of (R: x). We show that this search can be narrowed down finding the best exponent for power functions.
We find the optimal exponent parameter to more closely fit the original weight distribution compared with existing (e.g. uniform and logarithmic) baselines. To do so, we propose to optimize the quantization reconstruction error. We show that this problem is locally convex and admits unique solution.
In practice, we show that the proposed approach, dubbed PowerQuant, only requires simple modifications in the quantized DNN activation functions and accumulation. Furthermore, we demonstrate through extensive experimentation that our method achieves outstanding results various and challenging benchmarks with negligible computational overhead.
RELATED WORK
2.1 QUANTIZATION
In this section, we provide background on the current state of DNNs quantization. Notice that while certain approaches are geared towards memory footprint reduction (c.g. without quantizing inputs and activations) (Chen et al. 2015 Gong al. 2014 Han al., 2016 Zhou et al. 2017).
in what follows, we essentially focus methods that aim at reducing the inference time. In par-
ticular, motivated by the growing concerns for privacy and security, data-free quantization methods
(Banner et al., 2019 Cai et al.) 2020; Choukroun al. 2019 Fang al.) 2020 Garg et al. 2021 Zhao et al. 2019 Nagel et al. 2019 Cong et al. 2022) are emerging and have significantly im- proved over the recent years. The first breakthrough in data-free quantization (Nagel et al,, 2019)
was based on two mathematical ingenuities. First, they exploited the mathematical properties of piece-wise affine activation functions (such e.g. ReLU based DNNs) in order to balance the per-
2

---

Published as a conference paper at ICLR 2023
channel weight distributions by iteratively applying scaling factors to consecutive layers. Second, they proposed a bias correction scheme that consists in updating the bias terms of the layers with the difference between the expected quantized prediction and the original predictions. They achieved near full-precision accuracy in int8 quantization. Since this seminal work, two main categories of data-free quantization methods have emerged. First, data-generation based methods, such Cai iet al.] (2020); Garg et al.] (2021). that used samples generated by Generative Adversarial Networks (GANs) (Goodfellow et al., al.]] 2014)) as samples fine-tune the quantized model through knowledge distillation (Hinton et al. 2014). Nevertheless, these methods are time-consuming and require sig- nificantly more computational resources. Other methods, such as Banner al. (2019); Choukroun et al. 2019); Fang et al. (2020); Zhao et al. (2019); [agel et al. (2019); Cong et al. (2022). focus on improving the quantization operator but usually achieve lower accuracy. One limitation of these approaches is that they are essentially restricted to uniform quantization, while considering non- uniform mappings between the floating point and low-bit representation might be key to superior
performance.
2.2 NON-UNIFORM QUANTIZATION
Indeed, in uniform settings, continuous variables are mapped to an equally-spaced grid in the orig- inal, floating point space. Such mapping introduces error: however, applying such uniform mapping to an priori non-uniform weight distribution is likely to be suboptimal in the general case. To circumvent this limitation, non-uniform quantization has been introduced (Banner et al. Hubara et al. [016 [Joon et al. 2020 Wu et al. 2016)] Zhang et al., 2018 Miyashita et al., Zhou et al.j 2017) and (Zhang et al. 2021a Li et al., 2019). We distinguish two categories of non-uniform quantization approaches. First, methods that introduce code- base and require very sophisticated implementations for actual inference benefits such as Banner et al. (2019); Hubara et al. 2006))(Jeon et al. (2020); Wu et al. (2016); Zhang et al. (2018). Second, methods that simply modify the quantization operator such as Miyashita al. (2016); Zhou et al. (2017). In particular, Zhang et al. 2021a) propose log-quantization technique. Similarly, Li et al. (Li al. 2019) use log quantization with basis 2. In both cases, in practice, such logarithmic quantization scheme changes the nature of the mathematical operations involved, with multiplications being replaced by bit shifts. Nevertheless, one limitation of this approach is that because the very nature of the mathematical operations is intrinsically altered, in practice, it is hard leverage without dedicated hardware and implementation. Instead of transforming floating point multiplications in integer mul- tiplications, they change floating point multiplications into bit-shifts or even look up tables (LUTss.. Some of these operations are very specific to some hardware (e.g. LUTs are thought for FPGAs) and may not be well supported on most hardware. Conversely, in this work, we propose non-uniform quantization scheme that preserves the nature of the mathematical operations by mapping floating point multiplications to standard integer multiplications. As result, our approach boils down to simple modifications of the computations in the quantized DNN, hence allowing higher accuracies than uniform quantization methods while leading to straightforward, ready-to-use inference speed gains. Below we describe the methodology behind the proposed approach.
2019
2016
3 METHODOLOGY
Let F be a trained feed forward neural network with L layers, each comprising a weight tensor Wj. Let Q be a quantization operator such that the quantized weights Q(W,) are represented b bits. The most popular such operator is the uniform one. We argue that, despite its simplicity, the choice of such a uniform operator is responsible for a significant part of the quantization error. In fact, the weights W; most often follow a bell-shaped distribution for which uniform quantization is ill-suited: intuitively, such a case, we would want to quantize more precisely the small weights on the peak of the distribution. For this reason, the most popular non-uniform quantization scheme is logarithmic quantization, outputting superior performance. Practically speaking, however, it consists in replacing the quantized multiplications by bit-shift operations. As result, these methods have limited adaptability as the increment speed is hardware dependent. To adress this problem, we look for the non-uniform quantization operators that preserve the nature of matrix multiplications. Formally, taking aside the rounding operation in quantization, we want to define the space Q of functions such that
AQ V --- E s.t. Vo, v (Q())))) *
= x
(1)
3

---

Published as a conference paper at ICLR 2023
square root distribution
uniform distribution
square distribution
Figure 2:
Influence of the power parameter the quantized distribution for weights distributed following Gaussian prior. In such case, the reconstruction error typically minimized for a
where * is the intern composition law of the quantized space and x is the standard multiplication, and Q. Q- are the quantization and de-quantization operators, respectively. In the case of uniform quantization and our work * will be the multiplication while in other non-uniform works it often corresponds other operations that are harder leverage, e.g. bit-shift. Recall that, for now, omit the rounding operator. The proposed PowerQuant method consists in the search for the best suited operator Q for a given trained neural network and input statistics.
3.1 AUTOMORPHISMS OF (R2, x)
Let Q be transformation from R., to R... In this case, *, the intern composition law in quantized space in (T) simply denote the scalar multiplication operator x and (I) becomes Q(x) x Q(y) Q(x y) Var, y E TR2. In order define de-quantization operation, i.e. Q is bijective. Thus, by definition. Q is group automorphism of (R x). Thus, quantization operators that preserve the nature of multiplications are restricted automorphisms of (R", x). The following lemma further restricts the search to power functions.
need Q to be defined
Lemma 1. The set of continuous automorphisms of [R4, x) is defined by the set of power functions
Q {Q x: x= E R..
A proof of this result can be found in Appendix A For the sake of clarity, we will now include the rounding operation in the quantization operators.
Q { Qa @Q:WH+
(2 WWn x Wi
la R
(2)
max W
where W is a tensor and all the operations are performed element-wise. As functions of W, the quantization operators defined in equation are (signed) power functions. Fig illustrates the effect of the power parameter on quantization (vertical bars). Uniform quantization and = 1,
are equivalent and correspond the quantization is more fine-grained on weight values with low absolute value and coarser on high absolute values. Conversely, for 1, the quantization becomes more fine- fine-grained on high absolute values. We now define the search protocol in the proposed search space Q.
quantization invariant to the weight distribution. For
3.2 AUTOMORPHISH SEARCH AS MINIMIZATION PROBLEM
We propose to use the error introduced by quantization on the weights as proxy on the distance between the quantized and the original model.
The operator Qa is not bijection. Thus, quantization introduces reconstruction error summed over all the layers of the network, and defined as follows:
Reconstruction Error Minimization:
el
c(F, a)
Σ ||, (
where || 1lp denotes the IP vector norm (in practice p 2, see appendix and the de-quantization
operator Q-1 1 is defined as:
4

---

Published as a conference paper at ICLR 2023
Qw) = (W) x W x max [W|
(4)
2 -1
In practice, the problem of finding the best exponent a" argmin, (F, a) in is locally convex
optimization problem (Appendix(C.) which has unique minimum (see Appendix C.2). We find the optimal value for using the Nelder-Mead method (Nelder Mead 1965) which solves problems for which derivatives may not be known or, in our case, are almost-surely zero (due to the rounding operation). In practice, more recent solvers are not required in order to reach the optimal solution (see Appendix D). Lastly, we discuss the limitations of the proposed metric in Appendix H
3.3
FUSED DE- DE-QUANTIZATION AND ACTIVATION FUNCTION
Based on equation 2 the quantization process of the weights necessitates the storage and multipli- cation of W along with signs tensor, which is memory and computationally intensive. For the weights. however, this can be computed once during the quantization process, inducing no overhead during inference. As for activations, we do not have to store the sign of ReLU activations they are always positive. In this case, the power function has to be computed at inference time (see al- gorithm 2). However, it can be efficiently computed Kim al. (2021), using Newton's method to approximate continuous functions in integer-only arithmetic. This method is very efficient in prac- tice it converges in 2 steps for low bit representations (four steps for int32). Thus, PowerQuant leads to significant accuracy gains with limited computational overhead. Conversely, for non-ReLU feed forward networks such as EfficientNets (SiLU) Image Transformers (GeLU), activations signed. This can be tackled using asymmetric quantization which consists in the use of a zero-point. In general, asymmetric quantization allows one to have a better coverage of the quantized values support. In our case, we use asymmetric quantization to work with positive values only. Formally, for both SiLU and GeLU, the activations are analytically bounded below by CSILU 0.27846 and CGa. = 0.169971 respectively. Consequently, assuming a layer with SiLU activation with input and weights W, we have:
Q=¹ (Qa(æ + CSIL)) ~ ((x ((s))" = xW + CSiLUW
(5)
The bias term CSILU W induces very slight computation overhead which is standard in asymmetric quantization. We provide detailed empirical evaluation of this cost in Appendix G Using the adequate value for the bias corrector, can generalize equation to any activation function a. The quantization process and inference with the quantized DNN are summarized Algorithm T and [ The proposed representation is fully compatible with integer multiplication defined in Jacob al. 2018), thus it is fully compatible with integer only inference (see appendix F for more details).
Algorithm 1 Weight Quantization Algorithm
Require: trained neural network F with L layers to quantize, number of bits b > in practice we use the Nelder-Mead method
+ (errermmimerro((FF a)))
for 1 E (1, .1. I.] do
Wsign 4- WW,
D save the sign of the scalar values in W
Wi + Wsign |W/]
>> power transformation >> get quantization scale gdefine Q and Q-1
Wil
8 - 2
Q: W1 + [ and Q-! W + Wsign W x s/a
end for
4 EXPERIMENTS
In this section, we empirically validate our method. First, we discuss the optimization of the expo- nent parameter a of PowerQuant using the reconstruction error, showing its interest as proxy for the quantized model accuracy from an experimental standpoint. We show that the proposed approach preserves this reconstruction error significantly better, allowing closer fit to the original weight dis- tribution through non- uniform quantization. Second, we show through a variety of benchmarks that the proposed approach significantly outperforms of-te-of-the-art data-free methods, thanks to more efficient power function quantization with optimized exponent. Third, we show that the proposed approach comes negligible cost in term of inference speed.
5

---

Published as a conference paper at ICLR 2023
Algorithm 2 Simulated Inference Algorithm
Require: trained neural network F quantized with L layers, input X and exponent a"
for / {l....: L} do
X f- X
X is assumed positive (see equation ( where sx is a scale in the imput range where we applied + the accumulation D where is the activation function and sw the weight scale
X + [Xax]
0 R((X)
X + a(0)
sxsw
end for
ResNet-Se 50 W4/A4
DenseNel-12 121 W4/14
80
60
Resomitnationn anor
60
2,008-2
1,00E-2
Acuracy
Aosurac
1,505-2
1,00E
6,00E
20
20
5,00E-3
0.008-0
5,005-0
Exponenta
Exponenta
Figure 3: Accuracylreconstruutionn error relationship ResNet and DenseNet quantized W4/A4.
4.1 DATASETS AND IMPLEMENTATION DETAILS
We validate the proposed PowerQuant method on ImageNet classification (Deng et al. 2009)
(== 1.2M images train/50k test). In our experiments we used pre-trained MobileNets (Sandler et al. 2018), ResNets (He et al. 2016). EfficientNets (Tan & Le 2019) and DenseNets (Huang al. 2017). We used Tensorflow implementations of the baseline models from official repositories, achieving standard baseline accuracies. The quantization process was done using Numpy library. Activations are quantized as unsigned integers and weights are quantized using a symmetric repre- sentation. We fold aatch-normalization layers in Yvinec et al. (2022a). We performed ablation study using the uniform quantization operator over weight values from Krishnamoorthi (2018) and logarithmic quantization from Miyashita et al. (016)). For our comparison with state-of-lhe of-the-art ap- proaches in data-free quantization, we implemented the more complex quantization operator from SQuant (Cong al. 2022). To compare with strong baselines, also implement bias correction (Nagel et al. (2019) (which measures the expected difference between the outputs of the original and quantized models and updates the biases terms to compensate for this difference) as well input weight quantization (agel et al. 2019).
4.2 EXPONENT PARAMETER FITTING
Fig 3 illustrates the evolution of both the accuracy of the whole DNN and the reconstruction error summed over all the layers of the network, functions of the exponent parameter Our target is the highest accuracy with respect to the value of a: however, in data-free context, we only have access the reconstruction error. Nevertheless, shown Fig 3 these metrics strongly anti- correlated. Furthermore, while the reconstruction curve is not convex it behaves well for simplex based optimization method such as the Nelder-Mead method. This is due to two properties: locally convex (Appendix C.1 and has unique minimum (Appendix
Empirically, optimal values a* for the exponent parameter are centered on 0.55, which approxi- mately corresponds to the first distribution in Fig 2 Still, as shown on Table [ we observe some variations on the best value for a which motivates the optimization of for each network and bit- width. Furthermore, quantization. In the pruning literature (Han et al. 2015 Frankle & Carbin 2018 Molchanov et al. 2019), the baseline method consists in setting the smallest scalar weight values zero and keeping unchanged the highest non-zero values, assuming that small weights contribute less to the network prediction. In similar vein, logarithmic power quantization with > 1 roughly quantizes
results provide novel insight on the difference between pruning and

---

Published as a conference paper at ICLR 2023
Table 1: Comparison between logarithmic, uniform and the proposed quantization scheme ResNet 50 trained for ImageNet classification task. We report for different quantization configuration (weights noted and activations noted A) both the top1 accuracy and the reconstruction emoor (equation
Architecture
Method
W-bit
A-bit
a"
Accuracy
Reconstruction Error
Baseline
32
32
76. 15
uniform
8
76.15
1.1 x10
logarithmic
8
8
76. 12
2.0 x10-
ResNet 50
PowerQuant
8
0.55
76.15
1.0 x10
uniform
4
54.68
3.5 x10
logarithmic
4
57.07
2.1 x10
PowerQuant
4
0.55
70.29
1.9 x10-3
Table 2: Comparison between state-of satte-of-thh-ar tre-hh--rttt post training quantization techniques on ResNet 50 on ImageNet. We distinguish methods relying on data (synthetic or real) or not. In addition to being fully data-free, our approach significantly outperforms existing methods.
Architecture
Method
Data
W-bit
A-bit
Accuracy
gap
Bascline
32
32
76.15
DFQ Nagel et al.
2019
No
8
75.45
-0.70
ZeroQ Cai al. 2020)
Synthetic
8
75.89
-0.26
DSG Zhang et al. 202Tb
Synthetic
8
75.87
-0.28
GDFQ
2020)
Synthetic
8
75.71
-0.44
SQuant Cong et al.
(2022
No
8
8
76.04
-0.11
ResNet 50
PowerQuam
No
76.15
0.00
DFQ Nagel et al.
(2019)
No
0.10
76.05
Zero Cai ai al.
2020)
Synthetic
7.75
-68.40
DSG Zhang et al. 202 bb)
Synthetic
4
4
23.10
-53.05
GDFQ et al.
2020
Synthetic
55.65
-20.50
SQuant Cong
(2022
No
4
4
68.60
-7.55
PowerQuant
No
70.53
-5.62
(almost zeroing it out) small scalar values to better preserve the precision larger values. In prac- tice, in our case, lower reconstruction errors, and better accuracies, are achieved by setting a 1: this suggests that the assumption behind pruning can't be straightforwardly applied to quantization, where fact we argue that finely quantizing smaller weights paramount preserve the patterns learned at each layer, and the representation power of the whole network.
Another approach that puts more emphasis on the nuances between low valued weights is logarith- mic based non-uniform quantization. In Table D and Appendix we compare the proposed power method to both uniform and logarithmic approaches. By definition, the proposed power method nec- essarily outperforms the uniform method in every scenario uniform quantization is included in the search space. For instance, in int4, the proposed method improves the accuracy by 13.22 points on ResNet 50. This improvement can also be attributed a better input quantization of each layer, especially on ResNet 50 where the gap in the reconstruction error (over the weights) is smaller.
4.3 COMPARISON WITH DATA-FREE QUANTIZATION METHODS
In table we report the performance of several data-free quantization approaches on ResNet 50. Although no real training data is involved in these methods, some approaches such as ZeroQ (Cai al. 2020). DSG (Zhang et al.) 2021b) or GDFQ (Xu et al. 2020) rely on data generation (DG) in order to calibrate parameters of the method or apply fine-tuning to preserve the accuracy through quantization. As shown table 2 in the W8/A8 setup, the proposed PowerQuant method outper- forms other data-free solutions, fully preserving the accuracy of the floating point model. The gap is even wider on the more challenging low bit quantization W4/A4 setup. where the PowerQuant im- proves the accuracy by 1.93 points over SQuant (Cong et al. (222)) and by 14.88 points over GDFQ. This shows the effectiveness of the method on ResNet 50. We provide more results on DenseNet (Huang et al. 2017), MobileNet (Sandler et al. 2018). Efficient Net (Tan & Le 2019) in Appendix

---

Published as a conference paper at ICLR 2023
Table 3: Comparison of data-frce free quantization methods VTT and DeiT trained ImageNet.
model
method
W A
accuracy
model
method
W A
accuracy
baseline
-/-
78.05%
baseline
-/-
72.21%
DEQ (ICCV 2019)
8/8
70.33%
DEQ (ICCV 2019)
8/8
71.32%
SQuant (ICLR 2022)
8/8
68. 68.85%
SQuant (ICLR 2022)
8/8
71.11%
PSAQ (arxiv 2022)
8/8
37.36%
PSAQ (arxiv 2022)
8/8
71.56%
ViT
PowerQuant
8/8
77.46%
T
PowerQuant
8/8
72.23%
DFQ (ICCV 2019)
478
66.63%
DFQ (ICCV 2019)
4/8
67.71%
SQuant (ICLR 2022)
4/8
64.62%
SQuant (ICLR 2022)
4/8
67.58%
PSAQ (arxiv 2022)
4/8
25.34%
PSAQ (arxiv 2022)
4/8
65.57%
PowerQuant
4/8
75.24%
PowerQuant
4/8
69.77%
(a) Evaluation for VIT Base
(b) Evaluation for DeiT Tiny
model
method
W W/A A accuracy
model
method
W/A A accuracy
baseline
-/-
79.85%
baseline
-/-
81.85%
DFQ (ICCV 2019)
8/8
78.76%
DFQ (ICCV 2019)
8/8
80.72%
SQuant (ICLR 2022)
8/8
78.94%
SQuant (ICLR 2022)
8/8
80.60%
PSAQ (arxiv 2022)
8/8
76.92%
PSAQ (arxiv 2022)
8/8
79.10%
DeiT
PowerQuant
8/8
79.33%
DeiT B
PowerQuant
8/8
81.26 %
(ICV 2019)
4/8
76.75%
DFQ (ICCV 2019)
4/8
79.41%
SQuant (ICLR 2022)
4/8
76.61%
SQuant (ICLR 2022)
4/8
79.21% %
PSAQ (arxiv 2022)
4/8
73.23%
PSAQ (arxiv 2022)
4/8
77.05%
PowerQuant
4/8
78.16%
PowerQuant
4/8
80.67%
(c) Evaluation for DeiT Small
(d) Evaluation for DeiT Base
These results demonstrate the versatility the method on both large and very compact convnets. In summary, the proposed PowerQuant vastly outperforms other data-free quantization schemes. Last but not least, when compared to recent QAT methods such as OCTAV Sakr al. (2022). Pow- erQuant achieves competitive results on both ResNets and MobileNets using either both static or dynamic quantization. This remarkable since PowerQuant does not involve any fine-tuning of the network. We provide more details this benchamrk in Appendix In what follows, we evaluate PowerQuant on recent transformer architectures for both image and language applications.
4.4 EVALUATION ON TRANSFORMER ARCHITECTURES
In Table 3. we quantized the weight tensors a Dosovitty et al. (202T) with 85M parameters and baseline accuracy R 78 as well as DeiT T,S and B Touvron al. (2021) ) with baseline accuracies 72.2, 79.9 and 81.8 and x 5M, 23 22M, x 87M parameters respectively. Similarly to ConvNets, the image transformer is better quantized using PowerQuant rather than standard uniform quantization schemes such as DFQ. Furthermore, more complex and recent data-free quantization schemes such the novel Transformer architectures as compared to ConvNets.
SQuant, tend to under-perfore This is not the case for PowerQuant which maintains its very high performance even in low bit representations. This is best illustrated VIT where PowerQuant W4/A8 out performs both DFQ and SQuant even when they are allowed 8 bits for the weights (88/88) by a whopping 4.91 points. The proposed PowerQuant even outperforms methods dedicated transformer quantization such as PSAQ Li et al.] (2022) on every image transformer tested. We further compare the proposed power quantization, W4/A8, on natural language processing (NLP) tasks and report results Table We evaluate BERT model (Devlin et al. 2018) GLUE (Wang et al.) 2018) and report both the original (reference) and our reproduced (baseline) results. We compare the three quantization processes: uniform, logarithmic and PowerQuant. Similarly to computer vision tasks, the power quantization outperforms the other methods in every instances which further confirms its ability to generalize well to transformers and NLP tasks. In what follows, show experimentally that our approach induces very negligible overhead at inference time, making this accuracy enhancement virtually free from computational standpoint.
8

---

Published as a conference paper at ICLR 2023
Table Complementary Benchmarks the GLUE task with the BERT transformer architecture quantized in W4/A8. provide the original performance (from the article) as well our reproduced results (baseline).
task
original
baseline
uniform
log
SQuant
PowerQuant
CoLA 49.23
47.90
45.60
45.67
46.88
47.11
SST-2
91.97
92.32
91.81
91.53
91.09
92.23
MRPC
89.47/85.29
89.32/85.41
88.24/84.49
86.54/82.69
88.78/85.24
89.26/85.34
STS-B
83.95/83.70
84.01/83.87
83.89/83.85
84.01/83.81
83.80/83.65
84.01/83.87
QQP
88.40/84.31
90.77/84.65
89.56/83.65
90.30/84.04
90.34/84.32
90.61/84.45
MNLI
80.61/81.08
80.54/80.71
78.96/ 96796/99.1
78.96/79.71
78.35/79.56
79.02/80.28
QNLI
87,46
91,47
89.36
89.52
90.08
90.23
RTE
61.73
61.82
60.96
60.46
60.21
61.45
WNLI
45.07
43.76
39.06
42.19
42.56
42.72
Table 5: ACE cost of the overhead computations introduced by PowerQuant.
Architecture
overhead cost
accuracy W6/A6
ResNet 50
0.63%
75.07
DenseNet 121
0.97%
72.71
MobileNet V2
0.57%
52.20
EfficientNet BO
0.80%
58.24
4.5 INFERENCE COST AND PROCESSING TIME
The ACE metrics was recently introduced in Zhang et al. (2022) to provide a hardware-agnostic measurement of the overhead computation cost in quantized neural networks. In Table evaluate the cost the inference graph due to the change the activation function. We observe very similar results Table The proposed changes are negligible terms of computational cost all tested networks. Furthermore, DenseNet has the highest cost due to its very dense connectivity. On the other hand, using this metric it seems that the overhead cost due the zero-point technique from section 3.3 for EfficientNet has no significant impact as compared to MobileNet and ResNet. In addition, we discuss the inference and processing cost of PowerQuant on specific hardware using dedicated tools in Appendix K
5 CONCLUSION
In this paper, we pinpointed the uniformity of the quantization as a limitation of existing data- free methods. To address this limitation, proposed a novel data-free method for non-uniform quantization of trained neural networks for computer vision tasks, with an emphasis on not chang- ing the nature of the mathematical operations involved (e.g. matrix multiplication). This led to search among the continuous automorphisms of (RI, x), which are restricted to the power functions + xa We proposed an optimization of this exponent parameter based upon the reconstruction er- between the original floating point weights and the quantized ones. We show that this procedure locally convex and admits a unique solution. At inference time, the proposed approach, dubbed PowerQuant, involves only very simple modifications in the quantized DNN activation functions. We empirically demonstrate that PowerQuant allows a closer to the original weight distributions compared with uniform or logarithmic baselines, and significantly outperforms existing methods in a variety of benchmarks with only negligible computational overhead at inference time. In addi- tion, also discussed and addressed some of the limitations in terms of optimization (per-layer or global) and generalization (non-ReLU networks). Future work involves the search of better proxy error as compared with the proposed weight reconstruction error as well as the extension of the search space to other composition laws of R, that are suited for efficient calculus and inference.
ACKNOWLEDGMENTS
This work has been supported by the french National Association for Research and Technology (ANRT), the company Datakalab (CIFRE convention C20/1396) and by the French National Agency (ANR) (FacIL, project ANR- NORR-17-CE3--A 7-CE33-0002). This work was granted access to the HPC resources of IDRIS under the allocation 2022-AD01 1013388 made by GENCI.

---

Published as a conference paper at ICLR 2023
REFERENCES
Ron Banner, Yury Nahshan, and Daniel Soudry. Post training 4-bit quantization of convolutional networks for rapid-deployment. NeurIPS, pp. 7950-7958, 2019.
Yash Bhalgat, Jinwon Lee, Markus Nagel, Tijmen Blankevoort, and Nojun Kwak. Lsq+: Improving low-bit quantization through learnable offsets and better initialization. CVPR Workshops, pp- 696-697, 2020.
Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Zeroq: novel zero shot quantization framework. CVPR, pp. 13169-13178, 2020.
Wenlin Chen, James Wilson, et al. Compressing neural networks with the hashing trick. JCML, pp. 2285-22944 2015.
Vladimir Chikin and Vladimir Kryzhanovskiy. Channel balancing for accurate quantization of wino- grad convolutions. In CVPR, pp. 12507-12516, 2022.
Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural net- works for efficient inference. ICCV Workshops, pp. 3009-3018, 2019.
Guo Cong, Qiu Yuxian, Leng Jingwen, Gao Xiaotian, Zhang Chen, Liu Yunxin, Yang Fan, Zhu Yuhao, and Guo Minyi Squant: On-the-fly data-free quantization via diagonal hessian approxi- mation. ICLR, 2022.
Andrew R Conn, Katya Scheinberg, and Ph L Toint. On the convergence of derivative-free methods for unconstrained optimization. Approximation theory and optimization: tributes to MJD Powell, pp. 83-108, 1997.
Deng, Dong, et ImageNet: A Large-Scale Hierarchical Image Database. CVPR, 2009.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 18X.v:/810.04:00 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
Jun Fang, Ali Shafiee, Hamzah Abdel-Aziz, David Thorsley, Georgios Georgiadis, and Joseph H Hassoun. Post-training piecewise linear quantization for deep neural networks. ECCV, pp. 69- 86, 2020.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. ICLR, 2018
Sahaj Garg, Anirudh Jain, Joe Lou, and Mitchell Nahmias. Confounding tradeoff's for neural net- work quantization. arXiv preprint arXiv:2102.063666 2021.
Amir Gholami, Schoon Kim, Zhen Dong. Zhewei Yao, Michael Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. arXiv preprint 103.v:2003.136300 2021.
Yunchao Gong, Liu Liu, Ming Yang, and Lubomir Bourdev. Compressing deep convolutional net- works using vector quantization. arXiv preprint arXiv: 151..6155. 2014.
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurlPS, 27, 2014.
Zbigniew Hajduk. High accuracy fpga activation function implementation for neural networks.
Neurocomputing, 247:59-61. 2017.
Kun Han, Yuxuan Wang, DeLiang Wang. William Woods, Ivo Merks, and Tao Zhang. Learning spectral mapping for speech dereverberation and denoising. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 2386)9982 2015.
10

---

Published as a conference paper at ICLR 2023
Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. ICLR, 2016.
Kaiming He, Xiangyu Zhang, et al. Deep residual learning for image recognition. CVPR, pp. 770-778, 2016.
Horst Herrlich. Axiom of choice, volume 1876. Springer, 2006.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. NeurlPS, 2014.
Distilling the knowledge a neural network.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. CVPR, pp- 4700-4708, 2017.
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El- El-Yaniv, and Yoshua Bengio. Binarized neural networks. NewrIPS, 29, 2016.
Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient intege-arithmetic-onf inference. CVPR, pp. 2704-2713, 2018.
Yongkweon Jeon, Baeseong Park, Se Jung Kwon, Byeongwook Kim, Jeongin Yun, and Dongsoo Lee. Biggemm: matrix multiplication with lookup table for binary-coding-basee quantized dans. SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-14, 2020.
Yongkweon Jeon, Chungman Lee, Euirang Cho, and Yeonju Ro. Mr. biq: Post-training non-uniform quantization based on minimizing the reconstruction error. CVPR, pp. 12329-12338, 2022.
Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. I-bert: Integer- only bert quantization. In International conference on machine learning, pp. 5506-5518, PMLR, 2021.
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint vrXiv:1806.08344 2018.
Maximilian Lam, Michael Mitzenmacher, Vijay Janapa Reddi, GeoYeen Wei, and David Brooks. Tabula: Efficiently computing nonlinear activation functions for secure neural network inference. arXiv preprint 1:XXv:2203.0283: 2022.
Yuhang Li, Xin Dong, and Wei Wang. Additive powers-of-two quantization: An efficient non- uniform discretization for neural networks. arXiv preprint 1rXiv:1909.13144, 2019.
Zhikai Li, Liping Ma, Mengjuan Chen, Junrui Xiao, and Qingyi Gu. Patch similarity aware data-free quantization for vision transformers. arXiv preprint arXiv:2203.02255, 2022.
Daisuke Miyashita, Edward H Lee, and Boris Murmann. Convolutional neural networks using logarithmic data representation. arXiv preprint arXiv: 1603.000033 2016.
Pavlo Molchanov, Arun Mallya, Stephen Tyree, luri Frosio, and Jan Kautz. Importance estimation for neural network pruning. CVPR, pp. 11264-11272, 2019.
Markus Nagel, Mart van Baalen, et al. Data-free quantization through weight equalization and bias correction. ICCV, pp. 1324-1334, 2019.
John A Nelder and Roger Mead. A simplex method for function minimization. The computer journal, 7(4):308-315 -03,333 1965.
Michael JD Powell. An efficient method for finding the minimum of function of several variables without calculating derivatives. The computer journal, 1(2):155-162, 1964.
Charbel Sakr, Steve Dai, Rangha Venkatesan, Brian Zimmer, William Dally, and Brucek Khailany. Optimal clipping and magnitude-awar differentiation for improved quantization- aware training. In ICML, pp. 19123-191388 PMLR, 2022.
11

---

Published as a conference paper at ICLR 2023
Mark Sandler, Andrew Howard, et al. Mobilenetv2: Inverted residuals and linear bottlenecks. CVPR, pp. 4510-4520, 2018.
Mingxing Tan and Quoe V Le. Efficientnet: Rethinking model scaling for convolutional neural networks. ICML, pp. 6105-6114, 2019.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, pp. 10347-10357. PMLR, 2021.
Mart van Baalen, Brian Kahne, Eric Mahurin, Andrey Kuzmin, Andrii Skliar, Markus Nagel, and Tijmen Blankevoort. Simulated quantization, real power savings. In CVPR, pp. 2757-2761, 2022.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceed- ings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353-355, Brussels, Belgium, November 2018. Association for Computational Lin- guistics. doi: 10.18653/v1/118-544
Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, and Jian Cheng. Quantized convolutional neural networks for mobile devices. CVPR. pp. 4820-4828, 2016.
Shoukai Xu, Haokun Li, Bohan Zhuang, Jing Liu, Jiezhang Cao, Chuangrun Liang, and Mingkui Tan. Generative low-bitwidth data free quantization. ECCV, pp. 1-17, 2020.
Edouard Yvinec, Arnaud Dapogny, and Kevin Bailly. To fold or not to fold: a necessary and sufficient condition on batch nomaalaomaaiiztioon layers folding. IJCAI, 2022a.
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly. Spiq: Data-free per-channel static input quantization. arXiv preprint arXiv:2203. 14642, 2022b.
Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua. Lq-nets: Learned quantization for highly accurate and compact deep neural networks. ECCV, pp. 365-382, 2018.
Sai Qian Zhang, Bradley McDanel, HT Kung, and Xin Dong. Training for multi restutors inference using reusable quantization terms. Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 845-860, 202
Xiangguo Zhang, Haotong Qin, Yifu Ding, Ruihao Gong, Qinghua Yan, Renshuai Tao, Yuhang Li, Fengwei Yu, and Xianglong Liu. Diversifying sample generation for accurate data-free quantiza- tion. CVPR. pp. 15658-15667, 2021b.
Yichi Zhang, Zhiru Zhang, and Lukasz Lew. Pokebnn: A binary pursuit of lightweight accuracy. In CVPR, pp. 12475-12485, 2022.
Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Chris De Sa, and Zhiru Zhang. Improving neural network quantization without retraining using outlier channel splitting. ICML, pp. 7543-7552, 2019.
Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, and Yurong Chen. Incremental network quantiza- tion: Towards lossless cnns with low-precision weights. ICLR, 2017.
Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa-net: Train- ing low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint 1oop:066606666 2016.
12

---

Published as a conference paper at ICLR 2023
A PROOF OF LEMMA I
In this section, we provide simple proof for lemma T as well discussion the continuity hypothesis.
Proof. We have that Ve R+,Q(x)) Q(0) Q(0) and v R4. ((x) Q(1) Q(x) which
induces that Q is either the constant or Q(0) = 0 and Q(1) = Because Q is an automorphism we can eliminate the first option. Now, will demonstrate that is necessarily power function. Let n be an integer, then
Q(=))= = Q(x) x Q(x Q(x)² x =-
(6)
Similarly, for fractions, we get Q(==) ) x .... continuous, we deduce that for any rational I. we have
... x (x=+)(=+=))))))
) = Q(x) + Q(x=t Q(x)=. Assuming Q
=(=")= (x))
In order to verify that the solution is limited to power functions, Assume Q is not a power function. Therefore, there exists y) E R and a E R such that Q(x) A xd and Q(y) y". By definition of the logarithm, there exists such that -¹ y. We get the following contradiction, from (7).
reductio ad absurdum.
Q(x4)) Q(19) = yd 10xbba) Q(240)) Q(a9)) 60 * =1a)=1²²)
(8)
Consequently, the suited functions Q are limited power functions ie. = 1Q + x R].
We would also like to put the emphasis the fact that there are other Automorphisms of (R, x
However, the construction of such automorphisms require the axiom of choice Herrlich (2006). Such automorphisms automorphism rather than the continuous property.
applicable in case which is why the key constraint is being
B
NORM SELECTION
In the minimization objective, we need to select a norm to apply. In this section, we provide the- oretical arguments in favor of the 12 vector norm. Let F be feed forward neural network with L layers to quantize, each defined by a set of weights W = (wr)). E R and bias by E IR" We note (X(1)), the eigenvalues associated with Wi. We want study the distance d(F, between the predictive function F and its quantized version Fú defined as
d( F, F.) max FCD
[F(x) Fa (x)
where D is the domain of F. We prove that minimizing the reconstruction error with respect to is equivalent to minimizing d(F, F.) with respect to a. Assume L for the sake of simplicity and drop the notation l. With the proposed PowerQuant method, we minimize the vector norm
| Q- Q7
Σ max Wi.j - (Q
(10)
For 2, the euclidean norm is equal the spectral norm, thus minimizing | (Q.
equivalent to minimizing d(F, F.) for = 1. However, we know that minimizing for another value of p may result in different optimal solution and therefore not necessarily minimize d(F, F.).
In the context of data-free quantization, we want to avoid uncontrollable changes on F, which is why recommend the use of 2.
13

---

Published as a conference paper at ICLR 2023
C MATHEMATICAL PROPERTIES
C.I LOCAL CONVEXITY
We prove that the minimization problem defined in equation Bis locally convex around the solution a'. Formally we prove that
x - || Q₂ (Qo(z))
(11)
locally convex around a" defined as arg min, Lemma 2. The minimization problem defined as
|| Qu (Q.(z)))lp
arg min {|I -- (Qu()))))
(12)
locally convex around any solution a*.
0x Proof. We recall that da x log(z). The function ||> - Q- (Qa(z)) is differentiable. We
assume R, then can simplify the sign functions (assume positive without loss of generality) and note y = max [z), then
(Qa()) 0 0
2
(13)
Oa
Da
This simplifies to
BB(B) B
aQ ' (Qo(x)) da
(14)
da
with B 26-1 By using the standard differentiation rules, we know that the rounding operator has derivative a.e.. Consequently we get,
1
aQ_¹ (Qa(z)) aa
B
B
-a²y
log
(15)
B
B
Now we can compute the second derivative of Qn (Qa(z)).
1Q0'1QQl0 =ayy
B
B 12
log²
(16)
da²
B
B
From this expression, derive the second derivative, using the property (fog)" fo og g²2 +
I' o x g" and the derivatives
+ and
1
1-p læ 22 then for any E
8 x4 Q (Qn():))
[x; (Q.
[QQ=1 (Qa(z))
2
Da²
p (z, - (Qu
da
(17)
(x; Q- (Qu(()):)),;; (='(Qu(r:)) -- 8²Q (Qa(z))
p
da
1-p |z Q (Q.()))
(QQ (Q_
2
We now note the first term the previous addition
and
p Q (2,))
da
the second term as a product of = (#--Q (2.1) x; Q- (Q:)) times T3 (20,1
do
We know that Ti > 0 and T₃ > 0, consequently, and T2 is continuous in a. At a" the terms with b2y O- (v.)).)) are negligible in comparison with "² and (Qn()()
da
Consequently, there exists an open set around a" where Tj > [3 and 0a Q- (Qn()))) This concludes the proof.
O.
du
14

---

Published as a conference paper at ICLR 2023
Table 6: Minimization of the reconstruction MobileNet V2 for W6/A6 quantization with different solvers.
Solver
a²
reconstruction emror
accuracy
Nelder-Men
0.750
1.12
64.248
Powell (Powell 1964)
0.744
1.10
64.104
COBYLA Coon al. 1997)
0.752
1.11
64.364
C.2 UNIQUENESS OF THE SOLUTION
In this section provide the elements of proof on the uniqueness of the solution of the minimization of the quantization reconstruction error.
Lemma 3. The minimization problem over E RRN defined as
arg min [1 Q (Qa(z)))lp)
(18)
has almost surely unique global minimum a".
Proof. We assume that x can not exactly quantized, i.e. min, f1I-- (Qa(z)) which is true almost everywhere. We use reductio ad absurdum and assume that there ex- ist two optimal solutions a1 and ag to the optimization problem. We expand the expression r Qa (Qa(z)) Il, and get
>
sign(a)) |a:|0 max |ar (2 (2
--- (Qa(z)))l, (0, -
sign (I)
(19)
max 22 a
26-1
We note the rounding term R. and get
max |x:
|| Qu (Qa(z)))l,
R. 2
sign(x)
(20)
Assume Ra, Ray R, the minimization problem arg mina
R
max 1
sign(2))
is
20-1
p
convex and has unique solution, thus a₁ a2 Now assume Ra, o Raz
Let's denote ((R) the domain of power values over which we have (26-1-1) sign(=)
max [ R.
there is a value outside of D U D(Rag) such that R' has each of its coordinate strietly between the coordinates of Rg, and Ruz- then, without loss generality, assume that least half of the coordinates of Ra, are further away from the corresponding coordinates of x than one quan- tization step. This implies that there exists a value a' in D(I') such that || QT' (Q)) <
J - Q (Q,, (x)) Ilp which goes against our hypothesis. Thus, there are up to possible values for R. that minimize the problem which happens iff x satisfies at least one coordinate can be either ceiled or floored by the rounding. The set defined by this condition has zero measure.
D SOLVER FOR MINIMIZATION
In the main article we state that we can use Nelder-Mead (Nelder & Mead 1965) solver to find the optimal a" We tested several other solvers and report the results in Table 6. The empirical results show that basically any popular solver can be used, and that the Nelder-Mead solver is sufficient for the minimization problem.
E COMPARISON BETWEEN LOG, NAIVE AND POWER QUANTIZATION COMPLEMENTARY RESULTS
To complement the results provided in the main paper on ResNet 50, we list in Table 7 more quan- tization setups ResNet 50 as well DenseNet 121. To put in nutshell, The proposed power
15

---

Published as a conference paper at ICLR 2023
Table 7: Comparison between logarithmic, uniform and the proposed quantization scheme ResNet 50 and DenseNet 121 trained ImageNet classification task. We report different quantization configuration (weights noted W and activations noted A) both the top1 accuracy and the reconstruction error (equation
Architecture
Method
W-bit
A-bit
Accuracy
Reconstruction Error
Baseline
32
32
76.15
uniform
8
76.15
1.1 x10
logarithmic
8
8
76.12
2.0 x100
PowerQuant
8
0.55
76.15
1.0 x10
ResNet 50
uniform
6
75.07
8.0 x10+
logarithmic
6
6
75.37
4.6 x0000
power (ours)
6
0.50
75.95
4.3 x10-1
uniform
4
54.68
3.5 x10-3
logarithmic
4
4
57.07
2.1 x10-3
PowerQuant
4
4
0.55
70.29
1.9 x1003
Bascline
32
32
75.00
uniform
8
8
75.00
2,8 x10
logarithmic
8
74.91
2.5 1,5x100-
PowerQuant
8
0.60
75.00
2.2 x10000
DenseNet 121
uniform
6
6
74.47
1.1 x10
logarithmic
6
72.71
1.0 x10-3
power (ours)
6
6
0.50
74.84
0.7 x10-
uniform
4
4
54.83
4.7 x10-3
logarithmic
4
5.28
x10-3
PowerQuant
4
4
0.55
68.04
3.1 x10-3
quantization systematically achieves significantly higher accuracy and lower reconstruction error than the logarithmic and uniform quantization schemes. On side note, the poor performance of the logarithmic approach on DenseNet 121 can be attributed to the skewness of the weight distributions. Formally, ResNet 50 and DenseNet 121 weight values show similar average standard deviations across layers (0.0246 and 0.0264 respectively) as well as similar kurtosis (6. (6.905 and 6.870 respec- tively). However their skewness are significantly different: 0.238 for ResNet 50 and more than twice as much for DenseNet 121, with 0.489. The logarithmic quantization, that focuses on very small value is very sensible to asymmetry which explains the poor performance on DenseNet 121. In contrast, the proposed method offers robust performance in all situations.
F How TO PERFORM MATRIX MULTIPLICATION WITH POWERQUANT
The proposed PowerQuant method preserves the multiplication operations, i.e. a multiplication in the floating point space remains multiplication in the quantized space (integers). This allows one to leverage current implementations of uniform quantization available on most hardware Gholami al.] I); Zhou al. (016). However, while PowerQuant preserves multiplications it doesn't preserve additions which are significantly less costly than multiplications. Consequently, in order to infer under the PowerQuant transformation, instead accumulating the quantized products, as done in standard quantization Jacob al. (2018), need accumulate the powers of said products. Formally. let's consider two quantized weights W1, W2 and their respective quantized inputs Ø1, 22- The standard accumulation would be performed follows w 21 + W2Z2. In the case of PowerQuant, this would be done (w x + (w22z2) 1 Previous studies on quantization have demonstrated that such power functions can be computed with very high fidelity at almost no latency cost Kim et al. 2021
G OVERHEAD COST OF ZERO-POINTS IN ACTIVATION QUANTIZATION
The overhead cost introduced in equation is well known in general in quantization it arises from asymmetric quantization. Nonetheless, we share here (as well as in the article) some empirical values.
16

---

Published as a conference paper at ICLR 2023
Table Overhead induced by asymmetric quantization
Architecture
parameters overhead
run-time overhead (CPU intel-m3)
ResNet50
0,25%
4.35%
EfficientNet
0.20%
3.38%
ViT bl6
0.73%
5.14%
Table 9: Comparison between the per-layer and global method of power parameter fitting ResNet 5à trained for ImageNet classification lask.
Architecture
Method
W-bit
A-bit
Accuracy
Reconstruction Error
Baseline
32
76.15
per-layer
8
76. 14
0.8 x10
ResNet 50
global
8
76.15
1.0 x x10
per-layer
64,19
1.7 x10 3
global
4
70.29
1.9 x10
These empirical results from be quantized using asymmetric quantization although in our research, we only applied asymmetric quantization SilU and GeLU based architectures. We included these results the appendix of the revised article. It is worth noting that according to LSQ+ Bhalgat et al. (2020), asymmetric quantization can be achieved virtually not run- run-time cost.
implementation. We include ResNet50 as it also
H LIMITATIONS OF THE RECONSTRUCTION ERROR METRIC
In the proposed PowerQuant method, we fit the parameter based on the reconstruction error over all the weights, i.e. over layers the whole network. Then, we perform per-channel quantization layer by layer independently. However, if the final objective is to minimize the reconstruction error from equation (3). a more efficient approach would consist in fitting the parameter a separately for each layer. We note of such that for every layer 1 we have
[[W Q. (Qa'(Q
af arg min
(21)
Then the network (F, (ai))) quantized with a per-layer fit of the power parameter will satisfy
L Σ - QQ.
Σ
- - (Q(((()))))
(22)
and only if their exists at least one / such that di e a. Consequently, if the reconstruction error was perfect estimate of the resulting accuracy, the per-layer strategy would offer even higher accuracy than the proposed PowerQuant method. Unfortunately, the empirical evidence, in table 9 shows that the proposed PowerQuant method achieves better results in every benchmark. This observation demonstrates the limits of the measure of the reconstruction error. We explain this phe- nomenon by the importance of inputs and activations quantization. This can be seen as some form of overfitting the parameters aj on the weights which leads to poor performance on the activation quantization and prediction. In the general sens, this highlights the limitations of the reconstruction етrоr as a proxy for maximizing the accuracy. Previous results can be interpreted in a similar way. For instance, SQuant Cong al. (2022) the author claim that better to minimize the abso- lute sum of errors rather than the sum of absolute errors and achieve good performance in data-free quantization.
I IMPROVEMENT WITH RESPECT TO QAT
In the introduction, we argued that data-driven quantization schemes performance define upper- bound on data-free performance. Our goal was to narrow the resulting gap between these methods. In Table [0] report the evolution in the gap between data-free and data-driven quantization tech- niques. These empirical results validate the significant improvement the proposed method at narrowing the gap between data-free and data-driven quantization methods by 26.66% 29. 79.74.
17

---

Published as a conference paper at ICLR 2023
Table 10: Performance Gap compared to Data-driven techniques ResNet 50 quantization in W4/A4, The
relative gap improvement to the state-of-the-an SQuant [6]. is measured as g6-e2 with gs SQuant
and
gp PowerQuant where * is the performance of a data-driven method
data-driven method
SQuant
PowerQuant
relative gap
OCTAV Sakr et al. (2022)
(ICML)
8,72%
6,15 %
+29.47%
SQ van Baalen et (2022)
(CVPR)
8,64%
6,07 %
+29,74%
WinogradQ Chikin & Kryzhanovski
(2022) (CVPR)
9,55%
7,00 %
+26,66%
Mr Beo et (2022) (CVPR)
8,74%
6,17% %
+29,38%
Table 11: Performance gap between data-free PowerQuant and short-retraining OCTAV Sakr et al.
2022).
method
architecture
quantization
accuracy
PowerQuant
ResNet 50
W4/A4
70.53
OCTAV
ResNet 50
W4/A4
75.84
PowerQuant MobileNet V2
W4/A4
45.84
OCTAV
MobileNet V2
W4/A4
0.66
Table 12: Performance gap between PowerQuant and OCTAV retraining), both using dynamic range estimation.
Sakr al.
(022
(using additional short
method
architecture
quantization
accuracy
PowerQuant
ResNet 50
W4/A4
76.02
OCTAV
ResNet 50
W4/A4
76.46
PowerQuant
MobileNet V2
W4/A4
71.65
OCTAV
MobileNet V2
W4/A4
71.23
In order to complete comparison to QAT methods, considered the short-re-trainngg (30 epochs) regime from OCTAV in Table [1 We can draw two observations from this comparison. First, ResNet 50, OCTAV achieves remarkable results by reach full-precision accuracy. Still the proposed method does not fall too far back with only 5.31 points lower accuracy while being data-free. Second, on very small models such MobileNet V2, using a strong quantization oper- ator rather than a short re-training leads to huge accuracy improvement as PowerQuant achieves 45.18 points higher accuracy. This is also the finding of the author in OCTAV, as they conclude that models such as MobileNet tend to be very challenging to quantize using static quantization and short re-triining.
In Table 12 we draw comparison between the proposed PowerQuant and the QAT method OCTAV Sakr et al. (2022), both using dynamic quantization (i.e. estimating the ranges of the activations on- the-fly depending
the input). As expected, the use of dynamic ranges has considerable influence on the performance both quantization methods. As can be observed the QAT method OCTAV achieved very impressive results and even outperforming the full-precision model on ResNet 50. Nevertheless, it on MobileNet that the influence of dynamic ranges the most impressive. For OCTAV, we observe boost of almost 71 points going from almost random predictions near exact full-precision accuracy. It is to be noted that PowerQuant does not fall shy in front of these perfor- mances, as using static quantization we still manage preserve the model. Furthermore, using dynamic quantization. Powerquant achieves similar accuracies than OCTAV while not involving any fine-tuning, contrary to OCTAV.
of the predictive capability of
All in all, we can conclude that the proposed data-free method manages to hold close results to state-of-the--rt QAT method in some context. interesting future work could be the extension of PowerQuant QAT method and possibly learning the power parameter a that we use in our quantization operator.
J COMPARISON TO STATE-OF-THE-AN DATA-FREE QUANTIZATION ON OTHER CONVNETS
In addition to our evaluation on ResNet. we propose some complementary results on DenseNet in Table 13 as well the challenging and compact architectures MobileNet and EfficientNet in Table
18

---

Published as a conference paper at ICLR 2023
Table 13: Comparison between state-of-the- -hettarrt post-training quantization techniques on DenseNet 121 on Im-
ageNet. We distinguish methods relying data (synthetic real) or not. In addition to being fully data-free our approach significantly outperforms existing methods.
Architecture
Method
Data
W-bit
A-bit
Accuracy
gap
Bascline
32
32
75.00
DFQ Nagel et al. (2019)
No
8
8
74.75
-0.25
SQuant Cong al. (2022
No
8
74.70
-0.30
OMSE Choukroun et (211
Real
8
8
74.97
-0.03
SPIQ Vineec al. (0226b
No
8
75.00
-0.00
DenseNet 121
PowerQuaa want
No
8
8
75.00
-0.00
DFQ Nagel et al.
(2019)
No
0.10
74.990
SQuant Cong
al.
(2022)
No
4
4
47.14
-27.86
SPIQ Vvnnc et al. (0022)
No
4
51.83
-23.17
OMSE
Choukroun et (2019)
Real
4
4
57.07
-17.93
PowerQuant
No
4
4
69.37
-5.63
Table 14: Complementary Benchmarks ImageNet
Architecture
Method
Data
W-bit
A-bit
Accuracy
gap
Baseline
32
32
71.80
DFQ (ICCV 2019)
No
8
70.92
-0.88
SQuant (ICLR 2022)
No
8
71.68
-0.12
SPIQ (WACV 2023)
No
8
8
71.79
-0.01
MobileNet V2
PowerQuant
No
8
71.81
+0.01
DFQ (ICCV 2019)
No
4
27.1
-44.70
SQuant (ICLR 2022)
No.
4
28.21
-43.59
SPIQ (WACV 2023)
No
4
31.28
-40.52
PowerQuant
No
45.84 4
25.96
Baseline
32
32
77.10
DFQ (ICCV 2019)
No
8
76.89
-0.21
SQuant (ICLR 2022)
No
8
76.93
-0.17
SPIQ (WACV 2023)
No
8
77.02
-0.08
EfficientNet BO
PowerQuant
No
8
77.05
-0.55
DFQ (ICCV 2019)
No
6
43.08
-3.022
SQuant (ICLR 2022)
No
6
54.51
-32.59
SPIQ (WACV 2023)
No
6 74.67
-2.43
PowerQuant
No
6
75.13
-1.97
14 as well weights only for Bert Table T6 In table 13 we report the performance other data-free quantization processes DenseNet 121. The OMSE method (Choukroun et al.) 2019) a post-training quantization method that leverages validation examples during quantization, thus cannot be labelled data-free. Yet, include this work in comparison they show strong performance in terms of accuracy at very low usage of real data. As showcased in table [13] the proposed PowerQuant method almost preserves the floating point accuracy in W8/A8 quantization. Additionally, on the challenging W4/A4 setup, our approach improves the accuracy by remarkable 12.30 points over OMSE and 17.54 points over SQuant. This is due to the overall better efficiency of non-uninn non-uniform quantization. that allows a theoretically closer fit to the weight distributions of each DNN layer. The results MobileNet and EfficientNet from Table confirm our previous findings. We observe significant boost in performance from PowerQuant compared to the other very competitive data-free solutions.
K OVERHEAD COST DISCUSSION
In this section, we provide more empirical results on the inference cost the proposed method. Table 17 shows the inference time of DNNs quantized with our approach (which only implies modi- fications the activation function and a bias correction-see Section(.3). For DenseNet. ResNet and MobileNet V2, the baseline activation function the ReL.U, which is particularly fast compute.
19

---

Published as a conference paper at ICLR 2023
Table 15: Complementary Benchmarks Vision Transformers for ImageNet
Architecture
Method
Data
W-bit
A-bit
Accuracy
gap
Baseline
32
32
78.524
DFQ (ICCV 2019)
No
8
77.612
-0.912
SQuant (ICLR 2022)
No
77.638 8
-0.886
CaiT xxs24
PowerQuant
No
8
77.718 8
-0.806
DFQ (ICCV 2019)
No
8
74.192
4.332
SQuant (ICLR 2022)
No
4
8
74.224
-4.00
PowerQuant
No
8
75.104
-3,420
Baseline
32
32
79.760
DFQ (ICCV 2019)
No
8
79.000
-0.760
SQuant (ICLR 2022)
No
8 78.914
-0.846
CaiT xxs36
PowerQuant
No
79.150 8
-0.610
DFQ (ICCV 2019)
No
8 76.906
-2.854
SQuant (ICLR 2022)
No
8
76.896
2.864
PowerQuant
No
8
77.702
2.058
Baseline
32
32
83.368
DFQ (ICCV 2019)
No
8
82.802
-0.566
SQuant (ICLR 2022)
No
8
82.784
-0.584
CaiT s24
PowerQuant
No
8
82.766
-0.602
DFQ (ICCV 2019)
No
8
81.474
-1,894
SQuant (ICLR 2022)
No
8
81,486
-1.882
PowerQuant
No
8
81.612
-1.756
Table 16: Complementary Benchmarks on the GLUE task. We consider the BERT transformer architecture.
We provide the reference performance of BERT GLUE as well reproduced results (baseline).
task
(reference)
baseline
uniform
log
power
CoLA 49.23
47.90
46.24
46.98
47.77
SST-2
91.97
92.32
91.28
91.85
92.32
MRPC
89.47/85.29
89.32/85.41
86.49/81.37
86.65/82.86
89.32/85.41
STS-B
83.95/83.70
84.01/83.87
83.25/83.14 14
84.01/83.81
84.01/83.87
QQP
88.40/84.31 90.77/84.65
90.23/84.61
90.76/84.65
90.77/84.65
MNLI
80.61/81.08
80.54/80.71
79.72/79.13
79.22/79.71
80.54/80.71
QNLI
87.46
91.47
90.32
91.43
91.47
RTE
61.73
61.82
59.23
61.27
61.68
WNLI
45.07
43.76
40.85
42.80
42.85
Nevertheless, results show that our approach leads to only increasing by 1% the whole inference time on most networks. More precisely, in the case of ResNet 50, the change in activation function induces slowdown of 0. 55. The largest runtime increase is obtained DenseNet with a 3.4% in- crease. Lastly, note that our approach is also particularly fast and efficient on EfficientNet BO, which uses SiLU activation, thanks the bias correction technique introduced in Section 3.3 Overall, the proposed approach can be easily implemented and induces negligible overhead in inference on GPU. To furthermore justify the practicality of the proposed quantization process, practicality concern that may arise is on the activation function as the other operations are strictly identical standard uniform quantization. According to Ko/Kim al. (202T) efficient power functions can be implemented for generic hardware long as they support standard integer arithmetic, i.e. as long as they support uniform quantization. When comes to Field-Prov Pooid--Pggaamaable Gate Array (FPGA), activation functions implemented using look-up tables (LUT) detailed in Hajduk 2017). More precisely, they are pre-computed using Padé approximation which are quotients of polynomial functions. Consequently the proposed approach would simply change the polynomial values but not the inference time as would still rely on the same number of LUTs.
recall that the only
In general, activation functions that non-linear be very effectively implemented in quantiza- tion runtime Lam et al. (2022). However these considerations are hardware agnostic. In order to circumvent this limitation and address any concerns to best, conducted small study using
20

---

Published as a conference paper at ICLR 2023
Table 17: Inference time, in seconds, ImageNet using batches of size 16 of several networks 2070
RTX GPU. We also report accuracy W6/A6 quantization setup.
Architecture
Method
inference time (gap)
accuracy
Uniform
164
75.07
ResNet 50
Power Function
164 (+0.2)
75.95
Uniform
162
72.71
DenseNet 121
Power Function
167 (+4.8)
74.84
Uniform
85
52.20
MobileNet V2
Power Function
86 (+0.7)
64.09
Uniform
125
58.24
EfficientNet BO
Power Function
127 (+2.2)
66.38
Table 18: Inference cost each component of convolutional layer and percentage of total in terms of number of cycles wide range of simulated hardware using nntool from GreenWavee
operation
number of cycles
number of
% of total cycles % of total ops
convolution bias
22950
442368
85.482%
99.310%
2033
1024
7.573%
0.229%
relu
924
1024
3.442%
0.229%
power function
940
1024
3.502%
0.229%
Table 19: We report the processing time in seconds an Intel(R) Core(TM) 19-9900K CPU) required quantize trained neural network such as ResNet, MobileNet, DenseNet EfficientNet.
Architecture
GDFQ
SQuant
Uniform
Power
MobileNet V2
7.10*
134
<1
<1
ResNet 50
11.10
320
<1
1.3
the simulation tool antool from GreenWaves, risc-v chips manufacturer that enables to simulate inference cost of quantized neural networks on their gap unit. We tested a single convolutional layer with bias and relu activation plus our power quantization operation and reported the number of cycles and operations. These results demonstrate that even without any optimization the proposed method has a marginal computational cost on MCU inference which corroborates our previous em- pirical results. We would like to put the emphasis on the fact that this cost could be further reduced optimizing computation of the power function using existing methods such as Kim et al.) (222T) Similarly, measure the empirical time required to perform the proposed quantization method on several neural networks and report the results table 19 These results show that the proposed PowerQuant method offers outstanding trade-off's in terms of compression and accuracy at virtually no cost over the processing and inference time as compared to other data-free quantiza- tion methods. For instance, SQuant is sophisticated method that requires heavy lifting in order to efficiently process a neural network. On a CPU, it requires at least 100 times more time to reach a lower accuracy than the proposed method will showcase in our comparison state-of-the-art quantization schemes.
21


Reviews:
[{"review_id": "qbKkIFr_1wm", "reviewer": "Reviewer_6h14", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to transform weights with power operation before quantization to improve the quantized accuracy. The motivation behind this is the bell-shaped distribution of weights, indicating that fine-grained resolution should be allocated around zero. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method against several strong baselines. ", "summary_of_the_review": "Both methods and results are interesting and significant. I am looking forward to feedback from the authors on my questions. ", "strength_and_weaknesses": "### Strength \n* This paper is well-motivated. Bell-shaped distribution of weights is widely observed. Power transformation addresses this issue well. \n* Continuous automorphisms are highly appreciated by quantization and hardware implementation. This method leverages continuous automorphisms in a smart way. \n* Extensive experiments are conducted on various architectures and datasets with significant improvements. \n\n\n### Weaknesses\n* Is $W$ in Eq. 2 a matrix? What is the precise definition of $|\\cdot|^a$?\n* \"for instance, for both SiLU (EfficientNets) and GeLU (Image Transformers), the activations are bounded below\" Are they empirical observations? How many examples are tested?\n* Although the authors mentioned the overhead of power transformation (against uniform quantization) in Eq (5), it should be further analyzed in detail to help understand the trade-off. \n* Missing reference on power-based quantization [1] [2]\n\n[1] Zhang, Sai Qian, et al. \"Training for multi-resolution inference using reusable quantization terms.\" Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 2021.\n[2] Li, Yuhang, Xin Dong, and Wei Wang. \"Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks.\" ICLR (2020).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "qGApnAJFgl", "reviewer": "Reviewer_TJBp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper looks at data free techniques for quantization. The framework is to look for automorphisms on the (R,x) which the authors show is essentially power functions. Then exponential representation are optimized to minimize reconstruction error. Finally, experimental results are presented to show the usefulness of the work.", "summary_of_the_review": "The paper proposes a new quantization method, provides analyses and empirical evaluations. \n\nDuring the rebuttal and through discussions with the authors, I have raised my score to a 6.", "strength_and_weaknesses": "Strengths: - The paper proposes a novel quantization technique based on power functions. - Some theoretical analysis is performed to describe the method. - Extensive experiments are performed to evaluate the method. Weaknesses: - The mathematical derivations are hard to follow. I think there is room for improvement in terms of clarity. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "UPJyW1ApfaS", "reviewer": "Reviewer_Htvb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposed a novel non-uniform data-free quantization method.\nThe main Idea (which is very interesting) is to find the best non-uniform quantization intervals for both weights and activations by minimizing the reconstruction error using the proposed quantization and de-quantization method. \nThe paper provides extensive experimentation on various datasets and tasks (including Image-classification and NLP) and archives promising results (outperforms  SOTA). ", "summary_of_the_review": "As mentioned in the Strength And Weaknesses section, I did not understand the quantization of activations. Hence, I need to rely on the answer from the authors and other reviewers comments. I am open to change my score in future. ", "strength_and_weaknesses": "Strength: \n\n- Novelty \n- The claims are backed by mathematical explanation \n- promising results\n\nWeaknesses: \n\n- The paper is heavy and very hard to read. To someone like me who is a non-native English speaker, the paper looks like a poorly translated novel that consists of unnecessary fancy words and grammars. It was hard to keep reading the paper and not getting distracted. \n- Section \"3.3 FUSED DE-QUANTIZATION AND ACTIVATION FUNCTION\" needs more clarification. I could not understand how the quantization works for the activations. Do we calculate $x^{\\alpha}$ online during the inference for the activations? If yes (and I think the answer is yes) then how is it calculated? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster