Original: Leveraging Inter-Layer Dependency for Post -Training Quantization
Revised: Leveraging Inter-Layer Dependency for Post -Training Quantization

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Prior works on Post-training Quantization (PTQ) typically separate a neural network into sub-nets and quantize them sequentially.
Revised: Prior works on Post-training Quantization (PTQ) typically separate a neural network into sub-nets and quantize them sequentially.

Original: This process pays little attentionto the dependency across the sub-nets, hence is less optimal.
Revised: This process pays little attentionto the dependency across the sub-nets, hence is less optimal.

Original: In this paper, wepropose a novel Network-Wise Quantization (NWQ) approach to fully leveraginginter-layer dependency.
Revised: In this paper, wepropose a novel Network-Wise Quantization (NWQ) approach to fully leveraginginter-layer dependency.

Original: NWQ faces a larger scale combinatorial optimization problem of discrete variables than in previous works, which raises two major challenges:over-fitting and discrete optimization problem.
Revised: NWQ faces a larger scale combinatorial optimization problem of discrete variables than in previous works, which raises two major challenges:over-fitting and discrete optimization problem.

Original: NWQ alleviates over fitting viaa Activation Regularization (AR) technique, which better controls the activationdistribution.
Revised: NWQ alleviates over-fitting viaa Activation Regularization (AR) technique, which better controls the activationdistribution.

Original: To optimize discrete variables, NWQ introduces Annealing Softmax(ASoftmax) and Annealing Mixup (AMixup) to progressively transition quantizedweights and activations from continuity to discretization, respectively.
Revised: To optimize discrete variables, NWQ introduces Annealing Softmax(ASoftmax) and Annealing Mixup (AMixup) to progressively transition quantizedweights and activations from continuity to discretization, respectively.

Original: Extensiveexperiments demonstrates that NWQ outperforms prior state-of-the-art approachesby a large margin: 20.24% for the challenging configuration of MobileNetV2 with2 bits on ImageNet, pushing extremely low-bit PTQ from feasibility to usability.
Revised: Extensiveexperiments demonstrate that NWQ outperforms previous state-of-the-art by alarge margin: 20.24% for the challenging configuration of MobileNetV2 with 2bits on ImageNet, pushing extremely low-bit PTQ from feasibility to usability.

Original: In addition, NWQ is able to achieve competitiveor better results with only 10%computation cost of previous works.
Revised: Inaddition, NWQ is able to achieve competitive results with only 10% computationcost of previous works.

Original:  Introduction
Revised:  Introduction

Original: Deep learning has seen a boom of architectures with ever-increasing capabilities and capacity.
Revised: Deep learning has seen a boom of architectures with ever-increasing capabilities and capacity.

Original: Hugecapacity, however, always comes with boosting parameters, resulting in large computation costand memory footprint.
Revised: Hugecapacity, however, always comes with boosting parameters, resulting in large computation costand memory footprint.

Original: Researchers have utilized strategies like neural architecture search (NAS)[4, 11, 29, 49], network pruning [12, 19] and knowledge distillation (KD)
Revised: Researchers have utilized strategies like neural architecture search (NAS)[4, 11, 29, 50], network pruning [12, 19] and knowledge distillation (KD)

Original: [18, 21] to design compactnetworks and improve their accuracy.
Revised: [18, 21] to design compactnetworks and improve their accuracy.

Original: Furthermore, neural network quantization [7, 15, 22, 25, 28]is typically applied along with the methods mentioned above to reduce the computation cost andmemory footprint.
Revised: Furthermore, neural network quantization [7, 15, 22, 25, 28]is typically applied along with the methods mentioned above to reduce the computation cost andmemory footprint.

Original: Quantization represents neural networks with lower bit precision, usually 8 bits inpractice and saves 75% of memory while offering 2 ∼ 4 × speedup[26].
Revised: Quantization represents neural networks with lower bit precision, usually 8 bits inpractice, and saves 75% of memory while offering 2 ∼ 4 × speedup[26].

Original: Network quantization techniques are typically categorized into Quantization Aware Training (QAT)and Post-training Quantization(PTQ) based on the dependency on labeled data and the necessity ofweight tuning.
Revised: Network quantization techniques are typically categorized into Quantization Aware Training (QAT)and Post-training Quantization(PTQ) based on the dependency on labeled data and the necessity ofweight tuning.

Original: Generally, PTQ techniques are preferred in industry because they rely less on labeleddata and take into account privacy concerns as well as compute resources.
Revised: Generally, PTQ techniques are preferred in industry because they rely less on labeleddata and take into account privacy concerns as well as compute resources.

Original: This paper focuses on
Revised: This paper focuses on

Original: PTQ.
Revised: PTQ.

Original: PTQ usually suffers from significant accuracy degradation, especially for low-bit quantization whichoffers much more memory reduction and speedup.
Revised: PTQ usually suffers from significant accuracy degradation, especially for low-bit quantization whichoffers much more memory reduction and speedup.

Original: To improve the accuracy of PTQ, much efforthas been devoted [5, 32, 34, 35, 39].
Revised: To improve the accuracy of PTQ, much efforthas been devoted [5, 32, 34, 36, 40].

Original: AdaRound[32] has recently discovered that round-to-nearestappears to be optimal for single weight but not for the entire layer.
Revised: AdaRound[32] has recently discovered that round-to-nearestappears to be optimal for single weight but not for the entire layer.

Original: The rounding errors of weights ina single layer have the capability of compensating for each other, contributing to a smaller layer-wiselayerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (a) layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (b) block-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (c) network-wise quantization error.
Revised: The rounding errors of weights ina single layer have the capability of compensating for each other, contributing to a smaller layer-wise quantization error.

Original: We term this property as intra-layer dependency .
Revised: We term this property as intra-layer dependency .

Original: Motivated by this property,
Revised: Motivated by this property,

Original: AdaRound[32] proposes to learn the round policy layer by layer to improve accuracy.
Revised: AdaRound[32] proposes to learn the round policy layer by layer to improve accuracy.

Original: Inspired by AdaRound[32], we hypothesize that the quantization errors of all layers in a network areable to compensate for each other, resulting in a reduced quantization error.
Revised: Inspired by AdaRound[32], we hypothesize that the quantization errors of all layers in a network isable to compensate for each other, resulting in a reduced quantization error.

Original: We refer to this capabilityas inter-layer dependency . However, AdaRound[32] assumes that layers are mutual-independentand derives a layer-diagonal Hessian, leading to a layer-wise solution(Figure 1a). BRECQ[27]points out that the dependency should not be ignored and the assumption of AdaRound[32] is notaccurate, they instead utilize the inter-layer dependency within each block to build a block-wiseapproach(Figure 1b). However, BRECQ[27] assumes that blocks are mutual-independent.
Revised: We refer to this capabilityas inter-layer dependency , which attracts little attention in previous works.

Original: Weargue that neither the approximations of AdaRound nor that of BRECQ are accurate enough, theinter-layer dependency should be leveraged in a network-wise manner.

Original: To achieve this, we propose a Network-Wise Quantization (NWQ, Figure 1c) scheme to fully leveragethe inter-layer dependency. AdaRound and BRECQ separate a network into sub-nets and train them sequentially, this processmakes the preceding layers unaware of succeeding layers. NWQ solves this problem via trainingthe entire network end-to-end, where layers are aware of each other and they are able to cooperatefor a more optimal solution.
Revised: Based on that, we proposea Network-Wise Quantization (NWQ) scheme, which is different from previous layer-wise[23, 32]and block-wise [27, 41] methods, to fully leverage this capability.

Original: However, quantizing the entire network end-to-end is a combinatorialoptimization problem where huge number of discrete parameters from all layers are combinedtogether for optimization.
Revised: However, quantizing the entirenetwork is a combinatorial optimization problem where huge number of discrete parameters fromall layers are combined together for optimization.

Original: This optimization problem will raise two major challenges that result insignificant accuracy degradation.
Revised: This optimization problem will raise two majorchallenges that result in significant accuracy degradation.

Original: The first challenge is over-fitting.
Revised: The first challenge is over-fitting.

Original: Extending the reconstruction unit from layer/block to entire networkwill increase parameters dramatically and thus raises the risk of over-fitting over a typically smallcalibration set.
Revised: Extending the reconstruction unit from layer/block to entire networkwill increase parameters dramatically and thus raises the risk of over-fitting over a typically smallcalibration set.

Original: To alleviate this issue, NWQ applies Activation Regularization (AR), a variant of
Revised: To alleviate this issue, NWQ applies Activation Regularization (AR), a variant of

Original: L2 regularization, on intermediate representation to keep the quantized activations as close to thefloating-point counterpart as possible.
Revised: L2 regularization, on intermediate representation to keep the quantized activations as close to thefloating-point counterpart as possible.

Original: The benefits of AR are two folds.
Revised: The benefits of AR are two folds.

Original: First, AR regularizesquantized activation flow to prevent over-fitting.
Revised: First, AR regularizesquantized activation flow to prevent over-fitting.

Original: Second, it works as a quantization loss function toencourage minimal intermediate quantization error.
Revised: Second, it works as a quantization loss function toencourage minimal intermediate quantization error.

Original: AR turns out to be both simple and effective.
Revised: AR turns out to be both simple and effective.

Original: Another challenge is discrete optimization.
Revised: Another challenge is discrete optimization.

Original: Based on AdaRound, NWQ learns the weight roundpolicy and activation quantization parameters, both of which involve in discrete variables.
Revised: Based on AdaRound, NWQ learns the weight round policyand activation quantization parameters, both of which involve in discrete variables.

Original: However,the substantially increased depth and discrete variables make it more difficult to converge comparedto layer-wise[23, 32] and block-wise[27, 40] approach.
Revised: However, thesubstantially increased depth and discrete variables make it more difficult to convergence comparedto layer-wise[23, 32] and block-wise[27, 41] approach.

Original: The insufficient convergence will leave thediscrete variables (weight round policy) in a state of continuity rather than discretization, leadingto great performance drop.
Revised: The insufficient convergence will leave thediscrete variables (weight round policy) in a state of continuity rather than discretization, leadingto great performance drop.

Original: To solve this problem, we propose Annealing Sofmtax (ASoftmax)and Annealing Mixup (AMixup) tobetter optimize discrete weights and activations.
Revised: To solve this problem, we propose Annealing Sofmtax (ASoftmax) and Annealing Mixup (AMixup) to optimize discrete weights and activations via continuous relaxation[35].

Original: Specifically, ASoftmax provides a mechanism to guarantee the discritization of round policylearning via annealingtemperature.
Revised: Specifically, ASoftmax optimizes weight round policy via annealing the temperature to transitionthe continuous probability vector to one-hot vector.

Original: AMixup resolves the train-test inconsistency problem indtroduced by QDROP[40] viagradually decreasing the floating-point percentage in mixed activations.
Revised: AMixup randomly mixes up the pre-quantizationactivations with the post-quantization counterpart and gradually decreases the pre-quantizationpercentage in mixed activations 1 .

Revised: The benefit of ASoftmax and AMixup is that they guarantee thedescretization after optimization while offering continuous relaxation.

Original: Equipped with AR, ASoftmax and AMixup, we are able to build our NWQ framework that fullyleverages inter-layer dependency to achieve higher accuracy than existing works.
Revised: Equipped with AR, ASoftmax and AMixup, we are able to build our NWQ framework that fullyleverages inter-layer dependency to achieve higher accuracy than existing works.

Original: To summarize, this paper makes the following contributions:
Revised: To summarize, this paper makes the following contributions:

Original: We propose a novel network-wise quantization framework for PTQ to fully leverage interlayer dependency .
Revised: We propose a novel network-wise quantization framework for PTQ to fully leverage inter- layer dependency .

Original: We analyze the combinatorial optimization problem introduced by network-wise quan- tization and propose AR, ASoftmax and AMixup to solve the over-fitting and discreteoptimization problem.
Revised: We analyze the combinatorial optimization problem introduced by network-wise quantizationand propose AR, ASoftmax and AMixup to solve the over-fitting and discrete optimization problem.3.

Original: We establish a new state-of-the-art through significantly improving PTQ by a large margin:20.24% for MobileNetV2 with 2 bits and thus push the limit of extremely low-bit PTQ fromfeasibility to usability.
Revised: We establish a new state-of-the-art through significantly improving PTQ by a large margin:20.24% for MobileNetV2 with 2 bits and thus push the limit of extremely low-bit PTQ fromfeasibility to usability.

Original:  Related Work
Revised:  Related Work

Original: Network quantization is commonly divided into QAT [7, 15, 22, 25, 48] and PTQ [27, 32, 33, 47].
Revised: Network quantization is commonly divided into QAT [7, 15, 22, 25, 49] and PTQ [27, 32, 33, 48].

Original: QAT methods tend to minimize task error by training weights and quantization parameters jointlywith the entire labeled dataset and thus require a large number of computing resources, whereas PTQmethods focus on finding optimal quantization parameters for off-the-shelf well-trained networks withmuch less data and compute resources than QAT.
Revised: QAT methods tend to minimize task error by training weights and quantization parameters jointlywith the entire labeled dataset and thus require a large number of computing resources, whereas PTQmethods focus on finding optimal quantization parameters for off-the-shelf well-trained networks withmuch less data and compute resources than QAT.

Original: PTQ can further be classified into statistic-based
Revised: PTQ can further be classified into statistic-based

Original: PTQ and learning-based PTQ .
Revised: PTQ and learning-based PTQ .

Original: Statistic-based PTQ methods [33, 35, 47] first estimate the distributionof activation on calibration data, and then find the optimal quantization parameters that minimizelocal quantization errors heuristically.
Revised: Statistic-based PTQ methods [33, 36, 48] first estimate the distributionof activation on calibration data, and then find the optimal quantization parameters that minimizelocal quantization errors heuristically.

Original: Learning-based PTQ methods[23, 27, 32, 40] dedicate toslightly weight tuning and quantization parameters learning.
Revised: Learning-based PTQ methods[23, 27, 32, 41] dedicate toslightly weight tuning and quantization parameters learning.

Original: Our work follows the learning-based
Revised: Our work follows the learning-based

Original: PTQ scheme.
Revised: PTQ scheme.

Revised: [Table]

Revised: [Figure]

Revised: … 𝐴𝑅𝐿 :;<=> + … 𝐴𝑅 ?

Revised: @ 𝑙𝑜𝑠𝑠

Revised: [Figure]

Original:  Learning-based PTQ : AdaRound[32] adopts the regime of learning the optimal round policy layerby layer.
Revised: ASoftmax 𝐿 <A + [Figure] Learning-based PTQ : AdaRound[32] adopts the regime of learning the optimal round policy layerby layer.

Original: BRECQ [27] succeeds in extending AdaRound [32] to a block-wise reconstruction andconcludes that block-wise quantization is superior to network-wise method.
Revised: BRECQ [27] succeeds in extending AdaRound [32] to a block-wise reconstruction andconcludes that block-wise quantization is superior to network-wise method.

Original: We argure that bothlayer-wise and block-wise approach is sub-optimal because the preceding layers/blocks are unawareof succeeding layers/block, thus the dependency they leveraged is unidirectional.
Revised: We argure that bothlayer-wise and block-wise approach is sub-optimal because the preceding layers/blocks are unawareof succeeding layers/block, thus the dependency they leveraged is unidirectional.

Original: Some works focuson the sequence of weight and activation quantization.
Revised: Some works focuson the sequence of weight and activation quantization.

Original: AdaQuant [23] proposes to tune weights andactivation quantization parameters separately for each layer to reduce over-fitting, while QDROP[40] finds that incorporating activation quantization into weight tuning can achieve higher accuracy.
Revised: AdaQuant [23] proposes to tune weights andactivation quantization parameters separately for each layer to reduce over-fitting, while QDROP[41] finds that incorporating activation quantization into weight tuning can achieve higher accuracy.

Original: Activiation Regularization : Large values in the learned dense representation can be a sign of beingoverfit.
Revised: Activiation Regularization : Large values in the learned dense representation can be a sign ofbeing overfit.

Original: To encourage small activations, [31] employs L2 regularization on hidden state of recurrentneural networks (RNN), while various normalizations [2, 24, 38, 42] focus on centralize the activationdistribution to regularize large values.
Revised: To encourage small activations, [31] employs L2 regularization on hidden state ofrecurrent neural networks (RNN), while various normalizations [2, 24, 39, 43] focus on centralizethe activation distribution to regularize large values.

Original: Some prior works attempt to learn sparse representation.
Revised: Some prior works attempt to learn sparserepresentation.

Original: Dropout and its variants [1, 10,36, 37, 41] randomly drop activations from the neural network toencourage sparsity.
Revised: Dropout and its variants [1, 10, 37, 38, 42] randomly drop activations from the neuralnetwork to encourage sparsity.

Original: ReLUs[16, 20, 30, 43] dedicate to learning sparse representation via rectifyingnegative values.
Revised: ReLUs[16, 20, 30, 44] dedicate to learning sparse representation viarectifying negative values.

Original: These techniques aim to regularize activations when training a neural networkfrom scratch, which inapplicable for PTQ where the networks are off-the-shelf and well-trained,thus the activation distributions are deterministic and immutable.
Revised: These techniques aim to regularize activations when training a neuralnetwork from scratch, which inapplicable for PTQ where the networks are off-the-shelf andwelltrained, thus the activation distributions are deterministic and immutable.

Original: Instead of encouraging small andsparse activations, NWQ apply activation regularization on intermediate representation to encouragesmall offsets between quantized activations and the floating-point counterpart.
Revised: Instead of encouragingsmall and sparse activations, NWQ apply activation regularization on intermediate representation toencourage small offsets between quantized activations and the floating-point counterpart.

Original: AdaRound[32] and

Original: BRECQ [27] also employ quantization losses on intermediate activations but those losses works asquantization objectives rather than regularization, while the quantization objective of NWQ is tominimize the quantization error of the final network output and the regularization on intermediateactivations is to prevent optimization process from over-fitting.

Original: Discrete Optimization for Quantization : Network quantization optimizes discrete variables viagradient backpropagation.
Revised: Discrete Optimization for Quantization : Network quantization optimizes discrete variables viagradient backpropagation.

Original: However, the quantization process introduces a round function which isindifferentiable.
Revised: However, the quantization process introduces a round function which isindifferentiable.

Original: Some works on QAT focus on approximating the gradient through the round function[3, 6, 13, 14, 17, 44].
Revised: Some works on QAT focus on approximating the gradient through the round function[3, 6, 13, 14, 17, 45].

Original: For PTQ, AdaRound [32] introduces additional discrete parameters(the binaryround policy) and optimize them via Rectified Sigmoid with an explicit Regularization (RSeR) lossfunction.
Revised: For PTQ, AdaRound [32] introduces additional discrete parameters(the binaryround policy) and optimize them via Rectified Sigmoid with an explicit Regularization (RSeR) lossfunction.

Original: However, RSeR is adversarial to quantization loss and thus difficult to convergence innetwork-wise quantization.

Original: QDROP [40] proposes to randomly drop quantized activations to improvethe flatness of optimization landscape, but introcues train-test inconsistency problem which results inperformance degradation if used in network-wise PTQ. [Table] [Figure] … 𝐴𝑅
Revised: QDROP [41] proposes to randomly drop quantized activations to improve the flatnessof optimization landscape.

Original: [Figure]

Original: 𝐿 :;<=> + … 𝐴𝑅 ? @ 𝑙𝑜𝑠𝑠 [Figure] 𝐿 <A
Revised: These approaches are proposed to improve layer-wise and block-wisequantization and encounter failure when applied to network-wise quantization, since the discreteoptimization problem is much more difficult.

Original: +

Original: [Figure]

Original:  Approach
Revised:  Approach

Original: [Figure] Network-Wise Quantization (NWQ) is a simple end-to-end post-training quantization approach.
Revised: Network-Wise Quantization (NWQ) is a simple end-to-end post-training quantization approach.

Original: The structure of NWQ is illustrated in Figure 2.
Revised: The structure of NWQ is illustrated in Figure 1.

Original: The optimization objective is composed of outputquantization loss and layer-wise AR loss.
Revised: The optimization objective is composed of outputquantization loss and layer-wise AR loss.

Original: Meanwhile, the annealing approaches (Asoftmax and
Revised: Meanwhile, the annealing approaches (Asoftmax and

Original: AMixup) are introduced in training process, which enable the quantized weights and activations togently converge from continuous state to discrete state, lowering training complexity and improvingaccuracy.
Revised: AMixup) are introduced in training process, which enable the quantized weights and activations togently converge from continuous state to discrete state, lowering training complexity and improvingaccuracy.

Original: 3.1 Activation Regularization
Revised: 3.1 Activation Regularization

Original: NWQ leverages Activation Regularization (AR) to penalize quantized intermediate activations thatare substantially away from the floating-point counterparts, thereby encouraging the activations toremain unchanged.
Revised: NWQ leverages Activation Regularization (AR) to penalize quantized intermediate activations thatare substantially away from the floating-point counterparts, thereby encouraging the activations toremain unchanged.

Original: Formally, we define AR by:
Revised: Formally, we define AR by:

Original: [Equation]
Revised: [Equation]

Original: (1) while G represents floating-point network and G is the simulated quantized counterpart.
Revised: while G represents floating-point network and G is the simulated quantized counterpart.

Original: a l and a ldenote the floating-point and the simulated quantized activation of the l -th layer, respectively.
Revised: a l and a ldenote the floating-point and the simulated quantized activation of the l -th layer, respectively.

Original: Nrepresents the number of total layers and ∥·∥ F denotes the Frobenius norm.
Revised: Nrepresents the number of total layers and ∥·∥ F denotes the Frobenius norm.

Original: Our network quantizationerror L quant ( a N , a N ) is defined by:
Revised: Our network quantizationerror L quant ( a N , a N ) is defined by:

Original: [Equation]
Revised: [Equation]

Original: then the final objective can be formulated as:
Revised: then the final objective can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: Practically, L ar is not restricted to be utilized layer by layer, it can also be performed block byblock and stage by stage, as BRECQ [27] defined.
Revised: Practically, L ar is not restricted to be utilized layer by layer, it can also be performed block byblock and stage by stage, as BRECQ [27] defined.

Original: We will discuss how the granularity affects theperformance in Sec. 4.2.1.
Revised: We will discuss how the granularity affects theperformance in Sec. 4.2.1.

Original: By default, we employ layer-wise AR in our experiments.
Revised: By default, we employ layer-wise AR in our experiments.

Original: 3.2 Annealing Softmax
Revised: 3.2 Annealing Softmax

Revised: 3.2.1 Formulation

Original: For a weight vector w , AdaRound [32] proposes to learn the round policy by:
Revised: For a weight vector w , AdaRound [32] proposes to learn the round policy by:

Original: [Equation]
Revised: [Equation]

Original:  where ⌊·⌋ gives as output of the integral part and clip ( · ) clamps all values between q − and q + .
Revised: (3) where ⌊·⌋ gives as output of the integral part and clip ( · ) clamps all values between q − and q + .

Original: ˆ w idenotes the quantized weight for the i -th element of w .
Revised: ˆ w idenotes the quantized weight for the i -th element of w .

Original: We will omit i below for simplicity.
Revised: We will omit i below for simplicity.

Original: s is thequantizer step size.
Revised: s is thequantizer step size.

Original: v is a learnable continuous vector w.r.t.
Revised: v is a learnable continuous vector w.r.t.

Original: w and h ( v ) is recitified sigmoid function(see Eqn.(4)) which is regularized to converge to { 0 , 1 } .[Equation]
Revised: w and h ( v ) is recitified sigmoid function(see Eqn.(4)) which is regularized to converge to { 0 , 1 } .

Original: Based on Eqn.(4), AdaRound proposes Rectified Sigmoid with an explicit Regularization (RSeR)loss to discrete h ( v ) :

Original: [Equation]

Original: However, there are two limitations of RSeR. First, RSeR works only for binary round policy.

Original: Second,

Original: RSeR is adversarial to quantization loss.

Original: For example, for a linear transformation network, thequantization loss is

Original: [Equation]
Revised: [Equation]

Original: where x denotes the input and we ignore the clip function in Eqn.

Original: Eqn.

Original: (6) indicates that quantization loss will prevent L RSeR from convergence ( i.e. , L RSeR encourages h ( v ) to converge to 0 or 1 while L q prefers h ( v ) to remain w s − ⌊ w s ⌋ ).

Original: [-Besides, this adversarialbecome more severe when extending layer/block-wise methods to network-wise quantization(see Sec.4.2.
-]
Revised: [-Besides, this adversarialbecome more severe when extending layer/block-wise methods to network-wise quantization(see Sec.4.2.
-]

Original: and Table 4). 3.2.1 Formulation To resolve the two problems of RSeR, we propose Annealing Softmax (ASoftmax). First, we extendthe discrete optimization space of h ( · ) from { 0 , 1 } to K , where K is a integer range and defined by:
Revised: Based on Eqn.(3), we extend the discrete optimization space of h ( · ) from { 0 , 1 } to K , where K is ainteger range and defined by:

Original: [Equation]
Revised: [Equation]

Original: where n and m are hyper-parameters that specify the discrete optimization space.
Revised: where n and m are hyper-parameters that specify the discrete optimization space.

Original: Then, ASoftmaxcan be formulated as:
Revised: Then, ASoftmaxcan be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where V is a continuous learnable matrix and represents the logits of probability over K .
Revised: where V is a continuous learnable matrix and represents the logits of probability over K .

Original: h t ( V ) isdefined by:
Revised: h t ( V ) isdefined by:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where t represents for t -th iteration.
Revised: where t represents for t -th iteration.

Original: τ t is a temperature that decays linearly to transition h t ( V )from continuity to discretization.
Revised: τ t is a temperature that decays linearly to transition h t ( V )from continuity to discretization.

Original: For example, when τ t is large, p ( k | V , τ t ) is a continuous vectorof probability and h t ( V ) is the expectation of the discretization.
Revised: For example, when τ t is large, p ( k | V , τ t ) is a continuous vector of probability and h t ( V ) is the expectation of the discretization.

Original: When τ t turns tiny, p ( k | V , τ t )becomes a one-hot vector and h t ( V ) selects k by p ( k | V , τ t )
Revised: When τ t turns tiny, p ( k | V , τ t )becomes a one-hot vector and h t ( V ) selects k by p ( k | V , τ t )

Original: = 1 .
Revised: = 1 .

Original: τ t is simply decayed by:
Revised: τ t is simply decayed by:

Original: [Equation]
Revised: [Equation]

Original:  where τ 0 and τ T are hyper-parameters of initial and final temperature.
Revised: 9) where τ 0 and τ T are hyper-parameters of initial and final temperature.

Original: ASoftmax is a general formation of learning wider range of discrete parameters than binary.
Revised: ASoftmax is a general formation of learning wider range of discrete parameters than binary.

Original: It easilydegenerates to round policy learning by setting n = 0 and m = 1 .
Revised: It easilydegenerates to round policy learning by setting n = 0 and m = 1 .

Original: We use τ T = 0 .
Revised: We use τ T = 0 .

Original: 01 and τ 0 = 1 bydefault.
Revised: 01 and τ 0 = 1 bydefault.

Original: 3.2.2 Initialization
Revised: 3.2.2 Initialization

Original: According to our observations, initialization has significant impacts on learning round policy.
Revised: According to our observations, initialization has significant impacts on learning round policy.

Original: Giventhe weight quantizer step size s , which has been computed using statistic-based PTQ methods, we attempt to initialize V in order to make  s ·ˆ w 0 as close to floating-point w as possible at the start.
Revised: Giventhe weight quantizer step size s , which has been computed using statistic-based PTQ methods, weattempt to initialize V in order to make ˆ s · w 0 as close to floating-point w as possible at the start.

Original: Toachieve this goal, we first calculate the un-normalized probability by:
Revised: Toachieve this goal, we first calculate the un-normalized probability by:

Original: [Equation]
Revised: [Equation]

Original: Then, we normalize the probability:
Revised: Then, we normalize the probability:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Finally, the initialization for V
Revised: Finally, the initialization for V

Original: i is log ( σ ( V ) i ) .
Revised: i is log ( σ ( V ) i ) .

Original: It can be easily proved (see Appendix A) that theinitialized  s ·ˆ w 0 equals w as long as the integer range K is symmetric w.r.t.
Revised: It can be easily proved (see Appendix A) that theinitialized ˆ s · w 0 equals w as long as the integer range K is symmetric w.r.t.

Original: 0.5 ( i.e. n = 1 − m ).
Revised: 0.5 ( i.e. n = 1 − m ).

Original: 3.3 Annealing Mixup
Revised: 3.3 Annealing Mixup

Original: NWQ considers activation quantization as a discrete optimization problem and solves it via continuousrelaxation.
Revised: NWQ considers activation quantization as a discrete optimization problem and solves it via continuousrelaxation.

Original: Different from ASoftmax which is applied on element-wise level, AMixup works ontensor-wise level: we randomly mix up the floating-point activations with the simulated quantizedactivations.
Revised: Different from ASoftmax which is applied on element-wise level, AMixup works on tensorwise level: we randomly mix up pre-quantization activations with post-quantization activations.

Original: To encourage continuous relaxation converge to discretization, NWQ progressivelydecreases the percentage of floating-point activations, so the mixed activations will become fullyquantized activation gradually.
Revised: Toencourage discretization, NWQ progressively decreases the percentage of pre-quantization activations,and the mixed activations will become fully quantized activation gradually.

Original: Formally, the percentage of floating-point activations  is defined by:
Revised: Formally, the percentage of pre-quantization activations for t -th iteration is defined by:

Original: [Equation]
Revised: [Equation]

Original: where P( t ) is the percentage of floating-point activation in the mixed activation for the t -th iteration. P s and P e denote the percentage at  start and  end respectively, and T is the number of iterations.
Revised: where P s and P e denote the percentage at the start and the end respectively, and T is the number ofiterations.

Original: Weuse P s = 0 .
Revised: We use P s = 0 .

Original: 5 and P e = 0 by default.
Revised: 5 and P e = 0 by default.

Original: AMixup is similar to QDROP [40] in the mixup operation.
Revised: AMixup is similar to QDROP [41] in the mixup operation.

Original: The key difference, however, is that
Revised: The key difference, however, is that

Original: AMixup introduces an annealing process.
Revised: AMixup introduces an annealing process.

Original: This is very important for NWQ.
Revised: This is very important for NWQ.

Original: QDROP trains with mixedactivations while testing with fully quantized activations, thus introducing inconsistency bias.
Revised: QDROP trains with mixedactivations while testing with fully quantized activations, thus introducing inconsistency bias.

Original: Thisbias accmulates as network depth increases and leads to great accuracy degradation in network-wisequantization.
Revised: Thisbias accmulates as network depth increases and leads to great accuracy degradation in network-wisequantization.

Original: AMixup is able to eliminate the inconsistency bias via the annealing process.
Revised: AMixup is able to eliminate the inconsistency bias via the annealing process.

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct extensive experiments with various architectures to verify our NWQ.
Revised: In this section, we conduct extensive experiments with various architectures to verify our NWQ.

Original: We first compare our method with previous state-of-the-art, and then we delve into our importantdesigns and fully explore the scalability, effectiveness, and efficiency of our solution.
Revised: We first compare our method with previous state-of-the-art, and then we delve into our importantdesigns and fully explore the scalability, effectiveness, and efficiency of our solution.

Original: We report top-1classification accuracy on the ImageNet[9] validation set.
Revised: We report top-1classification accuracy on the ImageNet[9] validation set.

Original: Implementation Details.
Revised: Implementation Details.

Original: Our code is based on open-source codes BRECQ and QDrop.
Revised: Our code is based on open-source codes BRECQ and QDrop.

Original: Weapply layer-wise AR.
Revised: We applylayer-wise AR.

Original: For ASoftmax, we set n = 0 , m = 1 and τ is decayed from 1 to 0 .
Revised: For ASoftmax, we set n = 0 , m = 1 and τ is decayed from 1 to 0 .

Original: For AMixup, we set default P s as 0.5 and P e as 0.
Revised: For AMixup, we setdefault P s as 0.5 and P e as 0.

Original: We randomly sample 1024 images from ImageNettrain set and employ  Cutmix [45] and Mixup[46] as data augmentation.
Revised: We randomly sample 1024 images from ImageNet train set and employ [Table] Cutmix [46] and Mixup[47] as data augmentation.

Original: The learning rates are0 .
Revised: The learning rates are 0 .

Original: 01 for round policy and 0 .[-
-]
Revised: 01 for round policy and0 .[-
-]

Original:  for activation quantizer step size.
Revised: 0004 for activation quantizer step size.

Original: We train for 20000 iterationswith a mini-batch size of 32 in 8{+
+} Tesla V100 GPUs, taking ∼ 30 minutes for ResNet18, whichis on par with BRECQ and QDROP.
Revised: We train for 20000 iterations with a mini-batch size of 32 in {+
+} Tesla V100 GPUs, taking ∼ 30 minutes for ResNet18, which is on par with BRECQ and QDROP.

Original: Our experiments are conducted on 5 architectures, including 
Revised: Ourexperiments are conducted on 5 architectures, including ResNet18(Res18), MobileNetV2(MNV2),

Original: ResNet18(Res18), MobileNetV2(MNV2), RegNet-600MF(Reg600MF), RegNet-3.2GF(Reg3.2GF)and MnasNet(Mnas).
Revised: RegNet-600MF(Reg600MF), RegNet-3.2GF(Reg3.2GF) and MnasNet(Mnas).

Original: Other settings remain same as QDROP[40] if not specified.
Revised: Other settings remainsame as QDROP[41] if not specified.

Original: 4.1 Comprehensive Comparison
Revised: 4.1 Comprehensive Comparison

Original: Table 1 compares our NWQ with several main-stream PTQ approaches.
Revised: Table 1 compares our NWQ with several main-stream PTQ approaches.

Original: Overall, NWQ outperformsprevious works across all neural network architectures and bit widths, and our performance improveseven more as the bit width decreases.
Revised: Overall, NWQ outperformsprevious works across all neural network architectures and bit widths, and our performance improveseven more as the bit width decreases.

Original: Classical Calibration Set.
Revised: Classicial Calibration Set.

Original: Following previous works, we compare the performance on a smallcalibration set of 1024 images.
Revised: Following previous works, we compare the performance on a smallcalibration set of 1024 images.

Original: For start, we achieve 0.2% ∼ 0.9% improvement even compared with [Table] strong baselines on W4A4, which are rather close to full precision accuracy.
Revised: For start, we achieve 0.2% ∼ 0.9% improvement even compared withstrong baselines on W4A4, which are rather close to full precision accuracy.

Original: For W3A3, NWQ yields0.83% ∼ 3.26% and reduces the gap between W3A3 and W4A4.
Revised: For W3A3, NWQ yields0.83% ∼ 3.26% and reduces the gap between W3A3 and W4A4.

Original: For the challenging W2A2, ourmethod surpasses the previous state-of-the-art by a large margin of 13.37% for MobileNetV2 and12.4% for MnasNet, pushing the limit of extremely low-bit PTQ step further.
Revised: For the challenging W2A2, ourmethod surpasses the previous state-of-the-art by a large margin of 13.37% for MobileNetV2 and12.4% for MnasNet, pushing the limit of extremely low-bit PTQ step further.

Original: Scaled-up Calibration Set.
Revised: Scaled-up Calibration Set.

Original: To further explore the potential of NWQ, we scale up the calibrationset by 10 × and reproduce QDROP on the scaled-up calibration set, building a very strong baseline.
Revised: To further explore the potential of NWQ, we scale up the calibrationset by 10 × and reproduce QDROP on the scaled-up calibration set, building a very strong baseline.

Original: Interestingly, NWQ outperforms QDROP even with one-tenth data.
Revised: Interestingly, NWQ outperforms QDROP even with one-tenth data.

Original: When equipping NWQ withthe scaled-up calibration set, we observe significant improvement under all settings.
Revised: When equipping NWQ withthe scaled-up calibration set, we observe significant improvement under all settings.

Original: Particularly,the accuracy gaps of W2A2 between NWQ and QDROP are enlarged even more as the calibrationset scales up.
Revised: Particularly,the accuracy gaps of W2A2 between NWQ and QDROP are enlarged even more as the calibrationset scales up.

Original: For example, the gap of MnasNet increases from 12.4% to 19.36%, and that of
Revised: For example, the gap of MnasNet increases from 12.4% to 19.36%, and that of

Original: MobileNetV2 increases from 13.37% to 20.24%, showing the excellent scalability of our method. 
Revised: MobileNetV2 increases from 13.37% to 20.24%, showing the excellent scalability of our method. [Table] [Table] [Table]

Revised: [Table]

Revised: [Figure]

Original: 4.2 Ablation Study
Revised: 4.2 Ablation Study

Revised: [Figure]

Original: 4.2.1 AR
Revised: 4.2.1 AR

Original: [Figure]
Revised: [Figure]

Original: The Effect of AR.
Revised: The Effect of AR.

Original: We study the effect of AR via comparing the experiments with layer-wise AR to the experimentswithout AR on W2A2.
Revised: We study the effect of AR viacomparing the experiments with layer-wise AR tothe experiments without AR on W2A2.

Original: Table 2 shows that layer-wise ARimproves the accuracy significantly.
Revised: Table 2shows that layer-wise AR improves the accuracysignificantly.

Original: It’s worth notingthat layer-wise AR improves 5.22% for [-MobileNetV
 and10.42%-] for MnasNet.
Revised: It’s worth noting that layer-wise ARimproves 5.22% for [-MobileNetV
 and10.42%-]MobileNetV2 and 10.42% for MnasNet.

Original: To understand how AR works, weanalyze the loss curves during training in Figure 3.
Revised: To understand how AR works, we analyze the loss curves during training in Figure 2.

Original: First,our AR experiment consistently converges to lower losson both the train and test datasets, resulting in higher accuracy.
Revised: First, our AR experiment consistently convergesto lower loss on both the train and test datasets,resulting in higher accuracy.

Original: Second, AR accelerates convergence significantlyand thus allowsNWQ to reach competitive accuracy at a [Table] [Table]
Revised: Second, AR accelerates convergence significantly and thus allows

Original: [Table] [Table] [Table] substantially lower computing cost.
Revised: NWQ to reach competitive accuracy at a substantially lower computing cost.

Original: Third, even though AR does not totally eliminate over-fitting, iteffectively bridges the loss gap between train and test datasets.
Revised: Third, even though AR does not totally eliminate over-fitting, it effectively bridges the loss gap between train and testdatasets.

Original: The Effect of Granularity.
Revised: The Effect of Granularity.

Original: Table 2 studies how AR granularity affects accuracy.
Revised: Table 2 studies how AR granularity affects accuracy.

Original: AR granularityrepresents for applying AR after each layer/block/stage, while training(reconstruction) granularityrepresents for optimizing each layer/block/stage sequentially and independently.

Original: We discover thatlightweight networks prefer coarse-grained AR, whereas heavyweight networks prefer fine-grained AR.
Revised: We discover that lightweight networks prefer coarse-grained AR,whereas heavyweight networks prefer fine-grained AR.

Original: MobileNetV2, RegNet-600MF, and MnasNet, for example, achieve the best performance withblock-wise AR, whereas ResNet18 and RegNet-3.2GF achieve the maximum accuracy with layer-wise AR.
Revised: MobileNetV2, RegNet-600MF, and MnasNet,for example, achieve the best performance with block-wise AR, whereas ResNet18 and RegNet-3.2GFachieve the maximum accuracy with layer-wise AR.

Original: Comparison with Data Augmentation.
Revised: Comparison with Data Augmentation.

Original: Data augmentation is a classical strategy used to relieveover-fitting due to its simplicity and effectiveness.
Revised: Data augmentation is a classical strategy used to relieveover-fitting due to its simplicity and effectiveness.

Original: We compare AR with strong augmentation ( e.g.
Revised: We compare AR with strong augmentation ( e.g.

Original: AutoAugment[8]) in Table 3.
Revised: AutoAugment[8]) in Table 3.

Original: Notably, both AR and AutoAugment significantly improve accuracy,and AR contributes more than AutoAugment, particularly for sparse networks like MobileNetV2 and MnasNet.
Revised: Notably, both AR and AutoAugment significantly improve accuracy,with AR contributing more than AutoAugment, particularly for sparse networks like MobileNetV2and MnasNet.

Original: Coupling these two techniques enables us to achieve better performance.
Revised: Coupling these two techniques enables us to achieve better performance.

Original: 4.2.2 ASoftmax Study
Revised: 4.2.2 ASoftmax Study

Original: The Effect of ASoftmax.
Revised: The Effect of ASoftmax.

Original: Table 4 compares ASoftmax to Rectified Sigmoid with explicit Regularization (RSeR) which is used in AdaRound.
Revised: Table 5 compares ASoftmax to Rectified Sigmoid with explicit Regularization (RSeR) which is used in AdaRound.

Original: We simply replace ASoftmax with RSeR in our NWQ andrun a grid search on regularization loss weight to explore the potential of RSeR.
Revised: We simply replace ASoftmax with RSeR in our NWQ and [Table] [Table] run a grid search on regularization loss weight to explore the potential of RSeR.

Original: We discover that
Revised: We discover that

Original: RSeR requires a huge loss weight for NWQ, which is 10 3 ∼ 10 5 × larger than that of subnet-wiseapproaches.
Revised: RSeR requires a huge loss weight for NWQ, which is 10 3 ∼ 10 5 × larger than that of subnet-wiseapproaches.

Original: Even with exhausting hyper-parameter search, RSeR does not perform as well as in thesubnet-wise method.
Revised: Even with exhausting hyper-parameter search, RSeR does not perform as well as in thesubnet-wise method.

Original: ASoftmax, on the other hand, outperforms RSeR by 12.58% and 21.56% on
Revised: ASoftmax, on the other hand, outperforms RSeR by 12.58% and 21.56% on

Original: ResNet18 and MobileNetV2, respectively, without any additional hyper-parameter tuning.
Revised: ResNet18 and MobileNetV2, respectively, without any additional hyper-parameter tuning.

Original: The Effect of Initialization.

Original: Table 5 demonstrates the effectiveness of our initialization, which keepsthe quantization model close to the float-point at the start to steady the training.

Original: It is shown that ourinitialization method significantly outperforms random initialization.

Original: The Robustness of τ .
Revised: The Robustness of τ .

Original: We investigate the robustness of τ in Table 6.
Revised: We investigate the robustness of τ in Table 6.

Original: τ starts with a large numberto encourage continuous relaxation and ends with a small value to favor one-hot distribution.
Revised: τ starts with a large numberto encourage continuous relaxation and ends with a small value to favor one-hot distribution.

Original: Thefinal τ T determines the decaying speed of τ .
Revised: Thefinal τ T determines the decaying speed of τ .

Original: It can be observed that τ T has a robustness range of10 − 2 ∼ 10 − 4 , making it easily adaptable to a wider range of neural network architectures.
Revised: It can be observed that τ T has a robustness range of10 − 2 ∼ 10 − 4 , making it easily adaptable to a wider range of neural network architectures.

Original: The Extensiveness of ASoftmax.
Revised: The Extensiveness of ASoftmax.

Original: As described in Sec.3.2.1, ASoftmax can easily extend the discreteparameter space from { 0 , 1 } to a larger integer range, offering the opportunity for a better optimum.
Revised: As described in Sec. 3.2.1, ASoftmax can easily extend thediscrete parameter space from { 0 , 1 } to a larger integer range, offering the opportunity for a betteroptimum.

Original: However, extending discrete range benefits MobileNetV2 and RegNet-600MF while degrading [Table] [Table] [Table]
Revised: However, extending discrete range benefits MobileNetV2 and RegNet-600MF whiledegrading RegNet-3.2GF on a small calibration set, as shown in Table 7 upper group.

Original: [Table]

Original: [Table]

Original: RegNet-3.2GF on a small calibration set, as shown in Table 7 upper group.

Original: We argue that the enlargeddiscrete range raises the risk of over-fitting for large networks.
Revised: We argue thatthe enlarged discrete range raises the risk of over-fitting for large networks.

Original: When we scale up  thecalibration set,the performance no longer degrades and the performance improves even more.
Revised: When we scale up thecalibration set, the performance no longer degrades and the performance improves even more.

Original: 4.2.3 AMixup Study
Revised: 4.2.3 AMixup Study

Original: The Effect of AMixup.
Revised: The Effect of AMixup.

Original: QDROP presents good performance in block-wise reconstruction, but failsin our network-wise scheme.
Revised: QDROP presents good performance in block-wise reconstruction, but failsin our network-wise scheme.

Original: As illustrated in Table 8, we achieve 1.95% improvement for ResNet18and 19.96% for MobileNetV2 by simply removing QDROP (w/o Drop row).
Revised: As illustrated in Table 4, we achieve 1.95% improvement for ResNet18and 19.96% for MobileNetV2 by simply removing QDROP.

Original: We hypothesizethat randomly mixing activations up introduces inconsistency bias between the training and testphase, which is concealed by the improvement of flatness in block-wise reconstruction.
Revised: We hypothesize that randomly mixingactivations up introduces inconsistency bias between the training and test phase, which is concealedby the improvement of flatness in block-wise reconstruction.

Original: However,as the reconstruction unit goes deeper (from block to network) and sparser, the problem becomesmore pronounced, resulting in significant degeneration.
Revised: However, as the reconstruction unitgoes deeper (from block to network) and sparser, the problem becomes more severe, resulting insignificant degeneration.

Original: Over-parameterized ResNet18 is able tocompensate for the side effect, but MobileNetV2 suffers a great performance drop.
Revised: Over-parameterized ResNet18 is able to compensate for the side effect, but MobileNetV2 suffers a great performance drop.

Original: Remarkably, our AMixup is able to overcome the inconsistency while taking advantage of the flatness, contributing tohigher accuracy. 
Revised: Remarkably, our AMixup is able to overcome theinconsistency while taking advantage of the flatness, contributing to higher accuracy. [Table]

Original: The Policy of Decay.
Revised: The Policy of Decay.

Original: In Table 9, we run a grid search for P s to explores the influence of differentdecay policies. 
Revised: Table 8 explores the influence of different decay policies. We run a gridsearch for P s while freezing P e as 0.

Original: It’s worth noting that MobileNetV2 and MnasNet prefer 0.5, whereas ResNet18 and RegNet prefer 1.
Revised: It’s worthnoting that MobileNetV2 and MnasNet prefer 0.5,whereas ResNet18 and RegNet prefer 1.

Original: The basic law is very similar to that of AR granularity, which states that heavyweightnetworks require more regularization, implying that AMixup and AR are both capable of regularizingaccuracy.
Revised: The basic law is very similar to that of AR granularity,which states that heavyweight networks requiremore regularization, implying that AMixup and AR are both capable of regularizing optimization.

Original: 4.3 Efficiency
Revised: 4.3 Efficiency

Original: [Figure]

Original: 20 Iterations/

Original: [Figure] [Figure] [Figure] We compare the efficiency of several approacheson ResNet18.
Revised: We compare the efficiency of several approaches on ResNet18.

Original: According to Figure 4, NWQ outperforms QDROP and BRECQ under various iterations.
Revised: According to Figure 3, NWQoutperforms QDROP and BRECQ under various iterations.

Original: Specifically, NWQ achieve competitiveresultswith only 10% computation cost, demonstrating the efficiency of NWQ.
Revised: Specifically, NWQ achieve competitiveresults only 10% computation cost, demonstrating the efficiency of NWQ.

Original: As the number ofiterations increases, NWQ continues to improveaccuracy, whereas BRECQ and QDROP reachtheir saturation at ∼ 1 .
Revised: As the number of iterationsincreases, NWQ continues to improve accuracy, whereas BRECQ and QDROP reach their saturationat ∼ 1 .

Original: 6 × 10 4 iterations.
Revised: 6 × 10 4 iterations.

Revised: [Figure]

Revised: [Figure]

Original:  Discussion and Conclusion
Revised:  Discussion and Conclusion

Original: Simple algorithms that scale well are the core ofdeep learning.
Revised: Simple algorithms that scale well are the core of deep learning.

Original: Prior works tend to quantize networks in layer-wise and block-wise manner.
Revised: Prior works tend to quantize networksin layer-wise and block-wise manner.

Original: We prove that network-wise quantization is a simplerand more effective way than layer/block-wise solutions.
Revised: We prove that network-wise quantization is a simpler andmore effective way than layer/block-wise solutions.

Original: To accomplish this, we analyze the problemsintroduced by the network-wise regime and propose three key designs to address them.
Revised: To accomplish this, we analyze the problemsintroduced by the network-wise regime and propose three key designs to address them.

Original: Based on that,we construct a simple and effective end-to-end network-wise PTQ framework.
Revised: Based on that,we construct a simple and effective end-to-end network-wise PTQ framework.

Original: Extensive experimentsdemonstrate that our solution is scalable, effective, and efficient.
Revised: Extensive experimentsdemonstrate that our solution is scalable, effective, and efficient.

Original: However, there is still a largeaccuracy gap between low-bit PTQ and full precision network, and we believe that there is still muchroom for further improvement for PTQ, which is a key direction for future research.
Revised: However, there is still a largeaccuracy gap between low-bit PTQ and full precision network, and we believe that there is still muchroom for further improvement for PTQ, which is a key direction for future research.

Original: Broder impacts.
Revised: Broder impacts.

Original: The proposed method uses GPU resources for training, thus resulting in a certainamount of energy consumption and carbon footprints.
Revised: The proposed method uses GPU resources for training, thus resulting in a certainamount of energy consumption and carbon footprints.

Original: For example, it takes about 4 GPU-hours forour method to quantize ResNet18, which would consume around 1 kWh and produce around 1 lbs of
Revised: For example, it takes about 4 GPU-hours forour method to quantize ResNet18, which would consume around 1 kWh and produce around 1 lbs of

Original: CO2, accelerating climate change and global warming in some degree.
Revised: CO2, accelerating climate change and global warming in some degree.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See
Revised: [Yes] See

Original: broader impacts.
Revised: broader impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes](b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes](b) Did you specify all the training details (e
Revised: [Yes] (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,
Revised: g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi- ments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We report error bars in Table 1.
Revised: [Yes] We report error bars in Table 1.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: of GPUs, internal cluster, or cloud provider)?
Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (eg.
Revised: If you are using existing assets (e

Original: , code, data, models) or curating [Table]
Revised: g., code, data, models) or curating/releasing new assets...

Original: releasing [Table] assets... (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [No]
Revised: [No]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [No] We use public datasets only.
Revised: [No] We use public datasets only.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A ASoftmax Initialization
Revised: A ASoftmax Initialization

Original: First of all, We define the fractional part of x as: { x } = x − ⌊ x ⌋ , and we use w 0 to represent initiallysimulated quantization of w .
Revised: First of all, We define the fractional part of x as: { x } = x − ⌊ x ⌋ , and we use w 0 to represent initiallysimulated quantization of w .

Original: According to equation (9),(10), (12) and (13), we get:[Equation]
Revised: According to equation (7),(8), (10) and (11), we get:

Original: [Equation]
Revised: [Equation]

Original: If we disregard the clip function, then
Revised: If we disregard the clip function, then

Original: [Equation]
Revised: [Equation]