Original: Leveraging Inter-Layer Dependency for Post -Training Quantization
Revised: Leveraging Inter-Layer Dependency for Post -Training Quantization

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Prior works on Post-training Quantization (PTQ) typically separate a neural network into sub-nets and quantize them sequentially.
Revised: Prior works on Post-training Quantization (PTQ) typically separate a neural network into sub-nets and quantize them sequentially.

Original: This process pays little attentionto the dependency across the sub-nets, hence is less optimal.
Revised: This process pays little attentionto the dependency across the sub-nets, hence is less optimal.

Original: In this paper, wepropose a novel Network-Wise Quantization (NWQ) approach to fully leveraginginter-layer dependency.
Revised: In this paper, wepropose a novel Network-Wise Quantization (NWQ) approach to fully leveraginginter-layer dependency.

Original: NWQ faces a larger scale combinatorial optimization problem of discrete variables than in previous works, which raises two major challenges:over-fitting and discrete optimization problem.
Revised: NWQ faces a larger scale combinatorial optimization problem of discrete variables than in previous works, which raises two major challenges:over-fitting and discrete optimization problem.

Original: NWQ alleviates over-fitting viaa Activation Regularization (AR) technique, which better controls the activationdistribution.
Revised: NWQ alleviates over fitting viaa Activation Regularization (AR) technique, which better controls the activationdistribution.

Original: To optimize discrete variables, NWQ introduces Annealing Softmax(ASoftmax) and Annealing Mixup (AMixup) to progressively transition quantizedweights and activations from continuity to discretization, respectively.
Revised: To optimize discrete variables, NWQ introduces Annealing Softmax(ASoftmax) and Annealing Mixup (AMixup) to progressively transition quantizedweights and activations from continuity to discretization, respectively.

Original: Extensiveexperiments demonstrate that NWQ outperforms previous state-of-the-art by alarge margin: 20.24% for the challenging configuration of MobileNetV2 with 2bits on ImageNet, pushing extremely low-bit PTQ from feasibility to usability.
Revised: Extensiveexperiments demonstrates that NWQ outperforms prior state-of-the-art approachesby a large margin: 20.24% for the challenging configuration of MobileNetV2 with2 bits on ImageNet, pushing extremely low-bit PTQ from feasibility to usability.

Original: Inaddition, NWQ is able to achieve competitive  results with only 10% computationcost of previous works.
Revised: In addition, NWQ is able to achieve competitive or better results with only 10%computation cost of previous works.

Original:  Introduction
Revised:  Introduction

Original: Deep learning has seen a boom of architectures with ever-increasing capabilities and capacity.
Revised: Deep learning has seen a boom of architectures with ever-increasing capabilities and capacity.

Original: Hugecapacity, however, always comes with boosting parameters, resulting in large computation costand memory footprint.
Revised: Hugecapacity, however, always comes with boosting parameters, resulting in large computation costand memory footprint.

Original: Researchers have utilized strategies like neural architecture search (NAS)[4, 11, 29, 50], network pruning [12, 19] and knowledge distillation (KD)
Revised: Researchers have utilized strategies like neural architecture search (NAS)[4, 11, 29, 50], network pruning [12, 19] and knowledge distillation (KD)

Original: [18, 21] to design compactnetworks and improve their accuracy.
Revised: [18, 21] to design compactnetworks and improve their accuracy.

Original: Furthermore, neural network quantization [7, 15, 22, 25, 28]is typically applied along with the methods mentioned above to reduce the computation cost andmemory footprint.
Revised: Furthermore, neural network quantization [7, 15, 22, 25, 28]is typically applied along with the methods mentioned above to reduce the computation cost andmemory footprint.

Original: Quantization represents neural networks with lower bit precision, usually 8 bits inpractice, and saves 75% of memory while offering 2 ∼ 4 × speedup[26].
Revised: Quantization represents neural networks with lower bit precision, usually 8 bits inpractice and saves 75% of memory while offering 2 ∼ 4 × speedup[26].

Original: Network quantization techniques are typically categorized into Quantization Aware Training (QAT)and Post-training Quantization(PTQ) based on the dependency on labeled data and the necessity ofweight tuning.
Revised: Network quantization techniques are typically categorized into Quantization Aware Training (QAT)and Post-training Quantization(PTQ) based on the dependency on labeled data and the necessity ofweight tuning.

Original: Generally, PTQ techniques are preferred in industry because they rely less on labeleddata and take into account privacy concerns as well as compute resources.
Revised: Generally, PTQ techniques are preferred in industry because they rely less on labeleddata and take into account privacy concerns as well as compute resources.

Original: This paper focuses on
Revised: This paper focuses on

Original: PTQ.
Revised: PTQ.

Original: PTQ usually suffers from significant accuracy degradation, especially for low-bit quantization whichoffers much more memory reduction and speedup.
Revised: PTQ usually suffers from significant accuracy degradation, especially for low-bit quantization whichoffers much more memory reduction and speedup.

Original: To improve the accuracy of PTQ, much efforthas been devoted [5, 32, 34, 36, 40].
Revised: To improve the accuracy of PTQ, much efforthas been devoted [5, 32, 34, 36, 40].

Original: AdaRound[32] has recently discovered that round-to-nearestappears to be optimal for single weight but not for the entire layer.
Revised: AdaRound[32] has recently discovered that round-to-nearestappears to be optimal for single weight but not for the entire layer.

Original: The rounding errors of weights ina single layer have the capability of compensating for each other, contributing to a smaller layer-wise  quantization error.
Revised: The rounding errors of weights ina single layer have the capability of compensating for each other, contributing to a smaller layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (a) layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (b) block-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (c) network-wise quantization error.

Original: We term this property as intra-layer dependency .
Revised: We term this property as intra-layer dependency .

Original: Motivated by this property,
Revised: Motivated by this property,

Original: AdaRound[32] proposes to learn the round policy layer by layer to improve accuracy.
Revised: AdaRound[32] proposes to learn the round policy layer by layer to improve accuracy.

Original: Inspired by AdaRound[32], we hypothesize that the quantization errors of all layers in a network isable to compensate for each other, resulting in a reduced quantization error.
Revised: Inspired by AdaRound[32], we hypothesize that the quantization errors of all layers in a network isable to compensate for each other, resulting in a reduced quantization error.

Original: We refer to this capabilityas inter-layer dependency , which attracts little attention in previous works.
Revised: We refer to this capabilityas inter-layer dependency . However, AdaRound assumes that layers are mutual-independent andderives a layer-diagonal Hessian, leading to a layer-wise solution(Figure 1a). BRECQ points out thatthe approximation of AdaRound is not accurate, they restrict the inter-layer dependency within a blockand assume that blocks are mutual-independent, thus extend the Hessian to a block-diagonal matrixfor a block-wise approach(Figure 1b). We argue that neither the approximations of AdaRound nor thatof BRECQ are accurate enough, the inter-layer dependency should be leveraged in a network-wisemanner.

Original: Based on that, we proposea Network-Wise Quantization (NWQ) scheme, which is different from previous layer-wise[23, 32]and block-wise [27, 41] methods, to fully leverage  this capability.
Revised: To achieve this, we propose a Network-Wise Quantization (NWQ, Figure 1c) scheme tofully leverage the inter-layer dependency. AdaRound and BRECQ separate a network into sub-netsand train them sequentially, this process makes the preceding layers unaware of succeeding layers. NWQ solves this problem via training the entire network end-to-end, where layers are aware of eachother and they are able to cooperate for optimal solution.

Original: However, quantizing the entirenetwork is a combinatorial optimization problem where huge number of discrete parameters fromall layers are combined together for optimization.
Revised: However, quantizing the entire networkend-to-end is a combinatorial optimization problem where huge number of discrete parameters fromall layers are combined together for optimization.

Original: This optimization problem will raise two majorchallenges that result in significant accuracy degradation.
Revised: This optimization problem will raise two majorchallenges that result in significant accuracy degradation.

Original: The first challenge is over-fitting.
Revised: The first challenge is over-fitting.

Original: Extending the reconstruction unit from layer/block to entire networkwill increase parameters dramatically and thus raises the risk of over-fitting over a typically smallcalibration set.
Revised: Extending the reconstruction unit from layer/block to entire networkwill increase parameters dramatically and thus raises the risk of over-fitting over a typically smallcalibration set.

Original: To alleviate this issue, NWQ applies Activation Regularization (AR), a variant of
Revised: To alleviate this issue, NWQ applies Activation Regularization (AR), a variant of

Original: L2 regularization, on intermediate representation to keep the quantized activations as close to thefloating-point counterpart as possible.
Revised: L2 regularization, on intermediate representation to keep the quantized activations as close to thefloating-point counterpart as possible.

Original: The benefits of AR are two folds.
Revised: The benefits of AR are two folds.

Original: First, AR regularizesquantized activation flow to prevent over-fitting.
Revised: First, AR regularizesquantized activation flow to prevent over-fitting.

Original: Second, it works as a quantization loss function toencourage minimal intermediate quantization error.
Revised: Second, it works as a quantization loss function toencourage minimal intermediate quantization error.

Original: AR turns out to be both simple and effective.
Revised: AR turns out to be both simple and effective.

Original: Another challenge is discrete optimization.
Revised: Another challenge is discrete optimization.

Original: Based on AdaRound, NWQ learns the weight round policyand activation quantization parameters, both of which involve in discrete variables.
Revised: Based on AdaRound, NWQ learns the weight round policyand activation quantization parameters, both of which involve in discrete variables.

Original: However, thesubstantially increased depth and discrete variables make it more difficult to convergence comparedto layer-wise[23, 32] and block-wise[27, 41] approach.
Revised: However, thesubstantially increased depth and discrete variables make it more difficult to convergence comparedto layer-wise[23, 32] and block-wise[27, 41] approach.

Original: The insufficient convergence will leave thediscrete variables (weight round policy) in a state of continuity rather than discretization, leadingto great performance drop.
Revised: The insufficient convergence will leave thediscrete variables (weight round policy) in a state of continuity rather than discretization, leadingto great performance drop.

Original: To solve this problem, we propose Annealing Sofmtax (ASoftmax) and
Revised: To solve this problem, we propose Annealing Sofmtax (ASoftmax) and

Original: Annealing Mixup (AMixup) to optimize discrete weights and activations via continuous relaxation[35].
Revised: Annealing Mixup (AMixup) to optimize discrete weights and activations via continuous relaxation[35].

Original: Specifically, ASoftmax optimizes weight round policy via annealing the temperature to transitionthe continuous probability vector to one-hot vector.
Revised: Specifically, ASoftmax optimizes weight round policy via annealing the temperature to transitionthe continuous probability vector to one-hot vector.

Original: AMixup randomly mixes up the pre-quantizationactivations with the post-quantization counterpart and gradually decreases the pre-quantizationpercentage in mixed activations 1 .
Revised: AMixup randomly mixes up the floating-pointactivations and the quantized counterpart and gradually decreasing the floating-point percentage.

Original: The benefit of ASoftmax and AMixup is that they guarantee thedescretization after optimization while offering continuous relaxation.
Revised: Thebenefit of ASoftmax and AMixup is that they guarantee the descretization after optimization whileoffering continuous relaxation.

Original: Equipped with AR, ASoftmax and AMixup, we are able to build our NWQ framework that fullyleverages inter-layer dependency to achieve higher accuracy than existing works.
Revised: Equipped with AR, ASoftmax and AMixup, we are able to build our NWQ framework that fullyleverages inter-layer dependency to achieve higher accuracy than existing works.

Original: To summarize, this paper makes the following contributions:
Revised: To summarize, this paper makes the following contributions:

Original: We propose a novel network-wise quantization framework for PTQ to fully leverage inter- layer dependency .
Revised: We propose a novel network-wise quantization framework for PTQ to fully leverage interlayer dependency .

Original: We analyze the combinatorial optimization problem introduced by network-wise quantizationand propose AR, ASoftmax and AMixup to solve the over-fitting and discrete optimization problem.3.
Revised: We analyze the combinatorial optimization problem introduced by network-wise quantization and propose AR, ASoftmax and AMixup to solve the over-fitting and discreteoptimization problem.3.

Original: We establish a new state-of-the-art through significantly improving PTQ by a large margin:20.24% for MobileNetV2 with 2 bits and thus push the limit of extremely low-bit PTQ fromfeasibility to usability.
Revised: We establish a new state-of-the-art through significantly improving PTQ by a large margin:20.24% for MobileNetV2 with 2 bits and thus push the limit of extremely low-bit PTQ fromfeasibility to usability.

Original:  Related Work
Revised:  Related Work

Original: Network quantization is commonly divided into QAT [7, 15, 22, 25, 49] and PTQ [27, 32, 33, 48].
Revised: Network quantization is commonly divided into QAT [7, 15, 22, 25, 49] and PTQ [27, 32, 33, 48].

Original: QAT methods tend to minimize task error by training weights and quantization parameters jointlywith the entire labeled dataset and thus require a large number of computing resources, whereas PTQmethods focus on finding optimal quantization parameters for off-the-shelf well-trained networks withmuch less data and compute resources than QAT.
Revised: QAT methods tend to minimize task error by training weights and quantization parameters jointlywith the entire labeled dataset and thus require a large number of computing resources, whereas PTQmethods focus on finding optimal quantization parameters for off-the-shelf well-trained networks withmuch less data and compute resources than QAT.

Original: PTQ can further be classified into statistic-based
Revised: PTQ can further be classified into statistic-based

Original: PTQ and learning-based PTQ .
Revised: PTQ and learning-based PTQ .

Original: Statistic-based PTQ methods [33, 36, 48] first estimate the distributionof activation on calibration data, and then find the optimal quantization parameters that minimizelocal quantization errors heuristically.
Revised: Statistic-based PTQ methods [33, 36, 48] first estimate the distributionof activation on calibration data, and then find the optimal quantization parameters that minimizelocal quantization errors heuristically.

Original: Learning-based PTQ methods[23, 27, 32, 41] dedicate toslightly weight tuning and quantization parameters learning.
Revised: Learning-based PTQ methods[23, 27, 32, 41] dedicate toslightly weight tuning and quantization parameters learning.

Original: Our work follows the learning-based
Revised: Our work follows the learning-based

Original: PTQ scheme.
Revised: PTQ scheme.

Original: [Table]

Original: [Figure]

Original: … 𝐴𝑅𝐿 :;<=> + … 𝐴𝑅 ?

Original: @ 𝑙𝑜𝑠𝑠

Original: [Figure]

Original: ASoftmax 𝐿 <A + [Figure] Learning-based PTQ : AdaRound[32] adopts the regime of learning the optimal round policy layerby layer.
Revised: Learning-based PTQ : AdaRound[32] adopts the regime of learning the optimal round policy layerby layer.

Original: BRECQ [27] succeeds in extending AdaRound [32] to a block-wise reconstruction andconcludes that block-wise quantization is superior to network-wise method.
Revised: BRECQ [27] succeeds in extending AdaRound [32] to a block-wise reconstruction andconcludes that block-wise quantization is superior to network-wise method.

Original: We argure that bothlayer-wise and block-wise approach is sub-optimal because the preceding layers/blocks are unawareof succeeding layers/block, thus the dependency they leveraged is unidirectional.
Revised: We argure that bothlayer-wise and block-wise approach is sub-optimal because the preceding layers/blocks are unawareof succeeding layers/block, thus the dependency they leveraged is unidirectional.

Original: Some works focuson the sequence of weight and activation quantization.
Revised: Some works focuson the sequence of weight and activation quantization.

Original: AdaQuant [23] proposes to tune weights andactivation quantization parameters separately for each layer to reduce over-fitting, while QDROP[41] finds that incorporating activation quantization into weight tuning can achieve higher accuracy.
Revised: AdaQuant [23] proposes to tune weights andactivation quantization parameters separately for each layer to reduce over-fitting, while QDROP[41] finds that incorporating activation quantization into weight tuning can achieve higher accuracy.

Original: Activiation Regularization : Large values in the learned dense representation can be a sign ofbeing overfit.
Revised: Activiation Regularization : Large values in the learned dense representation can be a sign of beingoverfit.

Original: To encourage small activations, [31] employs L2 regularization on hidden state ofrecurrent neural networks (RNN), while various normalizations [2, 24, 39, 43] focus on centralizethe activation distribution to regularize large values.
Revised: To encourage small activations, [31] employs L2 regularization on hidden state of recurrentneural networks (RNN), while various normalizations [2, 24, 39, 43] focus on centralize the activationdistribution to regularize large values.

Original: Some prior works attempt to learn sparserepresentation.
Revised: Some prior works attempt to learn sparse representation.

Original: Dropout and its variants [1, 10, 37, 38, 42] randomly drop activations from the neuralnetwork to encourage sparsity.
Revised: Dropout and its variants [1, 10, 37, 38, 42] randomly drop activations from the neural network toencourage sparsity.

Original: ReLUs[16, 20, 30, 44] dedicate to learning sparse representation viarectifying negative values.
Revised: ReLUs[16, 20, 30, 44] dedicate to learning sparse representation via rectifyingnegative values.

Original: These techniques aim to regularize activations when training a neuralnetwork from scratch, which inapplicable for PTQ where the networks are off-the-shelf andwelltrained, thus the activation distributions are deterministic and immutable.
Revised: These techniques aim to regularize activations when training a neural networkfrom scratch, which inapplicable for PTQ where the networks are off-the-shelf and well-trained,thus the activation distributions are deterministic and immutable.

Original: Instead of encouragingsmall and sparse activations, NWQ apply activation regularization on intermediate representation toencourage small offsets between quantized activations and the floating-point counterpart.
Revised: Instead of encouraging small andsparse activations, NWQ apply activation regularization on intermediate representation to encouragesmall offsets between quantized activations and the floating-point counterpart.

Revised: AdaRound[32] and

Revised: BRECQ [27] also employ quantization losses on intermediate activations but those losses worksas optimization objectives rather than regularization, while objective of NWQ is to minimize thequantization error of the final network output and the regularization on intermediation activations isto prevent optmization process from over-fitting.

Original: Discrete Optimization for Quantization : Network quantization optimizes discrete variables viagradient backpropagation.
Revised: Discrete Optimization for Quantization : Network quantization optimizes discrete variables viagradient backpropagation.

Original: However, the quantization process introduces a round function which isindifferentiable.
Revised: However, the quantization process introduces a round function which isindifferentiable.

Original: Some works on QAT focus on approximating the gradient through the round function[3, 6, 13, 14, 17, 45].
Revised: Some works on QAT focus on approximating the gradient through the round function[3, 6, 13, 14, 17, 45].

Original: For PTQ, AdaRound [32] introduces additional discrete parameters(the binaryround policy) and optimize them via Rectified Sigmoid with an explicit Regularization (RSeR) lossfunction.
Revised: For PTQ, AdaRound [32] introduces additional discrete parameters(the binaryround policy) and optimize them via Rectified Sigmoid with an explicit Regularization (RSeR) lossfunction.

Revised: However, RSeR is adversarial to quantization loss and thus difficult to convergence innetwork-wise quantization.

Original: QDROP [41] proposes to randomly drop quantized activations to improve the flatnessof optimization landscape.
Revised: QDROP [41] proposes to randomly drop quantized activations to improvethe flatness of optimization landscape, but introcues train-test inconsistency problem which results inperformance degradation if used in network-wise PTQ. [Table] [Figure] … 𝐴𝑅

Revised: [Figure]

Original: These approaches are proposed to improve layer-wise and block-wisequantization and encounter failure when applied to network-wise quantization, since the discreteoptimization problem is much more difficult.
Revised: 𝐿 :;<=> + … 𝐴𝑅 ? @ 𝑙𝑜𝑠𝑠 [Figure] 𝐿 <A

Revised: +

Revised: [Figure]

Original:  Approach
Revised:  Approach

Original:  Network-Wise Quantization (NWQ) is a simple end-to-end post-training quantization approach.
Revised: [Figure] Network-Wise Quantization (NWQ) is a simple end-to-end post-training quantization approach.

Original: The structure of NWQ is illustrated in Figure 1.
Revised: The structure of NWQ is illustrated in Figure 2.

Original: The optimization objective is composed of outputquantization loss and layer-wise AR loss.
Revised: The optimization objective is composed of outputquantization loss and layer-wise AR loss.

Original: Meanwhile, the annealing approaches (Asoftmax and
Revised: Meanwhile, the annealing approaches (Asoftmax and

Original: AMixup) are introduced in training process, which enable the quantized weights and activations togently converge from continuous state to discrete state, lowering training complexity and improvingaccuracy.
Revised: AMixup) are introduced in training process, which enable the quantized weights and activations togently converge from continuous state to discrete state, lowering training complexity and improvingaccuracy.

Original: 3.1 Activation Regularization
Revised: 3.1 Activation Regularization

Original: NWQ leverages Activation Regularization (AR) to penalize quantized intermediate activations thatare substantially away from the floating-point counterparts, thereby encouraging the activations toremain unchanged.
Revised: NWQ leverages Activation Regularization (AR) to penalize quantized intermediate activations thatare substantially away from the floating-point counterparts, thereby encouraging the activations toremain unchanged.

Original: Formally, we define AR by:
Revised: Formally, we define AR by:

Original: [Equation]
Revised: [Equation]

Original:  while G represents floating-point network and G is the simulated quantized counterpart.
Revised: (1) while G represents floating-point network and G is the simulated quantized counterpart.

Original: a l and a ldenote the floating-point and the simulated quantized activation of the l -th layer, respectively.
Revised: a l and a ldenote the floating-point and the simulated quantized activation of the l -th layer, respectively.

Original: Nrepresents the number of total layers and ∥·∥ F denotes the Frobenius norm.
Revised: Nrepresents the number of total layers and ∥·∥ F denotes the Frobenius norm.

Original: Our network quantizationerror L quant ( a N , a N ) is defined by:
Revised: Our network quantizationerror L quant ( a N , a N ) is defined by:

Original: [Equation]
Revised: [Equation]

Original: then the final objective can be formulated as:
Revised: then the final objective can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: Practically, L ar is not restricted to be utilized layer by layer, it can also be performed block byblock and stage by stage, as BRECQ [27] defined.
Revised: Practically, L ar is not restricted to be utilized layer by layer, it can also be performed block byblock and stage by stage, as BRECQ [27] defined.

Original: We will discuss how the granularity affects theperformance in Sec. 4.2.1.
Revised: We will discuss how the granularity affects theperformance in Sec. 4.2.1.

Original: By default, we employ layer-wise AR in our experiments.
Revised: By default, we employ layer-wise AR in our experiments.

Original: 3.2 Annealing Softmax
Revised: 3.2 Annealing Softmax

Original: 3.2.1 Formulation

Original: For a weight vector w , AdaRound [32] proposes to learn the round policy by:
Revised: For a weight vector w , AdaRound [32] proposes to learn the round policy by:

Original: [Equation]
Revised: [Equation]

Original: (3) where ⌊·⌋ gives as output of the integral part and clip ( · ) clamps all values between q − and q + .
Revised: where ⌊·⌋ gives as output of the integral part and clip ( · ) clamps all values between q − and q + .

Original: ˆ w idenotes the quantized weight for the i -th element of w .
Revised: ˆ w idenotes the quantized weight for the i -th element of w .

Original: We will omit i below for simplicity.
Revised: We will omit i below for simplicity.

Original: s is thequantizer step size.
Revised: s is thequantizer step size.

Original: v is a learnable continuous vector w.r.t.
Revised: v is a learnable continuous vector w.r.t.

Original: w and h ( v ) is recitified sigmoid function(see Eqn.(4)) which is regularized to converge to { 0 , 1 } . 
Revised: w and h ( v ) is recitified sigmoid function(see Eqn.(4)) which is regularized to converge to { 0 , 1 } . [Equation]

Revised: Based on Eqn.(4), AdaRound proposes Rectified Sigmoid with an explicit Regularization (RSeR) todiscrete h ( v ) :

Original: [Equation]
Revised: [Equation]

Revised: However, there are two limitations of RSeR. First, RSeR works only for binary round policy.

Revised: Second,

Original: Based on Eqn.(3), we extend the discrete optimization space of h ( · ) from { 0 , 1 } to K , where K is ainteger range and defined by:
Revised: RSeR is adversarial to quantization loss. For example, for a linear transformation network, thequantization loss is [Equation] where x denotes the input and we ignore the clip function in Eqn.

Revised: Eqn.

Revised: (6) indicates that quantization loss will prevent L RSeR from convergence ( i.e. , L RSeR encourages h ( v ) to binary while L q prefers w s − ⌊ w s ⌋ ).

Revised: Besides, this adversarial become more severe whenextending layer/block-wise methods to network-wise quantization(see Sec.

Revised: 4.2.2 and Table 4).

Revised: 3.2.1 Formulation

Revised: To resolve the two problems of RSeR, we propose Annealing Softmax (ASoftmax).

Revised: First, we extendthe discrete optimization space of h ( · ) from { 0 , 1 } to K , where K is a integer range and defined by:

Revised: K

Original: [Equation]
Revised: [Equation]

Original: where n and m are hyper-parameters that specify the discrete optimization space.
Revised: where n and m are hyper-parameters that specify the discrete optimization space.

Original: Then, ASoftmaxcan be formulated as:
Revised: Then, ASoftmaxcan be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where V is a continuous learnable matrix and represents the logits of probability over K .
Revised: where V is a continuous learnable matrix and represents the logits of probability over K .

Original: h t ( V ) isdefined by:
Revised: h t ( V ) isdefined by:

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where t represents for t -th iteration.
Revised: where t represents for t -th iteration.

Original: τ t is a temperature that decays linearly to transition h t ( V )from continuity to discretization.
Revised: τ t is a temperature that decays linearly to transition h t ( V )from continuity to discretization.

Original: For example, when τ t is large, p ( k | V , τ t ) is a continuous vector of probability and h t ( V ) is the expectation of the discretization.
Revised: For example, when τ t is large, p ( k | V , τ t ) is a continuous vectorof probability and h t ( V ) is the expectation of the discretization.

Original: When τ t turns tiny, p ( k | V , τ t )becomes a one-hot vector and h t ( V ) selects k by p ( k | V , τ t )
Revised: When τ t turns tiny, p ( k | V , τ t )becomes a one-hot vector and h t ( V ) selects k by p ( k | V , τ t )

Original: = 1 .
Revised: = 1 .

Original: τ t is simply decayed by:
Revised: τ t is simply decayed by:

Original: [Equation]
Revised: [Equation]

Original: 9) where τ 0 and τ T are hyper-parameters of initial and final temperature.
Revised: [Equation] where τ 0 and τ T are hyper-parameters of initial and final temperature.

Original: ASoftmax is a general formation of learning wider range of discrete parameters than binary.
Revised: ASoftmax is a general formation of learning wider range of discrete parameters than binary.

Original: It easilydegenerates to round policy learning by setting n = 0 and m = 1 .
Revised: It easilydegenerates to round policy learning by setting n = 0 and m = 1 .

Original: We use τ T = 0 .
Revised: We use τ T = 0 .

Original: 01 and τ 0 = 1 bydefault.
Revised: 01 and τ 0 = 1 bydefault.

Original: 3.2.2 Initialization
Revised: 3.2.2 Initialization

Original: According to our observations, initialization has significant impacts on learning round policy.
Revised: According to our observations, initialization has significant impacts on learning round policy.

Original: Giventhe weight quantizer step size s , which has been computed using statistic-based PTQ methods, weattempt to initialize V in order to makeˆ s ·  w 0 as close to floating-point w as possible at the start.
Revised: Giventhe weight quantizer step size s , which has been computed using statistic-based PTQ methods, weattempt to initialize V in order to make s · ˆ w 0 as close to floating-point w as possible at the start.

Original: Toachieve this goal, we first calculate the un-normalized probability by:
Revised: Toachieve this goal, we first calculate the un-normalized probability by:

Original: [Equation]
Revised: [Equation]

Original: Then, we normalize the probability:
Revised: Then, we normalize the probability:

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Finally, the initialization for V
Revised: Finally, the initialization for V

Original: i is log ( σ ( V ) i ) .
Revised: i is log ( σ ( V ) i ) .

Original: It can be easily proved (see Appendix A) that theinitializedˆ s ·  w 0 equals w as long as the integer range K is symmetric w.r.t.
Revised: It can be easily proved (see Appendix A) that theinitialized s · ˆ w 0 equals w as long as the integer range K is symmetric w.r.t.

Original: 0.5 ( i.e. n = 1 − m ).
Revised: 0.5 ( i.e. n = 1 − m ).

Original: 3.3 Annealing Mixup
Revised: 3.3 Annealing Mixup

Original: NWQ considers activation quantization as a discrete optimization problem and solves it via continuousrelaxation.
Revised: NWQ considers activation quantization as a discrete optimization problem and solves it via continuousrelaxation.

Original: Different from ASoftmax which is applied on element-wise level, AMixup works on tensorwise level: we randomly mix up pre-quantization activations with post-quantization activations.
Revised: Different from ASoftmax which is applied on element-wise level, AMixup works ontensor-wise level: we randomly mix up the floating-point activations with the simulated quantizedactivations.

Original: Toencourage discretization, NWQ progressively decreases the percentage of pre-quantization activations,and the mixed activations will become fully quantized activation gradually.
Revised: To encourage continuous relaxation converge to discretization, NWQ progressivelydecreases the percentage of floating-point activations, so the mixed activations will become fullyquantized activation gradually.

Original: Formally, the percentage of pre-quantization activationsfor t -th iteration is defined by:
Revised: Formally, the percentage of floating-point activations is defined by:

Original: [Equation]
Revised: [Equation]

Original: where P  s and P e denote the percentage atthe start andthe end respectively, and T is the number ofiterations.
Revised: where P ( t ) is the percentage of floating-point activation in the mixed activation for the t -th iteration. P s and P e denote the percentage at start and end respectively, and T is the number of iterations.

Original: We use P s = 0 .
Revised: Weuse P s = 0 .

Original: 5 and P e = 0 by default.
Revised: 5 and P e = 0 by default.

Original: AMixup is similar to QDROP [41] in the mixup operation.
Revised: AMixup is similar to QDROP [41] in the mixup operation.

Original: The key difference, however, is that
Revised: The key difference, however, is that

Original: AMixup introduces an annealing process.
Revised: AMixup introduces an annealing process.

Original: This is very important for NWQ.
Revised: This is very important for NWQ.

Original: QDROP trains with mixedactivations while testing with fully quantized activations, thus introducing inconsistency bias.
Revised: QDROP trains with mixedactivations while testing with fully quantized activations, thus introducing inconsistency bias.

Original: Thisbias accmulates as network depth increases and leads to great accuracy degradation in network-wisequantization.
Revised: Thisbias accmulates as network depth increases and leads to great accuracy degradation in network-wisequantization.

Original: AMixup is able to eliminate the inconsistency bias via the annealing process.
Revised: AMixup is able to eliminate the inconsistency bias via the annealing process.

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct extensive experiments with various architectures to verify our NWQ.
Revised: In this section, we conduct extensive experiments with various architectures to verify our NWQ.

Original: We first compare our method with previous state-of-the-art, and then we delve into our importantdesigns and fully explore the scalability, effectiveness, and efficiency of our solution.
Revised: We first compare our method with previous state-of-the-art, and then we delve into our importantdesigns and fully explore the scalability, effectiveness, and efficiency of our solution.

Original: We report top-1classification accuracy on the ImageNet[9] validation set.
Revised: We report top-1classification accuracy on the ImageNet[9] validation set.

Original: Implementation Details.
Revised: Implementation Details.

Original: Our code is based on open-source codes BRECQ and QDrop.
Revised: Our code is based on open-source codes BRECQ and QDrop.

Original: We applylayer-wise AR.
Revised: Weapply layer-wise AR.

Original: For ASoftmax, we set n = 0 , m = 1 and τ is decayed from 1 to 0 .
Revised: For ASoftmax, we set n = 0 , m = 1 and τ is decayed from 1 to 0 .

Original: For AMixup, we setdefault P s as 0.5 and P e as 0.
Revised: For AMixup, we set default P s as 0.5 and P e as 0.

Original: We randomly sample 1024 images from ImageNet train set and employ[Table] Cutmix [46] and Mixup[47] as data augmentation.
Revised: We randomly sample 1024 images from ImageNettrain set and employ Cutmix [46] and Mixup[47] as data augmentation.

Original: The learning rates are 0 .
Revised: The learning rates are0 .

Original: 01 for round policy and0 . {+
+}
Revised: 01 for round policy and 0 . {+
+}

Original: 0004 for activation quantizer step size.
Revised: for activation quantizer step size.

Original: We train for 20000 iterations with a mini-batch size of 32 in [-
-] Tesla V100 GPUs, taking ∼ 30 minutes for ResNet18, which is on par with BRECQ and QDROP.
Revised: We train for 20000 iterationswith a mini-batch size of 32 in [-
-]8 Tesla V100 GPUs, taking ∼ 30 minutes for ResNet18, whichis on par with BRECQ and QDROP.

Original: Ourexperiments are conducted on 5 architectures, includingResNet18(Res18), MobileNetV2(MNV2),
Revised: Our experiments are conducted on 5 architectures, including

Original:  RegNet-600MF(Reg600MF), RegNet-3.2GF(Reg3.2GF) and MnasNet(Mnas).
Revised: ResNet18(Res18), MobileNetV2(MNV2), RegNet-600MF(Reg600MF), RegNet-3.2GF(Reg3.2GF)nad MnasNet(Mnas).

Original: Other settings remainsame as QDROP[41] if not specified.
Revised: Other settings remain same as QDROP[41] if not specified.

Revised: We use abovehyper-parameters across the entire paper by default unless explicitly specified.

Original: 4.1 Comprehensive Comparison
Revised: 4.1 Comprehensive Comparison

Original: Table 1 compares our NWQ with several main-stream PTQ approaches.
Revised: Table 1 compares our NWQ with several main-stream PTQ approaches.

Original: Overall, NWQ outperformsprevious works across all neural network architectures and bit widths, and our performance improveseven more as the bit width decreases.
Revised: Overall, NWQ outperformsprevious works across all neural network architectures and bit widths, and our performance improveseven more as the bit width decreases.

Original: Classicial Calibration Set.
Revised: Classicial Calibration Set.

Original: Following previous works, we compare the performance on a smallcalibration set of 1024 images.
Revised: Following previous works, we compare the performance on a smallcalibration set of 1024 images.

Original: For start, we achieve 0.2% ∼ 0.9% improvement even compared withstrong baselines on W4A4, which are rather close to full precision accuracy.
Revised: For start, we achieve 0.2% ∼ 0.9% improvement even compared withstrong baselines on W4A4, which are rather close to full precision accuracy.

Original: For W3A3, NWQ yields0.83% ∼ 3.26% and reduces the gap between W3A3 and W4A4.
Revised: For W3A3, NWQ yields0.83% ∼ 3.26% and reduces the gap between W3A3 and W4A4.

Original: For the challenging W2A2, ourmethod surpasses the previous state-of-the-art by a large margin of 13.37% for MobileNetV2 and12.4% for MnasNet, pushing the limit of extremely low-bit PTQ step further. 
Revised: For the challenging W2A2, ourmethod surpasses the previous state-of-the-art by a large margin of 13.37% for MobileNetV2 and12.4% for MnasNet, pushing the limit of extremely low-bit PTQ step further. [Table]

Original: Scaled-up Calibration Set.
Revised: Scaled-up Calibration Set.

Original: To further explore the potential of NWQ, we scale up the calibrationset by 10 × and reproduce QDROP on the scaled-up calibration set, building a very strong baseline.
Revised: To further explore the potential of NWQ, we scale up the calibrationset by 10 × and reproduce QDROP on the scaled-up calibration set, building a very strong baseline.

Original: Interestingly, NWQ outperforms QDROP even with one-tenth data.
Revised: Interestingly, NWQ outperforms QDROP even with one-tenth data.

Original: When equipping NWQ withthe scaled-up calibration set, we observe significant improvement under all settings.
Revised: When equipping NWQ withthe scaled-up calibration set, we observe significant improvement under all settings.

Original: Particularly,the accuracy gaps of W2A2 between NWQ and QDROP are enlarged even more as the calibrationset scales up.
Revised: Particularly,the accuracy gaps of W2A2 between NWQ and QDROP are enlarged even more as the calibrationset scales up.

Original: For example, the gap of MnasNet increases from 12.4% to 19.36%, and that of
Revised: For example, the gap of MnasNet increases from 12.4% to 19.36%, and that of

Original: MobileNetV2 increases from 13.37% to 20.24%, showing the excellent scalability of our method.[Table] [Table] [Table]
Revised: MobileNetV2 increases from 13.37% to 20.24%, showing the excellent scalability of our method.

Original: [Table]

Original: [Figure]

Original: 4.2 Ablation Study
Revised: 4.2 Ablation Study

Original: [Figure]

Original: 4.2.1 AR
Revised: 4.2.1 AR

Original: [Figure]

Original: The Effect of AR.
Revised: The Effect of AR.

Original: We study the effect of AR viacomparing the experiments with layer-wise AR tothe experiments without AR on W2A2.
Revised: We study the effect of AR via comparing the experiments with layer-wise AR tothe experiments without AR on W2A2.

Original: Table 2shows that layer-wise AR improves the accuracysignificantly.
Revised: Table 2 shows that layer-wise AR improves the accuracysignificantly.

Original: It’s worth noting that layer-wise ARimproves 5.22% for MobileNetV2 and 10.42% for MnasNet.
Revised: It’s worth noting that layer-wise AR improves 5.22% for MobileNetV2 and 10.42%for MnasNet.

Original: To understand how AR works, we analyze the loss curves during training in Figure 2.
Revised: To understand how AR works, we analyze the loss curves during training in Figure3.

Original: First, our AR experiment consistently convergesto lower loss on both the train and test datasets,resulting in higher accuracy.
Revised: First, our AR experiment consistently converges to lower loss on both the train and test datasets,resulting in higher accuracy.

Original: Second, AR accelerates convergence significantly and thus allows NWQ to reach competitive accuracy at a substantially lower computing cost.
Revised: Second, AR accelerates convergence significantly and thus allows NWQto reach competitive accuracy at a substantially lower computing cost.

Original: Third, even though AR does not totally eliminate over-fitting, it effectively bridges the loss gap between train and testdatasets.
Revised: Third, even though AR doesnot totally eliminate over-fitting, it effectively bridges the loss gap between train and test datasets.

Original: The Effect of Granularity.
Revised: The Effect of Granularity.

Original: Table 2 studies how AR granularity affects accuracy.
Revised: Table 2 studies how AR granularity affects accuracy.

Original: We discover that lightweight networks prefer coarse-grained AR,whereas heavyweight networks prefer fine-grained AR.
Revised: We discoverthat lightweight networks prefer coarse-grained AR, whereas heavyweight networks prefer finegrained AR.

Original: MobileNetV2, RegNet-600MF, and MnasNet,for example, achieve the best performance with block-wise AR, whereas ResNet18 and RegNet-3.2GFachieve the maximum accuracy with layer-wise AR.
Revised: MobileNetV2, RegNet-600MF, and MnasNet, for example, achieve the best performancewith block-wise AR, whereas ResNet18 and RegNet-3.2GF achieve the maximum accuracy withlayer-wise AR.

Original: Comparison with Data Augmentation.
Revised: Comparison with Data Augmentation.

Original: Data augmentation is a classical strategy used to relieveover-fitting due to its simplicity and effectiveness.
Revised: Data augmentation is a classical strategy used to relieveover-fitting due to its simplicity and effectiveness.

Original: We compare AR with strong augmentation ( e.g. 
Revised: We compare AR with strong augmentation ( e.g. [Table] [Table]

Original:  AutoAugment[8]) in Table 3.
Revised: [Table] [Table] [Figure] AutoAugment[8]) in Table 3.

Original: Notably, both AR and AutoAugment significantly improve accuracy,with AR contributing more than AutoAugment, particularly for sparse networks like MobileNetV2and MnasNet.
Revised: Notably, both AR and AutoAugment significantly improve accuracy,with AR contributing more than AutoAugment, particularly for sparse networks like MobileNetV2and MnasNet.

Original: Coupling these two techniques enables us to achieve better performance.
Revised: Coupling these two techniques enables us to achieve better performance.

Original: 4.2.2 ASoftmax Study
Revised: 4.2.2 ASoftmax Study

Revised: [Figure]

Original: The Effect of ASoftmax.
Revised: The Effect of ASoftmax.

Original: Table 5 compares ASoftmax to Rectified Sigmoid with explicit Regularization (RSeR) which is used in AdaRound.
Revised: Table 4 compares ASoftmax to Rectified Sigmoid with explicit Regularization (RSeR) which is used in AdaRound.

Original: We simply replace ASoftmax with RSeR in ourNWQ and [Table] [Table]
Revised: We simply replace ASoftmax with RSeR in our

Original:  run a grid search on regularization loss weight to explore the potential of RSeR.
Revised: NWQ and run a grid search on regularization lossweight to explore the potential of RSeR.

Original: We discover that RSeR requires a huge loss weight for NWQ, which is 10 3 ∼ 10 5 × larger than that of subnet-wiseapproaches.
Revised: We discover that RSeR requires a huge loss weight for NWQ, which is 10 3 ∼ 10 5 × larger than that ofsubnet-wise approaches.

Original: Even with exhausting hyper-parameter search, RSeR does not perform as well as in thesubnet-wise method.
Revised: Even with exhaustinghyper-parameter search, RSeR does not performas well as in the subnet-wise method.

Original: ASoftmax, on the other hand, outperforms RSeR by 12.58% and 21.56% on ResNet18 and MobileNetV2, respectively, without any additional hyper-parameter tuning.
Revised: ASoftmax,on the other hand, outperforms RSeR by 12.58%and 21.56% on ResNet18 and MobileNetV2, respectively, without any additional hyper-parametertuning. The Effect of Initialization.

Revised: Table 5 demonstratesthe effectiveness of our initialization, which keepsthe quantization model close to the float-point atthe start to steady the training.

Original: The Robustness of τ .
Revised: The Robustness of τ .

Original: We investigate the robustness of τ in Table 6.
Revised: We investigate the robustness of τ in Table 6.

Original: τ starts with a large numberto encourage continuous relaxation and ends with a small value to favor one-hot distribution.
Revised: τ starts with a large numberto encourage continuous relaxation and ends with a small value to favor one-hot distribution.

Original: Thefinal τ T determines the decaying speed of τ .
Revised: Thefinal τ T determines the decaying speed of τ .

Original: It can be observed that τ T has a robustness range of10 − 2 ∼ 10 − 4 , making it easily adaptable to a wider range of neural network architectures.
Revised: It can be observed that τ T has a robustness range of10 − 2 ∼ 10 − 4 , making it easily adaptable to a wider range of neural network architectures.

Original: The Extensiveness of ASoftmax.
Revised: The Extensiveness of ASoftmax.

Original: As described in Sec.
Revised: As described in Sec.

Original: 3.2.1, ASoftmax can easily extend thediscrete parameter space from { 0 , 1 } to a larger integer range, offering the opportunity for a betteroptimum.
Revised: 3.2.1, ASoftmax can easily extend thediscrete parameter space from { 0 , 1 } to a larger integer range, offering the opportunity for a betteroptimum.

Original: However, extending discrete range benefits MobileNetV2 and RegNet-600MF whiledegrading RegNet-3.2GF on a small calibration set, as shown in Table 7 upper group.
Revised: However, extending discrete range benefits MobileNetV2 and RegNet-600MF whiledegrading RegNet-3.2GF on a small calibration set, as shown in Table 7 upper group.

Original: We argue thatthe enlarged discrete range raises the risk of over-fitting for large networks.
Revised: We argue thatthe enlarged discrete range raises the risk of over-fitting for large networks.

Original: When we scale up thecalibration set, the performance no longer degrades and the performance improves even more. 
Revised: When we scale up thecalibration set, the performance no longer degrades and the performance improves even more. [Table] [Table] [Table]

Revised: [Table]

Original: 4.2.3 AMixup Study
Revised: 4.2.3 AMixup Study

Original: The Effect of AMixup.
Revised: The Effect of AMixup.

Original: QDROP presents good performance in block-wise reconstruction, but failsin our network-wise scheme.
Revised: QDROP presents good performance in block-wise reconstruction, but failsin our network-wise scheme.

Original: As illustrated in Table 4, we achieve 1.95% improvement for ResNet18and 19.96% for MobileNetV2 by simply removing QDROP.
Revised: As illustrated in Table 8, we achieve 1.95% improvement for ResNet18and 19.96% for MobileNetV2 by simply removing QDROP.

Original: We hypothesize that randomly mixingactivations up introduces inconsistency bias between the training and test phase, which is concealedby the improvement of flatness in block-wise reconstruction.
Revised: We hypothesize that randomly mixingactivations up introduces inconsistency bias between the training and test phase, which is concealedby the improvement of flatness in block-wise reconstruction.

Original: However, as the reconstruction unitgoes deeper (from block to network) and sparser, the problem becomes more severe, resulting insignificant degeneration.
Revised: However, as the reconstruction unitgoes deeper (from block to network) and sparser, the problem becomes more pronounced, resultingin significant degeneration.

Original: Over-parameterized ResNet18 is able to compensate for the side effect, but MobileNetV2 suffers a great performance drop.
Revised: Over-parameterized ResNet18 is able to compensate for the side effect,but MobileNetV2 suffers a great performance drop.

Original: Remarkably, our AMixup is able to overcome theinconsistency while taking advantage of the flatness, contributing to higher accuracy.[Table]
Revised: Remarkably, our AMixup is able to overcomethe inconsistency while taking advantage of the flatness, contributing to higher accuracy.

Original: The Policy of Decay.
Revised: The Policy of Decay.

Original: Table 8 explores the influence of different decay policies.
Revised: Table 9 explores the influence of different decay policies.

Original: We run a gridsearch for P s while freezing P e as 0.
Revised: We run a grid searchfor P s while freezing P e as 0.

Original: It’s worthnoting that MobileNetV2 and MnasNet prefer 0.5,whereas ResNet18 and RegNet prefer 1.
Revised: It’s worth noting that MobileNetV2 and MnasNet prefer 0.5, whereas ResNet18 and RegNet prefer 1.

Original: The basic law is very similar to that of AR granularity,which states that heavyweight networks requiremore regularization, implying that AMixup and AR are both capable of regularizing optimization.
Revised: The basic law is very similar to that of AR granularity, which statesthat heavyweight networks require more regularization, implying that AMixup and AR are bothcapable of regularizing accuracy.

Original: 4.3 Efficiency
Revised: 4.3 Efficiency

Original: We compare the efficiency of several approaches on ResNet18.
Revised: We compare the efficiency of several approaches on ResNet18.

Original: According to Figure 3, NWQoutperforms QDROP and BRECQ under various iterations.
Revised: According to Figure 4, NWQoutperforms QDROP and BRECQ under various iterations.

Original: Specifically, NWQ achieve competitiveresults only 10% computation cost, demonstrating the efficiency of NWQ.
Revised: Specifically, NWQ achieve competitiveresults only 10% computation cost, demonstrating the efficiency of NWQ.

Original: As the number of iterationsincreases, NWQ continues to improve accuracy, whereas BRECQ and QDROP reach their saturationat ∼ 1 .
Revised: As the number of iterationsincreases, NWQ continues to improve accuracy, whereas BRECQ and QDROP reach their saturationat ∼ 1 .

Original: 6 × 10 4 iterations.
Revised: 6 × 10 4 iterations.

Revised:  Discussion and Conclusion

Original: {+20 Iterations/
+}
Revised: {+20 Iterations/
+}

Original: [Figure]
Revised: [Figure]

Revised: cc

Original: [Figure]
Revised: [Figure]

Original: Discussion and Conclusion
Revised: NWQ(Ours) QDROP BRECQ

Original: Simple algorithms that scale well are the core of deep learning.
Revised: Simple algorithms that scale well are the core ofdeep learning.

Original: Prior works tend to quantize networksin layer-wise and block-wise manner.
Revised: Prior works tend to quantize networks in layer-wise and block-wise manner.

Original: We prove that network-wise quantization is a simpler andmore effective way than layer/block-wise solutions.
Revised: Weprove that network-wise quantization is a simplerand more effective way than layer/block-wise solutions.

Original: To accomplish this, we analyze the problemsintroduced by the network-wise regime and propose three key designs to address them.
Revised: To accomplish this, we analyze the problems introduced by the network-wise regime andpropose three key designs to address them.

Original: Based on that,we construct a simple and effective end-to-end network-wise PTQ framework.
Revised: Basedon that, we construct a simple and effective endto-end network-wise PTQ framework.

Original: Extensive experimentsdemonstrate that our solution is scalable, effective, and efficient.
Revised: Extensiveexperiments demonstrate that our solution is scalable, effective, and efficient.

Original: However, there is still a largeaccuracy gap between low-bit PTQ and full precision network, and we believe that there is still muchroom for further improvement for PTQ, which is a key direction for future research.
Revised: However, there isstill a large accuracy gap between low-bit PTQand full precision network, and we believe thatthere is still much room for further improvementfor PTQ, which is a key direction for future research.

Original: Broder impacts.
Revised: Broder impacts.

Original: The proposed method uses GPU resources for training, thus resulting in a certainamount of energy consumption and carbon footprints.
Revised: The proposed method uses GPU resources for training, thus resulting in a certainamount of energy consumption and carbon footprints.

Original: For example, it takes about 4 GPU-hours forour method to quantize ResNet18, which would consume around 1 kWh and produce around 1 lbs of
Revised: For example, it takes about 4 GPU-hours forour method to quantize ResNet18, which would consume around 1 kWh and produce around 1 lbs of

Original: CO2, accelerating climate change and global warming in some degree.
Revised: CO2, accelerating climate change and global warming in some degree.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See broader impacts.
Revised: [Yes] Seebroader impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?

Original: [Yes] We report error bars in Table1.
Revised: [Yes] We report error bars in Table

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?
Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [No]
Revised: [No]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [No] We use public datasets only.
Revised: [No] We use public datasets only.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A ASoftmax Initialization
Revised: A ASoftmax Initialization

Original: First of all, We define the fractional part of x as: { x } = x − ⌊ x ⌋ , and we use w 0 to represent initiallysimulated quantization of w .
Revised: First of all, We define the fractional part of x as: { x } = x − ⌊ x ⌋ , and we use w 0 to represent initiallysimulated quantization of w .

Original: According to equation (7),(8), (10) and (11), we get:
Revised: According to equation (9),(10), (12) and (13), we get:

Original: [Equation]
Revised: [Equation]

Revised: K i

Revised: [Equation]

Revised: [Equation]

Original: {+= (cid:80) m − n i =
+}
Revised: {+= (cid:80) m − n i =
+}

Revised: ˆ σ 0 ( V )

Revised: i

Revised: [Equation]

Revised: n

Revised: [Equation]

Revised: [Equation]

Revised: = (cid:80) m − n i =

Revised: [Equation]

Revised: σ

Revised: [Equation]

Revised: ) i − n

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: = {

Revised: [Equation]

Original: {+} (cid:80) m − n i =
+}
Revised: {+} (cid:80) m − n i =
+}

Revised: ˆ σ 0 ( V ) i − n (cid:88)

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: i

Revised: [Equation]

Revised: (cid:17)

Revised: = {

Revised: [Equation]

Original: {+} (cid:80) m − n i =
+}
Revised: {+} (cid:80) m − n i =
+}

Revised: ˆ σ 0 ( V ) i − n

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: = {

Revised: [Equation]

Original: {+(cid:80) m − n i =
+}
Revised: {+(cid:80) m − n i =
+}

Revised: ˆ σ 0 ( V ) i m − n (cid:88) i

Revised: =

Revised: ˆ

Revised: [Equation]

Revised: 0 ( V ) i = { w

Revised: [Equation]

Revised: [Equation]

Original: If we disregard the clip function, then
Revised: If we disregard the clip function, then

Original: [Equation]
Revised: [Equation]